1008851 - build fails due to syntax error in config.status

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[maillist-mozilla]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:28.0) Gecko/20100101 Firefox/28.0 (Beta/Release)
Build ID: 20140512024306

Steps to reproduce:

build 29.0.1 from source


Actual results:

checking for posix_fallocate... yes
updating cache ./config.cache
creating ./config.status
Traceback (most recent call last):
  File "./config.status", line 561, in <module>
    (''' JS_SHARED_LIBRARY ''', r'''  '''),
TypeError: 'tuple' object is not callable
*** Fix above errors and then restart with\
               "make -f client.mk build"
make[2]: *** [configure] Error 1


the generated config.status has these lines in it:
...
    (''' NONASCII ''', r''' .. ''')                                              
    (''' JS_SHARED_LIBRARY ''', r'''  '''),                                      
...

as one can see, the line with NONASCII is missing the comma

Comment 1

[maillist-mozilla]

reduced testcase from "configure" code

#!/bin/sh
NONASCII=$'\241\241'
sed 's/$/,/' >> /tmp/config.status <<EOF
    (''' NONASCII ''', r''' $NONASCII ''')
    (''' JS_SHARED_LIBRARY ''', r''' $JS_SHARED_LIBRARY ''')
EOF

it's a bug to print this invalid byte sequence and expect sed to ignore it:
http://lists.busybox.net/pipermail/busybox/2014-May/080856.html

Comment 2

[maillist-mozilla]

this patch seems to fix it

--- mozilla-release.org/configure
+++ mozilla-release/configure
@@ -25339,7 +25339,7 @@
   fi
 fi
 
-NONASCII=$'\241\241'
+NONASCII='\241\241'
 
 
 # Static build options
@@ -28174,7 +28174,7 @@
     (''' EXPAND_LIBS_LIST_STYLE ''', r''' $EXPAND_LIBS_LIST_STYLE ''')
     (''' EXPAND_LIBS_ORDER_STYLE ''', r''' $EXPAND_LIBS_ORDER_STYLE ''')
     (''' CL_INCLUDES_PREFIX ''', r''' $CL_INCLUDES_PREFIX ''')
-    (''' NONASCII ''', r''' $NONASCII ''')
+    (''' NONASCII ''', "$NONASCII")
     (''' JS_SHARED_LIBRARY ''', r''' $JS_SHARED_LIBRARY ''')
     (''' LIBXUL_LIBS ''', r''' $LIBXUL_LIBS ''')
     (''' GKMEDIAS_SHARED_LIBRARY ''', r''' $GKMEDIAS_SHARED_LIBRARY ''')
--- mozilla-release.org/js/src/configure
+++ mozilla-release/js/src/configure
@@ -14968,7 +14968,7 @@
   fi
 fi
 
-NONASCII=$'\241\241'
+NONASCII='\241\241'
 
 
 if test -n "$GNU_CC" -a -z "$CLANG_CC"; then
@@ -16306,7 +16306,7 @@
     (''' EXPAND_LIBS_LIST_STYLE ''', r''' $EXPAND_LIBS_LIST_STYLE ''')
     (''' EXPAND_LIBS_ORDER_STYLE ''', r''' $EXPAND_LIBS_ORDER_STYLE ''')
     (''' CL_INCLUDES_PREFIX ''', r''' $CL_INCLUDES_PREFIX ''')
-    (''' NONASCII ''', r''' $NONASCII ''')
+    (''' NONASCII ''', "$NONASCII")
     (''' JS_NATIVE_EDITLINE ''', r''' $JS_NATIVE_EDITLINE ''')
     (''' JS_DISABLE_SHELL ''', r''' $JS_DISABLE_SHELL ''')
     (''' EDITLINE_LIBS ''', r''' $EDITLINE_LIBS ''')

Comment 3

[:Gijs (away/busy until Mar 10)]

Do you have this issue when building mozilla-central (32), too? What toolchain are you using to build?

Comment 4

[maillist-mozilla]

(In reply to :Gijs Kruitbosch from comment #3)
> Do you have this issue when building mozilla-central (32), too? What
> toolchain are you using to build?

if mozilla-central uses the same configure script, then yes.

i use the default toolchain of sabotage linux.
gcc 4.7.3 (irrelevant)
busybox sed 1.21.1 (relevant)
musl libc 1.1.0 (relevant) - treats invalid byte sequences as errors in regcomp (used by busybox sed) as it ought to do per the POSIX specs, unlike glibc/gnu sed.
did you see the busybox bug report linked in comment 1?

Comment 5

[:Gijs (away/busy until Mar 10)]

(In reply to maillist-mozilla from comment #4)
> (In reply to :Gijs Kruitbosch from comment #3)
> > Do you have this issue when building mozilla-central (32), too? What
> > toolchain are you using to build?
> 
> if mozilla-central uses the same configure script, then yes.
> 
> i use the default toolchain of sabotage linux.
> gcc 4.7.3 (irrelevant)
> busybox sed 1.21.1 (relevant)
> musl libc 1.1.0 (relevant) - treats invalid byte sequences as errors in
> regcomp (used by busybox sed) as it ought to do per the POSIX specs, unlike
> glibc/gnu sed.
> did you see the busybox bug report linked in comment 1?

Yes, but I am somewhat out of my depth here. The above was helpful whichever way. :-)

In any case, the relevant bits seem to have been added by emk and gps, so I'm CC-ing them and marking the dep relationship between the bugs. Masatoshi-san, can you have a look at whether this is fixable? From bug 948405 comment 4, it seems the choice was intentional, but you also intended this to be windows-specific, so I'm not sure what to do about this issue...

Comment 6

[Masatoshi Kimura [:emk]]

It's just I was too lazy to convince the reviewer. We should move NONASCII back into the if block because we found real harm.

Comment 7

[Masatoshi Kimura [:emk]]

Attachment: Move NONASCII into |if| block to avoid errors on POSIX-violation intolerant environments

Comment 8

[Michael Shal [:mshal]]

Comment on attachment 8421701 [details] [diff] [review]
Move NONASCII into |if| block to avoid errors on POSIX-violation intolerant environments

(stealing review from gps)

>+    dnl Make sure that the build system can handle non-ASCII characters
>+    dnl in environment variables to prevent it from breking silently on

breking -> breaking

I can't seem to reproduce the issue with busybox + musl, but from the comments this sounds fine.

Comment 9

[Masatoshi Kimura [:emk]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/afff04328433
Landed with the typo fix.

Comment 10

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/afff04328433

Comment 11

[maillist-mozilla]

firefox 31.0 is still broken

creating ./config.status
Traceback (most recent call last):
  File "./config.status", line 598, in <module>
    (''' LIBXUL_LIBS ''', r''' $(XPCOM_FROZEN_LDOPTS)  '''),
TypeError: 'tuple' object is not callable
*** Fix above errors and then restart with\
               "make -f client.mk build"
make[2]: *** [configure] Error 1

Comment 12

[maillist-mozilla]

the bug is now in 2 files
mozilla-release/configure
mozilla-release/js/src/configure

patch
--- mozilla-release.org/configure
+++ mozilla-release/configure
@@ -25848,7 +25848,7 @@
   fi
 fi

-NONASCII=$'\241\241'
+NONASCII='\241\241'


 # Static build options
@@ -28922,7 +28922,7 @@
     (''' EXPAND_LIBS_LIST_STYLE ''', r''' $EXPAND_LIBS_LIST_STYLE ''')
     (''' EXPAND_LIBS_ORDER_STYLE ''', r''' $EXPAND_LIBS_ORDER_STYLE ''')
     (''' CL_INCLUDES_PREFIX ''', r''' $CL_INCLUDES_PREFIX ''')
-    (''' NONASCII ''', r''' $NONASCII ''')
+    (''' NONASCII ''', r''' "$NONASCII" ''')
     (''' LIBXUL_LIBS ''', r''' $LIBXUL_LIBS ''')
     (''' GKMEDIAS_SHARED_LIBRARY ''', r''' $GKMEDIAS_SHARED_LIBRARY ''')
     (''' ZLIB_IN_MOZGLUE ''', r''' $ZLIB_IN_MOZGLUE ''')
--- mozilla-release.org/js/src/configure
+++ mozilla-release/js/src/configure
@@ -14858,7 +14858,7 @@
   fi
 fi

-NONASCII=$'\241\241'
+NONASCII='\241\241'


 if test -n "$GNU_CC" -a -z "$CLANG_CC"; then
@@ -16451,7 +16451,7 @@
     (''' EXPAND_LIBS_LIST_STYLE ''', r''' $EXPAND_LIBS_LIST_STYLE ''')
     (''' EXPAND_LIBS_ORDER_STYLE ''', r''' $EXPAND_LIBS_ORDER_STYLE ''')
     (''' CL_INCLUDES_PREFIX ''', r''' $CL_INCLUDES_PREFIX ''')
-    (''' NONASCII ''', r''' $NONASCII ''')
+    (''' NONASCII ''', r''' "$NONASCII" ''')
     (''' JS_NATIVE_EDITLINE ''', r''' $JS_NATIVE_EDITLINE ''')
     (''' JS_DISABLE_SHELL ''', r''' $JS_DISABLE_SHELL ''')
     (''' EDITLINE_LIBS ''', r''' $EDITLINE_LIBS ''')

Comment 13

[Masatoshi Kimura [:emk]]

See the Target Milestone. The fix is not taken into Firefox 31.

Comment 14

[Nathan Froyd [:froydnj]]

(In reply to Masatoshi Kimura [:emk] from comment #13)
> See the Target Milestone. The fix is not taken into Firefox 31.

Could you request uplift?

Comment 15

[Masatoshi Kimura [:emk]]

(In reply to Nathan Froyd (:froydnj) from comment #14)
> Could you request uplift?

Is it still possible? Firefox 31 is already released. Firefox 32 (beta) has a fix.

Comment 16

[Mike Hommey [:glandium]]

31 is ESR, it's going to be supported for about a year.

Comment 17

[Masatoshi Kimura [:emk]]

Comment on attachment 8421701 [details] [diff] [review]
Move NONASCII into |if| block to avoid errors on POSIX-violation intolerant environments

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration: Some busybox users cannot build ESR for about a year.
User impact if declined: No (build only)
Fix Landed on Version: 32
Risk to taking this patch (and alternatives if risky): Very low
String or UUID changes made by this patch: None

See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.

Comment 18

[Masatoshi Kimura [:emk]]

https://hg.mozilla.org/releases/mozilla-esr31/rev/944a768e236a