perl syntax error from rules.mk: make 3.81 changes backslash-newline behavior inside single-quoted strings

RESOLVED FIXED in 3.11.3

Status

NSS
Build
RESOLVED FIXED
12 years ago
11 years ago

People

(Reporter: Tuukka Tolvanen (sp3000), Assigned: Pawel Chmielowski)

Tracking

({fixed1.8.1})

unspecified
3.11.3
x86
Linux
fixed1.8.1

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(3 attachments, 1 obsolete attachment)

(Reporter)

Description

12 years ago
GNU Make 3.81beta4 (debian 3.80+3.81.b4-1)

http://lxr.mozilla.org/mozilla/source/security/coreconf/rules.mk#874
http://lists.gnu.org/archive/html/bug-make/2005-12/msg00039.html

removing the @ from @perl -e in rules.mk shows backslashes inside the perl -e argument:

/usr/bin/make -C /home/tt/src/mozilla/HEAD/mozilla/security/coreconf MAKE="/usr/bin/make -j1" -j1 CC="gcc" MOZILLA_INCLUDES=-I/var/scratch/tt/mozilla/obj/thunderbird-debug/dist/include/dbm SOURCE_MD_DIR=/var/scratch/tt/mozilla/obj/thunderbird-debug/dist DIST=/var/scratch/tt/mozilla/obj/thunderbird-debug/dist NSPR_INCLUDE_DIR=/var/scratch/tt/mozilla/obj/thunderbird-debug/dist/include/nspr NSPR_LIB_DIR=/var/scratch/tt/mozilla/obj/thunderbird-debug/dist/lib MOZILLA_CLIENT=1 NO_MDUPDATE=1 BUILD_TREE=/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug NS_USE_GCC=1 NS_USE_NATIVE=
make: Siirrytään hakemistoon "/home/tt/src/mozilla/HEAD/mozilla/security/coreconf"
perl -e '                                                            \
            open(MD, "< /home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md");                    \
            while (<MD>) {                                                    \
                if (m@ \.*/*/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md @) {                                 \
                    $found = 1;                                              \
                    last;                                                     \
                }                                                             \
            }                                                                 \
            if ($found) {                                                    \
                print "Removing stale dependency /home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md from /home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md\n";  \
                seek(MD, 0, 0);                                               \
                $tmpname = "/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/fix.md" . $$;                   \
                open(TMD, "> " . $tmpname);                                  \
                while (<MD>) {                                                \
                    s@ \.*/*/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md @ @;                                 \
                    if (!print TMD "$_") {                                   \
                        unlink(($tmpname));                                  \
                        exit(1);                                              \
                    }                                                         \
                }                                                             \
                close(TMD);                                                   \
                if (!rename($tmpname, "/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md")) {                  \
                    unlink(($tmpname));                                      \
                }                                                             \
            } elsif ("/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md" ne "/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md") {                             \
                print "/usr/bin/make -j1: *** No rule to make target /home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md.  Stop.\n";     \
                exit(1);                                                      \
            }'
syntax error at -e line 3, near "while"
syntax error at -e line 7, near "}"
Execution of -e aborted due to compilation errors.
(Reporter)

Comment 1

12 years ago
Created attachment 210062 [details]
comment 0, LANG=C

ungh, bugzilla vs charset detection... here's the command getting executed, with LANG=C just to be sure
(Reporter)

Updated

12 years ago
Summary: syntax error in rules.mk: make 3.81 changes backslash-newline behavior inside single-quoted strings → perl syntax error from rules.mk: make 3.81 changes backslash-newline behavior inside single-quoted strings
(Assignee)

Comment 2

12 years ago
Created attachment 210189 [details] [diff] [review]
v1

This patch fixes this issue for me.

Comment 3

12 years ago
Thanks for the bug report and patch.  What is the
new backslash-newline behavior?
(Assignee)

Comment 4

12 years ago
For text like:

'aaa \
bbb'

you get "aaa<backslash><newline>bbb". Text in "" works "normally", backslash and newline is striped.

Comment 5

12 years ago
There is some useful discussion about this at and near
 http://lists.debian.org/debian-devel/2005/12/msg00988.html
and in the GNU make info documentation for the affected (newer) makes.
Assignee: wtchang → nobody
QA Contact: wtchang → build

Comment 6

12 years ago
*** Bug 331815 has been marked as a duplicate of this bug. ***
Cygwin is now distributing make 3.81, so it would be great to get the patch reviewed and into the codebase.

Comment 8

11 years ago
*** Bug 342047 has been marked as a duplicate of this bug. ***
Pawel, are you going to ask for review on the patch?

Comment 10

11 years ago
Just updated my cygwin install and got a broken build. After much trial and error, downgrading to make 3.80 fixed things. I can only assume this bug was the culprit.

See this MozillaZine post; looks like other people are beginning to get bitten by the changes in make 3.81 as well:

http://forums.mozillazine.org/viewtopic.php?t=441031
I just got bitten by this too. Can we get this reviewed please?
I would point out that the patch here does not fix the main problems I found with make 3.81 with errors like:

../coreconf/rules.mk:365: target `c' doesn't match the target pattern 
Until such time as a patch is contributed that works with gmake 3.79.1,
3.80 *and* 3.81, the officially recommended workaround for this problem
will be to fall back to an older gmake that is known to work.
I see. Appologies for jumping the gun, and thanks for the clarification. Would it be possible to have a test for make 3.81 abort the build process early with a clear error message, rather than allow it to build for an hour before dying with confusing errors?
Write it, I'll review it...

Comment 16

11 years ago
Actually despite this bug and the error messages it generates such as 'syntax error at -e line 3, near "while"', firefox 2.0b1 still compiles correctly.
It doesn't seem to be a fatal error message as firefox will still continue to compile correctly with make 3.81

Comment 17

11 years ago
(In reply to comment #16)
> Actually despite this bug and the error messages it generates such as 'syntax
> error at -e line 3, near "while"', firefox 2.0b1 still compiles correctly.
> It doesn't seem to be a fatal error message as firefox will still continue to
> compile correctly with make 3.81
> 

Under what setup? For me on Windows XP + cygwin + VC8, the build stops with the errors in comment 12. That's building the trunk, haven't tried the 1.8.1 branch.
(Reporter)

Comment 18

11 years ago
this bug as reported is a non-fatal build issue that occurs on linux and likely on other platforms. it would probably be a lot more useful to file another bug for the fatal build issue on windows, which I haven't heard of anyone experiencing on other platforms, and I haven't experienced myself with make 3.81 on linux.
I agree that the windows issue should be a separate bug.
It should also include the relevant section of the output log.

If someone here will confirm that the patch "v1" above works with 
the 3 above-named versions of gmake on linux, I will proceed to commit it,
first on the trunk, and then on the branch (next day).
(Reporter)

Comment 20

11 years ago
tested 3.79.1 and 3.80 (sources), 3.81 (debian), all ok with attachment 210189 [details] [diff] [review]

Comment 21

11 years ago
I was responsible for hijacking this bug, so I've filed bug 345482 on the new issue. Sorry for the confusion.
(Assignee)

Updated

11 years ago
Attachment #210189 - Flags: review?(benjamin)
Attachment #210189 - Flags: review?(benjamin) → review+
(Assignee)

Updated

11 years ago
Status: NEW → ASSIGNED
Whiteboard: [checkin needed]
Comment on attachment 210189 [details] [diff] [review]
v1

Because this code is in NSS, it needs to be approved by an NSS peer.
Attachment #210189 - Flags: review?(wtchang)
(Assignee)

Updated

11 years ago
Assignee: nobody → prefiks
Status: ASSIGNED → NEW
Whiteboard: [checkin needed]
Because this is an NSS bug, it needs more than mere approval by an NSS peer.
It must be checked in by an NSS team member.  

Something upon which no-one has yet commented: The code being altered is dead.  

The dependency-maker program was removed from coreconf years ago because of 
license issues.  The makefile code cited here hasn't been used since then.  
A patch to remove dead code might be received more quickly.

Comment 24

11 years ago
Comment on attachment 210189 [details] [diff] [review]
v1

r=wtc.  I verified that in GNU make 3.80 this patch only
changes the white space inside the quoted string.
Attachment #210189 - Flags: review?(wtchang) → review+

Comment 25

11 years ago
Created attachment 233858 [details] [diff] [review]
v2 (as checked in)

I made some whitespace changes before checking in the v1 patch
on the NSS trunk (NSS 3.12) and NSS_3_11_BRANCH (NSS 3.11.3).

Checking in rules.mk;
/cvsroot/mozilla/security/coreconf/rules.mk,v  <--  rules.mk
new revision: 1.69; previous revision: 1.68
done

Checking in rules.mk;
/cvsroot/mozilla/security/coreconf/rules.mk,v  <--  rules.mk
new revision: 1.66.2.2; previous revision: 1.66.2.1
done
Attachment #210189 - Attachment is obsolete: true
Created attachment 233861 [details] [diff] [review]
patch - remove dead code

reduce maintenance burden by removing dead code.
Attachment #233861 - Flags: review?(wtchang)

Comment 27

11 years ago
As an NSS bug, this bug is fixed.  The patch v2 still needs to be
propagated to the Mozilla trunk and MOZILLA_1_8_BRANCH.

It turns out that that makefile rule is executed when you make a
target for which there is no rule.  For example, on Linux,

$ make foobar
make: *** No rule to make target foobar.  Stop.
make: *** [foobar] Error 1
$

The first error message and the "Error 1" exit code (of perl) both come
from this .DEFAULT rule.  If I remove the .DEFAULT rule and try again, I
get this instead:

$ make foobar
make: *** No rule to make target `foobar'.  Stop.
$

Note that the target is quoted.
Status: NEW → RESOLVED
Last Resolved: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → 3.11.3

Comment 28

11 years ago
Comment on attachment 233861 [details] [diff] [review]
patch - remove dead code

This patch is incomplete.  The "X dependency system"
comment should be removed.  There is dependency-related
code in other makefiles in coreconf.  The README file
also needs to be updated.  If you want to remove the
dead code, I suggest you open a new bug.  But weren't
you asking Gervase about resurrecting the mkdepend
program (which has a non-MPL open source license) in
mozilla/security/coreconf/mkdepend?
Attachment #233861 - Flags: review?(wtchang) → review-
Fixed on MOZILLA_1_8_BRANCH by bug 340724.
Keywords: fixed1.8.1
*** Bug 356224 has been marked as a duplicate of this bug. ***

Comment 31

11 years ago
Yes, the patch v2 fixed my problem described in bug #356224 with current cvs HEAD sources. Someone should commit the patch to HEAD as well to keep it in sync with 1_8 branch. Thanks.


$ /usr/bin/gmake -v
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i686-pc-linux-gnu
$ /usr/bin/make -v
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i686-pc-linux-gnu
$ 

Comment 32

11 years ago
The patch v2 is in the current CVS HEAD sources.  What's
the output of this command?

cvs status mozilla/security/coreconf/rules.mk
I'll guess that Martin pulled some file from $HEAD that, in turn, pulls NSS 
from some tag.  Maybe the fix isn't in that tag?  Maybe the tag needs to be
changed in that file?

Comment 34

11 years ago
I hope the version number did not get screwed by the patch. So, I have reverted the patch which applied well with some offset ...


vrapenec mozilla # patch -R < ~mmokrejs/tmp/2.diff 
can't find file to patch at input line 9
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
--------------------------
|Index: rules.mk
|===================================================================
|RCS file: /cvsroot/mozilla/security/coreconf/rules.mk,v
|retrieving revision 1.68
|retrieving revision 1.69
|diff -U8 -r1.68 -r1.69
|--- rules.mk   7 Apr 2006 04:10:28 -0000       1.68
|+++ rules.mk   15 Aug 2006 21:35:25 -0000      1.69
--------------------------
File to patch: security/coreconf/rules.mk
patching file security/coreconf/rules.mk
Hunk #1 succeeded at 865 (offset -3 lines).
Hunk #2 succeeded at 893 (offset -3 lines).
vrapenec mozilla # cvs status security/coreconf/rules.mk
===================================================================
File: rules.mk          Status: Up-to-date

   Working revision:    1.66
   Repository revision: 1.66    /cvsroot/mozilla/security/coreconf/rules.mk,v
   Sticky Tag:          NSS_CLIENT_TAG (revision: 1.66)
   Sticky Date:         (none)
   Sticky Options:      (none)

vrapenec mozilla # 


So what did I make wrong in converting the tarball to a cvs copy? :(
Mozilla trunk always pulls NSS_CLIENT_TAG.

Comment 36

11 years ago
In the current CVS HEAD sources, cvs status security/coreconf/rules.mk should say

   Working revision:    1.66.2.2
   Repository revision: 1.66.2.2    /cvsroot/mozilla/security/coreconf/rules.mk,v
   Sticky Tag:          NSS_3_11_20060929_TAG (revision: 1.66.2.2)

(We no longer use the moving static tag NSS_CLIENT_TAG now.)

You have rev. 1.66, which doesn't have the fix.  rev. 1.66.2.2
has the fix.

Comment 37

11 years ago
OK, I had somewhat failed to convert the tarball to proper cvs copy. Checking out all modules from cvs gave me the 1.66.2.2 version. So my report is "INVALID". :(
You need to log in before you can comment on or make changes to this bug.