325148 - perl syntax error from rules.mk: make 3.81 changes backslash-newline behavior inside single-quoted strings

Status: RESOLVED FIXED

(NSS :: Build, defect)

Description

[Tuukka Tolvanen (sp3000)]

GNU Make 3.81beta4 (debian 3.80+3.81.b4-1)

http://lxr.mozilla.org/mozilla/source/security/coreconf/rules.mk#874
http://lists.gnu.org/archive/html/bug-make/2005-12/msg00039.html

removing the @ from @perl -e in rules.mk shows backslashes inside the perl -e argument:

/usr/bin/make -C /home/tt/src/mozilla/HEAD/mozilla/security/coreconf MAKE="/usr/bin/make -j1" -j1 CC="gcc" MOZILLA_INCLUDES=-I/var/scratch/tt/mozilla/obj/thunderbird-debug/dist/include/dbm SOURCE_MD_DIR=/var/scratch/tt/mozilla/obj/thunderbird-debug/dist DIST=/var/scratch/tt/mozilla/obj/thunderbird-debug/dist NSPR_INCLUDE_DIR=/var/scratch/tt/mozilla/obj/thunderbird-debug/dist/include/nspr NSPR_LIB_DIR=/var/scratch/tt/mozilla/obj/thunderbird-debug/dist/lib MOZILLA_CLIENT=1 NO_MDUPDATE=1 BUILD_TREE=/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug NS_USE_GCC=1 NS_USE_NATIVE=
make: Siirrytään hakemistoon "/home/tt/src/mozilla/HEAD/mozilla/security/coreconf"
perl -e '                                                            \
            open(MD, "< /home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md");                    \
            while (<MD>) {                                                    \
                if (m@ \.*/*/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md @) {                                 \
                    $found = 1;                                              \
                    last;                                                     \
                }                                                             \
            }                                                                 \
            if ($found) {                                                    \
                print "Removing stale dependency /home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md from /home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md\n";  \
                seek(MD, 0, 0);                                               \
                $tmpname = "/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/fix.md" . $$;                   \
                open(TMD, "> " . $tmpname);                                  \
                while (<MD>) {                                                \
                    s@ \.*/*/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md @ @;                                 \
                    if (!print TMD "$_") {                                   \
                        unlink(($tmpname));                                  \
                        exit(1);                                              \
                    }                                                         \
                }                                                             \
                close(TMD);                                                   \
                if (!rename($tmpname, "/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md")) {                  \
                    unlink(($tmpname));                                      \
                }                                                             \
            } elsif ("/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md" ne "/home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md") {                             \
                print "/usr/bin/make -j1: *** No rule to make target /home/tt/src/mozilla/obj-i686-pc-linux-gnu/thunderbird-debug/nss/.md.  Stop.\n";     \
                exit(1);                                                      \
            }'
syntax error at -e line 3, near "while"
syntax error at -e line 7, near "}"
Execution of -e aborted due to compilation errors.

Comment 1

[Tuukka Tolvanen (sp3000)]

Attachment: comment 0, LANG=C

ungh, bugzilla vs charset detection... here's the command getting executed, with LANG=C just to be sure

Comment 2

[Pawel Chmielowski]

Attachment: v1

This patch fixes this issue for me.

Comment 3

[Wan-Teh Chang]

Thanks for the bug report and patch.  What is the
new backslash-newline behavior?

Comment 4

[Pawel Chmielowski]

For text like:

'aaa \
bbb'

you get "aaa<backslash><newline>bbb". Text in "" works "normally", backslash and newline is striped.

Comment 5

[Ian Jackson]

There is some useful discussion about this at and near
 http://lists.debian.org/debian-devel/2005/12/msg00988.html
and in the GNU make info documentation for the affected (newer) makes.

Comment 6

[Wan-Teh Chang]

*** Bug 331815 has been marked as a duplicate of this bug. ***

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Cygwin is now distributing make 3.81, so it would be great to get the patch reviewed and into the codebase.

Comment 8

[timeless]

*** Bug 342047 has been marked as a duplicate of this bug. ***

Comment 9

[Martijn Wargers (dead)]

Pawel, are you going to ask for review on the patch?

Comment 10

[Jon Henry]

Just updated my cygwin install and got a broken build. After much trial and error, downgrading to make 3.80 fixed things. I can only assume this bug was the culprit.

See this MozillaZine post; looks like other people are beginning to get bitten by the changes in make 3.81 as well:

http://forums.mozillazine.org/viewtopic.php?t=441031

Comment 11

[Jonathan Watt [:jwatt]]

I just got bitten by this too. Can we get this reviewed please?

Comment 12

[Dave Townsend [:mossop]]

I would point out that the patch here does not fix the main problems I found with make 3.81 with errors like:

../coreconf/rules.mk:365: target `c' doesn't match the target pattern 

Comment 13

[Nelson Bolyard (seldom reads bugmail)]

Until such time as a patch is contributed that works with gmake 3.79.1,
3.80 *and* 3.81, the officially recommended workaround for this problem
will be to fall back to an older gmake that is known to work.

Comment 14

[Jonathan Watt [:jwatt]]

I see. Appologies for jumping the gun, and thanks for the clarification. Would it be possible to have a test for make 3.81 abort the build process early with a clear error message, rather than allow it to build for an hour before dying with confusing errors?

Comment 15

[Benjamin Smedberg]

Write it, I'll review it...

Comment 16

[Hussam Al-Tayeb]

Actually despite this bug and the error messages it generates such as 'syntax error at -e line 3, near "while"', firefox 2.0b1 still compiles correctly.
It doesn't seem to be a fatal error message as firefox will still continue to compile correctly with make 3.81

Comment 17

[Jon Henry]

(In reply to comment #16)
> Actually despite this bug and the error messages it generates such as 'syntax
> error at -e line 3, near "while"', firefox 2.0b1 still compiles correctly.
> It doesn't seem to be a fatal error message as firefox will still continue to
> compile correctly with make 3.81
> 

Under what setup? For me on Windows XP + cygwin + VC8, the build stops with the errors in comment 12. That's building the trunk, haven't tried the 1.8.1 branch.

Comment 18

[Tuukka Tolvanen (sp3000)]

this bug as reported is a non-fatal build issue that occurs on linux and likely on other platforms. it would probably be a lot more useful to file another bug for the fatal build issue on windows, which I haven't heard of anyone experiencing on other platforms, and I haven't experienced myself with make 3.81 on linux.

Comment 19

[Nelson Bolyard (seldom reads bugmail)]

I agree that the windows issue should be a separate bug.
It should also include the relevant section of the output log.

If someone here will confirm that the patch "v1" above works with 
the 3 above-named versions of gmake on linux, I will proceed to commit it,
first on the trunk, and then on the branch (next day).

Comment 20

[Tuukka Tolvanen (sp3000)]

tested 3.79.1 and 3.80 (sources), 3.81 (debian), all ok with attachment 210189 [details] [diff] [review]

Comment 21

[Jon Henry]

I was responsible for hijacking this bug, so I've filed bug 345482 on the new issue. Sorry for the confusion.

Comment 22

[Benjamin Smedberg]

Comment on attachment 210189 [details] [diff] [review]
v1

Because this code is in NSS, it needs to be approved by an NSS peer.

Comment 23

[Nelson Bolyard (seldom reads bugmail)]

Because this is an NSS bug, it needs more than mere approval by an NSS peer.
It must be checked in by an NSS team member.  

Something upon which no-one has yet commented: The code being altered is dead.  

The dependency-maker program was removed from coreconf years ago because of 
license issues.  The makefile code cited here hasn't been used since then.  
A patch to remove dead code might be received more quickly.

Comment 24

[Wan-Teh Chang]

Comment on attachment 210189 [details] [diff] [review]
v1

r=wtc.  I verified that in GNU make 3.80 this patch only
changes the white space inside the quoted string.

Comment 25

[Wan-Teh Chang]

Attachment: v2 (as checked in)

I made some whitespace changes before checking in the v1 patch
on the NSS trunk (NSS 3.12) and NSS_3_11_BRANCH (NSS 3.11.3).

Checking in rules.mk;
/cvsroot/mozilla/security/coreconf/rules.mk,v  <--  rules.mk
new revision: 1.69; previous revision: 1.68
done

Checking in rules.mk;
/cvsroot/mozilla/security/coreconf/rules.mk,v  <--  rules.mk
new revision: 1.66.2.2; previous revision: 1.66.2.1
done

Comment 26

[Nelson Bolyard (seldom reads bugmail)]

Attachment: patch - remove dead code

reduce maintenance burden by removing dead code.

Comment 27

[Wan-Teh Chang]

As an NSS bug, this bug is fixed.  The patch v2 still needs to be
propagated to the Mozilla trunk and MOZILLA_1_8_BRANCH.

It turns out that that makefile rule is executed when you make a
target for which there is no rule.  For example, on Linux,

$ make foobar
make: *** No rule to make target foobar.  Stop.
make: *** [foobar] Error 1
$

The first error message and the "Error 1" exit code (of perl) both come
from this .DEFAULT rule.  If I remove the .DEFAULT rule and try again, I
get this instead:

$ make foobar
make: *** No rule to make target `foobar'.  Stop.
$

Note that the target is quoted.

Comment 28

[Wan-Teh Chang]

Comment on attachment 233861 [details] [diff] [review]
patch - remove dead code

This patch is incomplete.  The "X dependency system"
comment should be removed.  There is dependency-related
code in other makefiles in coreconf.  The README file
also needs to be updated.  If you want to remove the
dead code, I suggest you open a new bug.  But weren't
you asking Gervase about resurrecting the mkdepend
program (which has a non-MPL open source license) in
mozilla/security/coreconf/mkdepend?

Comment 29

[Nick Thomas [:nthomas] (UTC+12)]

Fixed on MOZILLA_1_8_BRANCH by bug 340724.

Comment 30

[Nelson Bolyard (seldom reads bugmail)]

*** Bug 356224 has been marked as a duplicate of this bug. ***

Comment 31

[Martin Mokrejs]

Yes, the patch v2 fixed my problem described in bug #356224 with current cvs HEAD sources. Someone should commit the patch to HEAD as well to keep it in sync with 1_8 branch. Thanks.


$ /usr/bin/gmake -v
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i686-pc-linux-gnu
$ /usr/bin/make -v
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i686-pc-linux-gnu
$ 

Comment 32

[Wan-Teh Chang]

The patch v2 is in the current CVS HEAD sources.  What's
the output of this command?

cvs status mozilla/security/coreconf/rules.mk

Comment 33

[Nelson Bolyard (seldom reads bugmail)]

I'll guess that Martin pulled some file from $HEAD that, in turn, pulls NSS 
from some tag.  Maybe the fix isn't in that tag?  Maybe the tag needs to be
changed in that file?

Comment 34

[Martin Mokrejs]

I hope the version number did not get screwed by the patch. So, I have reverted the patch which applied well with some offset ...


vrapenec mozilla # patch -R < ~mmokrejs/tmp/2.diff 
can't find file to patch at input line 9
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
--------------------------
|Index: rules.mk
|===================================================================
|RCS file: /cvsroot/mozilla/security/coreconf/rules.mk,v
|retrieving revision 1.68
|retrieving revision 1.69
|diff -U8 -r1.68 -r1.69
|--- rules.mk   7 Apr 2006 04:10:28 -0000       1.68
|+++ rules.mk   15 Aug 2006 21:35:25 -0000      1.69
--------------------------
File to patch: security/coreconf/rules.mk
patching file security/coreconf/rules.mk
Hunk #1 succeeded at 865 (offset -3 lines).
Hunk #2 succeeded at 893 (offset -3 lines).
vrapenec mozilla # cvs status security/coreconf/rules.mk
===================================================================
File: rules.mk          Status: Up-to-date

   Working revision:    1.66
   Repository revision: 1.66    /cvsroot/mozilla/security/coreconf/rules.mk,v
   Sticky Tag:          NSS_CLIENT_TAG (revision: 1.66)
   Sticky Date:         (none)
   Sticky Options:      (none)

vrapenec mozilla # 


So what did I make wrong in converting the tarball to a cvs copy? :(

Comment 35

[Benjamin Smedberg]

Mozilla trunk always pulls NSS_CLIENT_TAG.

Comment 36

[Wan-Teh Chang]

In the current CVS HEAD sources, cvs status security/coreconf/rules.mk should say

   Working revision:    1.66.2.2
   Repository revision: 1.66.2.2    /cvsroot/mozilla/security/coreconf/rules.mk,v
   Sticky Tag:          NSS_3_11_20060929_TAG (revision: 1.66.2.2)

(We no longer use the moving static tag NSS_CLIENT_TAG now.)

You have rev. 1.66, which doesn't have the fix.  rev. 1.66.2.2
has the fix.

Comment 37

[Martin Mokrejs]

OK, I had somewhat failed to convert the tarball to proper cvs copy. Checking out all modules from cvs gave me the 1.66.2.2 version. So my report is "INVALID". :(