Closed Bug 829793 Opened 11 years ago Closed 7 years ago

IDL generation busted after bug 826606

Categories

(Firefox Build System :: General, defect)

x86_64
Linux
defect
Not set
critical

Tracking

(Not tracked)

RESOLVED INCOMPLETE

People

(Reporter: mnyromyr, Unassigned)

References

Details

(Whiteboard: see comments 0, 3, 5, 6 )

Building current SeaMonkey trunk under Linux 64 bit with external linkage fails when generating IDLs:

nsIDOMUserProximityEvent.idl
Traceback (most recent call last):
  File "/home/kd/projects/mozilla/mozilla.org/src/trunk/mozilla/config/pythonpath.py", line 56, in <module>
    main(sys.argv[1:])
  File "/home/kd/projects/mozilla/mozilla.org/src/trunk/mozilla/config/pythonpath.py", line 48, in main
    execfile(script, frozenglobals)
  File "/home/kd/moz/obj/sr/mozilla/dist/sdk/bin/header.py", line 523, in <module>
    idl = p.parse(open(file).read(), filename=file)
  File "/home/kd/moz/obj/sr/mozilla/dist/sdk/bin/xpidl.py", line 1507, in parse
    idl = self.parser.parse(lexer=self)
  File "/home/kd/moz/obj/sr/mozilla/dist/sdk/bin/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/home/kd/moz/obj/sr/mozilla/dist/sdk/bin/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/home/kd/moz/obj/sr/mozilla/dist/sdk/bin/xpidl.py", line 1478, in p_error
    raise IDLError("invalid syntax", location)
xpidl.IDLError: error: invalid syntax, /home/kd/projects/mozilla/mozilla.org/src/trunk/mozilla/dom/interfaces/events/nsIDOMDeviceProximityEvent.idl line 24:18
   double value = Infinity;
                  ^
Non-external linkage build fail as well.
Clobbering doesn't help either.
Summary: External Linkage build busted after bug 826606 → IDL generation busted after bug 826606
Might be old .pyc files in your mozilla srcdir. Try hg st -in
I removed all .pyc files from my tree before, it didn't help.
I have the exact same issue on my buildbot - when trying to build tb from c-c. Removing objdir & all .pyc files from the src tree didnt help.

See http://buildbot.rhaalovely.net/builders/comm-central-amd64 for the logs. Last successful nightly build was on the 7 with rev a976311210e4e, so the breakage window is the same.
JDTR: cleaning out/turning off ccache doesn't help either.
So, it's not even an comm-central problem at all, since I fail to build FF with the very same error trace …
- Linux 64 bit on an AMD Athlon X2 double core
- Python 2.7.3
Product: MailNews Core → Firefox
Whiteboard: see comments 0, 3, 5, 6
Product: Firefox → Core
FWIW, I don't see the problem on OS X 10.6.8 with clang 3.1 and Python 2.6.1(?!).
I'm using gcc 4.7.2 on Linux.
Both Linux and Mac are debug builds.
I'm not sure why this is happening. My guess is that it has something to do with yacc. My only idea right now is that there are some yacc/lex cache files somewhere in your tree. Try removing any files matching xpidllex.py* and xpidlyacc.py*.
(In reply to William Chen [:wchen] from comment #8)

> [...] Try removing any files matching xpidllex.py*
> and xpidlyacc.py*.

This indeed fixes the build.

Fedora 17 x86_64, gcc 4.7.2, python 2.7.3.
(In reply to Ilja Sekler from comment #9)
> (In reply to William Chen [:wchen] from comment #8)
> 
> > [...] Try removing any files matching xpidllex.py*
> > and xpidlyacc.py*.
> 
> This indeed fixes the build.
> 
> Fedora 17 x86_64, gcc 4.7.2, python 2.7.3.
Do you remember where these files were located in your tree? I'm suspect that there is a problem in the build script. On my Mac and Linux machines, these cache files are located in the objdir so clobbering should have fixed the problem and it looks like there is some code to delete these files. Something is causing these old cache files to stick around.

I'm not familiar with the build system but I'm guessing the problem is around here:
http://dxr.mozilla.org/mozilla-central/xpcom/idl-parser/Makefile.in.html#l32
http://dxr.mozilla.org/mozilla-central/xpcom/idl-parser/header.py.html#l502
The list of files to remove in addition to the objdir might be those in $srcdir :

xpcom/idl-parser/xpidl.pyc
xpcom/idl-parser/xpidllex.py{,c}
xpcom/idl-parser/xpidlyacc.py{,c}

They're not under version control/hg but strangely hg status doesnt show them as untracked - although only .pyc files are ignored by .hgignore.. not the two xpidl{lex,yacc}.py files.
Okay, blowing away xpidllex.py/xpidlyacc.py as well lets me build again (I already deleted objdir and *.pyc in my source tree before, but to no avail).
The cache were located where gaston pointed out.
(In reply to Karsten Düsterloh from comment #12)
> The cache were located where gaston pointed out.

To clarify: I had two sets of these files, one set in $objdir/mozilla/dist/sdk/bin/, one set in $source/mozilla/xpcom/idl-parser/. The .py files in the objdir are symlinked to their source tree equivalents, the .pyc caches were local to both directories (and diffrent due to different paths and such included).
OK, so apparently between 2.15 and 2.16 the generation of the xpidl*.py* was moved from the srcdir to the objdir, so if you start with a clean 2.16 checkout you won't get the problem, but if you pull a tree previously used to build 2.15 or earlier then you will have those stale files and you need to remove them so that the versions in the objdir get used.
Just hit this one on Windows (Win 7 x64) trying to build SM trunk. Since even before that my build failed due to an old mozilla/build/mozinfo.pyc, I'll now bite the bullet and have each and every untracked file deleted before building. I don't understand why anything is created/modified outside the objdir at all, but since this is about the third time that I cannot build due to some old pyc's lying around and seemingly no attempt was made to prevent this, I'll now take care of this myself. Sad but true. :-(

Neil: Thanks for the hint in comment 2. Will use that command for rm list identification purposes.
Fixing the pyc problem requires writing a custom Python importer and is tracked in bug 795995. This is a hard problem, which is why it hasn't been implemented yet. While the problem is annoying, things only break maybe once every few months. There are much bigger fish to fry.
> things only break maybe once every few months

And whenever anyone tries to bisect.
We haven't seen these problems recently.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INCOMPLETE
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.