950332 - [trunk] Thunderbird build process is broken while processing codegen.pp : no such file or directory.

Reporter

Description

•

11 years ago

Noticed this "bug" very recently.

I'm using an external objdir directory to get a "clean" trunk source code for Thunderbird.

Here is the .mozconfig I'm using :

mk_add_options PYTHON=/usr/bin/python2
mk_add_options AUTOCONF=autoconf-2.13

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../objdir-tb

ac_add_options --enable-application=mail
ac_add_options --enable-optimize
ac_add_options --with-ccache
ac_add_options --disable-debug
ac_add_options --disable-tests
ac_add_options --disable-crashreporter
ac_add_options --disable-installer
ac_add_options --disable-warnings-as-errors
ac_add_options --disable-debug-symbols

When I type this morning make -f client.mk depend, it ends with this error log :

"Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/action/webidl.py", line 17, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/action/webidl.py", line 12, in main
    manager = BuildSystemWebIDL.from_environment().manager
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/base.py", line 175, in from_environment
    if not samepath(topobjdir, config_topobjdir) \
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/base.py", line 41, in samepath
    return os.path.samefile(path1, path2)
  File "/home/fred/logs/mail/objdir-tb/mozilla/_virtualenv/lib/python2.7/posixpath.py", line 163, in samefile
    s2 = os.stat(f2)
OSError: [Errno 2] No such file or directory: '/home/fred/logs/mail/src/objdir-tb'
Creating makedepend file .deps/xpcAccEvents.pp"

After looking at last changes for both python/mozbuild/mozbuild/base.py and python/mozbuild/mozbuild/action/webidl.py I found that bug 928195 could be "guilty".

Magnus Melin [:mkmelin]

Comment 1

•

11 years ago

I built it today successfully. Did you forget delete the old build dir first?

Magnus Melin [:mkmelin]

Comment 2

•

11 years ago

Old obj dir (if it wasn't clear). I think if in-tree obj dirs arenät even supported anymore, so your setup isn't unusual.

Frederic Bezies

Reporter

Comment 3

•

11 years ago

Removed old objdir, cleaned ccache. Tried and failed again before I report this bug.

Frederic Bezies

Reporter

Comment 4

•

11 years ago

Tried again. Still broken at the same point. Could it be related to python3 as usual on archlinux ?

Frederic Bezies

Reporter

Comment 5

•

11 years ago

And I'm getting into trouble with Mozilla Firefox too. Got this in ehe end of mozilla firefox building process :

44:01.94 /home/fred/logs/fox/src/browser/locales/en-US/searchplugins/yahoo.xml: WARNING: no preprocessor directives found
44:01.95 /home/fred/logs/fox/src/browser/locales/en-US/searchplugins/wikipedia.xml: WARNING: no preprocessor directives found
44:05.99 firefox
44:06.67 xpcshell
44:06.67 plugin-container
44:22.29 Traceback (most recent call last):
44:22.29   File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
44:22.30     "__main__", fname, loader, pkg_name)
44:22.30   File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
44:22.30     exec code in run_globals
44:22.30   File "/home/fred/logs/fox/src/python/mozbuild/mozbuild/action/webidl.py", line 17, in <module>
44:22.30     sys.exit(main(sys.argv[1:]))
44:22.30   File "/home/fred/logs/fox/src/python/mozbuild/mozbuild/action/webidl.py", line 13, in main
44:22.30     manager.generate_build_files()
44:22.30   File "/home/fred/logs/fox/src/dom/bindings/mozwebidlcodegen/__init__.py", line 287, in generate_build_files
44:22.30     written = self.generate_example_files(interface)
44:22.30   File "/home/fred/logs/fox/src/dom/bindings/mozwebidlcodegen/__init__.py", line 308, in generate_example_files
44:22.30     root = CGExampleRoot(self.config, interface)
44:22.30   File "/home/fred/logs/fox/src/dom/bindings/Codegen.py", line 10314, in __init__
44:22.30     descriptor = config.getDescriptor(interfaceName, False)
44:22.30   File "/home/fred/logs/fox/src/dom/bindings/Configuration.py", line 176, in getDescriptor
44:22.30     for d in self.descriptorsByName[interfaceName]:
44:22.30 KeyError: u'TestExampleProxyInterface'

Opening another bug could be useful I think...

Frederic Bezies

Reporter

Comment 6

•

11 years ago

Other news. Now it tells me it is broken in codegen.pp

Creating makedepend file .deps/event_impl_gen.pp
Makefile:69: codegen.pp: No such file or directory
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/action/webidl.py", line 17, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/action/webidl.py", line 12, in main
    manager = BuildSystemWebIDL.from_environment().manager
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/base.py", line 175, in from_environment
    if not samepath(topobjdir, config_topobjdir) \
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/base.py", line 41, in samepath
    return os.path.samefile(path1, path2)
  File "/home/fred/logs/mail/objdir-tb/mozilla/_virtualenv/lib/python2.7/posixpath.py", line 163, in samefile
    s2 = os.stat(f2)
OSError: [Errno 2] No such file or directory: '/home/fred/logs/mail/src/objdir-tb'
Makefile:72: recipe for target 'codegen.pp' failed
make[5]: *** [codegen.pp] Error 1
/home/fred/logs/mail/src/mozilla/config/recurse.mk:189: recipe for target 'bindings_export' failed
make[4]: *** [bindings_export] Error 2
make[4]: *** Waiting for unfinished jobs....
/home/fred/logs/mail/src/mozilla/config/recurse.mk:189: recipe for target 'export' failed
make[3]: *** [export] Error 2
/home/fred/logs/mail/src/mozilla/config/recurse.mk:161: recipe for target 'export' failed
make[2]: *** [export] Error 2
Makefile:52: recipe for target 'export' failed
make[1]: *** [export] Error 2
client.mk:374: recipe for target 'depend' failed
make: *** [depend] Error 2

Frederic Bezies

Reporter

Updated

•

11 years ago

Summary: Since landing of bug 928195, build process cannot use an external objdir directory → [trunk] Thunderbird build process is broken while processing codegen.pp : no such file or directory.

Frederic Bezies

Reporter

Comment 7

•

11 years ago

Replacing in my .mozconfig 

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../objdir-tb

by mk_add_options MOZ_OBJDIR=/home/fred/logs/mail/objdir-tb removed the bug, even if I still get Makefile:69: codegen.pp: No such file or directory

Gregory Szorc [:gps]

Comment 8

•

11 years ago

Comment #5 is due to --disable-tests and was fixed in bug 950736.

The other stacks with os.stat() in them seem interesting. Comment #7 is *very* interesting. Seems like an error in environment resolution.

Go to the dom/bindings directory in your objdir and run |mach environment|. Do you see anything unexpected?

Blocks: 928195

Flags: needinfo?(fredbezies)

Frederic Bezies

Reporter

Comment 9

•

11 years ago

I know about bug in comment #5. I followed it.

I had to copy mach.

Here is the result :

[fred@fredo-arch bindings]$ python2 mach environment
platform:
	Linux-3.12.5-1-ARCH-x86_64-with-glibc2.2.5
python version:
	2.7.6 (default, Nov 26 2013, 12:52:49) 
[GCC 4.8.2]
python prefix:
	/usr
mach cwd:
	/home/fred/logs/mail/objdir-tb/mozilla/dom/bindings
os cwd:
	/home/fred/logs/mail/objdir-tb/mozilla/dom/bindings
mach directory:
	/home/fred/logs/mail/src/mozilla
state directory:
	/home/fred/.mozbuild
mozconfig path:
	/home/fred/logs/mail/src/.mozconfig
object directory:
	/home/fred/logs/mail/objdir-tb/mozilla
mozconfig configure args:
	--enable-application=mail
	--enable-optimize
	--enable-calendar
	--with-ccache
	--disable-debug
	--disable-tests
	--disable-crashreporter
	--disable-installer
	--disable-warnings-as-errors
	--disable-debug-symbols
mozconfig extra make args:
	PYTHON=/usr/bin/python2
	AUTOCONF=autoconf-2.13
mozconfig make flags:
config topsrcdir:
	/home/fred/logs/mail/src/mozilla
config topobjdir:
	/home/fred/logs/mail/objdir-tb/mozilla

Flags: needinfo?(fredbezies)

Gregory Szorc [:gps]

Comment 10

•

11 years ago

OSError: [Errno 2] No such file or directory: '/home/fred/logs/mail/src/objdir-tb'

is worrisome. Looks like something is getting confused about the location of the objdir. That something seems to be in python/mozbuild/mozbuild/base.py:175.

Do you know how to debug Python scripts? Insert the following to enter an interactive debugger at run-time:

  import pdb; pdb.set_trace()

Then print out some variables and tell me what doesn't make sense.

jcranmer can likely help here.

Frederic Bezies

Reporter

Comment 11

•

11 years ago

I added  import pdb; pdb.set_trace() in top of python/mozbuild/mozbuild/base.py

Here is the log :

-> from mach.mixin.logging import LoggingMixin
(Pdb) 
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/action/webidl.py", line 7, in <module>
    from mozwebidlcodegen import BuildSystemWebIDL
  File "/home/fred/logs/mail/src/mozilla/dom/bindings/mozwebidlcodegen/__init__.py", line 20, in <module>
    from mozbuild.base import MozbuildObject
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/base.py", line 17, in <module>
    from mach.mixin.logging import LoggingMixin
  File "/home/fred/logs/mail/src/mozilla/python/mozbuild/mozbuild/base.py", line 17, in <module>
    from mach.mixin.logging import LoggingMixin
  File "/usr/lib64/python2.7/bdb.py", line 49, in trace_dispatch
    return self.dispatch_line(frame)
  File "/usr/lib64/python2.7/bdb.py", line 68, in dispatch_line
    if self.quitting: raise BdbQuit
bdb.BdbQuit

I don't know if it is really helpful, but this is what I get.

Gregory Szorc [:gps]

Comment 12

•

11 years ago

(In reply to Frederic Bezies from comment #7)
> Replacing in my .mozconfig 
> 
> mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../objdir-tb
> 
> by mk_add_options MOZ_OBJDIR=/home/fred/logs/mail/objdir-tb removed the bug,
> even if I still get Makefile:69: codegen.pp: No such file or directory

So, you have a valid workaround, right? (Don't use relative paths when defining MOZ_OBJDIR.)

The make warning about codegen.pp missing can be safely ignored. You'll see that on first build.

I'm tempted to add this to the long list of bugs on the "comm-central build system integration is fragile" list. The cure is to merge c-c into m-c. That's apparently being worked on.

Can we resolve this WORKSFORME?

Frederic Bezies

Reporter

Comment 13

•

11 years ago

(In reply to Gregory Szorc [:gps] from comment #12)
[...]
> So, you have a valid workaround, right? (Don't use relative paths when
> defining MOZ_OBJDIR.)

Yes. Kinda "ugly" but working one !

> 
> The make warning about codegen.pp missing can be safely ignored. You'll see
> that on first build.
> 
> I'm tempted to add this to the long list of bugs on the "comm-central build
> system integration is fragile" list. The cure is to merge c-c into m-c.
> That's apparently being worked on.

Great.

> 
> Can we resolve this WORKSFORME?

Yes. The only solution. Feel free to close it. And thanks for your help !

Gregory Szorc [:gps]

Updated

•

11 years ago

Status: NEW → RESOLVED

Closed: 11 years ago

Resolution: --- → WORKSFORME

Bugzilla

Quick Search

[trunk] Thunderbird build process is broken while processing codegen.pp : no such file or directory.

Categories

(Thunderbird :: Build Config, defect)

Tracking

(Not tracked)

People

(Reporter: fredbezies, Unassigned)

References

Details

Crash Data

Security

(public)

User Story

Description

Comment 1

Comment 2

Comment 3

Comment 4

Comment 5

Comment 6

Updated

Comment 7

Comment 8

Comment 9

Comment 10

Comment 11

Comment 12

Comment 13

Updated