754625 - No such file or directory: '../../../../xpcom/idl-parser/cache' after checking in eae1858ccc6d (Bug 736961)

Status: VERIFIED FIXED

(Firefox Build System :: General, defect, P1)

Description

[Takanori MATSUURA]

In the case of building Firefox on top of XULRunner, the build is failed as the following:

make[7]: Entering directory `/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/objdir/browser/components/feeds'
make -C public export
make[8]: Entering directory `/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/objdir/browser/components/feeds/public'
mkdir -p /foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/objdir/browser/components/feeds/public/.deps
mkdir -p _xpidlgen/
mkdir -p .deps/
nsIFeedResultService.idl
/usr/bin/python2.7 /foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/config/pythonpath.py \
   \
  -I/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/xpcom/idl-parser \
  /foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/xpcom/idl-parser/header.py --cachedir=../../../../xpcom/idl-parser/cache -I/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/browser/components/feeds/public -I../../../../dist/idl -I/usr/lib64/xulrunner-devel-15/idl /foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/browser/components/feeds/public/nsIFeedResultService.idl -d .deps/nsIFeedResultService.h.pp -o _xpidlgen/nsIFeedResultService.h
Traceback (most recent call last):
  File "/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/config/pythonpath.py", line 44, in main
    execfile(script, frozenglobals)
  File "/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/xpcom/idl-parser/header.py", line 499, in <module>
Traceback (most recent call last):
  File "/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/config/pythonpath.py", line 44, in main
    os.mkdir(options.cachedir)
OSError    : [Errno 2] No such file or directory: '../../../xpcom/idl-parser/cache'
execfile(script, frozenglobals)
  File "/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/xpcom/idl-parser/header.py", line 499, in <module>
    os.mkdir(options.cachedir)
OSError: [Errno 2] No such file or directory: '../../../../xpcom/idl-parser/cache'
make[8]: make[7]: *** [_xpidlgen/fuelIApplication.h] Error 1*** [_xpidlgen/nsIFeedResultService.h] Error 1

rm _xpidlgen/.mkdir.donerm _xpidlgen/.mkdir.done .deps/.mkdir.done .deps/.mkdir.done

make[7]: Leaving directory `/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/objdir/browser/fuel/public'
make[8]: Leaving directory `/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/objdir/browser/components/feeds/public'
make[6]: *** [export] Error 2
make[7]: *** [export] Error 2
make[6]: Leaving directory `/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/objdir/browser/fuel'
make[7]: Leaving directory `/foo/rpmbuild/BUILD/firefox-15.0/mozilla-central/objdir/browser/components/feeds'
make[5]: *** [fuel_export] Error 2

Comment 2

[:aceman]

I get bug 754642 too, on Linux, gcc.

Comment 3

[Karsten Düsterloh]

I was able to build SeaMonkey after:
- pulling current trunk
- cleaning my objdir
- deleting mozilla/xpcom and doing hg up -C to rewrite it

Comment 4

[Takanori MATSUURA]

There are two possible solutions:

1. Use os.mkdirs instead of os.mkdir to generate cachedir
xpcom/idl-parser/header.py:
     if options.cachedir is not None:
         if not os.path.isdir(options.cachedir):
             os.mkdir(options.cachedir)
         sys.path.append(options.cachedir)

2. "make export" at xpcom/idl-parser before parsing idl files
xpcom/idl-parser/Makefile.in:
# Generate the PLY lexer and parser.
export:: $(PARSER_SRCS) $(PLY_PROGS)
	$(PYTHON_PATH) \
	  $(PLY_INCLUDE) \
	  -I$(topsrcdir)/xpcom/idl-parser \
	  $(topsrcdir)/xpcom/idl-parser/header.py --cachedir=$(DEPTH)/xpcom/idl-parser/cache --regen

Comment 5

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

What is causing us to parse idl files before we get to 'export' in xpcom/idl-parser?

Comment 6

[Bengt-Erik Soderstrom]

(In reply to Karsten Düsterloh from comment #3)
> I was able to build SeaMonkey after:
> - pulling current trunk
> - cleaning my objdir
> - deleting mozilla/xpcom and doing hg up -C to rewrite it

This is what can be called a one-time-workaround. It works, but only once. The second time we make an update and build, we are back to the beginning again..

Comment 7

[Landry Breuil (:gaston)]

I realized that c-c config/rules.mk also contains two references to a cachedir pointing to $(MOZILLA_DIR)/xpcom/idl-parser which should probably be amended too. Dunno if that's the issue, trying to sync them too.

Comment 8

[Mike Hommey [:glandium]]

The easy way out would be to use the scripts from $(LIBXUL_DIST)/sdk/bin instead of $(topsrcdir)/xpcom/idl-parser, and not set a cachedir at all.

Comment 9

[Mike Hommey [:glandium]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #5)
> What is causing us to parse idl files before we get to 'export' in
> xpcom/idl-parser?

In case of Firefox-on-Xulrunner, we don't export in xpcom/idl-parser. The real solution is comment 8.

Comment 10

[Mike Hommey [:glandium]]

Attachment: Use idl-parser from $(LIBXUL_DIST)/sdk/bin

Comment 11

[Landry Breuil (:gaston)]

Attachment: Use idl-parser from $(LIBXUL_DIST)/sdk/bin in comm-central config/rules.mk

Here's the comm-central similar patch, currently building here..

Comment 12

[Landry Breuil (:gaston)]

(In reply to Mike Hommey [:glandium] from comment #10)
> Created attachment 623616 [details] [diff] [review]
> Use idl-parser from $(LIBXUL_DIST)/sdk/bin

Note that in addition to that patch i also needed to "fix" xpcom/typelib/xpidl/Makefile.in :

-  $(DEPTH)/xpcom/idl-parser/cache/xpidllex.py \
-  $(DEPTH)/xpcom/idl-parser/cache/xpidlyacc.py \
+  $(DEPTH)/xpcom/idl-parser/xpidllex.py \
+  $(DEPTH)/xpcom/idl-parser/xpidlyacc.py \

That properly installed those files to sdk/bin

Comment 13

[Mike Hommey [:glandium]]

Attachment: Use idl-parser from $(LIBXUL_DIST)/sdk/bin

Comment 14

[Serge Gautherie (:sgautherie)]

Thunderbird is (partly) affected too.

Comment 15

[:aceman]

I had to 'hg backout eae1858ccc6d' to make TB build again. What is the definition of "partly"?

Comment 16

[Serge Gautherie (:sgautherie)]

(In reply to :aceman from comment #15)
> What is the definition of "partly"?

I meant that not all (SM and TB) builds are broken: I'm guessing they haven't been clobbered (or whatever) yet...

Comment 17

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

How do these patches not reintroduce bug 736961?

Comment 18

[Takanori MATSUURA]

Attachment 623637 [details] [diff] works fine for Firefox on top of XULRunner.
And attachment 623627 [details] [diff] [review] works fine for Thunderbird (not on top of XULRunner because of bug 306324).

Comment 19

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Do they work in a build with no objdir?

Comment 20

[Mike Hommey [:glandium]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #19)
> Do they work in a build with no objdir?

They probably don't. --regen should just remove xpidllex and xpidlyacc, instead of *.

Comment 21

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

It seems to make more sense to me to back out bug 736961 and fix it differently then.

Comment 22

[Mike Hommey [:glandium]]

Attachment: Use idl-parser from $(LIBXUL_DIST)/sdk/bin

With restricted removals.

Comment 23

[Mike Hommey [:glandium]]

Landed the m-c part:
https://hg.mozilla.org/mozilla-central/rev/9e5b25074f9f

Comment 24

[Serge Gautherie (:sgautherie)]

SeaMonkey error has changed to
"OSError: [Errno 2] No such file or directory: './xpidllex.py'"

What about the c-c part?

Comment 25

[Landry Breuil (:gaston)]

(In reply to Serge Gautherie (:sgautherie) from comment #24)
> SeaMonkey error has changed to
> "OSError: [Errno 2] No such file or directory: './xpidllex.py'"
> 
> What about the c-c part?

Is this with or without attachement 623627 ?

Comment 26

[Serge Gautherie (:sgautherie)]

(In reply to Landry Breuil (:gaston) from comment #25)
> Is this with or without attachement 623627 ?

With comment 23 changeset, without c-c part which hasn't been pushed yet: hence my question.

Comment 27

[Mike Hommey [:glandium]]

There's a pending fixup for the m-c part anyways.

Comment 28

[Mike Hommey [:glandium]]

Fixup:
https://hg.mozilla.org/mozilla-central/rev/3bd6844f2898

Comment 29

[Mike Hommey [:glandium]]

c-c part:
https://hg.mozilla.org/comm-central/rev/e6c1bac57163

Comment 30

[Landry Breuil (:gaston)]

tbpl for tb-trunk seems happy with it :
https://tbpl.mozilla.org/?tree=Thunderbird-Trunk&rev=e6c1bac57163

Comment 31

[Landry Breuil (:gaston)]

argh.

Comment 32

[Serge Gautherie (:sgautherie)]

V.Fixed, per SeaMonkey TBPL too.

Comment 33

[:aceman]

Confirming fix in TB when I updated with hg.