Closed Bug 1254987 Opened 4 years ago Closed 4 years ago

Fix comm-central fallout from |Bug 1254410 - Include app-specific configure files according to --enable-application/--enable-project|

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(firefox48 affected)

RESOLVED FIXED
mozilla48
Tracking Status
firefox48 --- affected

People

(Reporter: clokep, Assigned: philip.chee)

References

Details

Attachments

(3 files, 8 obsolete files)

You end up with an error like:

0:00.22 /usr/bin/make -f client.mk -s
 0:00.39 Adding client.mk options from /home/abdelrhman/mozilla/comm-central/.mozconfig:
 0:00.39     FOUND_MOZCONFIG := /home/abdelrhman/mozilla/comm-central/.mozconfig
 0:00.49 cd /home/abdelrhman/mozilla/comm-central/obj-x86_64-unknown-linux-gnu
 0:00.49 /home/abdelrhman/mozilla/comm-central/configure
 0:00.50 loading cache ./config.cache
 0:00.60 Reexecuting in the virtualenv
 0:00.69 Adding configure options from /home/abdelrhman/mozilla/comm-central/.mozconfig
 0:00.69   --enable-application=im
 0:00.69 Cannot find project im
 0:00.71 *** Fix above errors and then restart with\
 0:00.71                "/usr/bin/make -f client.mk build"
 0:00.71 make[1]: *** [configure] Error 1
 0:00.71 make: *** [/home/abdelrhman/mozilla/comm-central/obj-x86_64-unknown-linux-gnu/Makefile] Error 2
 0:00.74 0 compiler warnings present.

This pretty much comes from build_env['TOPSRCDIR'] being the mozilla/ directory instead of comm-central, e.g. my build_env in the include_project_configure method is:
{u'DIST': u'/Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb/dist', u'TOPOBJDIR': u'/Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb', u'TOPSRCDIR': u'/Users/pcloke/mozilla/comm-central/mozilla', u'MOZ_BUILD_ROOT': u'/Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb'}

http://hg.mozilla.org/mozilla-central/file/tip/build/moz.configure/init.configure#l221

I'm a bit stumped on how to fix this though. (Well besides doing incredibly hacky things.)
Blocks: 1254410
(In reply to aleth [:aleth] from comment #1)
> See bug 1254410 comment 2...

Awesome. I'm going to try this now.
I'm making some progress on this.
Assignee: nobody → clokep
Status: NEW → ASSIGNED
Attached patch WIP (obsolete) — Splinter Review
I've unfortunately ran out of time to look at this at the moment. This actually does start to run configure, but bails out. Please let me know if you have any ideas. (Note that configure.py and configure.in are copied with a couple of paths changed. We used to have some magic inside of configure.in that this blows away...which is probably not good.)

 0:00.24 /usr/bin/make -f client.mk -s configure
 0:00.37 cd /Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb
 0:00.37 /Users/pcloke/mozilla/comm-central/configure
 0:00.44 Creating Python environment
 0:00.57 New python executable in /Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb/_virtualenv/bin/python2.7
 0:00.57 Not overwriting existing python script /Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb/_virtualenv/bin/python (you must use /Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb/_virtualenv/bin/python2.7)
 0:06.21 Installing setuptools, pip, wheel...done.
 0:06.59 running build_ext
 0:06.59 copying build/lib.macosx-10.10-x86_64-2.7/psutil/_psutil_osx.so -> psutil
 0:06.59 copying build/lib.macosx-10.10-x86_64-2.7/psutil/_psutil_posix.so -> psutil
 0:06.59 
 0:06.59 Reexecuting in the virtualenv
 0:06.74 Traceback (most recent call last):
 0:06.74   File "/Users/pcloke/mozilla/comm-central/configure.py", line 73, in <module>
 0:06.74     sys.exit(main(sys.argv))
 0:06.74   File "/Users/pcloke/mozilla/comm-central/configure.py", line 22, in main
 0:06.74     sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
 0:06.74   File "/Users/pcloke/mozilla/comm-central/mozilla/python/mozbuild/mozbuild/configure/__init__.py", line 197, in run
 0:06.74     % option.option
 0:06.74 mozbuild.configure.ConfigureError: Option `MOZILLABUILD` is not handled ; reference it with a @depends
 0:06.76 *** Fix above errors and then restart with               "/Applications/Xcode.app/Contents/Developer/usr/bin/make -f client.mk build"
 0:06.76 make: *** [configure] Error 1
Comment on attachment 8728536 [details] [diff] [review]
WIP2 Currently building (omg omg)

Review of attachment 8728536 [details] [diff] [review]:
-----------------------------------------------------------------

::: configure.py
@@ +18,5 @@
> +
> +def main(argv):
> +    config = {}
> +    sandbox = ConfigureSandbox(config, os.environ, argv)
> +    sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))

I was wondering whether we could do `sandbox.run(os.path.join(base_dir, 'moz.configure'))` here instead of copying moz.configure.
Looks like this doesn't work yet for IB:
 1:00.96 Makefile:16: warning: overriding commands for target `DeprecatedPremultiplyTables.h'
 1:00.96 backend.mk:8: warning: ignoring old commands for target `DeprecatedPremultiplyTables.h'
 1:01.00 test_necko.xpt
 1:01.06 Usage: genTables.py <header.h>
 1:01.06 make[4]: *** [DeprecatedPremultiplyTables.h] Error 1
 1:01.06 make[3]: *** [gfx/thebes/export] Error 2
 1:01.06 make[3]: *** Waiting for unfinished jobs....
(In reply to aleth [:aleth] from comment #7)
> Looks like this doesn't work yet for IB:
>  1:00.96 Makefile:16: warning: overriding commands for target
> `DeprecatedPremultiplyTables.h'
>  1:00.96 backend.mk:8: warning: ignoring old commands for target
> `DeprecatedPremultiplyTables.h'
>  1:01.00 test_necko.xpt
>  1:01.06 Usage: genTables.py <header.h>
>  1:01.06 make[4]: *** [DeprecatedPremultiplyTables.h] Error 1
>  1:01.06 make[3]: *** [gfx/thebes/export] Error 2
>  1:01.06 make[3]: *** Waiting for unfinished jobs....

Possibly related to bug 1252302?
Attached patch Patch v3 works. (obsolete) — Splinter Review
Looks like it works. But my build fails somewhere in nsGlobalWindow probably unreated.
Attachment #8728596 - Flags: review?(clokep)
Attachment #8728596 - Flags: review?(Pidgeot18)
(In reply to Philip Chee from comment #9)
> Created attachment 8728596 [details] [diff] [review]
> Patch v3 works.
> 
> Looks like it works. But my build fails somewhere in nsGlobalWindow probably
> unreated.

This isn't working for me. I applied the patch in bug 1254795, am on the tip of m-c and c-c with this applied to c-c. Anything else I'm supposed to be doing? (I still get a "Cannot find project mail") If I'm missing a step to test this, can yo please expand? Thanks!
Assignee: clokep → philip.chee
Comment on attachment 8728596 [details] [diff] [review]
Patch v3 works.

Review of attachment 8728596 [details] [diff] [review]:
-----------------------------------------------------------------

While running |mach configure| with a mozconfig of:

ac_add_options --enable-application=../mail
ac_add_options --enable-calendar

This sort of worked...still ran into "ERROR PROCESSING MOZBUILD FILE"

::: mail/moz.configure
@@ +3,5 @@
> +# This Source Code Form is subject to the terms of the Mozilla Public
> +# License, v. 2.0. If a copy of the MPL was not distributed with this
> +# file, You can obtain one at http://mozilla.org/MPL/2.0/.
> +
> +include('../toolkit/moz.configure')

I had to change each of these (with glandium's help) to `include('../mozilla/toolkit/moz.configure')`
Attachment #8728596 - Flags: review?(clokep) → review-
(In reply to Patrick Cloke [:clokep] from comment #11)
> Comment on attachment 8728596 [details] [diff] [review]
> Patch v3 works.
> 
> Review of attachment 8728596 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> While running |mach configure| with a mozconfig of:
> 
> ac_add_options --enable-application=../mail
> ac_add_options --enable-calendar

It passed configure for me with the standard
ac_add_options --enable-application=mail
A.T.M. It works with mozilla/mach configure && mozilla/mach build
but not with mozmake -f client.py configure
This was broken by bug 1253203, since the "internal" name of the function changed. Fortunately, it's easy to fix with a changed name.
Attachment #8728824 - Flags: review?(mh+mozilla)
Attachment #8728596 - Flags: review?(Pidgeot18) → review-
Comment on attachment 8728824 [details] [diff] [review]
[mozilla-central] Get the subconfigure working properly for mail/configure.in [checked in, comment #18 and #22]

With only your patch I get this:

> $ MOZCONFIG=/c/t1/hg/comm-central/mozconfig mozilla/mach configure
>  0:00.76 C:\DEV\mozilla-build\msys\local\bin\mozmake.EXE -f client.mk MOZ_PARALLEL_BUILD=6 -s configure
>  0:03.42 cd c:/t1/hg/objdir-sm
>  0:03.45 c:/t1/hg/comm-central/configure
>  0:03.99 loading cache ./config.cache
>  0:04.77 Reexecuting in the virtualenv
>  0:05.52 Adding configure options from c:\t1\hg\comm-central\mozconfig
>  0:05.52   --enable-application=suite
>  0:05.52   --enable-optimize
>  0:05.52   --disable-debug
>  0:05.52   --enable-debug-symbols
>  0:05.52   --enable-tests
>  0:05.52   --enable-jemalloc
>  0:05.52   --enable-calendar
>  0:05.52   --with-windows-version=603
>  0:05.52   --disable-startupcache
>  0:05.52 Cannot find project suite
>  0:05.55 *** Fix above errors and then restart with\
>  0:05.55                "C:/DEV/mozilla-build/msys/local/bin/mozmake.EXE -f client.mk build"
>  0:05.57 client.mk:363: recipe for target 'configure' failed
>  0:05.57 mozmake.EXE: *** [configure] Error 1
Flags: needinfo?(Pidgeot18)
Attachment #8728824 - Flags: review?(mh+mozilla) → review+
Attachment #8728596 - Attachment is obsolete: true
Blocks: 1255370
Duplicate of this bug: 1255370
https://hg.mozilla.org/integration/mozilla-inbound/rev/9f6559ebb5932cdf9e0bd24bf91700f2bfa1a1a4
Bug 1254987 - Get the subconfigure working properly for mail/configure.in. r=glandium
Depends on: 1255312
(In reply to Philip Chee from comment #15)
> With only your patch I get this:

It's not sufficient, but it is necessary to get the */configure.in working properly.
Flags: needinfo?(Pidgeot18)
Whiteboard: [leave open]
Duplicate of this bug: 1255635
The other m-c change we need is bug 1255312, plus a related change there that adds --without-external-source-dir. With those two fixes, js/src/configure can run, and then we can happily build comm-central, once the moz.configure stuff is ported into c-c.
Status update: With jcranmer's patch [1], OS X builds OK on current c-c [2], but there seems to be a path issue with the tests.

[1] https://hg.mozilla.org/try-comm-central/rev/c6ff7bf60172
[2] https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=5bff81c7c0b6
Attached patch Patch works on Windows (obsolete) — Splinter Review
Successfully Configured, build, and link.
Attachment #8728536 - Attachment is obsolete: true
Attachment #8730045 - Flags: review?(Pidgeot18)
(In reply to Philip Chee from comment #24)
> Created attachment 8730045 [details] [diff] [review]
> Patch works on Windows

On SM-Trunk x86_64:

[...]
Adding configure options from /home/hafi/moz-work/src/.mozconfig
  --enable-application=suite
  --enable-optimize
  --disable-tests
  --enable-calendar
  --enable-extensions=default
  --enable-gstreamer=1.0
  --disable-pulseaudio
  --disable-elf-hack
  --enable-default-toolkit=cairo-gtk2
[...]
js/src/ctypes/libffi> config.status: executing src commands

Traceback (most recent call last):
  File "/home/hafi/moz-work/src/configure.py", line 73, in <module>
    sys.exit(main(sys.argv))
  File "/home/hafi/moz-work/src/configure.py", line 22, in main
    sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
  File "/home/hafi/moz-work/src/mozilla/python/mozbuild/mozbuild/configure/__init__.py", line 190, in run
    raise InvalidOptionError('Unknown option: %s' % without_value)
mozbuild.configure.options.InvalidOptionError: Unknown option: --enable-gstreamer

After commenting out
ac_add_options --enable-gstreamer=1.0

the build is now running. Not finished yet.
(In reply to Hartmut Figge from comment #25)

> (In reply to Philip Chee from comment #24)

>     raise InvalidOptionError('Unknown option: %s' % without_value)
> mozbuild.configure.options.InvalidOptionError: Unknown option:
> --enable-gstreamer
> 
> After commenting out
> ac_add_options --enable-gstreamer=1.0
> 
> the build is now running. Not finished yet.

Perhaps --enable-gstreamer needs to be added here:
http://hg.mozilla.org/mozilla-central/rev/9fddd3fdc22a#l1.13
(In reply to Philip Chee from comment #26)

> Perhaps --enable-gstreamer needs to be added here:
> http://hg.mozilla.org/mozilla-central/rev/9fddd3fdc22a#l1.13

Yes, that works. Using the new build now.
(In reply to Hartmut Figge from comment #27)
> (In reply to Philip Chee from comment #26)
> 
> > Perhaps --enable-gstreamer needs to be added here:
> > http://hg.mozilla.org/mozilla-central/rev/9fddd3fdc22a#l1.13
> 
> Yes, that works. Using the new build now.

There is an issue with the new build. May be unrelated. The used URL is not shown in the Address Bar of the browser. Just noticed when I tried to copy it for a success message to de.comm.software.mozilla.nightly-builds. :)
Attached image missing.png (obsolete) —
Missing URl in Address Bar, missing Entries in Bookmarks Toolbar

The first picture is from my last successful build, 20160309, the picture below is taken from my current build, 20160314. This may have nothing to do with the bustage fix. May.
(In reply to Philip Chee from comment #26)
> (In reply to Hartmut Figge from comment #25)
> 
> > (In reply to Philip Chee from comment #24)
> 
> >     raise InvalidOptionError('Unknown option: %s' % without_value)
> > mozbuild.configure.options.InvalidOptionError: Unknown option:
> > --enable-gstreamer
> > 
> > After commenting out
> > ac_add_options --enable-gstreamer=1.0
> > 
> > the build is now running. Not finished yet.
> 
> Perhaps --enable-gstreamer needs to be added here:
> http://hg.mozilla.org/mozilla-central/rev/9fddd3fdc22a#l1.13

No, remove it from your mozconfig, it was removed in bug 1234092 and hasn't been doing anything since.
Mine failed with:

5:27.62 mozbuild.configure.options.InvalidOptionError: Unknown option: --enable-inspector-apis

Is --enable-inspector-apis also deprecated?
Flags: needinfo?(mh+mozilla)
Was able to build suite under Windows too. VS2915 x86 and x64 compiled fine. It didn't work right out of the box. Bug 1235805 needs to be applied (ios.newChannel2 fixes) and navigator.js needed 2 'this.' for getters / setters due to Bug 1254752. Noscript and Flashgot are broken right now and need to be fixed by the developer.

FRG
Built TB on Win10 with Ratty's patch and first quick check shows it's working.
>> 'this.' for getters / setters due to Bug 1254752. 

Typo. Make this: due to Bug 1253016.
Frank, that was rather cryptic for a mere user like me. ;) But I have managed. The issue of Comment 29 is gone now.
Hartmu,

it would have been easier if I hadn't bungled the bug number in the first place :) I filed bug 1256272 for Seamonkey. 

Noscript and Flashgot are currently broken, I think, because of this too. I am sure they will be fixed soon too.

FRG
Hi,

I am a little bit confused. This bug was supposed to be C-C TB issue, right?
The last several comments referred to Flash and NoScript: could it be related SeaMonkey's browser?

Also, my fresh fetch and trial to build locally failed again with 
"Cannot find project mail".
Hmm...
>> I am a little bit confused. This bug was supposed to be C-C TB issue, right?

Yes and suite and the other comm-central applications.

>> The last several comments referred to Flash and NoScript: could it be related SeaMonkey's browser?

Yes. Follow up bugs to get it running after a week of no builds which have nothing to do with this one. 

FRG
(In reply to Richard Marti (:Paenglab) from comment #31)
> Mine failed with:
> 
> 5:27.62 mozbuild.configure.options.InvalidOptionError: Unknown option:
> --enable-inspector-apis
> 
> Is --enable-inspector-apis also deprecated?

Removed in bug 513924
Flags: needinfo?(mh+mozilla)
Contrary to what I've claimed to jcranmer, we're going to have to modify configure.py, so it's better if you don't copy it. So what I'm going to do is make the part you'd rather avoid copying importable.
The newly landed bugs 1256568, 1256587 and 1256990 needs this additional fix.
Attachment #8731205 - Flags: review?(Pidgeot18)
Comment on attachment 8731205 [details] [diff] [review]
Additional fix for bugs 1256568, 1256587 and 1256990

Review of attachment 8731205 [details] [diff] [review]:
-----------------------------------------------------------------

Why are you copying moz.configure? Just create a moz.configure with this:

include('mozilla/moz.configure')
Attachment #8731205 - Flags: review?(Pidgeot18) → review-
Comment on attachment 8730045 [details] [diff] [review]
Patch works on Windows

Review of attachment 8730045 [details] [diff] [review]:
-----------------------------------------------------------------

::: configure.in
@@ +14,1 @@
>  AC_PREREQ(2.13)

No AC_ macros, please.

::: configure.py
@@ +24,5 @@
> +    if sandbox._help:
> +        return 0
> +
> +    # Sanitize config data to feed config.status
> +    sanitized_config = {}

See bug 1256574, you don't need to copy the entire file anymore.
Attachment #8730045 - Flags: review?(Pidgeot18) → review-
(In reply to Mike Hommey [:glandium] from comment #42)
> Comment on attachment 8731205 [details] [diff] [review]
> Additional fix for bugs 1256568, 1256587 and 1256990
> 
> Review of attachment 8731205 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Why are you copying moz.configure? Just create a moz.configure with this:
> 
> include('mozilla/moz.configure')


I tried the include('mozilla/moz.configure') at the beginning and the end of the file (and removed the direct includes) then the build failed. I think this is because all 'mozilla/moz.configure' files will be included in one rush and the c-c configurations are then too late or to early.

I think the order needs to be:

mozilla/build/moz.configure/init.configure
mozilla/build/moz.configure/checks.configure
c-c/moz.configure
mozilla/build/moz.configure/old.configure

to work correctly.

But I don't know really what I'm doing here and copying only the changes from m-c to c-c.
(In reply to Richard Marti (:Paenglab) from comment #44)
> (In reply to Mike Hommey [:glandium] from comment #42)
> > Comment on attachment 8731205 [details] [diff] [review]
> > Additional fix for bugs 1256568, 1256587 and 1256990
> > 
> > Review of attachment 8731205 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > Why are you copying moz.configure? Just create a moz.configure with this:
> > 
> > include('mozilla/moz.configure')
> 
> 
> I tried the include('mozilla/moz.configure') at the beginning and the end of
> the file (and removed the direct includes) then the build failed.

That you're talking about the beginning or the end of the file tells me you didn't try what I'm saying. What I'm saying is, litterally, remove everything from your moz.configure, and make it contain include('mozilla/moz.configure') and that's all.
(In reply to Mike Hommey [:glandium] from comment #45)
> That you're talking about the beginning or the end of the file tells me you
> didn't try what I'm saying. What I'm saying is, litterally, remove
> everything from your moz.configure, and make it contain
> include('mozilla/moz.configure') and that's all.

Now I understand. Yes, it works this way.

I'm trying to address the other comments of the other patch.

(In reply to Mike Hommey [:glandium] from comment #43)
> Comment on attachment 8730045 [details] [diff] [review]
> ::: configure.py
> @@ +24,5 @@
> > +    if sandbox._help:
> > +        return 0
> > +
> > +    # Sanitize config data to feed config.status
> > +    sanitized_config = {}
> 
> See bug 1256574, you don't need to copy the entire file anymore.

Never written a Python script. How do I import config_status(config) from your configure.py in c-c configure.py.
Attached patch Don't copy & paste config_status (obsolete) — Splinter Review
This is attachment 8731205 [details] [diff] [review] with some python import magic to get config_status from the Mozilla version. It's slightly gross since Mozilla's version is shadowed by the current file...required a little bit of import foo.
Attachment #8728490 - Attachment is obsolete: true
Attachment #8731205 - Attachment is obsolete: true
Attachment #8730102 - Attachment is obsolete: true
Comment on attachment 8731281 [details] [diff] [review]
Don't copy & paste config_status

All review comments addressed.

Locally it builds on Linux and Windows. Try fails for Linux and Windows, OS X has already one green. I think this is more a buildbot issue.
Attachment #8731281 - Flags: review?(mh+mozilla)
The Windows failure is "only" a build failure ("recipe for target 'icalderivedvalue.h' failed"), looks like what broke is similar to bug 1247282.

The Linux problem is that it's looking for gcc in the wrong dir.
This is set by
https://dxr.mozilla.org/comm-central/source/build/unix/mozconfig.linux#16
(Note these mozconfigs are enforced to match the ones in mozilla/build.)
So it looks like $topsrcdir isn't set correctly at the time when that is evaluated. Any ideas as to why? (It's set in configure.in.)
Flags: needinfo?(mh+mozilla)
> The Linux problem is that it's looking for gcc in the wrong dir.
Link to the logs for future reference: https://treeherder.mozilla.org/logviewer.html#?job_id=17253&repo=try-comm-central#L2269
what are c-c builds expecting $topsrcdir to be in mozconfig?
Flags: needinfo?(mh+mozilla)
(In reply to Mike Hommey [:glandium] from comment #51)
> what are c-c builds expecting $topsrcdir to be in mozconfig?

Maybe I'm misunderstanding the question, but since this [1] worked OK before the recent configure changes, and it's now looking for gcc in /builds/slave/tb-try-c-cen-l64-0000000000000/build/mozilla/gcc/bin/gcc, $topsrcdir must have pointed at the c-c dir before during this step, and now points at the m-c subdir.

[1] https://dxr.mozilla.org/comm-central/source/build/unix/mozconfig.linux#16
No longer blocks: 1255370
Flags: needinfo?(mh+mozilla)
Tried to build thunderbird using patch from comment 48, and I got this while building process is trying to "setup" libxul.so.

84:25.14 ../../mail/components/build/nsMailComps.o: In function `nsMailGNOMEIntegrationConstructor(nsISupports*, nsID const&, void**)':
84:25.14 nsMailComps.cpp:(.text._ZL33nsMailGNOMEIntegrationConstructorP11nsISupportsRK4nsIDPPv+0x30): undefined reference to `nsMailGNOMEIntegration::nsMailGNOMEIntegration()'
84:25.14 nsMailComps.cpp:(.text._ZL33nsMailGNOMEIntegrationConstructorP11nsISupportsRK4nsIDPPv+0x46): undefined reference to `nsMailGNOMEIntegration::Init()'
84:25.14 /usr/bin/ld: libxul.so: hidden symbol `_ZN22nsMailGNOMEIntegration4InitEv' isn't defined
84:25.14 /usr/bin/ld: final link failed: Bad value
84:25.14 collect2: error: ld returned 1 exit status
84:25.14 /home/fred/logs/mail/src/mozilla/config/rules.mk:808: recipe for target 'libxul.so' failed
84:25.14 make[4]: *** [libxul.so] Error 1
84:25.14 /home/fred/logs/mail/src/mozilla/config/recurse.mk:71: recipe for target 'toolkit/library/target' failed
84:25.14 make[3]: *** [toolkit/library/target] Error 2
84:25.14 /home/fred/logs/mail/src/mozilla/config/recurse.mk:32: recipe for target 'compile' failed
84:25.14 make[2]: *** [compile] Error 2
84:25.14 /home/fred/logs/mail/src/mozilla/config/rules.mk:531: recipe for target 'default' failed
84:25.15 make[1]: *** [default] Error 2
84:25.15 client.mk:406: recipe for target 'build' failed
84:25.15 make: *** [build] Error 2
84:25.15 1254 compiler warnings present.

I'm on Archlinux with gcc 5.3 and gtk+ 3.18.9. Using a blank new trunk code. A trunk code version of Mozilla Firefox was made on my computer a few hours ago.

Any ideas ?
(In reply to Frederic Bezies from comment #54)
> Tried to build thunderbird using patch from comment 48, and I got this while
> building process is trying to "setup" libxul.so.
> 
> 84:25.14 ../../mail/components/build/nsMailComps.o: In function
> `nsMailGNOMEIntegrationConstructor(nsISupports*, nsID const&, void**)':
> 84:25.14
> nsMailComps.cpp:(.text.
> _ZL33nsMailGNOMEIntegrationConstructorP11nsISupportsRK4nsIDPPv+0x30):
> undefined reference to `nsMailGNOMEIntegration::nsMailGNOMEIntegration()'
> 84:25.14
> nsMailComps.cpp:(.text.
> _ZL33nsMailGNOMEIntegrationConstructorP11nsISupportsRK4nsIDPPv+0x46):
> undefined reference to `nsMailGNOMEIntegration::Init()'
> 84:25.14 /usr/bin/ld: libxul.so: hidden symbol
> `_ZN22nsMailGNOMEIntegration4InitEv' isn't defined
> 
> Any ideas ?

I see this too, I isolated it to bug 1566988.
Yes it's bug 1256988, sorry.
(In reply to Richard Marti (:Paenglab) from comment #57)
> Yes it's bug 1256988, sorry.

There is a little problem. This bug is closed as resolved fixed. So it will be logical not to be blocked while libxul.so is processed. Are you 100% sure this is the right bug ? :D
(In reply to Frederic Bezies from comment #58)
> (In reply to Richard Marti (:Paenglab) from comment #57)
> > Yes it's bug 1256988, sorry.
> 
> There is a little problem. This bug is closed as resolved fixed. So it will
> be logical not to be blocked while libxul.so is processed. Are you 100% sure
> this is the right bug ? :D

It's resolved fixed in m-c. But this needs changes also in c-c. With m-c on the revision before this patch TB builds and with only bug 1256988 in it, it fails.

I'm trying now a patch and hope this fixes the build problem.
(In reply to Richard Marti (:Paenglab) from comment #59)
> (In reply to Frederic Bezies from comment #58)
> > (In reply to Richard Marti (:Paenglab) from comment #57)
> > > Yes it's bug 1256988, sorry.
> > 
> > There is a little problem. This bug is closed as resolved fixed. So it will
> > be logical not to be blocked while libxul.so is processed. Are you 100% sure
> > this is the right bug ? :D
> 
> It's resolved fixed in m-c. But this needs changes also in c-c. With m-c on
> the revision before this patch TB builds and with only bug 1256988 in it, it
> fails.
> 
> I'm trying now a patch and hope this fixes the build problem.

Oops ! I didn't notice it was added on m-c... Well, let's hope you'll get a patch soon :)
This patch let me build TB on Linux on top of the other patch with m-c tip. Tested is only TB but I patched also IB and SM.
Attachment #8732637 - Flags: review?(mh+mozilla)
Attachment #8730045 - Attachment is obsolete: true
(In reply to Richard Marti (:Paenglab) from comment #61)
> Created attachment 8732637 [details] [diff] [review]
> GTK patch
> 
> This patch let me build TB on Linux on top of the other patch with m-c tip.
> Tested is only TB but I patched also IB and SM.

It fixes Bug 1258208. I should have wait a bit longer to file that. Sigh.
Attachment #8728824 - Attachment description: [mozilla-central] Get the subconfigure working properly for mail/configure.in → [mozilla-central] Get the subconfigure working properly for mail/configure.in [checked in]
(In reply to Richard Marti (:Paenglab) from comment #61)
> Created attachment 8732637 [details] [diff] [review]
> GTK patch
> 
> This patch let me build TB on Linux on top of the other patch with m-c tip.
> Tested is only TB but I patched also IB and SM.

Thanks for this patch. It is great to get a working thunderbird homemade build after nearly two weeks.
Attachment #8728824 - Attachment description: [mozilla-central] Get the subconfigure working properly for mail/configure.in [checked in] → [mozilla-central] Get the subconfigure working properly for mail/configure.in [checked in, comment #18 and #22]
Are we saying that with the two patches awaiting review, all will work again? Let's try that:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=57de7ac6dd35

Other tries before didn't build on Windows.
Not successful. The issue on Windows is the icalderivedvalue.h issue was already noted in comment #49.
Sorry, I didn't contribute anything here really ;-(
I think this fixes the issues we are addressing here (Mac build where green, local builds are working). Further bustages should be addressed in followup builds. If we are waiting longer, more and more comes in we don't see in time.
(In reply to Richard Marti (:Paenglab) from comment #66)
> If we are waiting longer, more and more comes in we don't see in time.
I agree. But a tree that doesn't build on the server and has no tests doesn't help much either.
Comment on attachment 8732637 [details] [diff] [review]
GTK patch [checked in]

I can review simple porting of the GTK changes. Do we know what bug this is porting from?

I'd prefer to not continually stack more changes on this bug.

(In reply to Richard Marti (:Paenglab) from comment #66)
> I think this fixes the issues we are addressing here (Mac build where green,
> local builds are working). Further bustages should be addressed in followup
> builds. If we are waiting longer, more and more comes in we don't see in
> time.

Yes, we need to stop packing more changes onto this bug and get it landed. We should handle further follow-up (e.g. the ical issue) in a separate bug.
Attachment #8732637 - Flags: review?(mh+mozilla) → review?(clokep)
Blocks: 1258208
(In reply to Patrick Cloke [:clokep] from comment #68)
> Comment on attachment 8732637 [details] [diff] [review]
> GTK patch
> 
> I can review simple porting of the GTK changes. Do we know what bug this is
> porting from?

It's a port from bug 1256988, especially http://hg.mozilla.org/mozilla-central/rev/5d3e4758be9d and http://hg.mozilla.org/mozilla-central/rev/9a143b863326
Comment on attachment 8732637 [details] [diff] [review]
GTK patch [checked in]

Review of attachment 8732637 [details] [diff] [review]:
-----------------------------------------------------------------

Straight-forward port. Thanks! This changes some whitespace, can you upload a new patch which doesn't do that?
Attachment #8732637 - Flags: review?(clokep) → review+
(In reply to Patrick Cloke [:clokep] from comment #70)
> This changes some whitespace, can you upload a new patch which doesn't do that?
Richard is a good boy-scout and always removes trailing spaces when he sees them. I can see a few extra changes due to this. That's OK, isn't it?
Comment on attachment 8731281 [details] [diff] [review]
Don't copy & paste config_status

Review of attachment 8731281 [details] [diff] [review]:
-----------------------------------------------------------------

::: configure.py
@@ +7,5 @@
> +import codecs
> +import imp
> +import json
> +import os
> +import subprocess

You're not using codecs, json and subprocess. Please remove them.

@@ +18,5 @@
> +
> +# We can't just import config_status since configure is shadowed by this file!
> +f, pathname, desc = imp.find_module('configure',
> +                                    [os.path.join(base_dir, 'mozilla')])
> +config_status = imp.load_module('configure', f, pathname, desc).config_status

We /could/ add an empty __init__.py at the top-level of m-c, and then you could just do `from mozilla.configure import config_status`. Not sure how gps would like that though.

::: mailnews/mime/src/nsCMS.cpp
@@ +30,5 @@
>  using namespace mozilla::psm;
>  using namespace mozilla::pkix;
>  
>  #ifdef PR_LOGGING
> +extern mozilla::LazyLogModule gPIPNSSLog;

Not sure how this is related.
Attachment #8731281 - Flags: review?(mh+mozilla) → review+
:gps, see comment 72.
Flags: needinfo?(gps)
(In reply to aleth [:aleth] from comment #52)
> (In reply to Mike Hommey [:glandium] from comment #51)
> > what are c-c builds expecting $topsrcdir to be in mozconfig?
> 
> Maybe I'm misunderstanding the question, but since this [1] worked OK before
> the recent configure changes, and it's now looking for gcc in
> /builds/slave/tb-try-c-cen-l64-0000000000000/build/mozilla/gcc/bin/gcc,
> $topsrcdir must have pointed at the c-c dir before during this step, and now
> points at the m-c subdir.
> 
> [1] https://dxr.mozilla.org/comm-central/source/build/unix/mozconfig.linux#16

Can you make things work with $topsrcdir pointing at m-c? I'd rather not have to do another layer of hacks to support c-c.
Flags: needinfo?(mh+mozilla)
(In reply to Mike Hommey [:glandium] from comment #74)
> Can you make things work with $topsrcdir pointing at m-c? I'd rather not
> have to do another layer of hacks to support c-c.

I don't see a good way to do this, but jcranmer is the expert on this.
Flags: needinfo?(Pidgeot18)
(In reply to Patrick Cloke [:clokep] from comment #70)
> Comment on attachment 8732637 [details] [diff] [review]
> GTK patch
> 
> Review of attachment 8732637 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Straight-forward port. Thanks! This changes some whitespace, can you upload
> a new patch which doesn't do that?

My editor automatically removes the trailing whitspaces. I checked the changes and this removals don't affect the functionality. In m-c the whitespaces also don't exist: https://dxr.mozilla.org/comm-central/source/mozilla/browser/installer/Makefile.in#147
Since both patches have r+, are we going to land them (with nits fixed) and then worry about improving them later or are we aiming at having C-C broken for an entire month? More than two weeks so far.

If I understand it correctly, at least Mac will build, and with the icalderivedvalue.h issue fixed, Windows perhaps as well one day (soon).
Flags: needinfo?(aleth)
(In reply to Jorg K (GMT+1) from comment #77)
> Since both patches have r+, are we going to land them (with nits fixed) and
> then worry about improving them later or are we aiming at having C-C broken
> for an entire month? More than two weeks so far.

We are waiting on the outstanding needinfos.

If you'd like to help speed things up, you could investigate the test failures, as landing the patches in their present state will not give us correct test coverage. Probably you'll find it's the topsrcdir issue, but it would be good to check.

> If I understand it correctly, at least Mac will build, and with the
> icalderivedvalue.h issue fixed, Windows perhaps as well one day (soon).

You can build on all OS locally with these patches applied.
Flags: needinfo?(aleth)
Addressing comments about not needed imports and removing c++ changes.
Attachment #8731281 - Attachment is obsolete: true
Attachment #8733820 - Flags: review+
https://hg.mozilla.org/comm-central/rev/9962c522bb1dc8b4e3a298899185e1a2e348ea35
Bug 1254987 - Fix comm-central fallout from |Bug 1254410 - Include app-specific configure files according to --enable-application/--enable-project|. r=glandium

https://hg.mozilla.org/comm-central/rev/e8f8bbc4331848f42e66d86aaa03392d6626e2ca
Bug 1254987 - Port bug 1256988 to c-c, GTK changes. r=clokep a=aleth CLOSED TREE
This bug remains open to resolve the needinfos, and the tree remains closed.
Attachment #8732637 - Attachment description: GTK patch → GTK patch [checked in]
Attachment #8733820 - Attachment description: configFix.patch → configFix.patch [checked in]
For the record, the libical issue is also due to a relative path in configure.in involving ${_topsrcdir}.
There is still the checkin needed for Bug 1256272 to fix the issue mentioned in Comment 29.
(In reply to Hartmut Figge from comment #83)
> There is still the checkin needed for Bug 1256272 to fix the issue mentioned
> in Comment 29.

Please discuss issues related to that bug in that bug ;)
(In reply to Richard Marti (:Paenglab) from comment #79)
> Addressing comments about not needed imports and removing c++ changes.
What about the C++ changes? Were they required? Seems to compile without them on Mac (and that's the most fussy compiler).
(In reply to Jorg K (GMT+1) from comment #85)
> (In reply to Richard Marti (:Paenglab) from comment #79)
> > Addressing comments about not needed imports and removing c++ changes.
> What about the C++ changes? Were they required? Seems to compile without
> them on Mac (and that's the most fussy compiler).

They don't belong in this bug.
(In reply to aleth [:aleth] from comment #86)
> They don't belong in this bug.
Sure, they don't below here. I was asking whether they are/were required (to fix some other issue).
(In reply to aleth [:aleth] from comment #82)
> For the record, the libical issue is also due to a relative path in
> configure.in involving ${_topsrcdir}.

Hmm... I'm not sure about this. That path seems to be correct:
c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/mozilla/../mailnews/build/msys-perl-wrapper
The problem on Windows is really weird:

c:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/objdir-tb/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/mozilla/config/nsinstall.py -t -m 644 'AccessibleEditableText_i.c' '../../../dist/include'
Can't locate readvaluesfile.pl in @INC (@INC contains: . c /builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/calendar/libical/src/libical/../../scripts /usr/lib/perl5/5.6.1/msys /usr/lib/perl5/5.6.1 /usr/lib/perl5/site_perl/5.6.1/msys /usr/lib/perl5/site_perl/5.6.1 /usr/lib/perl5/site_perl) at c:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/calendar/libical/src/libical/../../scripts/mkderivedvalues.pl line 5.
Makefile:196: recipe for target 'icalderivedvalue.h' failed

readvaluesfile.pl *is* in the same directory as mkderivedvalues.pl, both are in calendar/libical/scripts.

@INC includes . and calendar/libical/src/libical/../../scripts = calendar/libical/scripts so it is beyond me why the mkderivedvalues.pl fails at line 5 with require 'readvaluesfile.pl';

mkderivedvalues.pl is also run on Mac and there it works, so what's the difference, I don' see it.

Oh, I see it now, it's the single quotes, other files use double quotes:
.\mkderivedparameters.pl:require "readvaluesfile.pl";
.\mkderivedproperties.pl:require "readvaluesfile.pl";
.\mkderivedvalues.pl:require 'readvaluesfile.pl';
According to the log it seems that perl might be called directly instead of being called through the msys-perl-wrapper that converts the include paths:
> mozmake.exe[5]: Entering directory 'c:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/objdir-tb/calendar/libical/src/libical'
> C:/mozilla-build/msys/bin/perl.EXE -Ic:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/calendar/libical/src/libical/../../scripts

Is the fix from Bug 1247282 still in place and working?
(In reply to Stefan Sitter from comment #91)
> Is the fix from Bug 1247282 still in place and working?
Good observation. The problem looks pretty much like what we had in bug 1247282.
Aleth tried to check this, but something went wrong:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=bb8cc73465e3
Let's move further discussion of the Windows libical failure to bug 1259090.
Comment on attachment 8733935 [details] [diff] [review]
Fiddle with mkderivedvalues.pl

(I clearly know nothing about it.)
Attachment #8733935 - Attachment is obsolete: true
Depends on: 1259090
Depends on: 1259174
Filed Bug 1259174 for the relative path test failures, which only leaves the needinfos for this bug (Comment 73, comment 75).
Depends on: 1259219
I'm not a fan of adding a __init__.py at the root level. Just use imp.
Flags: needinfo?(gps)
No longer depends on: 1259174
Can/should we disable updates until all the test failures are shaken out?
(In reply to Axel Hecht [:Pike] from comment #98)
> Can/should we disable updates until all the test failures are shaken out?

The real failures should be visible after the next m-c merge, hopefully. There aren't that many new ones, luckily.
Duplicate of this bug: 1260019
So it this bug still being actively pursued? Mac and Windows are up and running, but Linux still fails to find the gcc compiler:

checking for objcopy... /builds/slave/tb-c-cen-l64-d-000000000000000/build/gcc/bin/objcopy
checking for gcc... /builds/slave/tb-c-cen-l64-d-000000000000000/build/mozilla/gcc/bin/gcc
checking whether the C compiler (/builds/slave/tb-c-cen-l64-d-000000000000000/build/mozilla/gcc/bin/gcc  -L/builds/slave/tb-c-cen-l64-d-000000000000000/build/gtk3/usr/local/lib ) works... no

Looks like a "topsrcdir" problem since the "mozilla" should not be in the path.

Can't we ask any of the rel-eng people for help?
(In reply to Mike Hommey [:glandium] from comment #74)

So it's
  CC="$topsrcdir/gcc/bin/gcc"

(with topsrcdir which points to mozilla-central)

Can't we just change it to
  CC="$topsrcdir/../gcc/bin/gcc"

I know very little about build configs really, so sorry if it's stupid question :)
BTW, my LOCAL build seems to be working for linux64. ???
(In reply to ISHIKAWA, Chiaki from comment #104)
> BTW, my LOCAL build seems to be working for linux64. ???

Not oonly yours. ;)
User agent: Mozilla/5.0 (X11; Linux x86_64; rv:48.0) Gecko/2016033101 SeaMonkey/2.45a1-h
Yes local builds are fine. It's just automation that's busted.

Sent my hack suggestion from comment 102 to try - https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=e1beb105401f
No luck - looks to me that part is just ignored, or set later than the test that fails.
Depends on: 1259382
Oh, so bug 1262057 could actually fix this issue?
Let's close this bug here and pursue the Linux compile in bug 1262057.
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Flags: needinfo?(Pidgeot18)
Resolution: --- → FIXED
Whiteboard: [leave open]
Target Milestone: --- → mozilla48
Depends on: 1262057
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.