1254987 - Fix comm-central fallout from |Bug 1254410 - Include app-specific configure files according to --enable-application/--enable-project|

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Patrick Cloke [:clokep]]

You end up with an error like:

0:00.22 /usr/bin/make -f client.mk -s
 0:00.39 Adding client.mk options from /home/abdelrhman/mozilla/comm-central/.mozconfig:
 0:00.39     FOUND_MOZCONFIG := /home/abdelrhman/mozilla/comm-central/.mozconfig
 0:00.49 cd /home/abdelrhman/mozilla/comm-central/obj-x86_64-unknown-linux-gnu
 0:00.49 /home/abdelrhman/mozilla/comm-central/configure
 0:00.50 loading cache ./config.cache
 0:00.60 Reexecuting in the virtualenv
 0:00.69 Adding configure options from /home/abdelrhman/mozilla/comm-central/.mozconfig
 0:00.69   --enable-application=im
 0:00.69 Cannot find project im
 0:00.71 *** Fix above errors and then restart with\
 0:00.71                "/usr/bin/make -f client.mk build"
 0:00.71 make[1]: *** [configure] Error 1
 0:00.71 make: *** [/home/abdelrhman/mozilla/comm-central/obj-x86_64-unknown-linux-gnu/Makefile] Error 2
 0:00.74 0 compiler warnings present.

This pretty much comes from build_env['TOPSRCDIR'] being the mozilla/ directory instead of comm-central, e.g. my build_env in the include_project_configure method is:
{u'DIST': u'/Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb/dist', u'TOPOBJDIR': u'/Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb', u'TOPSRCDIR': u'/Users/pcloke/mozilla/comm-central/mozilla', u'MOZ_BUILD_ROOT': u'/Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb'}

http://hg.mozilla.org/mozilla-central/file/tip/build/moz.configure/init.configure#l221

I'm a bit stumped on how to fix this though. (Well besides doing incredibly hacky things.)

Comment 1

[aleth [:aleth]]

See bug 1254410 comment 2...

Comment 2

[Patrick Cloke [:clokep]]

(In reply to aleth [:aleth] from comment #1)
> See bug 1254410 comment 2...

Awesome. I'm going to try this now.

Comment 3

[Patrick Cloke [:clokep]]

I'm making some progress on this.

Comment 4

[Patrick Cloke [:clokep]]

Attachment: WIP

I've unfortunately ran out of time to look at this at the moment. This actually does start to run configure, but bails out. Please let me know if you have any ideas. (Note that configure.py and configure.in are copied with a couple of paths changed. We used to have some magic inside of configure.in that this blows away...which is probably not good.)

 0:00.24 /usr/bin/make -f client.mk -s configure
 0:00.37 cd /Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb
 0:00.37 /Users/pcloke/mozilla/comm-central/configure
 0:00.44 Creating Python environment
 0:00.57 New python executable in /Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb/_virtualenv/bin/python2.7
 0:00.57 Not overwriting existing python script /Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb/_virtualenv/bin/python (you must use /Users/pcloke/mozilla/comm-central/obj-x86_64-apple-darwin14.5.0-tb/_virtualenv/bin/python2.7)
 0:06.21 Installing setuptools, pip, wheel...done.
 0:06.59 running build_ext
 0:06.59 copying build/lib.macosx-10.10-x86_64-2.7/psutil/_psutil_osx.so -> psutil
 0:06.59 copying build/lib.macosx-10.10-x86_64-2.7/psutil/_psutil_posix.so -> psutil
 0:06.59 
 0:06.59 Reexecuting in the virtualenv
 0:06.74 Traceback (most recent call last):
 0:06.74   File "/Users/pcloke/mozilla/comm-central/configure.py", line 73, in <module>
 0:06.74     sys.exit(main(sys.argv))
 0:06.74   File "/Users/pcloke/mozilla/comm-central/configure.py", line 22, in main
 0:06.74     sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
 0:06.74   File "/Users/pcloke/mozilla/comm-central/mozilla/python/mozbuild/mozbuild/configure/__init__.py", line 197, in run
 0:06.74     % option.option
 0:06.74 mozbuild.configure.ConfigureError: Option `MOZILLABUILD` is not handled ; reference it with a @depends
 0:06.76 *** Fix above errors and then restart with               "/Applications/Xcode.app/Contents/Developer/usr/bin/make -f client.mk build"
 0:06.76 make: *** [configure] Error 1

Comment 5

[Philip Chee]

Attachment: WIP2 Currently building (omg omg)

Comment 6

[Patrick Cloke [:clokep]]

Comment on attachment 8728536 [details] [diff] [review]
WIP2 Currently building (omg omg)

Review of attachment 8728536 [details] [diff] [review]:
-----------------------------------------------------------------

::: configure.py
@@ +18,5 @@
> +
> +def main(argv):
> +    config = {}
> +    sandbox = ConfigureSandbox(config, os.environ, argv)
> +    sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))

I was wondering whether we could do `sandbox.run(os.path.join(base_dir, 'moz.configure'))` here instead of copying moz.configure.

Comment 7

[aleth [:aleth]]

Looks like this doesn't work yet for IB:
 1:00.96 Makefile:16: warning: overriding commands for target `DeprecatedPremultiplyTables.h'
 1:00.96 backend.mk:8: warning: ignoring old commands for target `DeprecatedPremultiplyTables.h'
 1:01.00 test_necko.xpt
 1:01.06 Usage: genTables.py <header.h>
 1:01.06 make[4]: *** [DeprecatedPremultiplyTables.h] Error 1
 1:01.06 make[3]: *** [gfx/thebes/export] Error 2
 1:01.06 make[3]: *** Waiting for unfinished jobs....

Comment 8

[Michael Shal [:mshal]]

(In reply to aleth [:aleth] from comment #7)
> Looks like this doesn't work yet for IB:
>  1:00.96 Makefile:16: warning: overriding commands for target
> `DeprecatedPremultiplyTables.h'
>  1:00.96 backend.mk:8: warning: ignoring old commands for target
> `DeprecatedPremultiplyTables.h'
>  1:01.00 test_necko.xpt
>  1:01.06 Usage: genTables.py <header.h>
>  1:01.06 make[4]: *** [DeprecatedPremultiplyTables.h] Error 1
>  1:01.06 make[3]: *** [gfx/thebes/export] Error 2
>  1:01.06 make[3]: *** Waiting for unfinished jobs....

Possibly related to bug 1252302?

Comment 9

[Philip Chee]

Attachment: Patch v3 works.

Looks like it works. But my build fails somewhere in nsGlobalWindow probably unreated.

Comment 10

[Patrick Cloke [:clokep]]

(In reply to Philip Chee from comment #9)
> Created attachment 8728596 [details] [diff] [review]
> Patch v3 works.
> 
> Looks like it works. But my build fails somewhere in nsGlobalWindow probably
> unreated.

This isn't working for me. I applied the patch in bug 1254795, am on the tip of m-c and c-c with this applied to c-c. Anything else I'm supposed to be doing? (I still get a "Cannot find project mail") If I'm missing a step to test this, can yo please expand? Thanks!

Comment 11

[Patrick Cloke [:clokep]]

Comment on attachment 8728596 [details] [diff] [review]
Patch v3 works.

Review of attachment 8728596 [details] [diff] [review]:
-----------------------------------------------------------------

While running |mach configure| with a mozconfig of:

ac_add_options --enable-application=../mail
ac_add_options --enable-calendar

This sort of worked...still ran into "ERROR PROCESSING MOZBUILD FILE"

::: mail/moz.configure
@@ +3,5 @@
> +# This Source Code Form is subject to the terms of the Mozilla Public
> +# License, v. 2.0. If a copy of the MPL was not distributed with this
> +# file, You can obtain one at http://mozilla.org/MPL/2.0/.
> +
> +include('../toolkit/moz.configure')

I had to change each of these (with glandium's help) to `include('../mozilla/toolkit/moz.configure')`

Comment 12

[aleth [:aleth]]

(In reply to Patrick Cloke [:clokep] from comment #11)
> Comment on attachment 8728596 [details] [diff] [review]
> Patch v3 works.
> 
> Review of attachment 8728596 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> While running |mach configure| with a mozconfig of:
> 
> ac_add_options --enable-application=../mail
> ac_add_options --enable-calendar

It passed configure for me with the standard
ac_add_options --enable-application=mail

Comment 13

[Philip Chee]

A.T.M. It works with mozilla/mach configure && mozilla/mach build
but not with mozmake -f client.py configure

Comment 14

[Joshua Cranmer [:jcranmer]]

Attachment: [mozilla-central] Get the subconfigure working properly for mail/configure.in [checked in, comment #18 and #22]

This was broken by bug 1253203, since the "internal" name of the function changed. Fortunately, it's easy to fix with a changed name.

Comment 15

[Philip Chee]

Comment on attachment 8728824 [details] [diff] [review]
[mozilla-central] Get the subconfigure working properly for mail/configure.in [checked in, comment #18 and #22]

With only your patch I get this:

> $ MOZCONFIG=/c/t1/hg/comm-central/mozconfig mozilla/mach configure
>  0:00.76 C:\DEV\mozilla-build\msys\local\bin\mozmake.EXE -f client.mk MOZ_PARALLEL_BUILD=6 -s configure
>  0:03.42 cd c:/t1/hg/objdir-sm
>  0:03.45 c:/t1/hg/comm-central/configure
>  0:03.99 loading cache ./config.cache
>  0:04.77 Reexecuting in the virtualenv
>  0:05.52 Adding configure options from c:\t1\hg\comm-central\mozconfig
>  0:05.52   --enable-application=suite
>  0:05.52   --enable-optimize
>  0:05.52   --disable-debug
>  0:05.52   --enable-debug-symbols
>  0:05.52   --enable-tests
>  0:05.52   --enable-jemalloc
>  0:05.52   --enable-calendar
>  0:05.52   --with-windows-version=603
>  0:05.52   --disable-startupcache
>  0:05.52 Cannot find project suite
>  0:05.55 *** Fix above errors and then restart with\
>  0:05.55                "C:/DEV/mozilla-build/msys/local/bin/mozmake.EXE -f client.mk build"
>  0:05.57 client.mk:363: recipe for target 'configure' failed
>  0:05.57 mozmake.EXE: *** [configure] Error 1

Comment 16

[Mike Hommey [:glandium]]

Try current m-i + bug 1255312

Comment 18

[aleth [:aleth]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/9f6559ebb5932cdf9e0bd24bf91700f2bfa1a1a4
Bug 1254987 - Get the subconfigure working properly for mail/configure.in. r=glandium

Comment 19

[Joshua Cranmer [:jcranmer]]

(In reply to Philip Chee from comment #15)
> With only your patch I get this:

It's not sufficient, but it is necessary to get the */configure.in working properly.

Comment 21

[Joshua Cranmer [:jcranmer]]

The other m-c change we need is bug 1255312, plus a related change there that adds --without-external-source-dir. With those two fixes, js/src/configure can run, and then we can happily build comm-central, once the moz.configure stuff is ported into c-c.

Comment 22

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/9f6559ebb593

Comment 23

[aleth [:aleth]]

Status update: With jcranmer's patch [1], OS X builds OK on current c-c [2], but there seems to be a path issue with the tests.

[1] https://hg.mozilla.org/try-comm-central/rev/c6ff7bf60172
[2] https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=5bff81c7c0b6

Comment 24

[Philip Chee]

Attachment: Patch works on Windows

Successfully Configured, build, and link.

Comment 25

[Hartmut Figge]

(In reply to Philip Chee from comment #24)
> Created attachment 8730045 [details] [diff] [review]
> Patch works on Windows

On SM-Trunk x86_64:

[...]
Adding configure options from /home/hafi/moz-work/src/.mozconfig
  --enable-application=suite
  --enable-optimize
  --disable-tests
  --enable-calendar
  --enable-extensions=default
  --enable-gstreamer=1.0
  --disable-pulseaudio
  --disable-elf-hack
  --enable-default-toolkit=cairo-gtk2
[...]
js/src/ctypes/libffi> config.status: executing src commands

Traceback (most recent call last):
  File "/home/hafi/moz-work/src/configure.py", line 73, in <module>
    sys.exit(main(sys.argv))
  File "/home/hafi/moz-work/src/configure.py", line 22, in main
    sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
  File "/home/hafi/moz-work/src/mozilla/python/mozbuild/mozbuild/configure/__init__.py", line 190, in run
    raise InvalidOptionError('Unknown option: %s' % without_value)
mozbuild.configure.options.InvalidOptionError: Unknown option: --enable-gstreamer

After commenting out
ac_add_options --enable-gstreamer=1.0

the build is now running. Not finished yet.

Comment 26

[Philip Chee]

(In reply to Hartmut Figge from comment #25)

> (In reply to Philip Chee from comment #24)

>     raise InvalidOptionError('Unknown option: %s' % without_value)
> mozbuild.configure.options.InvalidOptionError: Unknown option:
> --enable-gstreamer
> 
> After commenting out
> ac_add_options --enable-gstreamer=1.0
> 
> the build is now running. Not finished yet.

Perhaps --enable-gstreamer needs to be added here:
http://hg.mozilla.org/mozilla-central/rev/9fddd3fdc22a#l1.13

Comment 27

[Hartmut Figge]

(In reply to Philip Chee from comment #26)

> Perhaps --enable-gstreamer needs to be added here:
> http://hg.mozilla.org/mozilla-central/rev/9fddd3fdc22a#l1.13

Yes, that works. Using the new build now.

Comment 28

[Hartmut Figge]

(In reply to Hartmut Figge from comment #27)
> (In reply to Philip Chee from comment #26)
> 
> > Perhaps --enable-gstreamer needs to be added here:
> > http://hg.mozilla.org/mozilla-central/rev/9fddd3fdc22a#l1.13
> 
> Yes, that works. Using the new build now.

There is an issue with the new build. May be unrelated. The used URL is not shown in the Address Bar of the browser. Just noticed when I tried to copy it for a success message to de.comm.software.mozilla.nightly-builds. :)

Comment 29

[Hartmut Figge]

Attachment: missing.png

Missing URl in Address Bar, missing Entries in Bookmarks Toolbar

The first picture is from my last successful build, 20160309, the picture below is taken from my current build, 20160314. This may have nothing to do with the bustage fix. May.

Comment 30

[Mike Hommey [:glandium]]

(In reply to Philip Chee from comment #26)
> (In reply to Hartmut Figge from comment #25)
> 
> > (In reply to Philip Chee from comment #24)
> 
> >     raise InvalidOptionError('Unknown option: %s' % without_value)
> > mozbuild.configure.options.InvalidOptionError: Unknown option:
> > --enable-gstreamer
> > 
> > After commenting out
> > ac_add_options --enable-gstreamer=1.0
> > 
> > the build is now running. Not finished yet.
> 
> Perhaps --enable-gstreamer needs to be added here:
> http://hg.mozilla.org/mozilla-central/rev/9fddd3fdc22a#l1.13

No, remove it from your mozconfig, it was removed in bug 1234092 and hasn't been doing anything since.

Comment 31

[Richard Marti (:Paenglab)]

Mine failed with:

5:27.62 mozbuild.configure.options.InvalidOptionError: Unknown option: --enable-inspector-apis

Is --enable-inspector-apis also deprecated?

Comment 32

[Frank-Rainer Grahl (:frg)]

Was able to build suite under Windows too. VS2915 x86 and x64 compiled fine. It didn't work right out of the box. Bug 1235805 needs to be applied (ios.newChannel2 fixes) and navigator.js needed 2 'this.' for getters / setters due to Bug 1254752. Noscript and Flashgot are broken right now and need to be fixed by the developer.

FRG

Comment 33

[Richard Marti (:Paenglab)]

Built TB on Win10 with Ratty's patch and first quick check shows it's working.

Comment 34

[Frank-Rainer Grahl (:frg)]

>> 'this.' for getters / setters due to Bug 1254752. 

Typo. Make this: due to Bug 1253016.

Comment 35

[Hartmut Figge]

Frank, that was rather cryptic for a mere user like me. ;) But I have managed. The issue of Comment 29 is gone now.

Comment 36

[Frank-Rainer Grahl (:frg)]

Hartmu,

it would have been easier if I hadn't bungled the bug number in the first place :) I filed bug 1256272 for Seamonkey. 

Noscript and Flashgot are currently broken, I think, because of this too. I am sure they will be fixed soon too.

FRG

Comment 37

[ISHIKAWA, Chiaki (back from day job exhibition event.)]

Hi,

I am a little bit confused. This bug was supposed to be C-C TB issue, right?
The last several comments referred to Flash and NoScript: could it be related SeaMonkey's browser?

Also, my fresh fetch and trial to build locally failed again with 
"Cannot find project mail".
Hmm...

Comment 38

[Frank-Rainer Grahl (:frg)]

>> I am a little bit confused. This bug was supposed to be C-C TB issue, right?

Yes and suite and the other comm-central applications.

>> The last several comments referred to Flash and NoScript: could it be related SeaMonkey's browser?

Yes. Follow up bugs to get it running after a week of no builds which have nothing to do with this one. 

FRG

Comment 39

[Mike Hommey [:glandium]]

(In reply to Richard Marti (:Paenglab) from comment #31)
> Mine failed with:
> 
> 5:27.62 mozbuild.configure.options.InvalidOptionError: Unknown option:
> --enable-inspector-apis
> 
> Is --enable-inspector-apis also deprecated?

Removed in bug 513924

Comment 40

[Mike Hommey [:glandium]]

Contrary to what I've claimed to jcranmer, we're going to have to modify configure.py, so it's better if you don't copy it. So what I'm going to do is make the part you'd rather avoid copying importable.

Comment 41

[Richard Marti (:Paenglab)]

Attachment: Additional fix for bugs 1256568, 1256587 and 1256990

The newly landed bugs 1256568, 1256587 and 1256990 needs this additional fix.

Comment 42

[Mike Hommey [:glandium]]

Comment on attachment 8731205 [details] [diff] [review]
Additional fix for bugs 1256568, 1256587 and 1256990

Review of attachment 8731205 [details] [diff] [review]:
-----------------------------------------------------------------

Why are you copying moz.configure? Just create a moz.configure with this:

include('mozilla/moz.configure')

Comment 43

[Mike Hommey [:glandium]]

Comment on attachment 8730045 [details] [diff] [review]
Patch works on Windows

Review of attachment 8730045 [details] [diff] [review]:
-----------------------------------------------------------------

::: configure.in
@@ +14,1 @@
>  AC_PREREQ(2.13)

No AC_ macros, please.

::: configure.py
@@ +24,5 @@
> +    if sandbox._help:
> +        return 0
> +
> +    # Sanitize config data to feed config.status
> +    sanitized_config = {}

See bug 1256574, you don't need to copy the entire file anymore.

Comment 44

[Richard Marti (:Paenglab)]

(In reply to Mike Hommey [:glandium] from comment #42)
> Comment on attachment 8731205 [details] [diff] [review]
> Additional fix for bugs 1256568, 1256587 and 1256990
> 
> Review of attachment 8731205 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Why are you copying moz.configure? Just create a moz.configure with this:
> 
> include('mozilla/moz.configure')


I tried the include('mozilla/moz.configure') at the beginning and the end of the file (and removed the direct includes) then the build failed. I think this is because all 'mozilla/moz.configure' files will be included in one rush and the c-c configurations are then too late or to early.

I think the order needs to be:

mozilla/build/moz.configure/init.configure
mozilla/build/moz.configure/checks.configure
c-c/moz.configure
mozilla/build/moz.configure/old.configure

to work correctly.

But I don't know really what I'm doing here and copying only the changes from m-c to c-c.

Comment 45

[Mike Hommey [:glandium]]

(In reply to Richard Marti (:Paenglab) from comment #44)
> (In reply to Mike Hommey [:glandium] from comment #42)
> > Comment on attachment 8731205 [details] [diff] [review]
> > Additional fix for bugs 1256568, 1256587 and 1256990
> > 
> > Review of attachment 8731205 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > Why are you copying moz.configure? Just create a moz.configure with this:
> > 
> > include('mozilla/moz.configure')
> 
> 
> I tried the include('mozilla/moz.configure') at the beginning and the end of
> the file (and removed the direct includes) then the build failed.

That you're talking about the beginning or the end of the file tells me you didn't try what I'm saying. What I'm saying is, litterally, remove everything from your moz.configure, and make it contain include('mozilla/moz.configure') and that's all.

Comment 46

[Richard Marti (:Paenglab)]

(In reply to Mike Hommey [:glandium] from comment #45)
> That you're talking about the beginning or the end of the file tells me you
> didn't try what I'm saying. What I'm saying is, litterally, remove
> everything from your moz.configure, and make it contain
> include('mozilla/moz.configure') and that's all.

Now I understand. Yes, it works this way.

I'm trying to address the other comments of the other patch.

(In reply to Mike Hommey [:glandium] from comment #43)
> Comment on attachment 8730045 [details] [diff] [review]
> ::: configure.py
> @@ +24,5 @@
> > +    if sandbox._help:
> > +        return 0
> > +
> > +    # Sanitize config data to feed config.status
> > +    sanitized_config = {}
> 
> See bug 1256574, you don't need to copy the entire file anymore.

Never written a Python script. How do I import config_status(config) from your configure.py in c-c configure.py.

Comment 47

[Patrick Cloke [:clokep]]

Attachment: Don't copy & paste config_status

This is attachment 8731205 [details] [diff] [review] with some python import magic to get config_status from the Mozilla version. It's slightly gross since Mozilla's version is shadowed by the current file...required a little bit of import foo.

Comment 48

[Richard Marti (:Paenglab)]

Comment on attachment 8731281 [details] [diff] [review]
Don't copy & paste config_status

All review comments addressed.

Locally it builds on Linux and Windows. Try fails for Linux and Windows, OS X has already one green. I think this is more a buildbot issue.

Comment 49

[aleth [:aleth]]

The Windows failure is "only" a build failure ("recipe for target 'icalderivedvalue.h' failed"), looks like what broke is similar to bug 1247282.

The Linux problem is that it's looking for gcc in the wrong dir.
This is set by
https://dxr.mozilla.org/comm-central/source/build/unix/mozconfig.linux#16
(Note these mozconfigs are enforced to match the ones in mozilla/build.)
So it looks like $topsrcdir isn't set correctly at the time when that is evaluated. Any ideas as to why? (It's set in configure.in.)

Comment 50

[aleth [:aleth]]

> The Linux problem is that it's looking for gcc in the wrong dir.
Link to the logs for future reference: https://treeherder.mozilla.org/logviewer.html#?job_id=17253&repo=try-comm-central#L2269

Comment 51

[Mike Hommey [:glandium]]

what are c-c builds expecting $topsrcdir to be in mozconfig?

Comment 52

[aleth [:aleth]]

(In reply to Mike Hommey [:glandium] from comment #51)
> what are c-c builds expecting $topsrcdir to be in mozconfig?

Maybe I'm misunderstanding the question, but since this [1] worked OK before the recent configure changes, and it's now looking for gcc in /builds/slave/tb-try-c-cen-l64-0000000000000/build/mozilla/gcc/bin/gcc, $topsrcdir must have pointed at the c-c dir before during this step, and now points at the m-c subdir.

[1] https://dxr.mozilla.org/comm-central/source/build/unix/mozconfig.linux#16

Comment 53

[aleth [:aleth]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ce0d40ed7466

Comment 54

[Frederic Bezies]

Tried to build thunderbird using patch from comment 48, and I got this while building process is trying to "setup" libxul.so.

84:25.14 ../../mail/components/build/nsMailComps.o: In function `nsMailGNOMEIntegrationConstructor(nsISupports*, nsID const&, void**)':
84:25.14 nsMailComps.cpp:(.text._ZL33nsMailGNOMEIntegrationConstructorP11nsISupportsRK4nsIDPPv+0x30): undefined reference to `nsMailGNOMEIntegration::nsMailGNOMEIntegration()'
84:25.14 nsMailComps.cpp:(.text._ZL33nsMailGNOMEIntegrationConstructorP11nsISupportsRK4nsIDPPv+0x46): undefined reference to `nsMailGNOMEIntegration::Init()'
84:25.14 /usr/bin/ld: libxul.so: hidden symbol `_ZN22nsMailGNOMEIntegration4InitEv' isn't defined
84:25.14 /usr/bin/ld: final link failed: Bad value
84:25.14 collect2: error: ld returned 1 exit status
84:25.14 /home/fred/logs/mail/src/mozilla/config/rules.mk:808: recipe for target 'libxul.so' failed
84:25.14 make[4]: *** [libxul.so] Error 1
84:25.14 /home/fred/logs/mail/src/mozilla/config/recurse.mk:71: recipe for target 'toolkit/library/target' failed
84:25.14 make[3]: *** [toolkit/library/target] Error 2
84:25.14 /home/fred/logs/mail/src/mozilla/config/recurse.mk:32: recipe for target 'compile' failed
84:25.14 make[2]: *** [compile] Error 2
84:25.14 /home/fred/logs/mail/src/mozilla/config/rules.mk:531: recipe for target 'default' failed
84:25.15 make[1]: *** [default] Error 2
84:25.15 client.mk:406: recipe for target 'build' failed
84:25.15 make: *** [build] Error 2
84:25.15 1254 compiler warnings present.

I'm on Archlinux with gcc 5.3 and gtk+ 3.18.9. Using a blank new trunk code. A trunk code version of Mozilla Firefox was made on my computer a few hours ago.

Any ideas ?

Comment 55

[Richard Marti (:Paenglab)]

(In reply to Frederic Bezies from comment #54)
> Tried to build thunderbird using patch from comment 48, and I got this while
> building process is trying to "setup" libxul.so.
> 
> 84:25.14 ../../mail/components/build/nsMailComps.o: In function
> `nsMailGNOMEIntegrationConstructor(nsISupports*, nsID const&, void**)':
> 84:25.14
> nsMailComps.cpp:(.text.
> _ZL33nsMailGNOMEIntegrationConstructorP11nsISupportsRK4nsIDPPv+0x30):
> undefined reference to `nsMailGNOMEIntegration::nsMailGNOMEIntegration()'
> 84:25.14
> nsMailComps.cpp:(.text.
> _ZL33nsMailGNOMEIntegrationConstructorP11nsISupportsRK4nsIDPPv+0x46):
> undefined reference to `nsMailGNOMEIntegration::Init()'
> 84:25.14 /usr/bin/ld: libxul.so: hidden symbol
> `_ZN22nsMailGNOMEIntegration4InitEv' isn't defined
> 
> Any ideas ?

I see this too, I isolated it to bug 1566988.

Comment 56

[Jorg K (CEST = GMT+2)]

That bug number ain't right.

Comment 57

[Richard Marti (:Paenglab)]

Yes it's bug 1256988, sorry.

Comment 58

[Frederic Bezies]

(In reply to Richard Marti (:Paenglab) from comment #57)
> Yes it's bug 1256988, sorry.

There is a little problem. This bug is closed as resolved fixed. So it will be logical not to be blocked while libxul.so is processed. Are you 100% sure this is the right bug ? :D

Comment 59

[Richard Marti (:Paenglab)]

(In reply to Frederic Bezies from comment #58)
> (In reply to Richard Marti (:Paenglab) from comment #57)
> > Yes it's bug 1256988, sorry.
> 
> There is a little problem. This bug is closed as resolved fixed. So it will
> be logical not to be blocked while libxul.so is processed. Are you 100% sure
> this is the right bug ? :D

It's resolved fixed in m-c. But this needs changes also in c-c. With m-c on the revision before this patch TB builds and with only bug 1256988 in it, it fails.

I'm trying now a patch and hope this fixes the build problem.

Comment 60

[Frederic Bezies]

(In reply to Richard Marti (:Paenglab) from comment #59)
> (In reply to Frederic Bezies from comment #58)
> > (In reply to Richard Marti (:Paenglab) from comment #57)
> > > Yes it's bug 1256988, sorry.
> > 
> > There is a little problem. This bug is closed as resolved fixed. So it will
> > be logical not to be blocked while libxul.so is processed. Are you 100% sure
> > this is the right bug ? :D
> 
> It's resolved fixed in m-c. But this needs changes also in c-c. With m-c on
> the revision before this patch TB builds and with only bug 1256988 in it, it
> fails.
> 
> I'm trying now a patch and hope this fixes the build problem.

Oops ! I didn't notice it was added on m-c... Well, let's hope you'll get a patch soon :)

Comment 61

[Richard Marti (:Paenglab)]

Attachment: GTK patch [checked in]

This patch let me build TB on Linux on top of the other patch with m-c tip. Tested is only TB but I patched also IB and SM.

Comment 62

[Hartmut Figge]

(In reply to Richard Marti (:Paenglab) from comment #61)
> Created attachment 8732637 [details] [diff] [review]
> GTK patch
> 
> This patch let me build TB on Linux on top of the other patch with m-c tip.
> Tested is only TB but I patched also IB and SM.

It fixes Bug 1258208. I should have wait a bit longer to file that. Sigh.

Comment 63

[Frederic Bezies]

(In reply to Richard Marti (:Paenglab) from comment #61)
> Created attachment 8732637 [details] [diff] [review]
> GTK patch
> 
> This patch let me build TB on Linux on top of the other patch with m-c tip.
> Tested is only TB but I patched also IB and SM.

Thanks for this patch. It is great to get a working thunderbird homemade build after nearly two weeks.

Comment 64

[Jorg K (CEST = GMT+2)]

Are we saying that with the two patches awaiting review, all will work again? Let's try that:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=57de7ac6dd35

Other tries before didn't build on Windows.

Comment 65

[Jorg K (CEST = GMT+2)]

Not successful. The issue on Windows is the icalderivedvalue.h issue was already noted in comment #49.
Sorry, I didn't contribute anything here really ;-(

Comment 66

[Richard Marti (:Paenglab)]

I think this fixes the issues we are addressing here (Mac build where green, local builds are working). Further bustages should be addressed in followup builds. If we are waiting longer, more and more comes in we don't see in time.

Comment 67

[Jorg K (CEST = GMT+2)]

(In reply to Richard Marti (:Paenglab) from comment #66)
> If we are waiting longer, more and more comes in we don't see in time.
I agree. But a tree that doesn't build on the server and has no tests doesn't help much either.

Comment 68

[Patrick Cloke [:clokep]]

Comment on attachment 8732637 [details] [diff] [review]
GTK patch [checked in]

I can review simple porting of the GTK changes. Do we know what bug this is porting from?

I'd prefer to not continually stack more changes on this bug.

(In reply to Richard Marti (:Paenglab) from comment #66)
> I think this fixes the issues we are addressing here (Mac build where green,
> local builds are working). Further bustages should be addressed in followup
> builds. If we are waiting longer, more and more comes in we don't see in
> time.

Yes, we need to stop packing more changes onto this bug and get it landed. We should handle further follow-up (e.g. the ical issue) in a separate bug.

Comment 69

[Richard Marti (:Paenglab)]

(In reply to Patrick Cloke [:clokep] from comment #68)
> Comment on attachment 8732637 [details] [diff] [review]
> GTK patch
> 
> I can review simple porting of the GTK changes. Do we know what bug this is
> porting from?

It's a port from bug 1256988, especially http://hg.mozilla.org/mozilla-central/rev/5d3e4758be9d and http://hg.mozilla.org/mozilla-central/rev/9a143b863326

Comment 70

[Patrick Cloke [:clokep]]

Comment on attachment 8732637 [details] [diff] [review]
GTK patch [checked in]

Review of attachment 8732637 [details] [diff] [review]:
-----------------------------------------------------------------

Straight-forward port. Thanks! This changes some whitespace, can you upload a new patch which doesn't do that?

Comment 71

[Jorg K (CEST = GMT+2)]

(In reply to Patrick Cloke [:clokep] from comment #70)
> This changes some whitespace, can you upload a new patch which doesn't do that?
Richard is a good boy-scout and always removes trailing spaces when he sees them. I can see a few extra changes due to this. That's OK, isn't it?

Comment 72

[Mike Hommey [:glandium]]

Comment on attachment 8731281 [details] [diff] [review]
Don't copy & paste config_status

Review of attachment 8731281 [details] [diff] [review]:
-----------------------------------------------------------------

::: configure.py
@@ +7,5 @@
> +import codecs
> +import imp
> +import json
> +import os
> +import subprocess

You're not using codecs, json and subprocess. Please remove them.

@@ +18,5 @@
> +
> +# We can't just import config_status since configure is shadowed by this file!
> +f, pathname, desc = imp.find_module('configure',
> +                                    [os.path.join(base_dir, 'mozilla')])
> +config_status = imp.load_module('configure', f, pathname, desc).config_status

We /could/ add an empty __init__.py at the top-level of m-c, and then you could just do `from mozilla.configure import config_status`. Not sure how gps would like that though.

::: mailnews/mime/src/nsCMS.cpp
@@ +30,5 @@
>  using namespace mozilla::psm;
>  using namespace mozilla::pkix;
>  
>  #ifdef PR_LOGGING
> +extern mozilla::LazyLogModule gPIPNSSLog;

Not sure how this is related.

Comment 73

[Mike Hommey [:glandium]]

:gps, see comment 72.

Comment 74

[Mike Hommey [:glandium]]

(In reply to aleth [:aleth] from comment #52)
> (In reply to Mike Hommey [:glandium] from comment #51)
> > what are c-c builds expecting $topsrcdir to be in mozconfig?
> 
> Maybe I'm misunderstanding the question, but since this [1] worked OK before
> the recent configure changes, and it's now looking for gcc in
> /builds/slave/tb-try-c-cen-l64-0000000000000/build/mozilla/gcc/bin/gcc,
> $topsrcdir must have pointed at the c-c dir before during this step, and now
> points at the m-c subdir.
> 
> [1] https://dxr.mozilla.org/comm-central/source/build/unix/mozconfig.linux#16

Can you make things work with $topsrcdir pointing at m-c? I'd rather not have to do another layer of hacks to support c-c.

Comment 75

[aleth [:aleth]]

(In reply to Mike Hommey [:glandium] from comment #74)
> Can you make things work with $topsrcdir pointing at m-c? I'd rather not
> have to do another layer of hacks to support c-c.

I don't see a good way to do this, but jcranmer is the expert on this.

Comment 76

[Richard Marti (:Paenglab)]

(In reply to Patrick Cloke [:clokep] from comment #70)
> Comment on attachment 8732637 [details] [diff] [review]
> GTK patch
> 
> Review of attachment 8732637 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Straight-forward port. Thanks! This changes some whitespace, can you upload
> a new patch which doesn't do that?

My editor automatically removes the trailing whitspaces. I checked the changes and this removals don't affect the functionality. In m-c the whitespaces also don't exist: https://dxr.mozilla.org/comm-central/source/mozilla/browser/installer/Makefile.in#147

Comment 77

[Jorg K (CEST = GMT+2)]

Since both patches have r+, are we going to land them (with nits fixed) and then worry about improving them later or are we aiming at having C-C broken for an entire month? More than two weeks so far.

If I understand it correctly, at least Mac will build, and with the icalderivedvalue.h issue fixed, Windows perhaps as well one day (soon).

Comment 78

[aleth [:aleth]]

(In reply to Jorg K (GMT+1) from comment #77)
> Since both patches have r+, are we going to land them (with nits fixed) and
> then worry about improving them later or are we aiming at having C-C broken
> for an entire month? More than two weeks so far.

We are waiting on the outstanding needinfos.

If you'd like to help speed things up, you could investigate the test failures, as landing the patches in their present state will not give us correct test coverage. Probably you'll find it's the topsrcdir issue, but it would be good to check.

> If I understand it correctly, at least Mac will build, and with the
> icalderivedvalue.h issue fixed, Windows perhaps as well one day (soon).

You can build on all OS locally with these patches applied.

Comment 79

[Richard Marti (:Paenglab)]

Attachment: configFix.patch [checked in]

Addressing comments about not needed imports and removing c++ changes.

Comment 80

[aleth [:aleth]]

https://hg.mozilla.org/comm-central/rev/9962c522bb1dc8b4e3a298899185e1a2e348ea35
Bug 1254987 - Fix comm-central fallout from |Bug 1254410 - Include app-specific configure files according to --enable-application/--enable-project|. r=glandium

https://hg.mozilla.org/comm-central/rev/e8f8bbc4331848f42e66d86aaa03392d6626e2ca
Bug 1254987 - Port bug 1256988 to c-c, GTK changes. r=clokep a=aleth CLOSED TREE

Comment 81

[aleth [:aleth]]

This bug remains open to resolve the needinfos, and the tree remains closed.

Comment 82

[aleth [:aleth]]

For the record, the libical issue is also due to a relative path in configure.in involving ${_topsrcdir}.

Comment 83

[Hartmut Figge]

There is still the checkin needed for Bug 1256272 to fix the issue mentioned in Comment 29.

Comment 84

[aleth [:aleth]]

(In reply to Hartmut Figge from comment #83)
> There is still the checkin needed for Bug 1256272 to fix the issue mentioned
> in Comment 29.

Please discuss issues related to that bug in that bug ;)

Comment 85

[Jorg K (CEST = GMT+2)]

(In reply to Richard Marti (:Paenglab) from comment #79)
> Addressing comments about not needed imports and removing c++ changes.
What about the C++ changes? Were they required? Seems to compile without them on Mac (and that's the most fussy compiler).

Comment 86

[aleth [:aleth]]

(In reply to Jorg K (GMT+1) from comment #85)
> (In reply to Richard Marti (:Paenglab) from comment #79)
> > Addressing comments about not needed imports and removing c++ changes.
> What about the C++ changes? Were they required? Seems to compile without
> them on Mac (and that's the most fussy compiler).

They don't belong in this bug.

Comment 87

[Jorg K (CEST = GMT+2)]

(In reply to aleth [:aleth] from comment #86)
> They don't belong in this bug.
Sure, they don't below here. I was asking whether they are/were required (to fix some other issue).

Comment 88

[aleth [:aleth]]

(In reply to aleth [:aleth] from comment #82)
> For the record, the libical issue is also due to a relative path in
> configure.in involving ${_topsrcdir}.

Hmm... I'm not sure about this. That path seems to be correct:
c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/mozilla/../mailnews/build/msys-perl-wrapper

Comment 89

[Jorg K (CEST = GMT+2)]

The problem on Windows is really weird:

c:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/objdir-tb/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/mozilla/config/nsinstall.py -t -m 644 'AccessibleEditableText_i.c' '../../../dist/include'
Can't locate readvaluesfile.pl in @INC (@INC contains: . c /builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/calendar/libical/src/libical/../../scripts /usr/lib/perl5/5.6.1/msys /usr/lib/perl5/5.6.1 /usr/lib/perl5/site_perl/5.6.1/msys /usr/lib/perl5/site_perl/5.6.1 /usr/lib/perl5/site_perl) at c:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/calendar/libical/src/libical/../../scripts/mkderivedvalues.pl line 5.
Makefile:196: recipe for target 'icalderivedvalue.h' failed

readvaluesfile.pl *is* in the same directory as mkderivedvalues.pl, both are in calendar/libical/scripts.

@INC includes . and calendar/libical/src/libical/../../scripts = calendar/libical/scripts so it is beyond me why the mkderivedvalues.pl fails at line 5 with require 'readvaluesfile.pl';

mkderivedvalues.pl is also run on Mac and there it works, so what's the difference, I don' see it.

Oh, I see it now, it's the single quotes, other files use double quotes:
.\mkderivedparameters.pl:require "readvaluesfile.pl";
.\mkderivedproperties.pl:require "readvaluesfile.pl";
.\mkderivedvalues.pl:require 'readvaluesfile.pl';

Comment 90

[Jorg K (CEST = GMT+2)]

Or the use lib '.', hmm.

Comment 91

[Stefan Sitter [:ssitter]]

According to the log it seems that perl might be called directly instead of being called through the msys-perl-wrapper that converts the include paths:
> mozmake.exe[5]: Entering directory 'c:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/objdir-tb/calendar/libical/src/libical'
> C:/mozilla-build/msys/bin/perl.EXE -Ic:/builds/moz2_slave/tb-c-cen-w32-00000000000000000/build/calendar/libical/src/libical/../../scripts

Is the fix from Bug 1247282 still in place and working?

Comment 92

[Jorg K (CEST = GMT+2)]

Attachment: Fiddle with mkderivedvalues.pl

See whether that improves things:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=615466591125

Comment 93

[Jorg K (CEST = GMT+2)]

(In reply to Stefan Sitter from comment #91)
> Is the fix from Bug 1247282 still in place and working?
Good observation. The problem looks pretty much like what we had in bug 1247282.
Aleth tried to check this, but something went wrong:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=bb8cc73465e3

Comment 94

[aleth [:aleth]]

Let's move further discussion of the Windows libical failure to bug 1259090.

Comment 95

[Jorg K (CEST = GMT+2)]

Comment on attachment 8733935 [details] [diff] [review]
Fiddle with mkderivedvalues.pl

(I clearly know nothing about it.)

Comment 96

[aleth [:aleth]]

Filed Bug 1259174 for the relative path test failures, which only leaves the needinfos for this bug (Comment 73, comment 75).

Comment 97

[Gregory Szorc [:gps]]

I'm not a fan of adding a __init__.py at the root level. Just use imp.

Comment 98

[Axel Hecht [:Pike]]

Can/should we disable updates until all the test failures are shaken out?

Comment 99

[aleth [:aleth]]

(In reply to Axel Hecht [:Pike] from comment #98)
> Can/should we disable updates until all the test failures are shaken out?

The real failures should be visible after the next m-c merge, hopefully. There aren't that many new ones, luckily.

Comment 101

[Jorg K (CEST = GMT+2)]

So it this bug still being actively pursued? Mac and Windows are up and running, but Linux still fails to find the gcc compiler:

checking for objcopy... /builds/slave/tb-c-cen-l64-d-000000000000000/build/gcc/bin/objcopy
checking for gcc... /builds/slave/tb-c-cen-l64-d-000000000000000/build/mozilla/gcc/bin/gcc
checking whether the C compiler (/builds/slave/tb-c-cen-l64-d-000000000000000/build/mozilla/gcc/bin/gcc  -L/builds/slave/tb-c-cen-l64-d-000000000000000/build/gtk3/usr/local/lib ) works... no

Looks like a "topsrcdir" problem since the "mozilla" should not be in the path.

Can't we ask any of the rel-eng people for help?

Comment 102

[Magnus Melin [:mkmelin]]

(In reply to Mike Hommey [:glandium] from comment #74)

So it's
  CC="$topsrcdir/gcc/bin/gcc"

(with topsrcdir which points to mozilla-central)

Can't we just change it to
  CC="$topsrcdir/../gcc/bin/gcc"

I know very little about build configs really, so sorry if it's stupid question :)

Comment 103

[Jorg K (CEST = GMT+2)]

I was looking at that too:
https://dxr.mozilla.org/comm-central/source/build/unix/mozconfig.linux#16

Comment 104

[ISHIKAWA, Chiaki (back from day job exhibition event.)]

BTW, my LOCAL build seems to be working for linux64. ???

Comment 105

[Hartmut Figge]

(In reply to ISHIKAWA, Chiaki from comment #104)
> BTW, my LOCAL build seems to be working for linux64. ???

Not oonly yours. ;)
User agent: Mozilla/5.0 (X11; Linux x86_64; rv:48.0) Gecko/2016033101 SeaMonkey/2.45a1-h

Comment 106

[Magnus Melin [:mkmelin]]

Yes local builds are fine. It's just automation that's busted.

Sent my hack suggestion from comment 102 to try - https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=e1beb105401f

Comment 107

[Magnus Melin [:mkmelin]]

No luck - looks to me that part is just ignored, or set later than the test that fails.

Comment 108

[Edmund Wong (:ewong)]

Oh, so bug 1262057 could actually fix this issue?

Comment 109

[Jorg K (CEST = GMT+2)]

Let's close this bug here and pursue the Linux compile in bug 1262057.