Builds fail on Solaris9 with distribution .mozconfig with MOZ_MAKE_FLAGS=-j4

RESOLVED FIXED in mozilla1.4alpha

Status

SeaMonkey
Build Config
--
blocker
RESOLVED FIXED
15 years ago
13 years ago

People

(Reporter: Donnie Cranford, Assigned: hacker formerly known as seawood@netscape.com)

Tracking

Trunk
mozilla1.4alpha
Sun
SunOS

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments, 1 obsolete attachment)

(Reporter)

Description

15 years ago
I have been building mozilla using the .mozconfig from
http://www.mozilla.org/build/distribution.html

# cat .mozconfig
# sh
# Build configuration script
#
# See http://www.mozilla.org/build/unix.html for build instructions.
#

# Options for client.mk.
mk_add_options MOZ_MAKE_FLAGS=-j4

# Options for 'configure' (same as command-line options).
ac_add_options --enable-crypto
ac_add_options --disable-tests
ac_add_options --enable-js-ultrasparc
ac_add_options --disable-debug
ac_add_options --enable-optimize=-xO2
ac_add_options --enable-strip
ac_add_options --without-system-nspr
ac_add_options --without-system-zlib
ac_add_options --without-system-jpeg
ac_add_options --without-system-png
ac_add_options --without-system-mng



The problem is that mozilla will not build using this....
it dies with the following message

"xpt_struct.c", line 939: warning: syntax error:  empty declaration

However if I build without a .mozconfig everything builds fine....


Need to get this fixed....


dcran

Comment 1

15 years ago
Created attachment 108061 [details] [diff] [review]
Fixes the "warning"

There's two problems where. One is that the macro ends with a semicolon and the
use of that macro ends in a semicolon. That normally generates a warning. And
this compiler seems to generate that warning as well, but apparently treats it
as an error. Someone should probably file a bug with Sun about this.

So I say lets correct the code and be done with this. It beats trying each of
the options in .mozconfig and figuring out where Sun's bug is ;-)
-> xpidl
Assignee: seawood → dbradley
Component: Build Config → xpidl
QA Contact: granrose → pschwartau

Comment 3

15 years ago
Was talking with reporter on IRC. The source of the problem is the -J4 option in
.mozconfig. The warning/error he described originally was noise diverting his
attention from it.

This patch is valid, and would at least get rid of some noisy warnings, but it's
not going to fix this problem.

So I added you to the cc list Chris in case you might have some insite into this
problem. Just don't use -J4?
Summary: mozilla doesnt build on Solaris9 with distribution .mozconfig → Builds fail on Solaris9 with distribution .mozconfig with MOZ_MAKE_FLAGS=-j4
(Reporter)

Comment 4

15 years ago
David,

Thats exactly right...the -j4 option is not working, however I cant get -j2 to
work either, so I kind of lose the speed of my 2 processor Sunblade 1000.

I imagine it might have something to do with a malformed makefile maybe???
What's the exact error that make is returning when 'make -j4' is used?  Most of
our tinderboxes are compiling with some form of make -jx so I doubt that it's a
problem  is specific to that Makefile.

Comment 6

15 years ago
I know it was failing on creating a file, I think during dependency creation,
the file had an extension of .o.pp, I don't remember the specific file name. The
reporter will have to provide more details.
(Reporter)

Comment 7

15 years ago
"xpt_struct.c", line 939: warning: syntax error:  empty declaration

is what the original message was....

then it would just die trying to create deps


Here is the complete final compile string I get


gmake[6]: Entering directory `/export/home/mozilla/xpcom/typelib/xpt/src'
Creating .deps
xpt_arena.c
Building deps for xpt_arena.c
xpt_struct.c
touch: .deps/xpt_arena.o.pp cannot create
Building deps for xpt_struct.c
gmake[6]: *** [xpt_arena.o] Error 1
gmake[6]: *** Waiting for unfinished jobs....
touch: .deps/xpt_struct.o.pp cannot create
gmake[6]: *** [xpt_struct.o] Error 1
gmake[6]: Leaving directory `/export/home/mozilla/xpcom/typelib/xpt/src'
gmake[5]: *** [export] Error 2
gmake[5]: Leaving directory `/export/home/mozilla/xpcom/typelib/xpt'
gmake[4]: *** [export] Error 2
gmake[4]: Leaving directory `/export/home/mozilla/xpcom/typelib'
gmake[3]: *** [export] Error 2
gmake[3]: Leaving directory `/export/home/mozilla/xpcom'
gmake[2]: *** [tier_2] Error 2
gmake[2]: Leaving directory `/export/home/mozilla'
gmake[1]: *** [default] Error 2
gmake[1]: Leaving directory `/export/home/mozilla'
gmake: *** [build] Error 2
Is the .deps directory created and with the proper permissions?
(Reporter)

Comment 9

15 years ago
Christoper

Yes the directory is created with the correct permissions...I am compiling as
root so it would be hard for permissions to be wrong...but here are the
permissions on the directory

# pwd
/export/home/mozilla/xpcom/typelib/xpt/src
drwxr-xr-x   2 root     other        512 Dec  4 17:13 .deps

Comment 10

15 years ago
Anything new on this?
(Reporter)

Comment 11

15 years ago
its still broken   :(

dcran

Comment 12

15 years ago
This just happened on nebiros as well. cls, so was gmake upgraded on nebiros
recently?
Status: NEW → ASSIGNED
Yep, it was upgraded a couple of weeks ago for bug 187594.
(Reporter)

Comment 14

15 years ago
I knew I wasnt completely insane....it would rock to get this fixed as I cant do
a make -j anything and slows down compilation on my Blade 1000

Comment 15

15 years ago
I wonder if this is really a mozilla problem, it could very well be a bug in
gmake, since apparently previous versions worked fine. Also I searched around a
bit and came across a number of people having problems with parallel builds, but
didn't find any solutions.

Moving to build config, since I doubt this is xpidl's fault.
Assignee: dbradley → seawood
Status: ASSIGNED → NEW
Component: xpidl → Build Config

Comment 16

15 years ago
Setting default QA -
QA Contact: pschwartau → granrose

Comment 17

15 years ago
Created attachment 115352 [details]
example patch for broken parallel builds

I also have noticed that parallel builds fail on my MP machines and tracked it
down to the .deps directory being removed when it wasn't supposed to be.  The
rule for $(MDDEPDIR) in config/rules.mk appears to be the culprit, it removes
the .deps directory prior to creating one.  All sorts of race conditions are
possible.  I've attached the changes that work for me.
Comment on attachment 115352 [details]
example patch for broken parallel builds

I'm not convinced that $(MDDEPDIR) is entirely to blame as it only removes
attempts to remove whatever's named $(MDDEPDIR) if it's not a directory.  We
want to fail if it cannot create the $(MDDEPDIR) directory so ignoring the
output of mkdir is not an option.

Have you tried building with just $(MDDEPDIR) added to the dependency lists?
Attachment #115352 - Flags: review-

Comment 19

15 years ago
It also will try to remove it if it doesn't yet exist.  My failures were mostly
on either a first build or a clobber build.  How about only removing it if it
exists and isn't a directory?  Or use "rm -f"?  "rm -f" will remove
non-directory objects with that name and leave a directory alone (but will have
a non-zero exit status).

If two concurrent make processes are both discovering it doesn't exist then one
will create it and drive on just as another is removing it.  I build with "-j3"
so there's plenty of opportunity for races, the reporter is using -j4.
(Reporter)

Comment 20

15 years ago
I patched my cvs build from yesterday with the attached patched and still died
in mozilla/xpcom/typelib/xpidl

/opt/SUNWspro/bin/cc -o xpidl_idl.o -c -DOSTYPE=\"SunOS5\" -DOSARCH=\"SunOS\"
-DOJI   -I../../../dist/include/xpcom -I../../../dist/include
-I/export/home/mozilla/dist/include/nspr      -I/usr/openwin/include   -KPIC
-I/usr/openwin/include -xstrconst -xbuiltin=%all -mt  -DNDEBUG -DTRIMMED -xO2
-I/usr/sfw/include/glib-1.2 -I/usr/sfw/lib/glib/include -I/usr/local/include
-I/usr/sfw/include/glib-1.2 -I/usr/sfw/lib/glib/include  -I/usr/openwin/include
-DSOLARIS=1 -DNSCAP_DISABLE_DEBUG_PTR_TYPES=1 -DD_INO=d_ino -DSTDC_HEADERS=1
-DHAVE_ST_BLKSIZE=1 -DHAVE_SIGINFO_T=1 -DHAVE_INT16_T=1 -DHAVE_INT32_T=1
-DHAVE_INT64_T=1 -DHAVE_UINT=1 -DHAVE_UINT_T=1 -DHAVE_UINT16_T=1
-DHAVE_WCRTOMB=1 -DHAVE_MBRTOWC=1 -DHAVE_DIRENT_H=1 -DHAVE_SYS_BYTEORDER_H=1
-DHAVE_MEMORY_H=1 -DHAVE_UNISTD_H=1 -DHAVE_NL_TYPES_H=1 -DHAVE_X11_XKBLIB_H=1
-DHAVE_SYS_STATVFS_H=1 -DHAVE_SYS_STATFS_H=1 -DHAVE_SYS_VFS_H=1
-DHAVE_SYS_MOUNT_H=1 -DNEW_H=\<new\> -DHAVE_LIBM=1 -DHAVE_LIBDL=1
-DHAVE_LIBSOCKET=1 -DFUNCPROTO=15 -DHAVE_XSHM=1 -D_REENTRANT=1 -DHAVE_RANDOM=1
-DHAVE_STRERROR=1 -DHAVE_LCHOWN=1 -DHAVE_FCHMOD=1 -DHAVE_SNPRINTF=1
-DHAVE_STATVFS=1 -DHAVE_MEMMOVE=1 -DHAVE_RINT=1 -DHAVE_NL_LANGINFO=1
-DHAVE_FLOCKFILE=1 -DHAVE_LOCALTIME_R=1 -DHAVE_STRTOK_R=1 -DHAVE_ICONV=1
-DHAVE_ICONV_WITH_CONST_INPUT=1 -DHAVE_IOS_BINARY=1 -DHAVE_CPP_EXPLICIT=1
-DHAVE_CPP_MODERN_SPECIALIZE_TEMPLATE_SYNTAX=1
-DHAVE_CPP_PARTIAL_SPECIALIZATION=1 -DHAVE_CPP_ACCESS_CHANGING_USING=1
-DHAVE_CPP_AMBIGUITY_RESOLVING_USING=1 -DHAVE_CPP_NAMESPACE_STD=1
-DHAVE_CPP_UNAMBIGUOUS_STD_NOTEQUAL=1 -DHAVE_CPP_NEW_CASTS=1
-DHAVE_CPP_DYNAMIC_CAST_TO_VOID_PTR=1 -DNEED_CPP_UNUSED_IMPLEMENTATIONS=1
-DHAVE_I18N_LC_MESSAGES=1 -DMOZ_DEFAULT_TOOLKIT=\"gtk\" -DMOZ_WIDGET_GTK=1
-DMOZ_ENABLE_XREMOTE=1 -DMOZ_X11=1 -DMOZ_ENABLE_COREXFONTS=1 -DIBMBIDI=1
-DACCESSIBILITY=1 -DMOZ_MATHML=1 -DMOZ_LOGGING=1 -DMOZ_USER_DIR=\".mozilla\"
-DCPP_THROW_NEW=throw\(\) -DMOZ_XUL=1 -DINCLUDE_XUL=1 -DNS_MT_SUPPORTED=1
-DMOZ_DLL_SUFFIX=\".so\" -DXP_UNIX=1 -DUNIX_ASYNC_DNS=1 -DJS_THREADSAFE=1
-DNS_PRINT_PREVIEW=1 -DNS_PRINTING=1 -DMOZILLA_VERSION=\"1.3b\" 
-D_MOZILLA_CONFIG_H_ -DMOZILLA_CLIENT xpidl_idl.c
gmake[5]: *** Waiting for unfinished jobs....
gmake[5]: Leaving directory `/export/home/mozilla/xpcom/typelib/xpidl'
gmake[4]: *** [export] Error 2
gmake[4]: Leaving directory `/export/home/mozilla/xpcom/typelib'
gmake[3]: *** [export] Error 2
gmake[3]: Leaving directory `/export/home/mozilla/xpcom'
gmake[2]: *** [tier_2] Error 2
gmake[2]: Leaving directory `/export/home/mozilla'
gmake[1]: *** [default] Error 2
gmake[1]: Leaving directory `/export/home/mozilla'
gmake: *** [build] Error 2
(Reporter)

Comment 21

15 years ago
Interesting development, after the build died in the previous comment, I
restarted  the build doing gmake -f client.mk build, and it continued
through......<SHRUG>

So it looks like using this patch now it will die the first time and then work
if you retry it, this is using -j4

dcran
Created attachment 118399 [details] [diff] [review]
Do not use treat mozilla build phase as dependency targets

Ok, after much testing, I discovered the real problem behind the bug.  We
aren't parallel-safe across build phases.  A typical Mozilla build does an
|export| pass over the tree which generates headers from idls and exports
headers.  Then it does a |libs| pass which builds the libraries & executables. 
In our build, for any given directory, we assume that the |export| phase will
have been run before the |libs| phase.	(This is a long-standing assumption and
one that's not likely to be fixed anytime soon.)

The problem is this custom ruleset that we use to generate xpidl during the
export phase:

export:: libs

In a normal build, that ruleset is fine as the |export| rules will be evaluated
in sequential order and then the |libs| rules will be evaluated since that
ruleset is at the end of the Makefile.	

However, in a parallel build, make jumps ahead and attempts to evaluate as many
of the dependencies in parallel as possible.  As a consequence, make starts
evaluating the dependency tree for the |libs| phase even though the |export|
phase hasn't been completed.	We ran into this problem a long time ago which
is why the |all| target in rules.mk invokes the individual build phases
separately rather than listing them as dependencies.
Attachment #115352 - Attachment is obsolete: true
Attachment #118399 - Flags: review?(bryner)
(Reporter)

Comment 23

15 years ago
I will be testing this patch against trunk and see if it fixes my problem. 

I now have a 4x 450Mz Ultra 80 at home now so I should be able to start doing a
good bit of work at home now as well on my Blade 1000 at work.

I will report on my findings with your patch Christopher

<dcran>
Created attachment 118510 [details] [diff] [review]
workaround for second type of bustage

This works around the second race condition which occurs when generating .idl
files that depend upon other .idl files in the current srcdir.	What's
happening is that the .idl files are being exported to $(IDLDIR) but they
aren't being completely written when xpidl attempts to generate .h from the
.idls.	Forcing xpidl to search in the current srcdir before IDLDIR works
around that race condition.
Attachment #118510 - Flags: review?(bryner)
(Reporter)

Comment 25

15 years ago
Christopher

I can confirm that after implementing both of these patches I was able to build
using -j4 successfully and the build did in fact launch with no problems at all.

in my mind this patch successfully does what is needed

r=dcran
Flags: blocking1.4a?
Attachment #118399 - Flags: review?(bryner) → review+
Attachment #118510 - Flags: review?(bryner) → review+
Attachment #118399 - Flags: approval1.4a?
Attachment #118510 - Flags: approval1.4a?

Comment 26

15 years ago
Comment on attachment 118399 [details] [diff] [review]
Do not use treat mozilla build phase as dependency targets

a=asa (on behalf of drivers) for checkin to 1.4a. Time is short so this needs
to land soon if it's gonna make it.
Attachment #118399 - Flags: approval1.4a? → approval1.4a+

Updated

15 years ago
Attachment #118510 - Flags: approval1.4a? → approval1.4a+
Patches have been checked in.
Status: NEW → RESOLVED
Last Resolved: 15 years ago
Flags: blocking1.4a?
Resolution: --- → FIXED
Target Milestone: --- → mozilla1.4alpha
Product: Browser → Seamonkey
You need to log in before you can comment on or make changes to this bug.