695 bytes, patch
|Details | Diff | Splinter Review|
1.46 KB, patch
Brian Ryner (not reading): review+
|Details | Diff | Splinter Review|
1014 bytes, patch
Brian Ryner (not reading): review+
|Details | Diff | Splinter Review|
I have been building mozilla using the .mozconfig from http://www.mozilla.org/build/distribution.html # cat .mozconfig # sh # Build configuration script # # See http://www.mozilla.org/build/unix.html for build instructions. # # Options for client.mk. mk_add_options MOZ_MAKE_FLAGS=-j4 # Options for 'configure' (same as command-line options). ac_add_options --enable-crypto ac_add_options --disable-tests ac_add_options --enable-js-ultrasparc ac_add_options --disable-debug ac_add_options --enable-optimize=-xO2 ac_add_options --enable-strip ac_add_options --without-system-nspr ac_add_options --without-system-zlib ac_add_options --without-system-jpeg ac_add_options --without-system-png ac_add_options --without-system-mng The problem is that mozilla will not build using this.... it dies with the following message "xpt_struct.c", line 939: warning: syntax error: empty declaration However if I build without a .mozconfig everything builds fine.... Need to get this fixed.... dcran
Created attachment 108061 [details] [diff] [review] Fixes the "warning" There's two problems where. One is that the macro ends with a semicolon and the use of that macro ends in a semicolon. That normally generates a warning. And this compiler seems to generate that warning as well, but apparently treats it as an error. Someone should probably file a bug with Sun about this. So I say lets correct the code and be done with this. It beats trying each of the options in .mozconfig and figuring out where Sun's bug is ;-)
Was talking with reporter on IRC. The source of the problem is the -J4 option in .mozconfig. The warning/error he described originally was noise diverting his attention from it. This patch is valid, and would at least get rid of some noisy warnings, but it's not going to fix this problem. So I added you to the cc list Chris in case you might have some insite into this problem. Just don't use -J4?
David, Thats exactly right...the -j4 option is not working, however I cant get -j2 to work either, so I kind of lose the speed of my 2 processor Sunblade 1000. I imagine it might have something to do with a malformed makefile maybe???
What's the exact error that make is returning when 'make -j4' is used? Most of our tinderboxes are compiling with some form of make -jx so I doubt that it's a problem is specific to that Makefile.
I know it was failing on creating a file, I think during dependency creation, the file had an extension of .o.pp, I don't remember the specific file name. The reporter will have to provide more details.
"xpt_struct.c", line 939: warning: syntax error: empty declaration is what the original message was.... then it would just die trying to create deps Here is the complete final compile string I get gmake: Entering directory `/export/home/mozilla/xpcom/typelib/xpt/src' Creating .deps xpt_arena.c Building deps for xpt_arena.c xpt_struct.c touch: .deps/xpt_arena.o.pp cannot create Building deps for xpt_struct.c gmake: *** [xpt_arena.o] Error 1 gmake: *** Waiting for unfinished jobs.... touch: .deps/xpt_struct.o.pp cannot create gmake: *** [xpt_struct.o] Error 1 gmake: Leaving directory `/export/home/mozilla/xpcom/typelib/xpt/src' gmake: *** [export] Error 2 gmake: Leaving directory `/export/home/mozilla/xpcom/typelib/xpt' gmake: *** [export] Error 2 gmake: Leaving directory `/export/home/mozilla/xpcom/typelib' gmake: *** [export] Error 2 gmake: Leaving directory `/export/home/mozilla/xpcom' gmake: *** [tier_2] Error 2 gmake: Leaving directory `/export/home/mozilla' gmake: *** [default] Error 2 gmake: Leaving directory `/export/home/mozilla' gmake: *** [build] Error 2
Is the .deps directory created and with the proper permissions?
Christoper Yes the directory is created with the correct permissions...I am compiling as root so it would be hard for permissions to be wrong...but here are the permissions on the directory # pwd /export/home/mozilla/xpcom/typelib/xpt/src drwxr-xr-x 2 root other 512 Dec 4 17:13 .deps
Anything new on this?
its still broken :( dcran
This just happened on nebiros as well. cls, so was gmake upgraded on nebiros recently?
Yep, it was upgraded a couple of weeks ago for bug 187594.
I knew I wasnt completely insane....it would rock to get this fixed as I cant do a make -j anything and slows down compilation on my Blade 1000
I wonder if this is really a mozilla problem, it could very well be a bug in gmake, since apparently previous versions worked fine. Also I searched around a bit and came across a number of people having problems with parallel builds, but didn't find any solutions. Moving to build config, since I doubt this is xpidl's fault.
Setting default QA -
Created attachment 115352 [details] example patch for broken parallel builds I also have noticed that parallel builds fail on my MP machines and tracked it down to the .deps directory being removed when it wasn't supposed to be. The rule for $(MDDEPDIR) in config/rules.mk appears to be the culprit, it removes the .deps directory prior to creating one. All sorts of race conditions are possible. I've attached the changes that work for me.
Comment on attachment 115352 [details] example patch for broken parallel builds I'm not convinced that $(MDDEPDIR) is entirely to blame as it only removes attempts to remove whatever's named $(MDDEPDIR) if it's not a directory. We want to fail if it cannot create the $(MDDEPDIR) directory so ignoring the output of mkdir is not an option. Have you tried building with just $(MDDEPDIR) added to the dependency lists?
It also will try to remove it if it doesn't yet exist. My failures were mostly on either a first build or a clobber build. How about only removing it if it exists and isn't a directory? Or use "rm -f"? "rm -f" will remove non-directory objects with that name and leave a directory alone (but will have a non-zero exit status). If two concurrent make processes are both discovering it doesn't exist then one will create it and drive on just as another is removing it. I build with "-j3" so there's plenty of opportunity for races, the reporter is using -j4.
I patched my cvs build from yesterday with the attached patched and still died in mozilla/xpcom/typelib/xpidl /opt/SUNWspro/bin/cc -o xpidl_idl.o -c -DOSTYPE=\"SunOS5\" -DOSARCH=\"SunOS\" -DOJI -I../../../dist/include/xpcom -I../../../dist/include -I/export/home/mozilla/dist/include/nspr -I/usr/openwin/include -KPIC -I/usr/openwin/include -xstrconst -xbuiltin=%all -mt -DNDEBUG -DTRIMMED -xO2 -I/usr/sfw/include/glib-1.2 -I/usr/sfw/lib/glib/include -I/usr/local/include -I/usr/sfw/include/glib-1.2 -I/usr/sfw/lib/glib/include -I/usr/openwin/include -DSOLARIS=1 -DNSCAP_DISABLE_DEBUG_PTR_TYPES=1 -DD_INO=d_ino -DSTDC_HEADERS=1 -DHAVE_ST_BLKSIZE=1 -DHAVE_SIGINFO_T=1 -DHAVE_INT16_T=1 -DHAVE_INT32_T=1 -DHAVE_INT64_T=1 -DHAVE_UINT=1 -DHAVE_UINT_T=1 -DHAVE_UINT16_T=1 -DHAVE_WCRTOMB=1 -DHAVE_MBRTOWC=1 -DHAVE_DIRENT_H=1 -DHAVE_SYS_BYTEORDER_H=1 -DHAVE_MEMORY_H=1 -DHAVE_UNISTD_H=1 -DHAVE_NL_TYPES_H=1 -DHAVE_X11_XKBLIB_H=1 -DHAVE_SYS_STATVFS_H=1 -DHAVE_SYS_STATFS_H=1 -DHAVE_SYS_VFS_H=1 -DHAVE_SYS_MOUNT_H=1 -DNEW_H=\<new\> -DHAVE_LIBM=1 -DHAVE_LIBDL=1 -DHAVE_LIBSOCKET=1 -DFUNCPROTO=15 -DHAVE_XSHM=1 -D_REENTRANT=1 -DHAVE_RANDOM=1 -DHAVE_STRERROR=1 -DHAVE_LCHOWN=1 -DHAVE_FCHMOD=1 -DHAVE_SNPRINTF=1 -DHAVE_STATVFS=1 -DHAVE_MEMMOVE=1 -DHAVE_RINT=1 -DHAVE_NL_LANGINFO=1 -DHAVE_FLOCKFILE=1 -DHAVE_LOCALTIME_R=1 -DHAVE_STRTOK_R=1 -DHAVE_ICONV=1 -DHAVE_ICONV_WITH_CONST_INPUT=1 -DHAVE_IOS_BINARY=1 -DHAVE_CPP_EXPLICIT=1 -DHAVE_CPP_MODERN_SPECIALIZE_TEMPLATE_SYNTAX=1 -DHAVE_CPP_PARTIAL_SPECIALIZATION=1 -DHAVE_CPP_ACCESS_CHANGING_USING=1 -DHAVE_CPP_AMBIGUITY_RESOLVING_USING=1 -DHAVE_CPP_NAMESPACE_STD=1 -DHAVE_CPP_UNAMBIGUOUS_STD_NOTEQUAL=1 -DHAVE_CPP_NEW_CASTS=1 -DHAVE_CPP_DYNAMIC_CAST_TO_VOID_PTR=1 -DNEED_CPP_UNUSED_IMPLEMENTATIONS=1 -DHAVE_I18N_LC_MESSAGES=1 -DMOZ_DEFAULT_TOOLKIT=\"gtk\" -DMOZ_WIDGET_GTK=1 -DMOZ_ENABLE_XREMOTE=1 -DMOZ_X11=1 -DMOZ_ENABLE_COREXFONTS=1 -DIBMBIDI=1 -DACCESSIBILITY=1 -DMOZ_MATHML=1 -DMOZ_LOGGING=1 -DMOZ_USER_DIR=\".mozilla\" -DCPP_THROW_NEW=throw\(\) -DMOZ_XUL=1 -DINCLUDE_XUL=1 -DNS_MT_SUPPORTED=1 -DMOZ_DLL_SUFFIX=\".so\" -DXP_UNIX=1 -DUNIX_ASYNC_DNS=1 -DJS_THREADSAFE=1 -DNS_PRINT_PREVIEW=1 -DNS_PRINTING=1 -DMOZILLA_VERSION=\"1.3b\" -D_MOZILLA_CONFIG_H_ -DMOZILLA_CLIENT xpidl_idl.c gmake: *** Waiting for unfinished jobs.... gmake: Leaving directory `/export/home/mozilla/xpcom/typelib/xpidl' gmake: *** [export] Error 2 gmake: Leaving directory `/export/home/mozilla/xpcom/typelib' gmake: *** [export] Error 2 gmake: Leaving directory `/export/home/mozilla/xpcom' gmake: *** [tier_2] Error 2 gmake: Leaving directory `/export/home/mozilla' gmake: *** [default] Error 2 gmake: Leaving directory `/export/home/mozilla' gmake: *** [build] Error 2
Interesting development, after the build died in the previous comment, I restarted the build doing gmake -f client.mk build, and it continued through......<SHRUG> So it looks like using this patch now it will die the first time and then work if you retry it, this is using -j4 dcran
Created attachment 118399 [details] [diff] [review] Do not use treat mozilla build phase as dependency targets Ok, after much testing, I discovered the real problem behind the bug. We aren't parallel-safe across build phases. A typical Mozilla build does an |export| pass over the tree which generates headers from idls and exports headers. Then it does a |libs| pass which builds the libraries & executables. In our build, for any given directory, we assume that the |export| phase will have been run before the |libs| phase. (This is a long-standing assumption and one that's not likely to be fixed anytime soon.) The problem is this custom ruleset that we use to generate xpidl during the export phase: export:: libs In a normal build, that ruleset is fine as the |export| rules will be evaluated in sequential order and then the |libs| rules will be evaluated since that ruleset is at the end of the Makefile. However, in a parallel build, make jumps ahead and attempts to evaluate as many of the dependencies in parallel as possible. As a consequence, make starts evaluating the dependency tree for the |libs| phase even though the |export| phase hasn't been completed. We ran into this problem a long time ago which is why the |all| target in rules.mk invokes the individual build phases separately rather than listing them as dependencies.
I will be testing this patch against trunk and see if it fixes my problem. I now have a 4x 450Mz Ultra 80 at home now so I should be able to start doing a good bit of work at home now as well on my Blade 1000 at work. I will report on my findings with your patch Christopher <dcran>
Created attachment 118510 [details] [diff] [review] workaround for second type of bustage This works around the second race condition which occurs when generating .idl files that depend upon other .idl files in the current srcdir. What's happening is that the .idl files are being exported to $(IDLDIR) but they aren't being completely written when xpidl attempts to generate .h from the .idls. Forcing xpidl to search in the current srcdir before IDLDIR works around that race condition.
Christopher I can confirm that after implementing both of these patches I was able to build using -j4 successfully and the build did in fact launch with no problems at all. in my mind this patch successfully does what is needed r=dcran
Comment on attachment 118399 [details] [diff] [review] Do not use treat mozilla build phase as dependency targets a=asa (on behalf of drivers) for checkin to 1.4a. Time is short so this needs to land soon if it's gonna make it.
Patches have been checked in.