Nightly builds have been missing for some days, e.g. Lightning linux comm-1.9.1 since 25-Nov-2009. Currently 3 boxes are burning on <http://tinderbox.mozilla.org/showbuilds.cgi?tree=Sunbird> and 6 boxes are burning on <http://tinderbox.mozilla.org/showbuilds.cgi?tree=Calendar1.0> Error messages indicate that this is caused by insufficient disk space on both linux and win32 build servers.
In addition, Linux hourlies have now gone missing, see http://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/tinderbox-builds/comm-1.9.1-linux/ and http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/tinderbox-builds/comm-1.9.1/linux-xpi-linux/ which ought not to be empty.
Side effect of release builds eating up disk space and not releasing it, as we haven't managed to successfully complete the release. I'll have a quick look to see if there are any old, unused leftover build directories I could clean up.
Freed up some space on the linux builder, keeping the linux_build results. Size Used Free Use% 30G 23G 5.2G 82% /builds Hope it's enough.
(In reply to comment #3) > Freed up some space on the linux builder, keeping the linux_build results. > > Size Used Free Use% > 30G 23G 5.2G 82% /builds > > Hope it's enough. Well, it changed nothing about the nightlies (which aren't on the same machine IIUC), but at least the hourlies are indeed back, so "nightly testers" can use that for the time being. People won't all have the same build, but at least severe bugs can (I hope) be crushed in time.
(In reply to comment #4) > (In reply to comment #3) > > Freed up some space on the linux builder, keeping the linux_build results. > > > > Size Used Free Use% > > 30G 23G 5.2G 82% /builds > > > > Hope it's enough. > > Well, it changed nothing about the nightlies (which aren't on the same machine > IIUC) Nightlies are on the same shared machines, and should show up starting with this evenining. , but at least the hourlies are indeed back, so "nightly testers" can use > that for the time being. People won't all have the same build, but at least > severe bugs can (I hope) be crushed in time. Good, this should normally be a much smaller time window, but getting the first ever Sunbird/Lightninig build thru via release automation is proving to be more time consuming that hoped.
W32: nightlies built, 2009-12-03 06:57 PST Mac: nightlies built, 2009-12-03 08:48 PST Linux: still waiting, 2009-12-04 already.
Hourlies have stopped being built on all three Tier1 platforms. Latest tinderbox-builds directories: http://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/tinderbox-builds/comm-1.9.1-linux/1259853429/ dated 2009-12-03 07:23 PST http://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/tinderbox-builds/comm-1.9.1-macosx/1259838000/ dated 2009-12-03 08:48 PST http://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/tinderbox-builds/comm-1.9.1-win32/1259859318/ dated 2009-12-03 09:19 PST http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/tinderbox-builds/comm-1.9.1/linux-xpi-linux/1259912157/ dated 2009-12-03 23:46 PST http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/tinderbox-builds/comm-1.9.1/macosx-xpi-macosx/1259911532/ dated 2009-12-03 23:44 PST http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/tinderbox-builds/comm-1.9.1/win32-xpi-win32/1259912402/ dated 2009-12-03 23:58 PST No comm-central-sunbird-* left for any platform http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/tinderbox-builds/comm-central/linux-xpi-linux/1259924402/ dated 2009-12-04 04:28 PST (the most recent one of all ATM) http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/tinderbox-builds/comm-central/macosx-xpi-macosx/1259914126/ dated 2009-12-04 00:24 PST http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/tinderbox-builds/comm-central/win32-xpi-win32/1259914126/ dated 2009-12-04 00:29 PST This is no reproach but a status check. Let's hope the release finally happens, and that the builders can be restarted thereafter (possibly after finding some disk space for them somewhere).
Looks like this isn't directly related to space issues because of releases. Since the release has been delayed some and will need respinning, I cleaned up the build areas on each calendar build box, so they have plenty of space now. Looks like they are failing to compile alltogether. make: Leaving directory `/e/buildbot/comm-1.9.1-sunbird-win32-nightly/build/objdir-tb' make: *** No rule to make target `/e/buildbot/comm-1.9.1-sunbird-win32-nightly/build/calendar/sunbird/branding/nightly/Makefile.in', needed by `calendar/sunbird/branding/nightly/Makefile'. Stop. make: *** [export_tier_app] Error 2 for example: Win32: http://build.mozillamessaging.com/buildbot/production/builders/WINNT%205.2%20comm-1.9.1%20sunbird%20nightly/builds/19/steps/compile/logs/stdio Mac: http://build.mozillamessaging.com/buildbot/production/builders/MacOSX%2010.5%20comm-1.9.1%20sunbird%20nightly/builds/19/steps/compile/logs/stdio
I'm resolving this bug as WORKSFORME for now. Lightning builds are back and Sunbird is failing due to Bug 533070.
Reopening. Lightning builds for linux and win32 are failing again. > fatal error LNK1180: insufficient disk space to complete link http://tinderbox.mozilla.org/showlog.cgi?log=Calendar1.0/1260929681.1260930528.18150.gz#err0