Closed Bug 449630 Opened 16 years ago Closed 16 years ago

Provide Lightning / gdata provider nightly builds from comm-central

Categories

(Mozilla Messaging Graveyard :: Server Operations, defect)

Product:
Mozilla Messaging Graveyard
Component:
Server Operations
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: standard8, Assigned: gozer)

References

Details

The Sunbird boxes that build lightning (http://tinderbox.mozilla.org/Sunbird/) are currently building with Thunderbird from trunk cvs. Due to versioning issues, they need to be building from comm-central.

The proposal is that MoMo takes over building of the lightning/gdata provider extensions and posting them on the ftp server.

Note: this is just the trunk builds and not the 1.8 branch builds.

I believe we have two options for how we do this, though one may not work:

1) Use the existing build/nightly boxes and extend them to include building lightning and adding extra steps to post the xpis.

- We must be careful if we do this not to deliver lightning with Thunderbird as we don't want to have that at the moment.

2) Set up new build/nightly boxes (or buildbot slaves) to do the build and post the xpis.


Option 2 may be better until we decide to deliver Lightning in trunk with Thunderbird (just for keeping things separate and reporting to different trees), but I'm open to suggestions on that.

In the future we may need to think about Lightning doing l10n builds, but trunk isn't set up for that yet (even for Thunderbird) and hopefully we'll just be incorporating Lightning into Thunderbird anyway.


For gozer, the extra changes over a Thunderbird build would be:

a) add "ac_add_options --enable-calendar" to the mozconfigs
b) Upload the following files to the equivalent of ftp://ftp.mozilla.org/pub/calendar/lightning/nightly/latest-trunk/

<objdir>/mozilla/dist/xpi-stage/lightning.xpi
<objdir>/mozilla/dist/xpi-stage/gdata-provider.xpi


Please can folks fill in if I've got things wrong/missed anything.
Option #2 certainly sounds better, at the expense of wasted disk space & CPU.

Regarding option #1, if we build thunderbird with --enable-calendar, how safely can we remove lightning/gdata from the generated builds ?
Status: NEW → ASSIGNED
Maybe you'll have to set DISABLE_LIGHTNING_INSTALL as currently done on the Lightning tinderbox to make it work correctly.
http://mxr.mozilla.org/mozilla/search?string=DISABLE_LIGHTNING_INSTALL
(In reply to comment #2)
> Maybe you'll have to set DISABLE_LIGHTNING_INSTALL as currently done on the
> Lightning tinderbox to make it work correctly.
> http://mxr.mozilla.org/mozilla/search?string=DISABLE_LIGHTNING_INSTALL
> 
Ok, so someone changed the sense of that flag recently.

(In reply to comment #1)
> Option #2 certainly sounds better, at the expense of wasted disk space & CPU.
> 
> Regarding option #1, if we build thunderbird with --enable-calendar, how safely
> can we remove lightning/gdata from the generated builds ?

With the DISABLE_LIGHTNING_INSTALL flag set it should just generate the xpis and not install them into <objdir>/dist/bin.

On Windows even if something crept in we'd be fine (because there is a packages-static file that causes us only to package the required items). On Mac and Linux we're potentially slightly at risk, but I'm sure with DISABLE_LIGHTNING_INSTALL flag set we'd be fine.

I'll set off a couple of mac builds later and do a comparison.
Hmm, not if we did option 1 the results would be on the Thunderbird tinderbox page rather than the Sunbird tinderbox page. Obviously this will happen eventually, don't know if we want to do it just yet or not though.
So I've just done some builds to check this, one with and one without. With DISABLE_LIGHTNING_INSTALL set in the environment (I couldn't get it to work from .mozconfig) then the <objdir>/dist/bin is unaffected.

So he only issue with option 1 is that the Tinderboxes will report to the Thunderbird tree rather than the Sunbird tree. Not sure if that's a big problem really.
That might be the case right now, but could change by accident at some point in the future.

After thinking about this some more, the base choice of action is probably to just setup 3 new buildbots as dedicated Lightning/gdata builders. They can even possibly be running on the same hardware, just under a different build root.
(In reply to comment #6)
> After thinking about this some more, the base choice of action is probably to
> just setup 3 new buildbots as dedicated Lightning/gdata builders. They can even
> possibly be running on the same hardware, just under a different build root.

Hopefully some point in the future, we'll just have Lightning integrated, so we'll be able to drop the buildbots.

However lets get this going with 3 new buildbots, then we have something for the intermediate stage that will give us valid nightlies.
So, can somebody save me some trouble and attach the simplest .mozconfig that would build Lightning/gdata only.

I'll get some buildbot running in the meantime.
If DISABLE_INSTALL_LIGHTNING works as advertised, why shouldn't we go with it? Accidents could happen, yes, but since calendar heads towards thunderbird-integration anyway, I don't see a good reason to now double the tinderboxen, and consolidate them again in a few months.

There is also the issues that the build results for these builds would end up in the Thunderbird tinderbox page, not a  separate one. Possibly making it more difficult to separate bugs/build problems from the 2.

It's not a big problem to run new buildbots, and I don't mind killing them off once we are ready to consolidate.
(In reply to comment #8)
> So, can somebody save me some trouble and attach the simplest .mozconfig that
> would build Lightning/gdata only.

Copy the current Thunderbird mozconfigs from http://hg.mozilla.org/build/index.cgi/buildbot-configs/file/8e04b481f62c/thunderbird/

a) add the line:

ac_add_options --enable-calendar

(either include globally, or once for each platform).

b) On Mac you also need to add:

mk_add_options MOZ_POSTFLIGHT_ALL+=calendar/lightning/build/universal.mk
mk_add_options MOZ_POSTFLIGHT_ALL+=calendar/providers/gdata/universal.mk

c) Drop codesighs pull/build.

In the master.cfg file, you'll need to:

1) remove "--skip-calendar" from client.py arguments
2) Drop packaging/uploading the Thunderbird package
3) Drop the codesighs items
4) Upload the following files to the equivalent of ftp://ftp.mozilla.org/pub/calendar/lightning/nightly/latest-trunk/
<objdir>/mozilla/dist/xpi-stage/lightning.xpi
<objdir>/mozilla/dist/xpi-stage/gdata-provider.xpi

Hopefully this is enough...
> 4) Upload the following files to the equivalent of
> ftp://ftp.mozilla.org/pub/calendar/lightning/nightly/latest-trunk/

I think the current Lightning CVS Trunk tinderboxen should be disabled at the same time the new Lightning HG Trunk tinderboxen goes online to avoid overriding each others files.
Right, please align the timing with Kurt.Zenker@sun.com. He's jumping in while ause is on vacation.

Can't we just reuse the same hardware these tinderboxen are running on, like we did with the Thunderbird nightly builders? 

Otherwise, we will also need to sort out access to the new, momo-controlled buildbots, no ?
(In reply to comment #14)
> Can't we just reuse the same hardware these tinderboxen are running on, like we
> did with the Thunderbird nightly builders?

That's a possibility. I expect they will have to run both tinderbox & buildbot at the same time.

> Otherwise, we will also need to sort out access to the new, momo-controlled
> buildbots, no ?

I'm not sure what access is needed... We obviously want clobber/force build support as per the existing Thunderbird boxes (and maybe access to the waterfall page that you still need to set up).

So the only other things would be buildbot reconfig and restarts, or configuration changes? I don't see those happening frequently, but maybe others can advise.

You may need access to the appropriate account to be able to upload to the lightning staging area.
This feels like a gozer bug, so giving to him.  Feel free to object if that's not the right home for it...
Assignee: nobody → gozer
Status: ASSIGNED → NEW
Is there any estimation when this will be enabled?
Working on it right now, almost certainly by the end of the week.
Status: NEW → ASSIGNED
Update is that I've got a buildbot configuration that seems to be working pretty good, and I am getting ready to merge it into our master buildbot after a little more polishing.

Unfortunately, I can get past the building step, because of bug 453491.
Depends on: 453491
s/I can get/I can't get/ in comment #19
BTW, in mozilla/dist/xpi-stage/ there is also calendar-timezones.xpi, is that important ?
For comm-central builds, do we want to upload to ftp://ftp.mozilla.org/pub/calendar/lightning/nightly/latest-comm-central/, following the pattern SeaMonkey/Thunderbird have been using now ?
That directory name sounds reasonable to me.
(In reply to comment #21)
> BTW, in mozilla/dist/xpi-stage/ there is also calendar-timezones.xpi, is that
> important ?

Yes.
(In reply to comment #21)
> BTW, in mozilla/dist/xpi-stage/ there is also calendar-timezones.xpi, is that
> important ?

Currently not, because the timezones data is directly packaged into lightning.xpi. Sunbird already comes with preinstalled calendar-timezones.xpi. In the future this makes sense for Thunderbird, too (once it comes with integrated calendar), to notify/distribute timezone updates.
Depends on: 455512
Blocks: 456037
As can be seen here, the nightly are showing up correctly where we now expect them:

ftp://ftp.mozilla.org/pub/calendar/lightning/nightly/latest-comm-central/
Status: ASSIGNED → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
Where do I find the previously nightly builds now? From looking at <ftp://ftp.mozilla.org/pub/calendar/lightning/nightly/> there are no new build folders after 2008-09-19-03. 

New builds also seem to be uploaded to <ftp://ftp.mozilla.org/pub/calendar/lightning/nightly/win32-xpi> instead of the corresponding folder in <ftp://ftp.mozilla.org/pub/calendar/lightning/nightly/latest-trunk/windows-xpi/>
You need to log in before you can comment on or make changes to this bug.