415180 - Create XULRunner builds at major milestones of Firefox

Status: VERIFIED FIXED

(Release Engineering :: General, defect, P3)

Description

[Mark Finkle (:mfinkle) (use needinfo?)]

It would really help the XULRunner community if we could create XULRunner releases at the major milestones of Firefox (beta's and final releases). Windows, Linux & Mac platforms are the only ones needed. Including the XULRunner SDK is also important.

The builds should be pushed to FTP.

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

rhelmer, mfinkle, cblizzard, bsmedberg and myself just met on this. Here are some quick notes from the meeting:

1) This is for trunk/1.9, and later moz2. Explicitly, this is not for the 1.8 branch.

2) These XULrunner builds should not slowdown or block actual FF/TB releases. We can easily arrange for slave to be running later in process, while QA is testing.

3) The slaves used, and the code timestamp used, should be the same as was used for the FF/TB release.

4) We're busy with bringing up builds machines for some other platforms, but should be able to start looking at this around mid-Feb. Having these builds sometime around b4 or even rc1 was ok.

5) There are no locales involved, it is all en-US only.

6) There are no updates involved.

Comment 2

[Mark Finkle (:mfinkle) (use needinfo?)]

John - Any thoughts on if build will have cycles for this after Fx3.0b4?

Comment 3

[Jean-Marc Desperrier]

Given the discussion in "Is "Gecko" 1.9 only Firefox?"
http://groups.google.fr/group/mozilla.dev.platform/browse_thread/thread/4f1595449bf94cd2

Does it make much sense to work on this if XULRunner regressions even if major
will not be fixed in major milestones of Firefox ?

Bugzilla is not the right place for, and I don't intend to start a debate about
it, what I mean is maybe this bug needs to be frozen as long as is not taken a
decision that will make sure XULRunner builds synchronized with Firefox are
actually useful.

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to comment #3)
> Given the discussion in "Is "Gecko" 1.9 only Firefox?"
> http://groups.google.fr/group/mozilla.dev.platform/browse_thread/thread/4f1595449bf94cd2
> 
> Does it make much sense to work on this if XULRunner regressions even if major
> will not be fixed in major milestones of Firefox ?
> 
> Bugzilla is not the right place for, and I don't intend to start a debate about
> it, what I mean is maybe this bug needs to be frozen as long as is not taken a
> decision that will make sure XULRunner builds synchronized with Firefox are
> actually useful.

I don't see why this bug should not be fixed. XULRunner and Gecko SDK releases based on the same code as Firefox would be very helpful to many people. The issue that the linked discussion is describing has nothing to do with this bug, but has to do with the triage process for Firefox releases.

Comment 5

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to comment #4)
> (In reply to comment #3)
> > Given the discussion in "Is "Gecko" 1.9 only Firefox?"
> > http://groups.google.fr/group/mozilla.dev.platform/browse_thread/thread/4f1595449bf94cd2
> > 
> > Does it make much sense to work on this if XULRunner regressions even if major
> > will not be fixed in major milestones of Firefox ?
> > 
> > Bugzilla is not the right place for, and I don't intend to start a debate about
> > it, what I mean is maybe this bug needs to be frozen as long as is not taken a
> > decision that will make sure XULRunner builds synchronized with Firefox are
> > actually useful.
> 
> I don't see why this bug should not be fixed. XULRunner and Gecko SDK releases
> based on the same code as Firefox would be very helpful to many people.
> 

Agreed. XULRunner builds synced with Firefox will be very useful. You can always chose not to use them if the build is not suitable for your needs.

Comment 6

[Robert Helmer [:rhelmer]]

Taking. First step is to get this into staging.

Comment 7

[Robert Helmer [:rhelmer]]

Doing this through bootstrap should be straightforward, I was thinking of trying this first. However, we could also use this as a guinea pig for doing more direct builds (without Tinderbox client or bootstrap); we'd need to somehow expose the info in bootstrap.cfg or duplicate the info in it to Buildbot, I guess.

Any thoughts? The downside to the bootstrap approach of course is that we'd need to have a separate xr-1.9-bootstrap.cfg that was updated for every release, as opposed to a builder which just does the right thing because it knows what tag to build from (I think tag is really the minimum info Buildbot would need to know about, version and everything else is checked into CVS).

Comment 8

[Benjamin Smedberg]

I'm happy to use XR as a guinea-pig for a tinderboxless build: I'm just worried that it will get complicated and then fall by the wayside, while there's an existing process which pretty much already works. How's that for studied ambivalence?

Comment 9

[Robert Helmer [:rhelmer]]

(In reply to comment #8)
> I'm happy to use XR as a guinea-pig for a tinderboxless build: I'm just worried
> that it will get complicated and then fall by the wayside, while there's an
> existing process which pretty much already works. How's that for studied
> ambivalence?

Not bad. How about this - I have a patch for using bootstrap (which is how we do Firefox builds), we can get that in and up on staging, then worry about making it Tinderboxless after that's up and running.

Like I said, we'll need to have a config file that gets bumped each release, but not much to it besides that. 

Comment 10

[Robert Helmer [:rhelmer]]

Attachment: [checked in] get paranoid with locks, and call bootstrap for XULRunner

Comment 11

[Robert Helmer [:rhelmer]]

Attachment: bootstrap config for xulrunner

based on fx-1.9-bootstrap.cfg

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 307407 [details] [diff] [review]
bootstrap config for xulrunner

>Index: xr-moz19-bootstrap.cfg
>===================================================================
...
>+# Absolute path to tinderbox build directory
>+# The win32 ones are kept short because of a long path issue detailed in
>+# bug# 400846
>+linux_buildDir       = /builds/tinderbox/Xr-Mozilla1.9-Release
>+macosx_buildDir      = /builds/tinderbox/Xr-Mozilla1.9-Release
>+win32_buildDir       = /e/fx19rel
>+linux_l10n_buildDir  = /builds/tinderbox/Xr-Mozilla1.9-l10n-Release
>+macosx_l10n_buildDir = /builds/tinderbox/Xr-Mozilla1.9-l10n-Release
>+win32_l10n_buildDir  = /e/fx19l10nrel

Need to have unique dirs for win32 too, eg xr19rel. Do the slaves have disk space for another build ? In particular fx-linux-1.9-slave2 given the problems over the last couple of days.

I wonder if you can get away with not defining *l10n_buildDir, *l10n_buildPlatform, patcher*, *verifyConfig, aus*

>+# Absolute path to store bootstrap's logs
>+linux_logDir    = /builds/logs
>+macosx_logDir   = /builds/logs
>+win32_logDir    = /builds/logs

Nit: A single logDir would be enough :-)

>+# symbol server variables
>+symbolServer     = stage.mozilla.org
>+symbolServerUser = ffxbld
>+symbolServerPath = /mnt/netapp/breakpad/symbols_ffx
>+win32_symbolServerKey  = /c/Documents and Settings/cltbld/.ssh/ffxbld_dsa
>+linux_symbolServerKey  = /home/cltbld/.ssh/ffxbld_dsa
>+macosx_symbolServerKey  = /Users/cltbld/.ssh/ffxbld_dsa

We don't currently push symbols for XULRunner, see bug 391718. Can we disable symbol push in the release config for tinderbox ? I guess s/ffxbld/xrbld/ in preparation for this being used, and add the key to the boxes.

Comment 13

[Robert Helmer [:rhelmer]]

(In reply to comment #12)
> (From update of attachment 307407 [details] [diff] [review])
> >Index: xr-moz19-bootstrap.cfg
> >===================================================================
> ...
> >+# Absolute path to tinderbox build directory
> >+# The win32 ones are kept short because of a long path issue detailed in
> >+# bug# 400846
> >+linux_buildDir       = /builds/tinderbox/Xr-Mozilla1.9-Release
> >+macosx_buildDir      = /builds/tinderbox/Xr-Mozilla1.9-Release
> >+win32_buildDir       = /e/fx19rel
> >+linux_l10n_buildDir  = /builds/tinderbox/Xr-Mozilla1.9-l10n-Release
> >+macosx_l10n_buildDir = /builds/tinderbox/Xr-Mozilla1.9-l10n-Release
> >+win32_l10n_buildDir  = /e/fx19l10nrel
> 
> Need to have unique dirs for win32 too, eg xr19rel. Do the slaves have disk
> space for another build ? In particular fx-linux-1.9-slave2 given the problems
> over the last couple of days.

Thanks, I missed that one :/

> I wonder if you can get away with not defining *l10n_buildDir,
> *l10n_buildPlatform, patcher*, *verifyConfig, aus*
> 
> >+# Absolute path to store bootstrap's logs
> >+linux_logDir    = /builds/logs
> >+macosx_logDir   = /builds/logs
> >+win32_logDir    = /builds/logs
> 
> Nit: A single logDir would be enough :-)
> 
> >+# symbol server variables
> >+symbolServer     = stage.mozilla.org
> >+symbolServerUser = ffxbld
> >+symbolServerPath = /mnt/netapp/breakpad/symbols_ffx
> >+win32_symbolServerKey  = /c/Documents and Settings/cltbld/.ssh/ffxbld_dsa
> >+linux_symbolServerKey  = /home/cltbld/.ssh/ffxbld_dsa
> >+macosx_symbolServerKey  = /Users/cltbld/.ssh/ffxbld_dsa
> 
> We don't currently push symbols for XULRunner, see bug 391718. Can we disable
> symbol push in the release config for tinderbox ? I guess s/ffxbld/xrbld/ in
> preparation for this being used, and add the key to the boxes.

Let's just disable in tinderbox, imho.

The reason I left the rest alone is so it could be easily sync'd against the Firefox config as that one changes. Do you think we should strip down this config and not worry about that?

Comment 14

[Robert Helmer [:rhelmer]]

Attachment: [checked in] more s/fx/xr/

Like I commented before, this is still based on fx-1.9-bootstrap.cfg, to make diffing between the two easier. If you want me to trim out the stuff that won't be used, let me know (I'm only planning on running Build and maybe Source steps for XR anytime soon).

Comment 15

[Robert Helmer [:rhelmer]]

Attachment: tinderconfig statements for xr tinderbox config

This would need to be landed on the "release" branch. 

The comment above the moz_cvsroot for the macosx config is weird, can we just fix that? :P No reason to make it share that key is there?

We can handle it in bootstrap.cfg regardless though.

Comment 16

[Robert Helmer [:rhelmer]]

Attachment: remove duplicate line

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 307561 [details] [diff] [review]
[checked in] more s/fx/xr/

r+ Easier diffing WFM, although I'm not sure why you changed *verifyConfig in that case.

Comment 18

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 307564 [details] [diff] [review]
remove duplicate line

>Index: linux/tinder-config.pl
...
> #$moz_cvsroot   = $ENV{CVSROOT};
>+# CONFIG: $moz_cvsroot   = '%mozillaCvsroot%';
> $moz_cvsroot   = ":ext:xrbld\@cvs.mozilla.org:/cvsroot";

cvsroot is odd on the mac config because that box does double duty for Thunderbird & XULRunner, and cvs/ssh picks up whatever id_dsa is symlinked to (tbirdbld_dsa in this case). Using one key for all CVS checkouts on multi-app boxes was my "solution" to not being able to go  CVS_RSH='ssh -i ~/.ssh/xrbld_dsa'. In this case you'll want to match whatever is used on the machine, cltbld in the case of the release automation. CONFIG will take care of it anyway.

Better solutions welcomed.

>@@ -204,14 +207,17 @@

Need to change $ReleaseToLatest to 0, modify $BuildNameExtra.

> $package_creation_path = "/xulrunner/installer";
> # needs setting for mac + talkback: $mac_bundle_path = "/browser/app";
> $ssh_version   = "2";
>+# CONFIG: $ssh_user      = "%sshUser%";
> $ssh_user      = "xrbld";
> $ssh_key       = "$ENV{HOME}/.ssh/xrbld_dsa";

ssh_key will need to go, as CONFIG will put in cltbld for ssh_user.

>+# CONFIG: $ssh_server    = "%sshServer%";
> $ssh_server    = "stage.mozilla.org";
> $ReleaseGroup  = "xulrunner";
> $ftp_path      = "/home/ftp/pub/xulrunner/nightly";
> $url_path      = "http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly";
> $tbox_ftp_path = "/home/ftp/pub/xulrunner/tinderbox-builds";
> $tbox_url_path = "http://ftp.mozilla.org/pub/mozilla.org/xulrunner/tinderbox-builds";
>+# CONFIG: $milestone     = "xulrunner%version%"
> #$milestone     = "trunk";
> $notify_list   = "build-announce\@mozilla.org";
> $stub_installer = 0;

Need to add 
# CONFIG: $ENV{'SYMBOL_SERVER_HOST'}    = '%symbolServer%';
$ENV{'SYMBOL_SERVER_HOST'}    = 'stage.mozilla.org';
# CONFIG: $ENV{'SYMBOL_SERVER_USER'}    = '%symbolServerUser%';
$ENV{'SYMBOL_SERVER_USER'}    = 'xrbld';
# CONFIG: $ENV{'SYMBOL_SERVER_PATH'}    = '%symbolServerPath%';
$ENV{'SYMBOL_SERVER_PATH'}    = '/mnt/netapp/breakpad/symbols_xr';
# CONFIG: $ENV{'SYMBOL_SERVER_SSH_KEY'} = '%symbolServerKey%';
$ENV{'SYMBOL_SERVER_SSH_KEY'} = '/home/cltbld/.ssh/xrbld_dsa';

Expect the same applies to the mac and win32 configs.

Comment 19

[Robert Helmer [:rhelmer]]

(In reply to comment #17)
> (From update of attachment 307561 [details] [diff] [review])
> r+ Easier diffing WFM, although I'm not sure why you changed *verifyConfig in
> that case.

I don't like to be too consistent, keeps people on edge :P

Seriously though, I dunno, just didn't like seeing "firefox" in there.. since we're not (currently?) doing l10n builds, just doesn't matter.

Thanks!

Comment 20

[Robert Helmer [:rhelmer]]

Comment on attachment 307561 [details] [diff] [review]
[checked in] more s/fx/xr/

RCS file: /cvsroot/mozilla/tools/release/configs/xr-moz19-bootstrap.cfg,v
done
Checking in xr-moz19-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/xr-moz19-bootstrap.cfg,v  <--  xr-moz19-bootstrap.cfg
initial revision: 1.1
done

Comment 21

[Robert Helmer [:rhelmer]]

(In reply to comment #18)
> (From update of attachment 307564 [details] [diff] [review])
> >Index: linux/tinder-config.pl
> ...
> > #$moz_cvsroot   = $ENV{CVSROOT};
> >+# CONFIG: $moz_cvsroot   = '%mozillaCvsroot%';
> > $moz_cvsroot   = ":ext:xrbld\@cvs.mozilla.org:/cvsroot";
> 
> cvsroot is odd on the mac config because that box does double duty for
> Thunderbird & XULRunner, and cvs/ssh picks up whatever id_dsa is symlinked to
> (tbirdbld_dsa in this case). Using one key for all CVS checkouts on multi-app
> boxes was my "solution" to not being able to go  CVS_RSH='ssh -i
> ~/.ssh/xrbld_dsa'. In this case you'll want to match whatever is used on the
> machine, cltbld in the case of the release automation. CONFIG will take care of
> it anyway.
> 
> Better solutions welcomed.

OK no worries then, we don't need to solve it here. One way I've worked around this in the past is via a shell script (CVS_RSH=/.../whatever.sh) and whatever.sh has whatever magical SSH incantation you want.

New patch coming up.

Comment 22

[Robert Helmer [:rhelmer]]

Attachment: [checked in] xulrunner tinder-config settings take 3

Comment 23

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 307639 [details] [diff] [review]
[checked in] xulrunner tinder-config settings take 3

>Index: linux/tinder-config.pl
...
>+$crashreporter_buildsymbols = 0;
>+$crashreporter_pushsymbols = 0;
>+# CONFIG: $ENV{'SYMBOL_SERVER_HOST'}    = '%symbolServer%';
>+$ENV{'SYMBOL_SERVER_HOST'}    = 'stage.mozilla.org';
>+# CONFIG: $ENV{'SYMBOL_SERVER_USER'}    = '%symbolServerUser%';
>+$ENV{'SYMBOL_SERVER_USER'}    = 'xrbld';
>+# CONFIG: $ENV{'SYMBOL_SERVER_PATH'}    = '%symbolServerPath%';
>+$ENV{'SYMBOL_SERVER_PATH'}    = '/mnt/netapp/breakpad/symbols_xr';
>+# CONFIG: $ENV{'SYMBOL_SERVER_SSH_KEY'} = '%symbolServerKey%';
>+$ENV{'SYMBOL_SERVER_SSH_KEY'} = '/home/cltbld/.ssh/xrbld_dsa';

Didn't mean to flip flop on this but there's been progress on bug 391718, so it should be possible to push XULRunner symbols now. $crashreporter_buildsymbols and _pushsymbols can be true.

>Index: macosx/tinder-config.pl
>===================================================================
>-$ssh_key       = "$ENV{HOME}/.ssh/xrbld_dsa";
>+#$ssh_key       = "$ENV{HOME}/.ssh/xrbld_dsa";
>+# CONFIG: $ssh_server    = "%sshServer%";
>+# CONFIG: $ssh_server    = "%sshServer%";

Nit: duplicated CONFIG lines

Comment 24

[Robert Helmer [:rhelmer]]

Comment on attachment 307639 [details] [diff] [review]
[checked in] xulrunner tinder-config settings take 3

Removed the duplicate CONFIG line, thanks!

Checking in linux/tinder-config.pl;
/cvsroot/mozilla/tools/tinderbox-configs/xulrunner/linux/tinder-config.pl,v  <--  tinder-config.pl
new revision: 1.8.2.1; previous revision: 1.8
done
Checking in macosx/tinder-config.pl;
/cvsroot/mozilla/tools/tinderbox-configs/xulrunner/macosx/tinder-config.pl,v  <--  tinder-config.pl
new revision: 1.6.2.1; previous revision: 1.6
done
Checking in win32/tinder-config.pl;
/cvsroot/mozilla/tools/tinderbox-configs/xulrunner/win32/tinder-config.pl,v  <--  tinder-config.pl
new revision: 1.11.2.1; previous revision: 1.11
done

Comment 25

[Robert Helmer [:rhelmer]]

Comment on attachment 307382 [details] [diff] [review]
[checked in] get paranoid with locks, and call bootstrap for XULRunner

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/automation/staging-1.9/master.cfg,v  <--  master.cfg
new revision: 1.26; previous revision: 1.25
done

Comment 26

[Robert Helmer [:rhelmer]]

Ok, going to try to get this going on staging then.

Comment 27

[Robert Helmer [:rhelmer]]

Just to clarify, this is not going to be ready in time for b4, I am shooting to have this in production by Fx b5 or RC1. 

Comment 28

[Robert Helmer [:rhelmer]]

This seems to work ok on staging, although XULRunner was burning on trunk when I tried :) Only one problem - the build ID is not printed to the log in the same way as Firefox and Thunderbird, I'm going to look into this a little more. We use this later for updates, so maybe we can just ignore for XULRunner, but I don't really want to special-case or make this optional.. more details in a bit.

Comment 29

[Robert Helmer [:rhelmer]]

Also, wanted to point out how I expect XULRunner releases to be on 1.9 for right now; I'm thinking of just setting this up to run automatically when the Firefox release has finished, and push the bits to e.g. http://stage.mozilla.org/pub/mozilla.org/xulrunner/nightly/3.0b5-candidates/rc1/

This is what we do for Firefox and Thunderbird. The builds are approved by QA, and the release team does some extra work like signing etc. and pushing to releases area. 

So, we need someone to test and push XULRunner releases to the final release area, I don't think that the Firefox/Thunderbird release and QA teams are prepared to take this on right now. This seems like something better done by interested XULRunner community members. mfinkle/bsmedberg, and ideas/volunteers?

Comment 30

[Robert Helmer [:rhelmer]]

(In reply to comment #28)
> This seems to work ok on staging, although XULRunner was burning on trunk when
> I tried :) Only one problem - the build ID is not printed to the log in the
> same way as Firefox and Thunderbird, I'm going to look into this a little more.
> We use this later for updates, so maybe we can just ignore for XULRunner, but I
> don't really want to special-case or make this optional.. more details in a
> bit.

Ok, so this is no big deal. I see in the XULRunner log:
buildid: 2008030612

The release automation stuff is looking for:
Got Build ID: 2007030311

The latter is from tinderbox post-mozilla-rel.pl, when complete updates are generated. It looks like the former is in post-mozilla-rel.pl main(), so it might actually be the better thing to look for in general.. I'll test both 1.8/1.9 branches and file a patch if so.

Comment 31

[Robert Helmer [:rhelmer]]

Attachment: [checked in] use buildid printed from post-mozilla-rel.pl::main()

The string we currently search for is only printed if complete updates are being generated. This one is always printed in release mode, we should use this instead.

Comment 32

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 308976 [details] [diff] [review]
[checked in] use buildid printed from post-mozilla-rel.pl::main()

Looks good.

Comment 33

[Robert Helmer [:rhelmer]]

Comment on attachment 308976 [details] [diff] [review]
[checked in] use buildid printed from post-mozilla-rel.pl::main()

Checking in TinderLogParse.pm;
/cvsroot/mozilla/tools/release/MozBuild/TinderLogParse.pm,v  <--  TinderLogParse.pm
new revision: 1.5; previous revision: 1.4
done

Comment 34

[Robert Helmer [:rhelmer]]

Attachment: [checked in] set JAVA_HOME for release configs in mozconfig

The JAVA_HOME isn't already set on these machines; should we just set it in mozconfig?

Comment 35

[Benjamin Smedberg]

Comment on attachment 310042 [details] [diff] [review]
[checked in] set JAVA_HOME for release configs in mozconfig

WFM

Comment 36

[Robert Helmer [:rhelmer]]

Comment on attachment 310042 [details] [diff] [review]
[checked in] set JAVA_HOME for release configs in mozconfig

Checking in linux/mozconfig;
/cvsroot/mozilla/tools/tinderbox-configs/xulrunner/linux/mozconfig,v  <--  mozconfig
new revision: 1.5.2.1; previous revision: 1.5
done
Checking in macosx/mozconfig;
/cvsroot/mozilla/tools/tinderbox-configs/xulrunner/macosx/mozconfig,v  <--  mozconfig
new revision: 1.3.4.1; previous revision: 1.3
done
Checking in win32/mozconfig;
/cvsroot/mozilla/tools/tinderbox-configs/xulrunner/win32/mozconfig,v  <--  mozconfig
new revision: 1.2.2.1; previous revision: 1.2
done

Comment 37

[Robert Helmer [:rhelmer]]

Had some problems on staging (including but not limited to having the rebuild the Linux builder), so going to do beta5 without this. I'll post an update once I get a clean staging run including XULRunner builds.

Comment 38

[Robert Helmer [:rhelmer]]

Looks like this is all working, except for the final SyncToStaging() call (which expects there to be a private staging area). Bug 415970 gets rid of this call, and I'm working on landing that soon so setting it as a dependency, instead of trying to work around it.

Bug 397554 (install, configure, update tinderbox automatically) would make this easier to roll this out, and I'm planning on landing that next.

We just need to carry the XULRunner staging config over when we merge the above to production, and I think we'll be all done here.

Comment 39

[Robert Helmer [:rhelmer]]

Attachment: xr staging bootstrap changes for bug 415970

Bug 415970 removes the need for the two sets of staging machines, and uses the linux slave as staging FTP/stage.m.o.

Comment 40

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 313667 [details] [diff] [review]
xr staging bootstrap changes for bug 415970

>Index: xr-moz19-staging-bootstrap.cfg
>===================================================================
>RCS file: /cvsroot/mozilla/tools/release/configs/xr-moz19-staging-bootstrap.cfg,v
>retrieving revision 1.2
>diff -u -r1.2 xr-moz19-staging-bootstrap.cfg
>--- xr-moz19-staging-bootstrap.cfg	28 Mar 2008 22:38:19 -0000	1.2
>+++ xr-moz19-staging-bootstrap.cfg	4 Apr 2008 18:49:48 -0000
>@@ -27,6 +27,7 @@
> linux_logDir    = /builds/logs
> macosx_logDir   = /builds/logs
> win32_logDir    = /builds/logs
>+sourceDir       = /buids/source
> mozillaCvsroot  = staging-1.9-master.build.mozilla.org:/builds/cvsmirror/cvsroot
> l10nCvsroot     = staging-1.9-master.build.mozilla.org:/builds/cvsmirror/l10n
> mofoCvsroot     = staging-1.9-master.build.mozilla.org:/builds/cvsmirror/mofo
>@@ -72,15 +73,13 @@
> buildTree       = MozillaTest
> # where QA updates/builds go
> stagingUser     = cltbld
>-stagingServer   = staging-1.9-master.build.mozilla.org
>-externalStagingUser     = cltbld
>-externalStagingServer   = fr-linux-1.9-slave1.build.mozilla.org
>+stagingServer   = fx-linux-1.9-slave1.build.mozilla.org
> # where beta updates/builds go
>-ftpServer       = staging-1.9-master.build.mozilla.org
>+ftpServer       = fx-linux-1.9-slave1.build.mozilla.org
> # where release updates/builds go
>-bouncerServer   = staging-1.9-master.build.mozilla.org
>+bouncerServer   = fx-linux-1.9-slave1.build.mozilla.org
> # username and server to push builds
> sshUser         = cltbld
>-sshServer       = staging-1.9-master.build.mozilla.org
>+sshServer       = fx-linux-1.9-slave1.build.mozilla.org
> useTalkback     = 0
> useCvsCompression = 1

Comment 41

[Nick Thomas [:nthomas] (UTC+12)]

Oops, had no comment to make there.

Comment 42

[Robert Helmer [:rhelmer]]

All the code from this has landed and I got green builds on staging last time I tried, just waiting on bug 427166 to merge all the recent staging work to production.

Comment 43

[Robert Helmer [:rhelmer]]

Verified Linux so far at least on production, using the existing config, which specified 3.0b4 (mozilla/tools/release/configs/xr-moz19-bootstra.cfg):

http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/3.0b4-candidates/rc1/

I'll check that Windows and Mac work as well.

Comment 44

[Robert Helmer [:rhelmer]]

Everything except the Mac build is working fine, which seems to be failing in packaging:

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaRelease/1208054584.1208056354.31719.gz&fulltext=1

Anyone know what's needed here? 

Comment 45

[Benjamin Smedberg]

probably the build machine doesn't have chown_root and chown_revert installed correctly, see http://wiki.mozilla.org/ReferencePlatforms/Mac#chown_scripts

The other possibility is some kind of fallout from https://bugzilla.mozilla.org/show_bug.cgi?id=410637 (which includes a tinderbox patch)

Comment 46

[Robert Helmer [:rhelmer]]

(In reply to comment #45)
> probably the build machine doesn't have chown_root and chown_revert installed
> correctly, see http://wiki.mozilla.org/ReferencePlatforms/Mac#chown_scripts

This was it; wrong permissions on the chown scripts. You can see all platforms here now (win32 goes into "unsigned"):

http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/3.0b4-candidates/rc1/

The "unsigned" dir, the 3.x version number, and the "_info.txt" files are all things needed for Firefox. I think the version number would be easy to change, but the others it's probably just best to ignore rather than special-casing the code, IMHO.

Comment 47

[Robert Helmer [:rhelmer]]

For XULRunner releases to happen moving forward, someone needs to:

* update the xulrunner bootstrap file, which is the way release automation is configured. This is in CVS at mozilla/tools/release/configs/xr-moz19-bootstrap.cfg

* trigger the {linux,win32,macosx}_xr_build builders on http://production-1.9-master.build:8812/ (requires VPN access). This could be made automatic, but it'd require that the xr-moz19-bootstrap.cfg is landed in time, so I left it manual for now.

* copy the files in e.g. http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/3.0b4-candidates/rc1/ to the releases area

For a general overview of how the release automation works, see:
http://wiki.mozilla.org/Build:Release_Automation