Closed Bug 391968 Opened 13 years ago Closed 13 years ago

improvements to bootstrap staging

Categories

(Release Engineering :: General, defect, P2)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: rhelmer, Assigned: rhelmer)

References

Details

Attachments

(4 files, 4 obsolete files)

There are several things that make running a full release in a staging environment difficult, for example:

* hardcoded hostnames
* manual staging setup/teardown
etc.

Most of these are inside bootstrap, but a few are in the build system and tinderbox client.
Attachment #276413 - Flags: review?(nrthomas)
this removes all hardcoded instances of ftp.m.o, stage.m.o and download.m.o and uses $ftpServer, $stagingServer and $bouncerServer instead.
Assignee: build → rhelmer
Status: NEW → ASSIGNED
Attachment #276414 - Flags: review?(nrthomas)
For your reviewing convenience, I merged this into one diff. This covers:

Firefox
 trunk
 1.8/l10n

Thunderbird
 1.8/l10n
 1.8.0/l10n
Attachment #276416 - Flags: review?(nrthomas)
Comment on attachment 276416 [details] [diff] [review]
make BuildTree configurable for all release configs

Looks like the tbox server will just bail out when it can't find MozillaRelease-{locale}, so changing l10n over is ok too. 

http://mxr.mozilla.org/mozilla/source/webtools/tinderbox/processbuild.pl#126
http://mxr.mozilla.org/mozilla/source/webtools/tinderbox/processbuild.pl#240
Attachment #276416 - Flags: review?(nrthomas) → review+
Comment on attachment 276414 [details] [diff] [review]
variablize all instances of (download|ftp|stage).mozilla.org

I think this is good thing to be doing, but I'm gonna minus it so that we can get some additional changes all in the same patch.


* There's this typo
  # where QA updates/builds go
  stagingServer   = staging.mozilla.org
ie use stage.mozilla.org

* There is a lot of overlap between sshServer and stagingServer. The current users of sshServer are all in Push()'s, so we could just move them over to stagingServer; similarly  sshUser. I like the more descriptive variable name.
* If we do the above, and have time to get this all tested before the next release, it would be great to integrate these changes into the existing release configs. This is mainly so that we don't forget, so a valid alternative would be to file the 2.0.0.7 tracker bug ahead of time and leave a note.
Attachment #276414 - Flags: review?(nrthomas) → review-
Comment on attachment 276413 [details] [diff] [review]
make building staging env more hands-off

>Index: Makefile
>===================================================================
...
> cvsmirror_main:
...
>+	chgrp -R cvs /builds/cvsmirror.clean/cvsroot /builds/cvsmirror/l10n
>+	chmod -R g+rw /builds/cvsmirror.clean/cvsroot /builds/cvsmirror/l10n

Missed the .clean insert on the final arguments here

>+	cvs -d /builds/cvsmirror.clean/cvsroot rtag -d FIREFOX_2_0_0_4_RELEASE mozilla
>+	cvs -d /builds/cvsmirror.clean/cvsroot rtag -d FIREFOX_2_0_0_4_RC1 mozilla
>+	cvs -d /builds/cvsmirror.clean/cvsroot rtag -d -B FIREFOX_2_0_0_4_MINIBRANCH mozilla
>+	cvs -d /builds/cvsmirror.clean/l10n rtag -d FIREFOX_2_0_0_4_RELEASE l10n
>+	cvs -d /builds/cvsmirror.clean/l10n rtag -d FIREFOX_2_0_0_4_RC1 l10n

Looks like there were three rc's for 2.0.0.4, which could have an affect on respin testing if the other RC tags aren't also deleted.

> cvsmirror_mofo:
...
>+	cd /builds/cvsmirror.clean/tmp/mofo && cvs -d cltbld@cvs.mozilla.org:/mofo export -r MOZILLA_1_8_0_BRANCH talkback

Any reason to be using the 1.8.0 branch ? Should switch to 1.8 me thinks.

>+	cd /builds/cvsmirror.clean/tmp/mofo && cvs -d cltbld@cvs.mozilla.org:/mofo export -r HEAD release
>+	rm -rf /builds/cvsmirror.clean/tmp/mofo/release/tinderbox-configs/
>+	cd /builds/cvsmirror.clean/tmp/mofo && cvs -d cltbld@cvs.mozilla.org:/mofo export -r MOZILLA_1_8_0_BRANCH_release release/tinderbox-configs/

All the bits here to do with tinderbox-configs and release automation are now in public cvs now and can be culled ? 

...
>+	chmod g+rwx /builds/cvsmirror.clean/mofo
>+	chmod -R g+rw /builds/cvsmirror.clean/mofo

Need both these commands ? Get away with one further up.

> clean_stage:
> 	rm -rf /builds/config/*
> 	rm -rf /builds/tags/*
>+	rm -rf /builds/logs/*
...
>+clean_logs:
>+	rm -rf /builds/logs/*
>+

What's the separate clean_logs target for ? My makefile foo is rusty, but can you avoid duplication by making clean_stage depend on clean_logs? Similarly I find the rm on clean_cvsmirror confusing. Feel free to slap me upside the head ;-)
Attachment #276413 - Flags: review?(nrthomas) → review-
(In reply to comment #5)
> (From update of attachment 276413 [details] [diff] [review])
> >Index: Makefile
> ...
> >+	chmod g+rwx /builds/cvsmirror.clean/mofo
> >+	chmod -R g+rw /builds/cvsmirror.clean/mofo
> 
> Need both these commands ? Get away with one further up.
> 


I want the target to work standalone as well as a dependency.

Everything else is spot-on, thanks!
Attachment #276413 - Attachment is obsolete: true
Attachment #276809 - Flags: review?(nrthomas)
Attachment #276809 - Flags: review?(nrthomas) → review+
Checking in Makefile;
/cvsroot/mozilla/tools/release/Makefile,v  <--  Makefile
new revision: 1.6; previous revision: 1.5
done
Ok:

* corrected stage.m.o in bootstrap example (made some other minor corrections too)
* s/sshUser/stagingUser/ and s/sshServer/stagingServer/
Attachment #276414 - Attachment is obsolete: true
Attachment #276817 - Flags: review?(nrthomas)
Comment on attachment 276817 [details] [diff] [review]
variabilize all download/ftp/stage take 2

Looks good, thanks! r+
Attachment #276817 - Flags: review?(nrthomas) → review+
Attached patch merge (obsolete) — Splinter Review
A little bitrotted, merged and testing this version now.
Attachment #276817 - Attachment is obsolete: true
Checking in bootstrap.cfg.example;
/cvsroot/mozilla/tools/release/bootstrap.cfg.example,v  <--  bootstrap.cfg.example
new revision: 1.5; previous revision: 1.4
done
Checking in Bootstrap/Step/Build.pm;
/cvsroot/mozilla/tools/release/Bootstrap/Step/Build.pm,v  <--  Build.pm
new revision: 1.15; previous revision: 1.14
done
Checking in Bootstrap/Step/PatcherConfig.pm;
/cvsroot/mozilla/tools/release/Bootstrap/Step/PatcherConfig.pm,v  <--  PatcherConfig.pm
new revision: 1.6; previous revision: 1.5
done
Checking in Bootstrap/Step/Repack.pm;
/cvsroot/mozilla/tools/release/Bootstrap/Step/Repack.pm,v  <--  Repack.pm
new revision: 1.16; previous revision: 1.15
done
Checking in Bootstrap/Step/Updates.pm;
/cvsroot/mozilla/tools/release/Bootstrap/Step/Updates.pm,v  <--  Updates.pm
new revision: 1.14; previous revision: 1.13
done
Attachment #276906 - Attachment is obsolete: true
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.