Closed Bug 487235 Opened 15 years ago Closed 15 years ago

Power off and recycle the last of the old Firefox2 machines

Categories

(Release Engineering :: General, defect, P3)

Product:
Release Engineering
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: joduinn, Assigned: joduinn)

References

Details

Attachments

(1 file)

Disable builds on bm-xserve05, and fix up monitoring
3.36 KB, patch
bhearsum
: review+
nthomas
: checked-in+
Details | Diff | Splinter Review 
After the FF2 EOL on 17dec2008, we're finally getting ready to power off
and recycle the FF2 machines listed below. This bug is to track that work.

While these machines could be restored from tape backup if needed, doing that is non-trivial, so should only be considered as last resort. 

What will change:
* no longer produce FF2.0.0.x incremental/depend/hourly builds
* no longer produce FF2.0.0.x clobber/nightly builds
* no longer produce FF2.0.0.x release builds
* remove the FF2.0.0.x waterfall page on tinderbox
(http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mozilla1.8), as it
will be empty.

What will *not* change:
* FF2.0.0.20 builds would still be available for download from
http://www.mozilla.com/en-US/firefox/all-older.html.
* existing update offers would still be available. For example:
** FF2.0.0.14 users who do check for update will still get updated to
FF2.0.0.20.
** FF2.0.0.20 users who do check for update will still get updated to
FF3.0.5.
* newly revised major update offers, like from FF2.0.0.20 -> FF3.0.9,
could still be produced as needed (because these are produced on the
FF3.0.x infrastructure, not on the powered off FF2 infrastructure.)
* Thunderbird2.0.0.x use 9 other machines for doing builds, etc. These
are not being touched, and will continue to be supported as usual.

Why do this:
* reuses some of these machines in production pool-of-slaves or try
pool-of-slaves, where there is more demand
* reduce support workload
* allows us speed up making changes to infrastructure code, as there's
now no longer a need to special-case and retest FF2 specific situations.


balsa-18branch
bm-xserve03
bm-xserve04
production-pacifica
production-pacifica-vm02
production-prometheus-vm
production-prometheus-vm02
staging-crazyhorse
staging-pacifica-vm
staging-pacifica-vm02
staging-patrocles
staging-prometheus-vm
staging-prometheus-vm02
(In reply to comment #0)
> * remove the FF2.0.0.x waterfall page on tinderbox
> (http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mozilla1.8), as it
> will be empty.

No, it won't be empty; Thunderbird 2.0.0.x nightly builds will still be reporting to the Mozilla1.8 tree.  

Note that at least one major security backport in the past couple of months has set that tree on fire (bug 478901), so as long as Gecko changes are landing for distros and Thunderbird, it's important for there to be a "common" tree for everyone to watch those changes on (ideally the distros would even have their Firefox builders reporting there, since they'll still be checking in for Firefox releases, but I know how that went with 1.8.0).
production-prometheus-vm we will have to keep too, we use it to for tag, source, updates, update verify, and stage. We could possibly consolidate nightly updates from proemtheus-vm onto this box, so remove one more VM.

production-pacifica-vm is also used for update verify, but we can move that to production-patrocles pretty easily.
Oh, and I'll need a couple of staging boxes for a short while to verify bug 481079 is OK.
(In reply to comment #3)
> Oh, and I'll need a couple of staging boxes for a short while to verify bug
> 481079 is OK.

In irc, nthomas just needs staging-prometheus-vm for a few days of testing, and will update this bug when its free to delete.
Why isn't bm-xserve05 on here as it's currently building Firefox on the 1.8 branch? Also, which machine will be used as bm-xserve02's backup? (Just curious, really.)

As Smokey said, please do not remove the Mozilla1.8 tinderbox page as that's where Thunderbird machines should continue to report to.
(In reply to comment #1)
> (In reply to comment #0)
> > * remove the FF2.0.0.x waterfall page on tinderbox
> > (http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mozilla1.8), as it
> > will be empty.
> No, it won't be empty; Thunderbird 2.0.0.x nightly builds will still be
> reporting to the Mozilla1.8 tree.  
Good catch, and yes, we'll keep it.


> Note that at least one major security backport in the past couple of months has
> set that tree on fire (bug 478901), so as long as Gecko changes are landing for
> distros and Thunderbird, it's important for there to be a "common" tree for
> everyone to watch those changes on (ideally the distros would even have their
> Firefox builders reporting there, since they'll still be checking in for
> Firefox releases, but I know how that went with 1.8.0).
If the distros want to post build info on the Mozilla1.8 tree, or even on the FirefoxPorts tree, that would be fine by me.
(In reply to comment #5)
> Why isn't bm-xserve05 on here as it's currently building Firefox on the 1.8
> branch? 
bm-xserve05 is also used as the Thunderbird release machine, hence its not on this list to power off. 

> Also, which machine will be used as bm-xserve02's backup? (Just 
> curious, really.)
bm-xserve01. It is the only other PPC xserve we have, and was freed up specifically just in case the production Thunderbird machine bm-xserve02 died.

> As Smokey said, please do not remove the Mozilla1.8 tinderbox page as that's
> where Thunderbird machines should continue to report to.
Noted.
(In reply to comment #2)
> production-prometheus-vm we will have to keep too, we use it to for tag,
> source, updates, update verify, and stage. We could possibly consolidate
> nightly updates from proemtheus-vm onto this box, so remove one more VM.
Filed bug#488194 to track this.

> production-pacifica-vm is also used for update verify, but we can move that to
> production-patrocles pretty easily.
Filed bug#487882 to track this.
Nagios now stopped watching these:

balsa-18branch
bm-xserve03
bm-xserve04
production-pacifica-vm02
production-prometheus-vm02
staging-crazyhorse
staging-pacifica-vm
staging-pacifica-vm02
staging-patrocles
staging-prometheus-vm
staging-prometheus-vm02
(In reply to comment #9)
> Nagios now stopped watching these:

These are now confirmed all powered off:
> balsa-18branch
> bm-xserve03
> bm-xserve04
> production-pacifica-vm02
> production-prometheus-vm02
> staging-prometheus-vm
> staging-crazyhorse
> staging-patrocles
> staging-pacifica-vm
> staging-pacifica-vm02
> staging-prometheus-vm02


I'll leave them powered off until for a few days in case of last minute surprises, but otherwise will delete them Monday (20th), assuming the backup can be done in time.

Meanwhile, I think we should backup every VM, except for the staging* VMs. Seem reasonable?
Depends on: 488651
Priority: -- → P3
bm-xserve05 is shared with release builds, so the first part of this disables the periodic scheduler and associated build factory for nightlies. 

Tier1_Mozilla1.8.txt controls the nagios check called "surf/tree - Mozilla1.8", which requires specific machines on a waterfall (rather than reporting coming and goings).
Attachment #373075 - Flags: review?(bhearsum)
Attachment #373075 - Flags: review?(bhearsum) → review+
Comment on attachment 373075 [details] [diff] [review]
Disable builds on bm-xserve05, and fix up monitoring

Landed and production-1.8-master:/builds/buildbot/Automation reconfig'd. In retrospect it may have been easier just to shut down this buildbot master instance.
Attachment #373075 - Flags: checked‑in+
Also emptied mozilla/tools/tinderbox-configs/monitoring/Firefox_mozilla1.8.txt so that we don't get squawked at for stale builds.
Depends on: 489282
Depends on: 488935
No longer depends on: 489282
Now deleted: 
> 
> balsa-18branch
> production-pacifica-vm02
> production-prometheus-vm02
> staging-crazyhorse
> staging-pacifica-vm
> staging-pacifica-vm02
> staging-patrocles
> staging-prometheus-vm
> staging-prometheus-vm02

...and filed bug#491879 to remove from DNS, inventory.
(In reply to comment #7)
> (In reply to comment #5)
> > Why isn't bm-xserve05 on here as it's currently building Firefox on the 1.8
> > branch? 
> bm-xserve05 is also used as the Thunderbird release machine, hence its not on
> this list to power off. 

fyi: bug#491077 is tracking consolidating the TB2.0.0.x processes running on bm-xserve02, bm-xserve05. Once this is done, it should free up one more PPC xserve.


> > Also, which machine will be used as bm-xserve02's backup? (Just 
> > curious, really.)
> bm-xserve01. It is the only other PPC xserve we have, and was freed up
> specifically just in case the production Thunderbird machine bm-xserve02 died.
Actually, bm-xserve03, 04 are also PPC xserves. I stopped thinking about them, as they were in active use. But, heh, they are now available! :-)
Remaining work already covered in separate bugs. All done here, so closing.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: