release clobberer STILL broken

RESOLVED FIXED

Status

P1
normal
RESOLVED FIXED
8 years ago
5 years ago

People

(Reporter: bhearsum, Assigned: bhearsum)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [automation])

Attachments

(3 attachments, 2 obsolete attachments)

There seems to be a couple of things wrong with it, even after the Great Purge in bug 626164:
1) Branches are never set for the release clobberer form
2) When a clobber is set, it's only set for the base builddir, eg, "tag", instead of "rel-191-tag".

The latter blocks clobberer from working at all for releases, the former makes it impossible to only clobber a single branches' builder(s).
Created attachment 506757 [details] [diff] [review]
fix clobberer release prefix

This seems to fix #2, and got release clobbers working again in staging. Doesn't affect branch clobberability.
Attachment #506757 - Flags: review?(catlee)
Created attachment 506814 [details] [diff] [review]
v2, fix the display, too

This patch gets us back into a state where all combinations of the release clobberer options work. I also fixed the "Regular Clobbers" display to not show release builds in any case, even if the user is special -- it's only confusing.

There's still an issue with tag and l10n though, we skip the "build" directory when clobbering to avoid deleting the "tools" clone, but that's also the directory where all the work happens....so we never actually clobber anything.
Attachment #506757 - Attachment is obsolete: true
Attachment #506814 - Flags: review?(catlee)
Attachment #506757 - Flags: review?(catlee)
Created attachment 506832 [details] [diff] [review]
make script factory more suitable for clobberer

This seems to be the best fix for the aforementioned problem with tag/l10n. With this, we can ignore "scripts" and "buildprops.json" when calling clobberer, and only clobber what we actually want to...

Catlee, what do you think about this approach?
Attachment #506832 - Flags: feedback?(catlee)
Blocks: 628666
Priority: -- → P1
Attachment #506814 - Flags: review?(catlee) → review+
Comment on attachment 506832 [details] [diff] [review]
make script factory more suitable for clobberer

So the gist here is to just change where scripts are run out of?  Instead of everything being under <builder_dir>/build/, it will now be under <builder_dir>?
(In reply to comment #4)
> Comment on attachment 506832 [details] [diff] [review]
> make script factory more suitable for clobberer
> 
> So the gist here is to just change where scripts are run out of?  Instead of
> everything being under <builder_dir>/build/, it will now be under
> <builder_dir>?

Yeah, that's right. Clobberer can only skip things that are in the root of the slavebuilddir, and this makes it easy for scripts to run out of it.
Comment on attachment 506814 [details] [diff] [review]
v2, fix the display, too

Landed this and updated the clobberer and stage-clobberer instances.
Attachment #506814 - Flags: checked-in+
Verified this in production by requesting a clobber for "any master", "mozilla-central", "tag". From mv-moz2-linux-ix-slave10's subsequent clobberer call:
Checking clobber URL: http://build.mozilla.org/clobberer/index.php?master=http%3A%2F%2Fproduction-master01.build.mozilla.org%3A8010%2F&slave=mv-moz2-linux-ix-slave10&builddir=tm-mb-br-lnx-ntly&branch=tracemonkey&buildername=Linux+Mobile+Desktop+tracemonkey+nightly
rel-cen-tag:Our last clobber date:  2011-01-21 15:38:15
rel-cen-tag:Server clobber date:    2011-01-27 06:12:20
rel-cen-tag:Server is forcing a clobber, initiated by bhearsum@mozilla.com
rel-cen-tag:Clobbering...
Skipping last-clobber
Removing build/
Attachment #506832 - Flags: feedback?(catlee) → feedback+
Created attachment 507559 [details] [diff] [review]
adjust clobberer invocations

Still need to test a couple of these, so far so good though.
hooray. l10n is still even more broken, probably because of the split builders. probably need to update the release builder names in index.php.
Actually...it looks like all builders that have shortened names can't be clobbered by their buildername. This is basically everything except "tag".
Created attachment 508017 [details] [diff] [review]
more fixes

So...this is a bit of a bigger change to the release clobberer, but I think it helps maintainability a lot. Here's the summary of this and the others parts of this patch:
- Switch to on-the-fly detection of release builddirs in Clobberer. Since *everything* is prefixed with "rel-", we don't need to list them out explicitly anymore. This is great, because it means the next time we change the build dirs, we won't break.
- Stop showing anything with the RELEASE_PREFIX in the main table
- Deleted testReleaseClobberPrefixBuilder, because it's unnecessary now (it tests the ability to clobber a builder on a branch, but we handle that through using the full builddir/buildername in the builder list now.)
- Update clobberer invocations/skip file/dirs for ScriptFactory changes.
Attachment #507559 - Attachment is obsolete: true
Attachment #508017 - Flags: review?(catlee)
(In reply to comment #11)
> Created attachment 508017 [details] [diff] [review]
> more fixes

A couple questions due to the impending beta 11:

1) is this likely to land before go-to-build tomorrow?
2) from reading your comments on the patch, it sounds like it should work for the first release after being landed.  Is this correct?
(In reply to comment #12)
> (In reply to comment #11)
> > Created attachment 508017 [details] [diff] [review]
> > more fixes
> 
> A couple questions due to the impending beta 11:
> 
> 1) is this likely to land before go-to-build tomorrow?

Ask Catlee :)

> 2) from reading your comments on the patch, it sounds like it should work for
> the first release after being landed.  Is this correct?

It should; you can set the clobber for mozilla-central as son as this lands to find out! I just updated the docs to make it clear how to do this (https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Documentation#Preparation). Once you mark things for a clobber try tracking down a couple of machines that did release builds in b10 to verify that they delete their release stuff in their next job. Let me know if you have issues with this.
Comment on attachment 506832 [details] [diff] [review]
make script factory more suitable for clobberer

From IRC: carry forward the feedback+ to r+

Landed on default: changeset:   1466:d01b1631d33a

Armen is planning a reconfig for this morning, so this should head to production-0.8 then.
Attachment #506832 - Flags: review+
Whiteboard: [automation[
Whiteboard: [automation[ → [automation]
Attachment #508017 - Flags: review?(catlee) → review+
Comment on attachment 508017 [details] [diff] [review]
more fixes

Landed this, updated clobberer and stage-clobberer instances.
Attachment #508017 - Flags: checked-in+
Comment on attachment 506832 [details] [diff] [review]
make script factory more suitable for clobberer

This made it to production today.
Attachment #506832 - Flags: checked-in+
We're all done here.
Status: ASSIGNED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Blocks: 627271
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.