The default bug view has changed. See this FAQ.

Rewrite Clobberer Service

RESOLVED FIXED

Status

Release Engineering
Tools
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: jhopkins, Assigned: mrrrgn)

Tracking

(Depends on: 2 bugs, Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(16 attachments, 1 obsolete attachment)

59 bytes, text/x-github-pull-request
Details | Review | Splinter Review
59 bytes, text/x-github-pull-request
Callek
: review+
mrrrgn
: checked-in+
Details | Review | Splinter Review
25.13 KB, patch
rail
: review+
Details | Diff | Splinter Review
4.63 KB, patch
rail
: review+
Details | Diff | Splinter Review
59 bytes, text/x-github-pull-request
mrrrgn
: checked-in+
Details | Review | Splinter Review
65 bytes, patch
mrrrgn
: review+
mrrrgn
: checked-in+
Details | Diff | Splinter Review
76 bytes, text/plain
bhearsum
: review+
mrrrgn
: checked-in+
Details
60 bytes, text/x-github-pull-request
dustin
: review+
mrrrgn
: checked-in+
Details | Review | Splinter Review
1.47 KB, patch
bhearsum
: review+
mrrrgn
: checked-in+
Details | Diff | Splinter Review
21.90 KB, patch
rail
: review+
mrrrgn
: checked-in+
Details | Diff | Splinter Review
14.57 KB, patch
rail
: review+
mrrrgn
: checked-in+
Details | Diff | Splinter Review
14.57 KB, patch
rail
: review+
mrrrgn
: checked-in+
Details | Diff | Splinter Review
1.08 KB, patch
rail
: review+
mrrrgn
: checked-in+
Details | Diff | Splinter Review
1.66 KB, patch
rail
: review+
mrrrgn
: checked-in+
Details | Diff | Splinter Review
1.64 KB, patch
rail
: review+
mrrrgn
: checked-in+
Details | Diff | Splinter Review
6.03 KB, patch
rail
: review+
mrrrgn
: checked-in+
Details | Diff | Splinter Review
(Reporter)

Description

3 years ago
Based on the objectives in:

 https://etherpad.mozilla.org/newclobberer

Code will live here initially:

 http://hg.mozilla.org/users/jhopkins_mozilla.com/newclobberer/
(Reporter)

Comment 1

3 years ago
Need to put this on the back burner while I work on bug 886640.  Happy to review any contributed code, however!
(Reporter)

Updated

3 years ago
Assignee: jhopkins → nobody

Updated

3 years ago
Duplicate of this bug: 1069539

Updated

3 years ago
Blocks: 785410
(Assignee)

Updated

3 years ago
Assignee: nobody → winter2718
(Assignee)

Comment 3

3 years ago
Created attachment 8504147 [details] [review]
GitHub PR

This minimal implementation has been tested with the client and the current clobberer db implementation for compatibility.
Attachment #8504147 - Flags: review?(dustin)
Attachment #8504147 - Flags: review?(bugspam.Callek)
Attachment #8504147 - Flags: review?(dustin) → review?(dustin)
Comment on attachment 8504147 [details] [review]
GitHub PR

(reviewed on github)
Attachment #8504147 - Flags: review?(dustin)
(Assignee)

Comment 5

3 years ago
Created attachment 8505939 [details] [review]
relengapi-clobberer endpoint which replicates always_clobber.php behavior

Please let me know if I'm driving you all nutty and should request reviews from more folks.
Attachment #8505939 - Flags: review?(dustin)
Attachment #8505939 - Flags: review?(bugspam.Callek)
Comment on attachment 8505939 [details] [review]
relengapi-clobberer endpoint which replicates always_clobber.php behavior

reviewed on github, r+
Attachment #8505939 - Flags: review?(dustin)
Attachment #8505939 - Flags: review?(bugspam.Callek)
Attachment #8505939 - Flags: review+

Updated

3 years ago
Attachment #8504147 - Flags: review?(bugspam.Callek)
(Assignee)

Comment 7

3 years ago
Created attachment 8506349 [details] [diff] [review]
swapping out clobberer api in staging configs
Attachment #8506349 - Flags: review?(rail)
Attachment #8506349 - Flags: review?(catlee)
Attachment #8506349 - Flags: review?(bhearsum)
(Assignee)

Comment 8

3 years ago
Client tests and tests using a private buildmaster are working with these changes: http://dev-master1.srv.releng.scl3.mozilla.com:8046/builders/WINNT%205.2%20try%20build/builds/21/steps/checking_clobber_times/logs/stdio
Attachment #8506349 - Flags: review?(rail) → review+
Comment on attachment 8506349 [details] [diff] [review]
swapping out clobberer api in staging configs

The patch is trivial, let's save some time for bhearsum/catlee
Attachment #8506349 - Flags: review?(catlee)
Attachment #8506349 - Flags: review?(bhearsum)
(Assignee)

Comment 10

3 years ago
Created attachment 8506364 [details] [diff] [review]
swapping out clobberer api in staging configs (mozharness)
Attachment #8506364 - Flags: review?(rail)
Comment on attachment 8506364 [details] [diff] [review]
swapping out clobberer api in staging configs (mozharness)

ship it!
Attachment #8506364 - Flags: review?(rail) → review+
(Assignee)

Comment 12

3 years ago
Created attachment 8507904 [details] [review]
Adds Angular UI

Just keeping things in sync between BZ and GH
Attachment #8507904 - Flags: review?(dustin)
Attachment #8507904 - Flags: review?(bugspam.Callek)
(Assignee)

Comment 13

2 years ago
Created attachment 8508808 [details] [diff] [review]
Fixes Specific Slave Clobbering Logic

This has been tested with old clobberer for backwards compatibility with good success.
Attachment #8508808 - Flags: review?(catlee)
(Assignee)

Comment 14

2 years ago
Created attachment 8509643 [details] [review]
Remove users' ability to see release builds

Ben, I got good feedback about the code; but would feel good if you gave this a final thumbs up before I merge since we discussed it on IRC.
Attachment #8509643 - Flags: review?(bhearsum)
Comment on attachment 8506349 [details] [diff] [review]
swapping out clobberer api in staging configs

https://hg.mozilla.org/build/buildbot-configs/rev/9ed7b194f428
Attachment #8506349 - Flags: checked-in+
Comment on attachment 8506364 [details] [diff] [review]
swapping out clobberer api in staging configs (mozharness)

https://hg.mozilla.org/build/mozharness/rev/1ad50eb7f82a
Attachment #8506364 - Flags: checked-in+
(In reply to Rail Aliiev [:rail] from comment #15)
> Comment on attachment 8506349 [details] [diff] [review]
> swapping out clobberer api in staging configs
> 
> https://hg.mozilla.org/build/buildbot-configs/rev/9ed7b194f428

FTR, I had to skip the esr24 related hunks. esr24 is dead! :)
(Assignee)

Updated

2 years ago
Attachment #8508808 - Flags: review?(catlee) → review+
In production: https://hg.mozilla.org/build/buildbot-configs/rev/9ed7b194f428
In production: https://hg.mozilla.org/build/mozharness/rev/1ad50eb7f82a
Comment on attachment 8509643 [details] [review]
Remove users' ability to see release builds

I think this is no longer needed because the PR is closed?
Attachment #8509643 - Flags: review?(bhearsum)
(Assignee)

Comment 21

2 years ago
Created attachment 8510396 [details]
Remove users' ability to see/clobber release builds

Ben, this took your suggestion to the previous PR into account. I caused a nasty merge situation in the other pull request and decided to abandon it (Travis wasn't working because of the merge errors).
Attachment #8509643 - Attachment is obsolete: true
Attachment #8510396 - Flags: review?(bhearsum)
Comment on attachment 8510396 [details]
Remove users' ability to see/clobber release builds

I left one comment on the PR, but nothing that blokcs.
Attachment #8510396 - Flags: review?(bhearsum) → review+
(Assignee)

Comment 23

2 years ago
Created attachment 8510602 [details] [review]
A Nice Search Feature for the UI
(Assignee)

Updated

2 years ago
Attachment #8510602 - Flags: review?(dustin)
Attachment #8510602 - Flags: review?(dustin) → review+
(Assignee)

Comment 24

2 years ago
Everything's done on clobberer that needs doing until the old clobberer is deprecated, you can see the status here: https://etherpad.mozilla.org/newclobberer
Comment on attachment 8507904 [details] [review]
Adds Angular UI

addressed in github
Attachment #8507904 - Flags: review?(dustin)
Attachment #8507904 - Flags: review?(bugspam.Callek)
(Assignee)

Comment 26

2 years ago
Created attachment 8511362 [details] [diff] [review]
Add deprecation message to current clobberer

Just a message in an h1 tag
Attachment #8511362 - Flags: review?(bhearsum)
Attachment #8511362 - Flags: review?(bhearsum) → review+
(Assignee)

Comment 27

2 years ago
Created attachment 8512019 [details] [diff] [review]
Buildbot Configs Change to New Clobberer (Prod)
Attachment #8512019 - Flags: review?(rail)
(Assignee)

Comment 28

2 years ago
Created attachment 8512022 [details] [diff] [review]
Mozharness to new prod clobberer
Attachment #8512022 - Flags: review?(rail)
(Assignee)

Comment 29

2 years ago
Created attachment 8512024 [details] [diff] [review]
Ash mozharness to new prod clobberer
Attachment #8512024 - Flags: review?(rail)
(Assignee)

Comment 30

2 years ago
Created attachment 8512026 [details] [diff] [review]
Puppet manifests to new prod clobberer
Attachment #8512026 - Flags: review?(rail)
(Assignee)

Comment 31

2 years ago
Created attachment 8512027 [details] [diff] [review]
Puppet to new prod clobberer
Attachment #8512027 - Flags: review?(rail)
Attachment #8512019 - Flags: review?(rail) → review+
Attachment #8512022 - Flags: review?(rail) → review+
Attachment #8512024 - Flags: review?(rail) → review+
Attachment #8512026 - Flags: review?(rail) → review+
Attachment #8512027 - Flags: review?(rail) → review+
(Assignee)

Comment 32

2 years ago
Created attachment 8512066 [details] [diff] [review]
relabs-puppet-clobberer-prod.diff
Attachment #8512066 - Flags: review?(rail)
(Assignee)

Comment 33

2 years ago
Created attachment 8512067 [details] [diff] [review]
tools-scripts-clobberer-prod.diff

I waited a bit on these because I wasn't familiar with them. Looked them over, and this seems right. Mostly these were just fallback values hard coded into the script.
Attachment #8512067 - Flags: review?(rail)
Attachment #8512066 - Flags: review?(rail) → review+
Attachment #8512067 - Flags: review?(rail) → review+
(Assignee)

Updated

2 years ago
Attachment #8508808 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8510396 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8510602 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8511362 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8512019 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8512022 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8512024 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8512026 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8512027 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8512066 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8512067 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8507904 - Flags: checked-in+
(Assignee)

Updated

2 years ago
Attachment #8505939 - Flags: checked-in+
In production: https://hg.mozilla.org/build/buildbot-configs/rev/7c491b4779c1
In production: https://hg.mozilla.org/build/mozharness/rev/dd409298ca1a
(Assignee)

Comment 36

2 years ago
Closing this task, since it's now a part of the production configs. I'll make smaller bugs to cover any small cleanup tasks.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
(Assignee)

Updated

2 years ago
Duplicate of this bug: 827790
Do we have bugs on file already to:

1) bring down old clobberer (https://secure.pub.build.mozilla.org/clobberer/)
2) remove old clobberer service from source control (under http://hg.mozilla.org/build/tools/file/e61d4d45346d/clobberer)
3) remove references to old clobber from the wiki (https://wiki.mozilla.org/ReleaseEngineering/Applications/Clobberer)
4) update mana pages:
       https://mana.mozilla.org/wiki/display/IT/Clobberer
       https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=24805981
5) Remove CNAME records from inventory (https://inventory.mozilla.org/en-US/core/search/#q=clobberer)
6) decommission old clobberer servers
7) clean up old nagios checks for the old clobberer environments

Other possible work:
1) I see a seamonkey-clobberer.mozilla.org CNAME in inventory, but I don't think we are hosting any seamonkey builders, can we decomm that too? (question for Callek)
2) I couldn't find references to the old clobberer service in our puppet repo - but is that managed by an IT puppet repo instead? If so, maybe we need to pull it too (or ask IT to pull it).
3) Are there any secrets stored anywhere which should be cleaned up relating to the old service?

Apologies if some or all of this is covered elsewhere already.

Thanks,
Pete
Flags: needinfo?(winter2718)
Flags: needinfo?(bugspam.Callek)
(In reply to Morgan Phillips [:mrrrgn] from comment #36)
> Closing this task, since it's now a part of the production configs. I'll
> make smaller bugs to cover any small cleanup tasks.

Sorry, only just seen this! :D

Hopefully the list is useful anyway.
(Assignee)

Updated

2 years ago
Flags: needinfo?(winter2718)
(In reply to Pete Moore [:pete][:pmoore] from comment #38)
> Other possible work:
> 1) I see a seamonkey-clobberer.mozilla.org CNAME in inventory, but I don't
> think we are hosting any seamonkey builders, can we decomm that too?
> (question for Callek)

Just to answer, we can't kill off that CNAME. Mozilla does host seamonkeys machines, but 'releng' does not, they are completely managed by community (which includes me though).

That CNAME is necessary last I checked, and doesn't point to any of the managed-by-releng systems.
Flags: needinfo?(bugspam.Callek)
(In reply to Justin Wood (:Callek) from comment #40)
> (In reply to Pete Moore [:pete][:pmoore] from comment #38)
> > Other possible work:
> > 1) I see a seamonkey-clobberer.mozilla.org CNAME in inventory, but I don't
> > think we are hosting any seamonkey builders, can we decomm that too?
> > (question for Callek)
> 
> Just to answer, we can't kill off that CNAME. Mozilla does host seamonkeys
> machines, but 'releng' does not, they are completely managed by community
> (which includes me though).
> 
> That CNAME is necessary last I checked, and doesn't point to any of the
> managed-by-releng systems.

Will it cause seamonkey any problems if we deprecate the old clobberer? For example, do they rely on our tools repo, or do they have their own fork?
Flags: needinfo?(bugspam.Callek)
It's forked.  The rest of this plan looked good to me
Flags: needinfo?(bugspam.Callek)
Blocks: 1090425
Depends on: 1115242
Depends on: 1115239
For anyone else who was struggling to find the new RoR (the repo in comment 0 is out of date, and the various Mana/wiki pages don't link to the source afaict), it's here:
https://github.com/mozilla/build-relengapi-clobberer

I've also filed an issue for adding links to the source in the footer of the clobberer:
https://github.com/mozilla/build-relengapi-clobberer/issues/31
Just as an fyi, all our repos are now listed at https://wiki.mozilla.org/ReleaseEngineering/Repositories as well
Depends on: 1129898
You need to log in before you can comment on or make changes to this bug.