1197978 - Add ability to retrigger all talos jobs X number of times

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[Mason Chang [Inactive] [:mchang]]

When using perfherder, I usually have to retrigger the talos jobs to get a good comparison. It'd be nice to just push once to try with talos jobs and say "run each talos job 5 times". Then when the two try pushes are done, it's a lot easier to compare. Right now, I have to wait for try to finish, then go through each talos job for each platform, and retrigger X times.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

simple way is to add: '--rebuild 5' to the try syntax.  We didn't get agreement from release engineering to do this by default, we have other options as well.  If --rebuild doesn't work for you or it is easy to forget, do let us know!

Comment 2

[Mason Chang [Inactive] [:mchang]]

Can we get a little button in try chooser as an option then?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

we have the --rebuild option as a text box in try chooser, I would be happy to default it to something if you are choosing talos jobs.

Comment 4

[Mason Chang [Inactive] [:mchang]]

(In reply to Joel Maher (:jmaher) from comment #3)
> we have the --rebuild option as a text box in try chooser, I would be happy
> to default it to something if you are choosing talos jobs.

Hmm, but the rebuild option runs all tests 5 times right? Can I do a try push that does all the unit tests once and only the talos jobs 5 times?

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

it is possible, but not via the try page- we could set up pulse actions to do this by default- I think it is the right thing to do- also we can make the run all talos jobs work for all folks on try.

adding :vaibhav1994 and :adusca as they are the masterminds of some of this stuff.

Comment 6

[Vaibhav (:vaibhav1994)]

We decided in bug 1194711 that we do not want to make triggering talos jobs 6 times by default (since we are constrained on talos machines). We have a --rebuild option in try syntax but that triggers all the jobs. So, we can have --rebuild-talos option in try-syntax that will trigger only talos jobs. This can be incorporated in trigger-bot since trigger-bot handles --rebuild option. Mason, Joel does that sound good to you?

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

yeah, I think --rebuild should work the same on everything and if there is a --rebuild-talos then that should be used for what talos has.

Comment 8

[Mason Chang [Inactive] [:mchang]]

(In reply to Vaibhav (:vaibhav1994) from comment #6)
> We decided in bug 1194711 that we do not want to make triggering talos jobs
> 6 times by default (since we are constrained on talos machines). We have a
> --rebuild option in try syntax but that triggers all the jobs. So, we can
> have --rebuild-talos option in try-syntax that will trigger only talos jobs.
> This can be incorporated in trigger-bot since trigger-bot handles --rebuild
> option. Mason, Joel does that sound good to you?

Yeah, a --rebuild-talos option would work!

Comment 9

[Vaibhav (:vaibhav1994)]

Attachment: MozReview Request: Bug 1197978 - Adding --rebuild-talos option in trychooser. r=jmaher

Bug 1197978 - Adding --rebuild-talos option in trychooser. r=jmaher

Comment 10

[Vaibhav (:vaibhav1994)]

This is the front-end, will land this once the required functionality is in trigger-bot.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8652422 [details]
MozReview Request: Bug 1197978 - Adding --rebuild-talos option in trychooser. r=jmaher

https://reviewboard.mozilla.org/r/17161/#review15259

::: trychooser/index.html:487
(Diff revision 1)
> +            <p>Retrigger all talos jobs (up to 20 times):</p>

we are sort of overloading something in here, in this case I would like to see us specify something other than what is here.  Visually this looks cluttered and confusing (not that the whole page isn't).

::: trychooser/trychooser.js:181
(Diff revision 1)
> +        args.push('--rebuild-talos ' + rebuild_talos);

what accepts --rebuild-talos?

Comment 12

[Vaibhav (:vaibhav1994)]

Comment on attachment 8652422 [details]
MozReview Request: Bug 1197978 - Adding --rebuild-talos option in trychooser. r=jmaher

Bug 1197978 - Adding --rebuild-talos option in trychooser. r=jmaher

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8652422 [details]
MozReview Request: Bug 1197978 - Adding --rebuild-talos option in trychooser. r=jmaher

https://reviewboard.mozilla.org/r/17161/#review15267

Ship It!

Comment 14

[Vaibhav (:vaibhav1994)]

Attachment: github-pr

Attaching github-pr.

Comment 15

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8652686 [details] [review]
github-pr

This is looking good. I'll find some time to deploy over the weekend.

Comment 16

[Vaibhav (:vaibhav1994)]

:chmanchester said that he has deployed the backend for --rebuild-talos in triggerbot. We can now land the front-end for the trychooser (mozreview patch). This patch does not require a try run since it is only a small change in trychooser page.

Comment 17

[Joel Maher ( :jmaher ) (UTC -8)]

landed:
https://hg.mozilla.org/build/tools/rev/df4e897f9bc7

Comment 18

[Vaibhav (:vaibhav1994)]

:Callek, can you update the trychooser page ( http://trychooser.pub.build.mozilla.org/ ) to have --rebuild-talos option showing to users?

Comment 19

[Justin Wood (:Callek)]

[[e-mail]3 trychooser]# ./update tip
pulling from http://hg.mozilla.org/build/tools
searching for changes
adding changesets
adding manifests
adding file changes
added 59 changesets with 113 changes to 35 files
(run 'hg update' to get a working copy)
35 files updated, 0 files merged, 0 files removed, 0 files unresolved
Deploying code...