1443773 - create ./mach try intermittent, that would take a bug id or test name and push to try the most frequently failing configs and try to reproduce

Status: NEW

(Testing :: General, enhancement, P3)

Description

[Joel Maher ( :jmaher ) (UTC -8) (PTO back normal Nov 17)]

:gijs has a suggestion in bug 1443364 to make it easier to push to try and reproduce an intermittent.  I know with ./mach test-info we have a good idea of the top failure configs and we should be able to figure out which jobs those run in with ./mach find-test-chunk; 

piecing that code together with job selection logic similar to ./mach try fuzzy- that would give us the ability to push to try and reproduce an issue, ideally automatically with --rebuild 20.

Ideal output would be:
./mach try intermittent 1443364

selecting jobs:
linux64/debug browser-chrome-e10s-4

rebuilding 20 times

We see 4 failures in 100 pushes, so you should retrigger the job 30 more times to get 50 data points where you should expect to see a failure.


something like this could really help people gather recent data and validate some fixes.

Comment 1

[Geoff Brown [:gbrown]]

As a reminder, to reproduce an intermittent on try, there are existing tools that can do much of this:

  mach try fuzzy <path> --artifact --rebuild 20

or

  Make a quick modification to the test and 
  mach try fuzzy --query test-verify --artifact