Open Bug 1443773 Opened 3 years ago Updated 3 years ago

create ./mach try intermittent, that would take a bug id or test name and push to try the most frequently failing configs and try to reproduce

Categories

(Testing :: General, enhancement, P3)

enhancement

Tracking

(Not tracked)

People

(Reporter: jmaher, Unassigned)

Details

:gijs has a suggestion in bug 1443364 to make it easier to push to try and reproduce an intermittent.  I know with ./mach test-info we have a good idea of the top failure configs and we should be able to figure out which jobs those run in with ./mach find-test-chunk; 

piecing that code together with job selection logic similar to ./mach try fuzzy- that would give us the ability to push to try and reproduce an issue, ideally automatically with --rebuild 20.

Ideal output would be:
./mach try intermittent 1443364

selecting jobs:
linux64/debug browser-chrome-e10s-4

rebuilding 20 times

We see 4 failures in 100 pushes, so you should retrigger the job 30 more times to get 50 data points where you should expect to see a failure.


something like this could really help people gather recent data and validate some fixes.
As a reminder, to reproduce an intermittent on try, there are existing tools that can do much of this:

  mach try fuzzy <path> --artifact --rebuild 20

or

  Make a quick modification to the test and 
  mach try fuzzy --query test-verify --artifact
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.