It looks like the way we handle timeouts in the jit-test.py script causes infinite retries in automation .  https://bugzilla.mozilla.org/show_bug.cgi?id=1098508#c4
gbrown located the magic in question: http://mxr.mozilla.org/build/source/buildbotcustom/status/errors.py#12
I've been trying two approaches here: The first is to handle the exception right at the point where the individual tests are run e.g. . It looks like if we hit one of these adb device timeouts, the remainder of the tests will timeout and the test run will be marked busted (red) rather than failed (orange). The second is to handle the exception in the generator that calls into the code above. This lets me end the test run prematurely e.g.  and  and seems like a better approach to me. The problem here is that either I'm missing something or I haven't had any luck getting one of adb device timeouts with this code despite many retriggers, so I'm not sure this works as intended. Terrence, any preference as to how I proceed here? Thanks!  https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=2d641f48db82  https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=18ccf7cd4a87  https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=9083c5fe9ac6
I agree that handling the error as in  is the better approach. I guess an ADB failure is an infra problem -- I thought those were supposed to show up as red?
Created attachment 8544604 [details] [diff] [review] Properly handle devicemanager exceptions Sorry for the long delay in posting this patch for review, I haven't been able to get the timeouts to reproduce on try since November. Rather than delay further I thought it best to get it reviewed and landed. I think it is better than what we have now. Recent try run here: https://tbpl.mozilla.org/?tree=Try&rev=c17a2cdcd382&showall=1
Comment on attachment 8544604 [details] [diff] [review] Properly handle devicemanager exceptions Review of attachment 8544604 [details] [diff] [review]: ----------------------------------------------------------------- Great! I expect the sheriffs will be able to tell us in a couple weeks if this worked or not.
Thanks, pushed to: https://hg.mozilla.org/integration/mozilla-inbound/rev/93640018725d