1401022 - Android debug mochitest, reftest max-run-time is excessive

Status: RESOLVED WONTFIX

(Firefox for Android Graveyard :: Testing, enhancement, P5)

Description

[Geoff Brown [:gbrown]]

The test task max-run-time parameter for Android Debug mochitests and reftests is an astounding 10800 seconds - much more than required. This may have been accidentally introduced in the buildbot -> taskcluster transition.

I think 5400 would be okay for everything but perhaps Android Debug mochitest-48, but let's try 7200 first.

(10800 means that we are waiting up to 3 hours for results on hung tasks, while most of these tasks normally complete in 30 to 60 minutes.)

Comment 1

[Geoff Brown [:gbrown]]

Complication: looks like the application timeout for debug mochitest is 5400 and debug reftest is 7200.

Comment 2

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #1)
> Complication: looks like the application timeout for debug mochitest is 5400
> and debug reftest is 7200.

No, it's 5400 for all Android opt and debug tests.  Sometimes it takes a few minutes longer to hit the application timeout ... bug 1401287? or inexact implementation??

Comment 3

[Geoff Brown [:gbrown]]

Attachment: reduce tc max-run-time by 1 hour

Seems OK:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9539e8c56e72ed69b49e202cdddf173403b19d01

I'm not done here. I'll leave-open, monitor, see if this can be reduced further in a week or so.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7c2ee4964472
Reduce taskcluster max-run-time for Android Debug mochitests and reftests; r=jmaher

Comment 5

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/7c2ee4964472

Comment 6

[Geoff Brown [:gbrown]]

This remains a concern but I'm not making progress. As more tests are run, more and more run time is required. What's worse is that some chunks are much slower than others. We can address that with more test chunks, but that adds some overhead and aws $. For now we run long chunks and keep max run time relatively large.

Will moving to geckoview-based testing help? Will we move to x86 kvm-accelerated emulators? Let's wait and see what efficiency improvements the future holds.

Comment 7

[Firefox Bug Husbandry Bot]

Re-triaging per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Needinfo :susheel if you think this bug should be re-triaged.

Comment 8

[Intermittent Failures Robot]

1 failures in 760 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1401022&startday=2018-07-23&endday=2018-07-29&tree=trunk

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The leave-open keyword is there and there is no activity for 6 months.
:gbrown, maybe it's time to close this bug?

Comment 10

[Geoff Brown [:gbrown]]

Sure. The max-run-time remains excessive but cannot be reduced without adding an unreasonable number of chunks. Eventually we will retire the android-em-4.3 tests.