Closed Bug 1436262 Opened 4 years ago Closed 4 years ago

With runByManifest, Android mochitest-chrome still runs longer

Categories

(Testing :: Mochitest, enhancement)

Product:
Testing
Component:
Mochitest
Version:
Version 3
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(firefox60 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla60
Tracking Flags:
Tracking Status
firefox60 --- fixed

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(1 file)

detect test completion more efficiently
862 bytes, patch
jmaher
: review+
Details | Diff | Splinter Review 
Even after bug 1435624, setting runByManifest still adds 10 to 20 minutes to each task run time for some mochitest-chrome jobs.

Opportunities for optimization:
 - do not start/stop the web server each time
 - faster detection of test completion
 - faster application startup?

...or we could just run in more chunks.
this is the same issue in desktop tests- we took a 30+ second hit for each directory, I expect that hit is amplified on android.
Firefox/test startup takes 30 to 60 seconds, but I don't think there's much that can be done to optimize that.

Web server start/stop takes 5 to 10 seconds and could perhaps be eliminated, but I kind-of like the additional sense of isolation with a new web server instance per manifest - I don't want to optimize it...at least not today.

There is a frequent 10 (sometimes 20) second delay after tests have completed and before the harness notices that Firefox has closed:

[task 2018-02-07T00:14:08.487Z] 00:14:08     INFO -  226 INFO Passed:  3151
[task 2018-02-07T00:14:08.487Z] 00:14:08     INFO -  227 INFO Failed:  0
[task 2018-02-07T00:14:08.488Z] 00:14:08     INFO -  228 INFO Todo:    3
[task 2018-02-07T00:14:08.488Z] 00:14:08     INFO -  229 INFO Mode:    non-e10s
[task 2018-02-07T00:14:08.488Z] 00:14:08     INFO -  230 INFO Slowest: 70755ms - chrome://mochitests/content/chrome/dom/animation/test/chrome/test_animation_performance_warning.html
[task 2018-02-07T00:14:08.488Z] 00:14:08     INFO -  231 INFO SimpleTest FINISHED
[task 2018-02-07T00:14:29.782Z] 00:14:29     INFO -  INFO | automation.py | Application ran for: 0:13:20.796362
[task 2018-02-07T00:14:29.782Z] 00:14:29     INFO -  INFO | zombiecheck | Reading PID log: /tmp/tmpxvipdwpidlog

This delay can be nearly eliminated with this simple change.
Attachment #8949151 - Flags: review?(jmaher)
Comment on attachment 8949151 [details] [diff] [review]
detect test completion more efficiently

Review of attachment 8949151 [details] [diff] [review]:
-----------------------------------------------------------------

nice easy review
Attachment #8949151 - Flags: review?(jmaher) → review+
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/705fdefaf5b1
Improve efficiency of detection of end of test Android test runs; r=jmaher
https://hg.mozilla.org/mozilla-central/rev/705fdefaf5b1
Status: NEW → RESOLVED
Closed: 4 years ago
status-firefox60: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
You need to log in before you can comment on or make changes to this bug.