Expose browsertime-* paths to Raptor test harness
Categories
(Testing :: Raptor, enhancement, P1)
Tracking
(firefox70 affected)
| Tracking | Status | |
|---|---|---|
| firefox70 | --- | affected |
People
(Reporter: rwood, Assigned: nalexander)
References
Details
Attachments
(4 files, 1 obsolete file)
Extend the existing raptor test harness to have the ability to run a page-load test via browsertime.
Add a command line argument indicating we are using browsertime (instead of the raptor web extension). Use mitmproxy (mozproxy) and an existing test page recording on tooltool for the page playback.
Launch browsertime with the test url and wait for the results. Browsertime results themselves will be grabbed and processed in Bug 1565316.
|
Assignee | |
Comment 1•5 years ago
|
||
This is better than yes n | mach raptor-test ....
Depends on D38773
|
|
Assignee | |
Comment 2•5 years ago
|
||
This will allow mozharness tooling, which does not run through mach,
to fish these paths.
Depends on D38774
|
|
Assignee | |
Comment 3•5 years ago
|
||
The goal is to configure browsertime in Raptor in two ways:
-
locally, just like
mach browsertimedoes; -
in automation, at taskgraph creation time, using fetches and
mozharness suite artifacts (for geckodriver).
It's possible for this to be done using mozharness config settings but
using command line options is more explicit and more likely to be easy
to remove later if and when we transition to a browsertime-specific
mozharness script.
Depends on D38775
|
|
Assignee | |
Comment 4•5 years ago
|
||
Depends on D38776
|
|
Assignee | |
Comment 5•5 years ago
|
||
The existing code parses an argument with 'chrome' in its name
incorrectly. That is, --arg-with-chrome value will make the old
code think the --app is 'chrome', which is not correct.
Argument parsing is subtle enough that we rely on argparse (which is
already imported and thus known to be available).
Depends on D38775
|
||
Updated•5 years ago
|
|
|
||
Updated•5 years ago
|
|
|
Assignee | |
Comment 6•5 years ago
|
||
Here's some smoke test Raptor try builds:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=71c0a9ec0beab55a9d801790501b3ee823f019bb
https://treeherder.mozilla.org/#/jobs?repo=try&revision=8974178835f894031a032cb7455d9f7f6bb6f78a
This is looking green enough to land. This ticket does less than what the original title suggests, so I've retitled it and we'll spin out dependents for the next round of harness work. Note that we're still deciding how to opt into browsertime; these flags are just preparation for that work.
Pushed by nalexander@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/7ade93b9f15c Pre: Allow to not re-install Android app in Raptor locally. r=perftest-reviewers,rwood https://hg.mozilla.org/integration/autoland/rev/03128c989471 Part 1: Expose browsertime helpers to Raptor harness. r=barret https://hg.mozilla.org/integration/autoland/rev/8606b27c66f6 Part 2: Allow argument names with 'chrome' in them. r=rwood https://hg.mozilla.org/integration/autoland/rev/0ee1357d92f8 Part 3: Add browsertime executable arguments to Raptor command line. r=rwood
|
||
Comment 8•5 years ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/7ade93b9f15c
https://hg.mozilla.org/mozilla-central/rev/03128c989471
https://hg.mozilla.org/mozilla-central/rev/8606b27c66f6
https://hg.mozilla.org/mozilla-central/rev/0ee1357d92f8
|
Reporter | |
Comment 9•5 years ago
|
||
Reopened as Part 4 (Comment 4) has not landed yet.
|
||
Updated•5 years ago
|
|
|
Reporter | |
Comment 10•5 years ago
|
||
Closing - the starting of building out the Browsertime class in Raptor will be handled in Bug 1572804.
|
|
||
Comment 11•5 years ago
|
||
Comment on attachment 9079536 [details]
Bug 1566171 - Part 4: Start to build Browsertime support into Raptor. f?rwood
Revision D38777 was moved to bug 1572804. Setting attachment 9079536 [details] to obsolete.
Description
•