1442289 - [meta] - tracking bug for tp7 work

Status: RESOLVED FIXED

(Testing :: Talos, enhancement)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

tp7 will be tp6++, a replacement for Hasal features that we find beneficial, supporting multiple browsers, and reference hardware.

Here is what is currently known to be next steps:
1) create web-extension that will take configuration input from a control webserver via xhr, load pages as specified in the configuration, collect metrics as specified in the configuration, and post results to the control webserver via xhr.  This should work in Firefox trunk and Chrome.
2) update mitmproxy recorded webpages (amazon.com, youtube.com, google search, facebook.com) to be most relevant versions of existing pages along with in page navigation as part of the recording.  In addition add new page for google docs.  While we would like to add gmail transaction, we had much difficulty in the past
3) document/script the process for recording webpages so this is easy to update on a regular schedule and access replay locally
4) add hero elements (basically observable events) to the recorded webpages when we hit sections of the pageload we want to measure.  There will be multiple elements per page and will be named to be unique.  This can be done with mitmproxy scripts to replace specific javascript functions (like onmouseover) to emit an event.
5) add the ability to support google chrome in our automation- this requires obtaining a copy of it regularly (probably nightly), supporting installing it on test machines (linux/osx/windows), configuring it to have specific preferences (turn off services like we do for Firefox) and test web extensions
6) add new control server+mitmproxy setup to work from the talos harness
7) adjust perfherder or find a way to use existing functionality to store multiple measurements per page as well as data from Google Chrome browser.
8) build a dashboard to show multiple datapoints and easy comparison between firefox/chrome for each of the scenarios
9) build a small pool (maybe 20) of reference hardware laptops that run windows10 and represent "average" hardware that many users would see.
10) add in page navigation to the scenarios instead of just page load.  This could be done with Marionette, Sikulu, or other tools.  This would require additional elements to be added to the recordings to support the full scenarios.

Of the above steps, I see 1-8 as being a top priority which means we will continue to focus on these steps until we have something running (early Q2).  Steps 9+10 will require more coordination and experimentation (think end of Q2)

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:jmaher is there anything remaining here? It seems that most of the requirements are covered by Raptor.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

agreed, the core work here is to get a new pageset with modern measurements, that is done.