1573521 - Test Enhanced Tracking Protection on by default versus current state (Tracking Protection on by default) before ship this feature.

Status: RESOLVED INACTIVE

(Testing :: Performance, task, P3)

Description

[Jean Gong :jgong]

Request fro Fenix team to test Enhanced Tracking Protection(ETP) on by default VS current state (TP on by default) in terms of performance before we ship this feature. Goal is to get the feature out to beta on 8/20, but testing could continue there if necessary. We're expecting a release to production date of 9/3.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Assuming you want to compare cold page load measurements of live sites with ETP on against the same with just TP on, I believe you can already do this by running Raptor tests against Fenix on try. When pushing to try, the latest Fenix build from the GitHub master branch is used. Preferences can be set to enable ETP (example), and live sites can be used via an optional command line argument. Without live sites enabled, Raptor uses pre-recorded requests and responses, therefore TP/ETP are unlikely to make much of a difference to the metrics (though I have no evidence of this).

I would suggest selecting a subset of sites and triggering try builds for baseline (TP on) and comparison (ETP on), and retriggering at least 5 times so are able to take the noise into account for the comparisons. You can use Perfherder's compare view to see the impact of your change. You might want to run the tests on both Moto G5 and Pixel 2, which are both available in our device lab.

Apologies if some of my assumptions are incorrect, the request is very late notice and lacking in necessary details. I'm going to be PTO from 15th-26th August. If you need any assistance with the above please reach out to :rwood.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dave, I assume this is not relevant anymore?

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Indeed. All the information to run such a test is in comment 1.