Lowered frame rate for low-end hardware analysis
Categories
(Data Science :: General, task)
Tracking
(Not tracked)
People
(Reporter: mconley, Assigned: flawrence)
References
Details
Brief description of the request:
The Firefox Desktop team would like to test whether or not lowering the default frame rate of the browser when running on low-end hardware will improve overall page load time and user perception of performance, while also not reducing retention.
Our hypothesis is that users that have machines with 2 or fewer cores, with clock speeds of 1.8Ghz or less will experience faster page loads if we reduce the global frame rate of the browser, as we believe this will free up the CPU to do more work to load web pages, and less work to animate things.
Link to any assets:
Is there a specific data scientist you would like or someone who has helped to triage this request:
None.
|
||
Updated•5 years ago
|
|
Assignee | |
Comment 1•5 years ago
|
||
It will be fairly straightforward to measure the impact on load time, and any large impacts on retention.
It will be difficult to measure the impact on user perception of performance, and the impact on page breakage. If either of these effects are really big then they might impact retention (positively or negatively) and be visible that way. But there's a definitely a potential for this change to make users happier or less happy in a way that we care about but can't detect through the load time or retention metrics.
If there are other performance metrics than page load (e.g. something that measures jank) then that would help.
Anti-tracking and I have been developing an add-on for detecting page-breakage caused by the change being tested. If you really want to be sure that you're not breaking more pages, then we should consider re-using that methodology here - which of course would delay things so it might not be an option?
If you're not too worried about the risks, then we could just go with the simple experiment, roll it out if the results are not alarming, and be prepared to roll it back if the amount of breakage is in the range where it didn't show up in the experiment's retention metrics but does generate bug reports when the feature is fully rolled out.
|
|
Assignee | |
Comment 2•5 years ago
|
||
Experimenter link. N.B. I made up placeholder dates.
|
|
Assignee | |
Comment 3•5 years ago
|
||
|
|
Assignee | |
Comment 4•5 years ago
|
||
Power analysis redux, now that we're targeting beta not release.
|
Reporter | |
Comment 5•5 years ago
|
||
Hi Felix,
Gijs and I have been getting some assistance from denispal from vchin's team to analyze our patch in a lab setting, and at this point, we're not convinced that there's anything here that's worth actually testing on our user population. I think it's safe to cancel this experiment for now. Sorry for the run-around!
Is there anything else we need to do to mothball this for now?
|
|
Assignee | |
Comment 6•5 years ago
|
||
No worries - glad a conclusion was reached without having to go to the effort of a full experiment.
Description
•