1355545 - consider running microsoft's BrowserEfficiencyTest in automation

Status: NEW

(Testing :: General, enhancement, P3)

Description

[Ben Kelly [:bkelly, not reviewing]]

Microsoft has been steadily improving the battery efficiency of edge.  One of the tools they are using to do this is a defined test suite:

https://github.com/MicrosoftEdge/BrowserEfficiencyTest/

While battery is not an immediate priority, perhaps we should consider getting this into automation at some point.  Once its there we can more easily make incremental improvements and at least understand if we are regressing.

Not sure what component to put this in.

Joel, what do you think?

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

thanks :bkelly for recommending this.  One interesting thing here is this requires xperf, wpr (windows performance recorder) and Elevator.  Luckily for windows 7 we have xperf installed and use that, so this might not be so scary.

In looking through the documentation I see that this navigates to live websites- that is not ideal for CI testing and we have locked down our machines to not access the real world.  I am sure this could be modified to load local pages, and likewise be automated to setup proper profiles, process management, and logging so that we could interpret it in treeherder and perfherder.

What specific gains would we get from this?  Maybe running this manually on a few versions of Firefox and seeing what numbers change could help us determine what value we might get from this?  Who would be responsible for this and helping out with regressions when they arise?

Would this be more important than writing new benchmarks to measure specific incoming features?  I like the idea of a end to end test.