Closed Bug 1533423 Opened 8 months ago Closed 8 months ago

Raptor cold page-load tests on Firefox android geckoview

Categories

(Testing :: Raptor, enhancement, P1)

Version 3
enhancement

Tracking

(firefox68 fixed)

RESOLVED FIXED
mozilla68
Tracking Status
firefox68 --- fixed

People

(Reporter: rwood, Assigned: rwood)

References

(Blocks 2 open bugs)

Details

Attachments

(1 file)

Add Raptor support to run the tp6m page-load test suites on the android geckoview_example app in cold-load mode i.e. restart the geckoview browser app between each page-cycle.

Raptor page-load tests will default to running in the new cold page-load mode, but will support running in the (current) warm page-load mode by adding 'warm = true' to the tp6m* test INI. Running in warm page-load mode will add '-warm' to the test name so the results can be differentiated in Perfherder. This way we won't be doubling the current Raptor android jobs running in production when this lands.

I guess the number of Raptor android jobs is a bottleneck?

It would be useful to have both cold and warm loads recorded since there can be independent regressions / improvements in either.

(In reply to Andrew Creskey from comment #2)

I guess the number of Raptor android jobs is a bottleneck?

It would be useful to have both cold and warm loads recorded since there can be independent regressions / improvements in either.

Yes correct due to device resources, see https://bugzilla.mozilla.org/show_bug.cgi?id=1529212#c8

(In reply to Robert Wood [:rwood] from comment #3)

Yes correct due to device resources, see https://bugzilla.mozilla.org/show_bug.cgi?id=1529212#c8

Gotcha, thanks Rob.

(In reply to Robert Wood [:rwood] from comment #1)

Raptor page-load tests will default to running in the new cold page-load mode, but will support running in the (current) warm page-load mode by adding 'warm = true' to the tp6m* test INI. Running in warm page-load mode will add '-warm' to the test name so the results can be differentiated in Perfherder. This way we won't be doubling the current Raptor android jobs running in production when this lands.

Correction, I'll do the opposite. Have 'cold = true' in the INI file, and add '-cold' to the test names. Reason is we want to be able to keep warm page-load results having the same existing test names, so that we can keep comparing with the existing / historical data in Perfherder.

Blocks: 1537941
Blocks: 1537944
Pushed by rwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6b770c5d755a
Raptor cold page-load support for Firefox android geckoview; r=jmaher
Status: ASSIGNED → RESOLVED
Closed: 8 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
Blocks: 1538786
Blocks: 1539474
Depends on: 1540241
You need to log in before you can comment on or make changes to this bug.