1874961 - Run nightly-as-release speedometer3 on mozilla-central instead of autoland

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P2)

Description

[Markus Stange [:mstange]]

We build Nightly-as-release builds on both mozilla-central and on autoland, but we only run perf tests on these builds on autoland.

On autoland, perf tests don't report replicates. This means that, in the graph for Speedometer 3 on Windows for Nightly-as-release, we only get one dot per build even though we run Speedometer 3 five times per build and so would like to have five dots per build.

Can we run perf tests on mozilla-central instead? Or alternatively get replicates enabled on autoland?

Comment 1

[Markus Stange [:mstange]]

The backstory is that this graph comparing Nightly with Nightly-as-release shows Nightly quite a bit ahead, and replicates might make an accurate assessment a bit easier.

Comment 2

[Greg Mierzwinski [:sparky]]

We can make the switch to mozilla-central. :mstange, is there any interest in having regression detection for tests using this build? If not, then we can disable it on autoland, but otherwise, we could run the tests on both branches.

Comment 3

[Markus Stange [:mstange]]

(In reply to Greg Mierzwinski [:sparky] from comment #2)

We can make the switch to mozilla-central. :mstange, is there any interest in having regression detection for tests using this build? If not, then we can disable it on autoland, but otherwise, we could run the tests on both branches.

I don't have a strong opinion either way. Both solutions would work for me.

Comment 4

[Florin Bilt]

Attachment: Bug 1874961 - Run nightly-as-release speedometer3 on mozilla-central instead of autoland r?#perftest

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/223dbcb9e4a5
Run nightly-as-release speedometer3 on mozilla-central instead of autoland r=perftest-reviewers,sparky

Comment 6

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/223dbcb9e4a5