1819740 - [meta] Use a "nightly as release" build on autoland instead of the nightly build (desktop)

Status: NEW

(Testing :: Performance, task, P3)

Description

[Greg Mierzwinski [:sparky]]

This task is for switching our autoland nightly builds to use a “nightly as release” (NaR) build. can provide information for where to get that build, and how to build it.

We’ll only need to replace the desktop builds on autoland, no where else (e.g. mozilla-central stays the same).

To start, we should enable all of our performance tests on these builds and run them for 2 weeks to see how stable they are. These should run at tier-3 to start. If we find that they are good, then we can disable the nightly variant tests and leave the new NaR build tests (and increase the tiers of the tests back to normal).

We’ll need to ensure that dashboards are updated to use this new build where appropriate as well.

Comment 1

[Greg Mierzwinski [:sparky]]

:emilio, can you provide information about how I could run this nightly-as-release build in CI, or locally?

Comment 2

[Emilio Cobos Álvarez [:emilio]]

Hey, sure. So for a local build you just need to add this to your mozconfig file:

ac_add_options --as-milestone=release

I don't think we have any mozconfig in the tree doing that but adding it should just work.

You can confirm it works looking at the version in the "About Nightly" window or so.

Comment 3

[Greg Mierzwinski [:sparky]]

Excellent, thanks!

Comment 4

[Greg Mierzwinski [:sparky]]

Something to note is that if we proceed with a change like this for autoland, it'll further diverge local results from CI results. I wonder if we need to start considering providing performance testing builds locally. We already have this to a certain extent with our perf-specific preference settings so this could be a continuation of that in a sense.

This brings up another question though which is if we need to consider functional testing on this build, or if testing on the other builds will be enough? For example, see the functional tests running on the same shippable build as the performance tests: https://treeherder.mozilla.org/jobs?repo=mozilla-central&searchStr=linux1804-64-shippable-qr

Comment 5

[Greg Mierzwinski [:sparky]]

:emilio, could you check if this build looks like what we're expecting? I've got a try run here with what might be a nightly-as-release build, but I'm not entirely sure what to expect from the about firefox window: https://treeherder.mozilla.org/jobs?repo=try&revision=296ae88de7f8a2618be8446f04b188f0185e1182

I have another try run here without artifact mode: https://treeherder.mozilla.org/jobs?repo=try&revision=4684f80c48b4aa16816586da9d42ce51d560e7b4

Comment 6

[Greg Mierzwinski [:sparky]]

Attachment: about firefox screenshot

Here's a screenshot of what I'm seeing on the page when I download the build, and run it locally.

Comment 7

[Greg Mierzwinski [:sparky]]

Something else to note is that the mozconfig this build is using inherits from the nightly one which adds the flag --with-branding=browser/branding/nightly

See my changes here: https://hg.mozilla.org/try/rev/6928211d658e6b16bddf8c105c3977f4c177c45d#l1.5

Comment 8

[Emilio Cobos Álvarez [:emilio]]

Yep, that looks good. Version is 112.0, instead of 112.0a1. On the non-artifact build you should also see nightly-only flags turned off, e.g. apz.record_checkerboarding should be true on nightly but false on nightly-as-release (as per here).

Comment 9

[Greg Mierzwinski [:sparky]]

Attachment: Screenshot from 2023-03-07 15-58-57.png

Nice, thanks for the quick response! I can confirm that I'm seeing that the pref is set to False on the non-artifact build (see attached screenshot).