1480576 - Rollout Windows Restart Manager to all Beta 62 via Normandy

Status: VERIFIED FIXED

(Core :: Widget: Win32, enhancement, P1)

Description

[Michael Cooper [:mythmon]]

To build confidence in the Release 61 rollout of Windows Restart Manager, we should use the same rollout mechanism to turn on the feature in Beta 62 as well.

We'll do this by creating a Normandy recipe targeting all builds of Beta 62.

The main benefit of this is that QA can test the feature exactly as it will be rolled out in Release, instead of having two different mechanisms for turning on the feature.

Once the recipe is created, QA can use the Normandy Devtools [0] to manually apply it, to test that the feature applies and acts correctly.

[0]: https://github.com/mozilla/normandy-devtools

Comment 1

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Hi Andrei, Tom, in addition to turning on Windows Restart manager with a pref flip (from bug 1480144), we also decided to turn on this feature via Normandy for beta users. Can your team verify that this recipe works as expected on Beta channel? We also need to do a smoke test on the feature before 62.0b14 sign off.

FYI CPeterson, Romain, Adam.

Comment 2

[Michael Cooper [:mythmon]]

The recipe has been created, approved by relman, and published. It is now active for 100% of Beta 62 (all builds).

Delivery Console (for users with access): https://delivery-console.prod.mozaws.net/recipe/532/
Public API: https://normandy.cdn.mozilla.net/api/v3/recipe/532/

To test:

Install the Normandy Devtools add-on [0]. Open them with the icon in the toolbar. Find the recipe in question in the list ("    Rollout Firefox Windows Restart Manager - Beta 62 [Bug 1480576]"). Click on the play icon on the right side of the recipe's row. Normandy will fetch and apply the recipe silently. From now on, the feature should be enabled.

[0]: https://github.com/mozilla/normandy-devtools

Comment 3

[Michael Cooper [:mythmon]]

Enrollment data for the rollout is available here: https://dbc-caf9527b-e073.cloud.databricks.com/#job/499/run/latestSuccess/resultsOnly

After about an hour in the field, we're up to over 12,000 enrollments and accelerating.

The graphs there should update once an hour. Note that the daily graph that shows the percent of enrollment won't be available until after UTC midnight.

Comment 4

[Andrei Vaida [:avaida]]

(In reply to Ritu Kothari (:ritu) from comment #1)
> Hi Andrei, Tom, in addition to turning on Windows Restart manager with a
> pref flip (from bug 1480144), we also decided to turn on this feature via
> Normandy for beta users. Can your team verify that this recipe works as
> expected on Beta channel? We also need to do a smoke test on the feature
> before 62.0b14 sign off.

Thank you for letting us know. Clearing our needinfos here, since Abe has already tested both the Normandy rollout and the actual functionality of the Windows Restart Manager on 62.0b13.

Abe, could you please post your test results here, for posterity? Thanks!

Comment 5

[Abe - QA (:Abe_LV)]

We tested windows restart manager pref flip on windows 10, preview,7 and 8 with beta 62.0b13. The pref, "toolkit.winRegisterApplicationRestart",is set to true by default (flipped) successfully by Normandy Recipe. We also conducted smoke tests on the feature and it works as intended. All our smoke test runs are green.

Test runs are here- https://testrail.stage.mozaws.net/index.php?/plans/view/11519

Comment 6

[Michael Cooper [:mythmon]]

Data for this rollout is available here: https://dbc-caf9527b-e073.cloud.databricks.com/#job/500/run/latestSuccess

It includes an hourly summary of total enrollment (users that have ever enrolled), as well as a daily summary of active enrollment (beta users with the rollout active right now).

Enrollment seems good so far. There is an expected amount of "graduate" and "would-be-no-op" events happening, as Beta 14 becomes more widespread. I'm going to modify this recipe to not apply to beta 14 and above, to quiet those events.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

Bug 1481278 is tracking this for 61. Updating the status accordingly.

Comment 8

[Abe - QA (:Abe_LV)]

This is verified based on comment 5. Closing it as verified-fixed.