Open Bug 1541488 Opened 6 months ago Updated 3 months ago

Gradual rollout of WebRender in release 67

Categories

(Core :: Graphics: WebRender, task, P1)

task

Tracking

()

Tracking Status
firefox67 + fixed

People

(Reporter: jrmuizel, Assigned: mythmon)

References

(Blocks 1 open bug)

Details

Aiming for full rollout in 1 week with this plan in mind:
Feature is off by default at mainline release go-live/launch.
Turn on feature at 25% update Monday post go-live.
Push to 50% 3 days later, Thursday
Push to 100% a week later IF there are no blocking issues.

Pascal, does this seem like a reasonable plan?

Flags: needinfo?(pascalc)
No longer depends on: 1541489
Type: defect → task
Priority: -- → P1

The pref to change from false to true to do the rollout is gfx.webrender.all.qualified.

(In reply to Jeff Muizelaar [:jrmuizel] from comment #1)

Pascal, does this seem like a reasonable plan?

sgtm, thanks.

Flags: needinfo?(pascalc)
Assignee: nobody → mcooper

I've prepared a Normandy recipe for this rollout. It is not automatically scheduled, so we will need to enable it during release week, and manually update it to increase the population size. I chose a manual schedule because of the impact of the feature: I think it's likely we'll want to tweak the schedule by hand anyways. If not, I can implement an automatic schedule as well.

I have not requested review for the recipe yet.

Normandy API: https://normandy.cdn.mozilla.net/api/v3/recipe/755/history/
Delivery Console: https://delivery-console.prod.mozaws.net/recipe/755/

Heads-up that we want to run a concurrent experiment in 67 to validate performance and stability in bug 1526094, with WR-enabled and WR-disabled branches for qualified users. That experiment will target the same pref. The experiment and the rollout might need to be aware of each other.

In light of comment 5, I propose that we modify the schedule of this rollout slightly. The experiment would occupy the range from 95% to 100%, making a total of a 5% sample. The rollout would start as 0%-25%, then 3 days later it would be 0%-50%, and finally a week later it would be 0%-95%. When we end the experiment in mid june, we can then expand the rollout to 100%.

The rollout is live at 25%. I'll update the recipe on Thursday to bump to 50%.

I have updated the recipe from 25% to 50%, and requested approval. Ritu, can you review? https://delivery-console.prod.mozaws.net/recipe/755/

Flags: needinfo?(rkothari)

I confirmed with Mythmon that we are seeing the expected level of activity with the 25% rollout so far, before going to 50%. He confirmed ~13.8M enrollments, and no significant amount of other events.

With that I have approved the recipe, lgtm.

Flags: needinfo?(rkothari)

I would like to hold off on bumping up the remainder of the rollout until Monday the 10th, just to make sure nothing pops up over the weekend.

The rollout continues to look healthy. We've seen ~52.5M enrollments, with a negligible error rate. I've queued a bump to a 95% sample rate for this rollout (as discussed in comment 6).

Ritu, can you review the change? https://delivery-console.prod.mozaws.net/recipe/755/

For reference, the experiment recipe that is taking up the 95%-100% range is the below. It is scheduled to end tomorrow.

Flags: needinfo?(rkothari)

Done. Approved. Thanks (as always) for sharing the recipe enrollment numbers.

Flags: needinfo?(rkothari)
See Also: → 1562966
You need to log in before you can comment on or make changes to this bug.