1413809 - investigate the value of generating a win32->win64 migration partial for 56.0->57.0 users

Status: RESOLVED WONTFIX

(Release Engineering :: Release Requests, enhancement)

Description

[Jordan Lund (:jlund)]

In bug 1405681 and bug 1411428 we generated partials outside of automation for win64 migrated users.

We should evaluate whether this should be done for 56.0->57.0 and discuss ways to reduce risk and plan ahead.

Comment 1

[Jordan Lund (:jlund)]

@bhearsum, @cpeterson - I saw in IRC you discussed this a bit. Do we have numbers for how many win32 users this would affect if we did not provide a partial? If not, could you point me in the right direction to find that out?

thanks in advance!

Comment 2

[[e-mail] (:bhearsum)]

(In reply to Jordan Lund (:jlund) from comment #1)
> @bhearsum, @cpeterson - I saw in IRC you discussed this a bit. Do we have
> numbers for how many win32 users this would affect if we did not provide a
> partial? If not, could you point me in the right direction to find that out?
> 
> thanks in advance!

I didn't end up doing anything here after jlorenzo told me that Chris already shared some numbers on release-drivers. If you want to do some querying yourself, this is well documented on https://wiki.mozilla.org/Balrog#Public_app.

Comment 3

[Chris Peterson [:cpeterson]]

(In reply to Jordan Lund (:jlund) from comment #1)
> @bhearsum, @cpeterson - I saw in IRC you discussed this a bit. Do we have
> numbers for how many win32 users this would affect if we did not provide a
> partial? If not, could you point me in the right direction to find that out?

Based on crash report ADIs [1], I estimate that roughly 9M users are still eligible for 64-bit migration. These are 64-bit Windows OS users running 32-bit Firefox <= 55.0.3 (about 3M) or exactly 56.0 (about 6M). (The crash report ADIs do not split out Win32 and Win64 OS users, so I multiplied the Windows ADIs by 0.70 because roughly 70% of Windows Firefox users run Win64 OS.)

I expect that many of these 56.0 users will be updated to 56.0.2 by the time 57.0 is released (November 14), making a 56.0 -> 57.0 migration partial less important. OTOH, these stragglers updating from Firefox <= 56.0 to 57.0 need all the help they can get to make updating easier, which would be an argument in favor of a migration partial.

I suggest that we watch the crash report ADIs [1] over the next week to see how quickly 56.0 users update to 56.0.2. We can decide then whether we want a 56.0 -> 57.0 migration partial. My hunch is that we can get away *without* the partial. If you need a decision sooner, then I say let's not bother generating the migration partial.

catlee said a partial mar from 32-bit 56.0 to 64-bit 57.0b12 is about 25MB in size. The complete update for 57.0b12 is about 40MB.

[1] https://crash-stats.mozilla.com/crashes-per-day/?p=Firefox&v=54.0.1&v=55.0.3&v=56.0&v=56.0.2&hang_type=any&os=Windows&date_start=2017-10-18&date_end=2017-11-02&submit=Generate

Comment 4

[Chris AtLee [:catlee]]

The partial mar from 32-bit 56.0 to 64-bit 57.0 RC1 is the same size as we estimated before - about 25MB vs 40MB for the complete.

Comment 5

[Jordan Lund (:jlund)]

Thanks for the findings catlee, cpeterson,

Based on this, I think we should do the following:

Proceed with generating the 56.0->57.0 partial because we shouldn't let releng technical requirements be a blocker here. However, we should leverage the fact that it makes minimal difference if we do it in time for Nov 14th.

So regardless if we use RC2 (gtb thurs) or require RC3 (~sun if needed), we wait until after Nov 14th, generate the partial next week, update the 57.0 migration blob, then have QE do a separate sign off for just 57.0 migrating users.

This reduces risk and work for 57 while still helping remaining eligible users migrate in time for when we go live at 100%.

As a bonus, this would allow more time for eligible migrating users to migrate before 57. And if the numbers are much less than 9million next week, we might not even need to do this at all.

Does that sound sane? Have I missed something?

Comment 6

[Chris Peterson [:cpeterson]]

This sounds like a good plan! We know these 9 million win32 users <= 56.0 do not update quickly (otherwise they would already be running 56.0.2), so:

1. They probably won't fetch a 57.0 full update during the throttled rollout.

2. But when they do get an update after throttling, the 25MB migration partial from 56.0->57.0 will improve the chance they get 57.0 sooner compared to a 40MB full update.

btw, my estimate of 9M win32 users <= 56.0 was an slight estimate because I inadvertently included Mac and Linux users. <:)

A closer estimate is about about 8.5M = 13M users <= 56.0 * 87% running Windows * 79% running 64-bit Windows * 95% with >= 3GB RAM (the migration minimum memory requirement).

Comment 7

[Jordan Lund (:jlund)]

57 is out! Let's evaluate if we need this.

cpeterson, could you have a look at numbers again to help us determine how many people we need to help?

Comment 8

[Jordan Lund (:jlund)]

(In reply to Jordan Lund (:jlund) from comment #7)
> 57 is out! Let's evaluate if we need this.
> 
> cpeterson, could you have a look at numbers again to help us determine how
> many people we need to help?

perhaps we should wait until we have some numbers after 100% roll out.

Comment 9

[Chris Peterson [:cpeterson]]

(In reply to Jordan Lund (:jlund) from comment #7)
> cpeterson, could you have a look at numbers again to help us determine how
> many people we need to help?

Based on crash ADIs, I estimate that we now have about 5M 32-bit users running Firefox <= 56.0 that are eligible to be migrated to 64-bit 57.0.

I don't think we need to bother creating a migration partial for 32-bit 56.0 to 64-bit 57.0. These 5M 32-bit users have had three weeks to update to 64-bit 56.0.2 and they are still running 56.0 or older. They won't be rushing to update to 57.0 so a full update is probably good enough.

Comment 10

[Chris Peterson [:cpeterson]]

So feel free to WONTFIX this bug unless someone on your team has free time on their hands to create the migration partial. :)

Comment 11

[Jordan Lund (:jlund)]

okay, thanks for looking into that. I suppose one benefit is we can do this at any time. Feel free to reopen if things change.