Open Bug 1556748 Opened 2 years ago Updated 1 month ago

Implement balrog rule to update 64-bit Windows OS users who are running 32-bit Thunderbird

Categories

(Thunderbird :: Build Config, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

People

(Reporter: rjl, Assigned: rjl)

References

(Blocks 1 open bug)

Details

Attachments

(6 files, 4 obsolete files)

Attached file win32Towin64_balrog.py (obsolete) —

The way this worked for Firefox, there's two Balrog rules:

  • Rule 685 acted as a watershed for all Windows users to install FF 56.0
  • Rule 681 then migrated only users running 56.0 and met the other criteria

Criteria for migrating to 64-bit:

  • Did not opt out with the registry setting created by the installation of 56.0
  • Running 32-bit app on 64-bit Windows
  • Have more than 2G of RAM

It looks like we need a watershed for Windows users on the next beta so the updater can set that registry key. We'll have to do the same for the release branch as well.

I've modified the script that releng used for Firefox to mangle the Balrog blob for a given release. It removes the non-windows data and sets the mapping for win32 on x64 users to get the 64 bit version.

For reference, this is the watershed rule that forced all users to install FF-56.0.

For reference, this is the rule that did the actual migration for eligible users.

Attached file TB_win32Towin64.md

Some notes on how to set up Balrog

The URLs that get generated by the Thunderbird updater should look something like (from a recent nightly):

https://aus5.mozilla.org/update/6/Thunderbird/69.0a1/20190526233640/WINNT_x86-msvc-x64/%LOCALE%/%CHANNEL%/Windows_NT 6.1.1.0.7601 (x64)/ISET:SSE4_2,MEM:32101/default/default/update.xml?mig64=1

The important pieces from a Balrog perspective:

  • build target: WINNT_x86-msvc-x64 that indicates an x86 build running on x64 OS
  • RAM: MEM:32101 (in MB, must be > 2048 to trigger the upgrade)
  • opt-in: mig64=1

I've got this working on the "nightlytest" update channel. You have to start off with a build < 20190604091847. You will get two updates, the first will go to Build 20190604091847 which will set the magic registry value, then on the next update if you meet the criteria you should go to Build 20190605101101 for win64.

I can put this on the regular nightly channel if desired.

Do we need to set the magic registry value also on the next, or one of the next, TB 60 updates to make it possible to update from 32bit TB 60 to 64bit TB68?

(In reply to Richard Marti (:Paenglab) from comment #5)

Do we need to set the magic registry value also on the next, or one of the next, TB 60 updates to make it possible to update from 32bit TB 60 to 64bit TB68?

Is there a process in place to go from 32 bit TB60 to 64 bit TB60? THAT would be the most logical first step; to only change one thing at a time. Otherwise, if something breaks, there are too many variables.

We could flip the registry bit for esr60 on the next release to get the process going. It doesn't do anything except send another parameter to the update server. Balrog will ignore it and continue sending win32 downloads unless we tell it otherwise.

Attached file win32Towin64_balrog.py

Updated script, works with a little less interaction from user now.

Attachment #9069658 - Attachment is obsolete: true

Just updated to 60.8.0 (32-bit). Still 32 bit.

(In reply to Worcester12345 from comment #9)

Just updated to 60.8.0 (32-bit). Still 32 bit.

Patience grasshopper. This bug is not fixed, so of course this is expected. Migration won't happen until the balrog piece described by this bug is done.

I think I read comment 4 and comment 5 together very quickly, and somehow thought this was done. Thanks for working on it.

This test document assumes a watershed for Windows users at 60.8 and that the win64 migration happens on 68.2.
I just verified that the script to create the special migration release in Balrog still works.

Attachment #9091141 - Flags: feedback?(vseerror)
Comment on attachment 9091141 [details]
Win32 to Win64 Automigrate Testing.html

I think win64 for 68.2 is reasonable, so SGTM.  I'd just double check with Magnus, so adding NI.

We still have Bug 1509918 - "use thunderbird as default client" prompt on every startup on WIndows 7 using TB 64bit.  However the issue also appears with 32bit Thunderbird so I don't think it blocks this bug - but we should still commit to fixing it.
Attachment #9091141 - Flags: feedback?(vseerror)
Attachment #9091141 - Flags: feedback?(mkmelin+mozilla)
Attachment #9091141 - Flags: feedback+
Comment on attachment 9091141 [details]
Win32 to Win64 Automigrate Testing.html

Looks good. 
While testing that migration works, can we also at the same time add an item to check that it found the existing profile after migration?
Attachment #9091141 - Flags: feedback?(mkmelin+mozilla) → feedback+

This title looks whacky.
"Implement balrog rule to update 32-bit Windows Thunderbird users running 64-bit OS"

Is this what it really wants to say?
"Implement balrog rule to update 64-bit Windows OS users who are running 32-bit Thunderbird"

Summary: Implement balrog rule to update 32-bit Windows Thunderbird users running 64-bit OS → Implement balrog rule to update 64-bit Windows OS users who are running 32-bit Thunderbird

Updated testing documentation version numbers, and added notes to check that existing profiles are used.

Attachment #9091141 - Attachment is obsolete: true

Last minute notes about testing setup. Scenario C can be tested effectively.

Attachment #9104031 - Attachment is obsolete: true
Attached image image.png

release-localtest is configured with automigration. The migration version being tested is 68.2.0. When 68.2.1 is released I will make the appropriate changes and copy to the regular release channel.

I've completed my testing of automigration using the release-locatest channel. Everything looks good to me, profiles remained in tact.
The opt out methods both work. I've updated the testing documentation with the actual registry keys that need to be set. Bypassing migration by installing the 32-bit version of 68.2.1 will be easiest for most people though.

Attachment #9104322 - Attachment is obsolete: true

What is left to get this bug closed out?

(In reply to Worcester12345 from comment #20)

What is left to get this bug closed out?

In general terms ... more important stuff which has emerged (profile issues), moving more users from 60 (because we don't want to move users from 60 and 64bit at the same time), and retesting. That's probably an incomplete list.

It's fluid and dynamic, and so it's not possible to give more specifics, dates or version numbers. And, to be clear about relative priorities, shipping 64bit hasn't been and still is not critical.

Blocks: 1598866
No longer blocks: 1598866
Duplicate of this bug: 1598866

(In reply to Wayne Mery (:wsmwk) from comment #21)

(In reply to Worcester12345 from comment #20)

What is left to get this bug closed out?

In general terms ... more important stuff which has emerged (profile issues), moving more users from 60 (because we don't want to move users from 60 and 64bit at the same time), and retesting. That's probably an incomplete list.

It's fluid and dynamic, and so it's not possible to give more specifics, dates or version numbers. And, to be clear about relative priorities, shipping 64bit hasn't been and still is not critical.

Are there any statistics on how many users are still on version 60? At what point does 64 bit BECOME more "critical"?

(In reply to Wayne Mery (:wsmwk) from comment #21)

(In reply to Worcester12345 from comment #20)

What is left to get this bug closed out?

In general terms ... more important stuff which has emerged (profile issues), moving more users from 60 (because we don't want to move users from 60 and 64bit at the same time), and retesting.

Would it be safe to say that the "move users from 60" is mostly done now?

(In reply to Worcester12345 from comment #25)

Are there any statistics on how many users are still on version 60? At what point does 64 bit BECOME more "critical"?

?

Duplicate of this bug: 1618961

Please add key word: "support-win64-thunderbird" from duplicate bug. Thank you.

(In reply to Worcester12345 from comment #28)

Please add key word: "support-win64-thunderbird" from duplicate bug. Thank you.

That's already used by bug 634233. Besides, we don't need an alias for this bug.

I think the original intent of this bug was to be identical to:

Migrate 32-bit Firefox (WOW64) users to 64-bit Firefox (Win64)
https://bugzilla.mozilla.org/show_bug.cgi?id=1274659

except for Thunderbird.

OK, what is still holding this bug back from installing 64 bit Thunderbird? Based on numerous crashes, is it time to move this forward?

From the submitted attachment 11 [details] months ago:
"The current plan is to enable automigration with the 68.2.1 release"

(In reply to Wayne Mery (:wsmwk) from comment #21)

(In reply to Worcester12345 from comment #20)

What is left to get this bug closed out?

In general terms ... more important stuff which has emerged (profile issues), moving more users from 60 (because we don't want to move users from 60 and 64bit at the same time), and retesting. That's probably an incomplete list.

So are users off 60 yet? Retesting done?

It's fluid and dynamic, and so it's not possible to give more specifics, dates or version numbers. And, to be clear about relative priorities, shipping 64bit hasn't been and still is not critical.

Another year has passed. And it was critical enough that you suggested it to me as a solution to many crashes I was having, and it actually did work as you suggested it might. I think it is time to ship this out. Put it on the schedule, so it will be ready before Halloween, or at the worst, Columbus Day. Thank you.

I recall providing a potential time line in a different bug report, so I don't see the value in asking the same question in a different bug report or setting arbitrary dates. Also note that migrating users to 64bit will itself create some initial instability for some users - at least that is what we see in support circles, some users encounter profiles issues. So additional testing will be required beyond what has already occurred.

I understand your passion for this issue. But those driving releases do not have capacity to do everything we would wish beyond the most important - currently that is fixing important version 78 bugs, bringing users to version 78, and solidifying Openpgp functionality, to name a few. Do you have an analysis and data which shows what specific user segment will benefit from 64bit, and that indicates how critical that need is?

(In reply to Wayne Mery (:wsmwk) from comment #34)

... I don't see the value in... setting arbitrary dates.

It is not arbitrary. It is a solid date, a month or two out. Most of the work is already done as of long ago.

I understand your passion for this issue.
Only from the frustration of repeated crashes. I wish you to spare others the same agony.

But those driving releases do not have capacity to do everything we would wish beyond the most important - currently that is fixing important version 78 bugs, bringing users to version 78,

But isn't this PRECISELY what you said about "moving more users from 60", literally right above here?

Do you have an analysis and data which shows what specific user segment will benefit from 64bit, and that indicates how critical that need is?

I'm sorry, I am not a software analyst nor a programmer. I am a volunteer "Tester", who is helping you with this little project. I was suckered into believing it was a "community project" or something like that, where everybody contributes somehow and does what they can to help out. I've sent in my share of bugs, crashes, suggestions, etc. over the years; for both Thunderbird, and Firefox, and even "Seamonkey" and "Sunbird" and "Lightning". As I said, it was YOUR suggestion for me to try 64-bit, which helped a lot.

Rob, I'm reevaluating whether we have any outstanding issues in bug 634233, etc. Assuming we find nothing major, and after retesting various scenarios users might encounter ...

For beta, what do you think about doing this during the 87 beta cycle, i.e. in March? (not at release, but during the cycle)
For release, what do you think about doing this ~8 weeks later during the 78.10, i.e in May. (not at release, but during the cycle)

In both cases, we probably want to avoid doing this very close to some other major change, for example enabling a major portion of e10s.
Is there any aspect that we'd want to discuss on tb-planning?

Flags: needinfo?(rob)

That timeline sounds good. I just verified that the 32to64 migration registry keys are set in current win32 on x64 installs and are sending the "mig64=1" parameter to Balrog.

During the release itself, after promotion but before calls for testing go out, I will need to mangle the release blob in Balrog. Then testing can proceed via the -localtest/-cdntest channels.

One thing I want to clarify: Thunderbird must have been installed using the installer. That's where the registry key to enable is created. ZIP installs don't get the registry key until they go through at least one upgrade.

Flags: needinfo?(rob)

Are you going to use the same process as Firefox to do this?

If so, you would need a Thunderbird version of this bug, among others:
https://bugzilla.mozilla.org/show_bug.cgi?id=1340936

No need to reinvent the wheel here, just follow the same process and steps.

We don't have a stub installer.

(In reply to Rob Lemley [:rjl] from comment #39)

We don't have a stub installer.

Here's the bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1558573

(In reply to Wayne Mery (:wsmwk) from comment #36)

Rob, I'm reevaluating whether we have any outstanding issues in bug 634233, etc. Assuming we find nothing major, and after retesting various scenarios users might encounter ...

For beta, what do you think about doing this during the 87 beta cycle, i.e. in March?
...

(In reply to Rob Lemley [:rjl] from comment #37)

That timeline sounds good.
...

Fantastic! Thank you. This should help others who were having issues similar to what I had, that putting on 64 bit build helped alleviate.

The "87 beta cycle, i.e. in March" has come and gone. Now on "89 beta cycle".

Probably want to change "version" up top from 68 also.

You need to log in before you can comment on or make changes to this bug.