Bug 1274659 (win64-migration)

Migrate 32-bit Firefox (WOW64) users to 64-bit Firefox (Win64)

NEW
Unassigned

Status

()

Firefox
General
P3
normal
a year ago
5 days ago

People

(Reporter: erahm, Unassigned)

Tracking

(Depends on: 2 bugs, Blocks: 1 bug, {64bit})

Trunk
x86_64
Windows
64bit
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

There was a rather lengthy discussion [1] on whether or not it's a good idea to update users of 32-bit Firefox a 64-bit version of Windows to a 64-bit version of Firefox.

The general benefits proposed are:
  - Performance: Guaranteed SSE2 support
  - Stability: No more OOMs due to address space fragmentation (this is a huge win)

Concerns voiced:

*Memory usage* 64-bit builds use more physical memory.

While this is true (probably about 20% looking at AWSY measurements), eliminating all OOMs related to address space fragmentation and exhaustion is a much higher priority than the potentially degraded performance from swapping.

Note: these are OOMs that happen even if there is plenty of physical memory left. The memshrink group has looked at this issue extensively and has concluded that the possible increase in physical memory usage is acceptable and amount of RAM a user has should not be taken into consideration.

*NPAPI plug-ins* If a user has 32-bit plug-ins we may not want to upgrade them.

- Flash and Silverlight both provide 64-bit plug-ins
- Once we drop support for NPAPI this becomes a non-issue

There were additional concerns that there are issues with our ability to sandbox the 64-bit version of flash. We can discuss that further in the comments.

[1] https://groups.google.com/forum/#!topic/mozilla.dev.platform/ANiu9qdUIwQ
From the app update technical side (not advocating for or against) of this:
1. We won't be able to change the install directory due to not being able to update shortcuts for other users on the same system along with a few other reasons.
2. Minimal testing of running Firefox x64 in a directory under C:\Program Files (x86)\ was performed a few years ago without any issues being found.
3. To accomplish this, releng would need to configure balrog to serve the update. No app update client changes should be required.
Depends on: 1123755, 1165891
(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #1)
> From the app update technical side (not advocating for or against) of this:
> 1. We won't be able to change the install directory due to not being able to
> update shortcuts for other users on the same system along with a few other
> reasons.
Could we install in C:\Program Files\ and put a stub in c:\Program Files (x86)\firefox.exe that just launches C:\Program Files\firefox.exe to catch those shortcuts?
(In reply to Brad Lassey [:blassey] (use needinfo?) from comment #2)
> (In reply to Robert Strong [:rstrong] (use needinfo to contact me) from
> comment #1)
> > From the app update technical side (not advocating for or against) of this:
> > 1. We won't be able to change the install directory due to not being able to
> > update shortcuts for other users on the same system along with a few other
> > reasons.
> Could we install in C:\Program Files\ and put a stub in c:\Program Files
> (x86)\firefox.exe that just launches C:\Program Files\firefox.exe to catch
> those shortcuts?

Starting with Vista, we could even do a symbolic link.
The updater intentionally does not touch anything on the filesystem outside of the install directory to mitigate exploits.

We could possibly do this in a synchronously in the post update code. Considering that this is just a cosmetic issue I really don't think we should add this type of complexity.
(In reply to Eric Rahm [:erahm] from comment #0)
> *NPAPI plug-ins* If a user has 32-bit plug-ins we may not want to upgrade
> them.
> 
> - Flash and Silverlight both provide 64-bit plug-ins
> - Once we drop support for NPAPI this becomes a non-issue

Bug 1269807 is the meta-bug tracking the removal of NPAPI plugins other than Flash.
Depends on: 1269807
This is not a scheduled part of the win64 deployment plan, so I'm on the fence about whether to mark this P5 or actually mark it INCOMPLETE. The staged rollout of win64 will focus first on new users via the installer, before we attempt to update existing users.
Priority: -- → P5
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #6)
> This is not a scheduled part of the win64 deployment plan, so I'm on the
> fence about whether to mark this P5 or actually mark it INCOMPLETE. The
> staged rollout of win64 will focus first on new users via the installer,
> before we attempt to update existing users.

To be clear: you explicitly do NOT want to move any users currently running the 32-bit Firefox on a 64-bit Windows to a 64-bit Firefox?
Flags: needinfo?(benjamin)
That is correct. This is effectively WONTFIX for a while.
Flags: needinfo?(benjamin)
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #8)
> That is correct. This is effectively WONTFIX for a while.

By *far* the top crasher on 46.0.1 is OOM small, virtually all of these are on 32-bit builds running on Windows. Including "OOM | unknown" from the top 15 we're looking at 10% of all crashes.

If we're serious about quality and project uptime this should be a much higher priority.
Blocks: 558448
rstrong recommends auto-upgrading in place ("pave over" install) in the 32-bit "C:\Program Files (x86)" directory instead of moving files to the 64-bit "C:\Program Files" directory. This would avoid breaking any user shortcuts that point to the original "C:\Program Files (x86)" directory.

The Win64-aware stub installer (bug 797208) should place new 64-bit installs in the 64-bit "C:\Program Files" directory, allowing 64-bit side-by-side with any existing 32-bit installs in "C:\Program Files (x86)".
Don't forget using 32-bit Firefox is a workaround for bug 1271398.
Depends on: 1271398
No longer blocks: 558448
Depends on: 558448
No longer depends on: 1123755, 1165891, 1269807, 1271398

Comment 12

a year ago
If we have the opportunity to eliminate those top crashers for 'free', it would be crazy to pass that up. We cannot afford to pass up that opportunity, it would be a mistake. Why would all that work and resources be put into making 64bit for Windows if there was no intention of moving people over to it?

And let's not forget about the security benefits, including this (link below), which for some reason never gets brought up http://news.softpedia.com/news/windows-legacy-layer-used-to-bypass-emet-security-measures-495691.shtml
status-firefox49: affected → ---
Keywords: 64bit
Summary: Update all WOW64 users to 64-bit Firefox → Update 32-bit Firefox (WOW64) users to 64-bit Firefox
mbest recommends that we migrate 32-bit Firefox users to 64-bit before making Flash click-to-play.
Blocks: 1317856
Priority: P5 → P3
Summary: Update 32-bit Firefox (WOW64) users to 64-bit Firefox → Migrate 32-bit Firefox (WOW64) users to 64-bit Firefox (Win64)

Updated

8 months ago
Depends on: 1318039
No longer depends on: 1318039

Updated

8 months ago
Depends on: 1322664

Updated

7 months ago
Depends on: 1324617
Depends on: 1324866

Updated

7 months ago
No longer depends on: 1324617
Alias: wow64
Blocks: 1325169
Depends on: 1323750
Depends on: 1340936

Updated

5 months ago
Depends on: 1345649

Comment 14

5 months ago
(In reply to Eric Rahm [:erahm] from comment #0)
> *Memory usage* 64-bit builds use more physical memory.
> 
> While this is true (probably about 20% looking at AWSY measurements),
> eliminating all OOMs related to address space fragmentation and exhaustion
> is a much higher priority than the potentially degraded performance from
> swapping.

Putting aside I can generally agree 32-64 bit memory difference is around 20%, at least in bug 1282322 we are talking of a minimum delta of 40%.

Comment 15

3 months ago
Google started moving eligible machines to 64-bit Chrome today and there is already an issue https://www.reddit.com/r/chrome/comments/68x0zd/help_aww_snap_after_upgrading_to_580302996/
(In reply to Will from comment #15)
> Google started moving eligible machines to 64-bit Chrome today and there is
> already an issue
> https://www.reddit.com/r/chrome/comments/68x0zd/
> help_aww_snap_after_upgrading_to_580302996/

Thanks, Will. That is very interesting! I will follow up with our Firefox enterprise users about testing Citrix XenApp.
(In reply to Chris Peterson [:cpeterson] from comment #16)
> Thanks, Will. That is very interesting! I will follow up with our Firefox
> enterprise users about testing Citrix XenApp.

Is it still a problem despite that we dropped support for NPAPI except Flash?
(In reply to Masatoshi Kimura [:emk] from comment #17)
> Is it still a problem despite that we dropped support for NPAPI except Flash?

AFAIK, this is not a plugin problem. According to the following Chrome KB article, 64-bit Chrome crashes when run as a Citrix remote desktop application due to some Citrix hooks injected into the Chrome process:

https://support.google.com/chrome/a/answer/7380899
Perhaps we can blocklist Citrix.

Comment 20

3 months ago
I don't know what the relationship with Google Chrome team is like but would they give any advice or recommendations so Firefox can avoid any issues when updating 32-bit machines to 64-bit Firefox? Maybe someone can reach out to them?
(In reply to Will from comment #20)
> I don't know what the relationship with Google Chrome team is like but would
> they give any advice or recommendations so Firefox can avoid any issues when
> updating 32-bit machines to 64-bit Firefox? Maybe someone can reach out to
> them?

We are closely watching Google's rollout of 64-bit Chrome to their 32-bit users. For example, that is how we found the cause because Citrix crash bug 1359443. :)
Alias: wow64 → wow64-migration
Depends on: 1359443

Comment 22

2 months ago
Yes, I notice quite a few issues on reddit too, such as Chrome not even starting up (this is different from the citrix problem) and re-installing the 32-bit version fixed it. 
I don't know if this is already being considered but whenever 64-bit is deemed ready (version 55?) there should be a bit of a push via blog posts, the firefox twitter account etc. to get technical users to opt-in to 64-bit to make sure there are no unexpected issus. Is there an easy way to get users to opt-in to 64-bit? Such as via the update window?

Comment 23

2 months ago
Created attachment 8869839 [details]
64bitoptin.png

I just made this terrible thing very quickly. Something like this would be ideal I think
(In reply to Will from comment #22)
> I don't know if this is already being considered but whenever 64-bit is
> deemed ready (version 55?) there should be a bit of a push via blog posts,
> the firefox twitter account etc. to get technical users to opt-in to 64-bit
> to make sure there are no unexpected issues. Is there an easy way to get
> users to opt-in to 64-bit? Such as via the update window?

Yeah, we plan to publish some blog posts promoting 64-bit Firefox 55. With the Firefox 56 release, we plan to automatically migrate existing eligible 32-bit users to 64-bit. Until the automatic migration, users need to manually re-run the Firefox installer to get 64-bit.

In fact, 79% of Nightly channel users and 7% of Release channel users on Windows are already running 64-bit Firefox, so we have pretty good test coverage. :)

We try to keep our release schedule on this wiki page up to date:

https://wiki.mozilla.org/Firefox/Win64#Schedule
Alias: wow64-migration → win64-migration

Comment 25

2 months ago
I'm not concerned that there's any issues with 64-bit Firefox itself, it's the act of upgrading 32-bit to 64-bit that I think there could be issues, such as Firefox not even starting up (like what's been happening to Chrome). Maybe in that blog post asking more technical users to opt-in and report any issues could be helpful. Google didn't do this and it caused some problems.

Comment 26

2 months ago
(In reply to Chris Peterson [:cpeterson] from bug 1366860, comment #1)
> For now, we don't need a minimum memory check in the 64-bit full installer.
> If a user went out of their way to download the 64-bit full installer, we
> should probably give it to them.

A really, really, relevant check to add for 64-bit _migration_ would be instead memory consumption average/Xpercentile. 
Or perhaps more specifically if this multiplied 1.5x is less than 50% of installed RAM. 

Ie: my use-case is that I have 4GB installed, and FF on *average* uses 2. 
(Unless bug 1125557 lands anytime soon) I think it's pretty glaring that I'd be screwed with x64. 

Your call on where to put the boundary then.
Depends on: 1366917
You need to log in before you can comment on or make changes to this bug.