Open Bug 1771638 Opened 2 years ago Updated 2 years ago

Running as admin causes crashes in the file picker/choose save location dialog

Categories

(Core :: Widget: Win32, defect, P2)

Product:
Core
Component:
Widget: Win32
Version:
Firefox 100
Platform:
Desktop
Windows 10
Type:
defect

Tracking

()

People

(Reporter: pridefulmizuki, Unassigned)

References

Details

(Keywords: crash, Whiteboard: [win:stability])

Crash Data

How to reproduce:

  • Downloading files with the folder setting sets to always ask
  • Attempting to set the default download folder
  • Uploading files
  • (anything causing the file picker/choose save location dialog to shows up)

I have submitted a crash report here: https://crash-stats.mozilla.org/report/index/96162a1c-581e-4422-bab9-24b120220528

From report: https://crash-stats.mozilla.org/report/index/96162a1c-581e-4422-bab9-24b120220528

Reason: EXCEPTION_ACCESS_VIOLATION_READ

Top 10 frames of crashing thread:

0 explorerframe.dll static int64_t CUniversalSearchBand::sUniversalSearchWndProc 
1 user32.dll int64_t UserCallWinProcCheckWow 
2 user32.dll DispatchClientMessage 
3 user32.dll _fnINOUTLPPOINT5 
4 ntdll.dll KiUserCallbackDispatch 
5 win32u.dll NtUserSetParent 
6 comctl32.dll int CReBar::_SetBandInfo 
7 comctl32.dll int CReBar::_InsertBand 
8 comctl32.dll int64_t CReBar::_WndProc 
9 comctl32.dll static int64_t CReBar::s_WndProc
Crash Signature: [@ CUniversalSearchBand::sUniversalSearchWndProc]
Keywords: crash

The bug has a crash signature, thus the bug will be considered confirmed.

Status: UNCONFIRMED → NEW
Ever confirmed: true

:gcp, are there any additional clues in here over bug 1629470, or extra details we can get from the reporter? Having this happen whenever you open a filepicker seems pretty bad.

Component: General → Widget: Win32
Flags: needinfo?(gpascutto)
Product: Firefox → Core
See Also: → 1629470

Not clear there's a relation between both bugs, but this specific one appears to have been filed 2 years ago against Thunderbird as well.

Flags: needinfo?(gpascutto)

This crash is very deep in Windows code and may not be actionable for us. These may be related to third party software hooking Explorer.

I notice from the crash dump Firefox has been hooked by "Letasoft Sound Booster" so getting rid of that might certainly be worth a try.

See Also: → 1677170

Having this happen whenever you open a filepicker seems pretty bad.

The problem with opening file pickers is that it invokes Windows code, and in many cases, random third party stuff that hooks the Windows file picker, which then crashes, taking us down with it. About the only thing we can do (see linked bug) is to open the file picker in a separate process and letting that crash so at least it doesn't take Firefox with it (but you won't get to pick any file if it happens).

(In reply to Gian-Carlo Pascutto [:gcp] from comment #6)

This crash is very deep in Windows code and may not be actionable for us. These may be related to third party software hooking Explorer.

I notice from the crash dump Firefox has been hooked by "Letasoft Sound Booster" so getting rid of that might certainly be worth a try.

It also crashes without that program (my apologies for not sending this one first)
https://crash-stats.mozilla.org/report/index/f156c976-81d5-41b0-bb13-de21a0220530

(In reply to pridefulmizuki from comment #8)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #6)

This crash is very deep in Windows code and may not be actionable for us. These may be related to third party software hooking Explorer.

I notice from the crash dump Firefox has been hooked by "Letasoft Sound Booster" so getting rid of that might certainly be worth a try.

It also crashes without that program (my apologies for not sending this one first)
https://crash-stats.mozilla.org/report/index/f156c976-81d5-41b0-bb13-de21a0220530

There is also a suspiciously named "hook64.dll" that is shown as a loaded module. A quick online search shows that this might be located in 'C:\Program Files (x86)\LG Soft India\EasySetPackage\bin' or similar. Does this ring a bell? Is that another application that you could uninstall and try again?

Flags: needinfo?(pridefulmizuki)

(In reply to Stephen A Pohl [:spohl] from comment #9)

(In reply to pridefulmizuki from comment #8)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #6)

This crash is very deep in Windows code and may not be actionable for us. These may be related to third party software hooking Explorer.

I notice from the crash dump Firefox has been hooked by "Letasoft Sound Booster" so getting rid of that might certainly be worth a try.

It also crashes without that program (my apologies for not sending this one first)
https://crash-stats.mozilla.org/report/index/f156c976-81d5-41b0-bb13-de21a0220530

There is also a suspiciously named "hook64.dll" that is shown as a loaded module. A quick online search shows that this might be located in 'C:\Program Files (x86)\LG Soft India\EasySetPackage\bin' or similar. Does this ring a bell? Is that another application that you could uninstall and try again?
nothing foreign anymore that I can see in the modules tab
https://crash-stats.mozilla.org/report/index/425ce8e2-8f65-4e53-bbb5-11a900220531#tab-modules

Flags: needinfo?(pridefulmizuki)

(In reply to pridefulmizuki from comment #10)

(In reply to Stephen A Pohl [:spohl] from comment #9)

(In reply to pridefulmizuki from comment #8)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #6)

This crash is very deep in Windows code and may not be actionable for us. These may be related to third party software hooking Explorer.

I notice from the crash dump Firefox has been hooked by "Letasoft Sound Booster" so getting rid of that might certainly be worth a try.

It also crashes without that program (my apologies for not sending this one first)
https://crash-stats.mozilla.org/report/index/f156c976-81d5-41b0-bb13-de21a0220530

There is also a suspiciously named "hook64.dll" that is shown as a loaded module. A quick online search shows that this might be located in 'C:\Program Files (x86)\LG Soft India\EasySetPackage\bin' or similar. Does this ring a bell? Is that another application that you could uninstall and try again?

nothing foreign anymore that I can see in the modules tab
https://crash-stats.mozilla.org/report/index/425ce8e2-8f65-4e53-bbb5-11a900220531#tab-modules

Thanks! So indeed this reproduces without visible injections.

I went through the Thunderbird reports with this signature, and there are many close to OOM, but yours isn't, and I found a few others that aren't either, for example: https://crash-stats.mozilla.org/report/index/bdeb3abd-7b2f-43eb-889d-dae1f0220531#tab-details

Looking at the stacks of other threads:

 	duser.dll 	int CoreSC::xwProcessNL(struct tagMSG*, struct HWND__*, unsigned int, unsigned int, unsigned int, unsigned int)

Googling wxProcessNL gives hits on other software hitting similar crashes: https://github.com/nvaccess/nvda/issues/6384 so presumably that's a worker thread for that dialog.

Of course that doesn't help us much further. Most of the hits on this problem are third-party issues (which we excluded). There's some hints to try resetting Explorer display options, e.g. https://superuser.com/questions/221720/why-does-notepad-crash-on-desktop-files-in-the-save-as-dialog/527285#527285 or even force-restarting Explorer itself: https://answers.microsoft.com/en-us/windows/forum/all/file-open-dialog-box-issues/be6101c8-fea4-4958-b2ea-30a6897c95d1

Finding out more what the exact CReBar::_SetBandInfo call does it does seem to expect to be passed a list of images: https://docs.microsoft.com/en-us/windows/win32/api/commctrl/ns-commctrl-rebarinfo
which is somewhat consistent with resetting the Folder View options potentially solving it (if that regenerates/removes some broken thumbnails...)

Reporter, are you able to try any of these steps? Are any other apps affected? I presume if you'd install an older Firefox release (or a Firefox ESR build), they crash in exactly the same way?

Flags: needinfo?(pridefulmizuki)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #13)

Reporter, are you able to try any of these steps? Are any other apps affected? I presume if you'd install an older Firefox release (or a Firefox ESR build), they crash in exactly the same way?
None of those works, but I managed to pinpoint FF67 as where it starts crashing (66.0.5 is fine, 67 crashes)
And no, it's only FF67+, everything else works fine

Flags: needinfo?(pridefulmizuki)

(In reply to pridefulmizuki from comment #14)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #13)

Reporter, are you able to try any of these steps? Are any other apps affected? I presume if you'd install an older Firefox release (or a Firefox ESR build), they crash in exactly the same way?

None of those works, but I managed to pinpoint FF67 as where it starts crashing (66.0.5 is fine, 67 crashes)
And no, it's only FF67+, everything else works fine

These may also be useful to see any not-so-obvious Explorer/Shell modifications:
https://superuser.com/a/286008

http://www.nirsoft.net/utils/shexview.html ---> http://www.nirsoft.net/utils/shexview-x64.zip

(66.0.5 is fine, 67 crashes)

Okay, that does give us something to go on! Are you able to use https://mozilla.github.io/mozregression/ ? It can identify the exact change that caused the issue.

(In reply to Gian-Carlo Pascutto [:gcp] from comment #16)

These may also be useful to see any not-so-obvious Explorer/Shell modifications:
https://superuser.com/a/286008

http://www.nirsoft.net/utils/shexview.html ---> http://www.nirsoft.net/utils/shexview-x64.zip

(66.0.5 is fine, 67 crashes)

Okay, that does give us something to go on! Are you able to use https://mozilla.github.io/mozregression/ ? It can identify the exact change that caused the issue.

25/9/18 nightly works, 26/9/18 crashes
which seems weird cause that's 64a1
nightly: 64 (after 26/9) to 67 all crashes
stable: 64 to 66 fine, 67 crashes
(if it matters, beta is same as stable)

(In reply to pridefulmizuki from comment #17)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #16)

These may also be useful to see any not-so-obvious Explorer/Shell modifications:
https://superuser.com/a/286008

http://www.nirsoft.net/utils/shexview.html ---> http://www.nirsoft.net/utils/shexview-x64.zip

(66.0.5 is fine, 67 crashes)

Okay, that does give us something to go on! Are you able to use https://mozilla.github.io/mozregression/ ? It can identify the exact change that caused the issue.

25/9/18 nightly works, 26/9/18 crashes
which seems weird cause that's 64a1
nightly: 64 (after 26/9) to 67 all crashes
stable: 64 to 66 fine, 67 crashes
(if it matters, beta is same as stable)

If I'm not mistaken that's https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ea12f08bca89233f4f1037b6061a3877ab81d841&tochange=48cc597db296ea5234a96321e2ce0c52094cc254 as a pushlog (possibly slightly too wide a window based on the dates, but better too much than missing stuff, I figured) - and the release/beta vs nightly thing points to something that rode the trains in 67 - but I don't see anything obvious in this set of changes. Nothing at all in widget/windows, no Windows sandbox changes afaict, and no relevant-looking XPCOM changes. I'm probably missing something - :gcp?

Flags: needinfo?(gpascutto)

I set a breakpoint on explorerframe!CUniversalSearchBand::sUniversalSearchWndProc+0x5e7 on Fx in WinDbg to see what kinds of values were supposed to be at the crash where it instead gets null. I see this in [%rcx]:

000001d3`bb2848c0  00007fff`e73f35e0 Windows_UI_FileExplorer!winrt::impl::heap_implements<CModernSearchBox>::`vftable'
000001d3`bb2848c8  00007fff`e73f35b0 Windows_UI_FileExplorer!winrt::impl::heap_implements<CModernSearchBox>::`vftable'
000001d3`bb2848d0  00007fff`e73f34c0 Windows_UI_FileExplorer!winrt::impl::heap_implements<CModernSearchBox>::`vftable'
000001d3`bb2848d8  00007fff`e73f3498 Windows_UI_FileExplorer!winrt::impl::heap_implements<CModernSearchBox>::`vftable'
000001d3`bb2848e0  00007fff`e73f3508 Windows_UI_FileExplorer!winrt::impl::heap_implements<CModernSearchBox>::`vftable'
000001d3`bb2848e8  00007fff`e73f34e8 Windows_UI_FileExplorer!winrt::impl::heap_implements<CModernSearchBox>::`vftable'
000001d3`bb2848f0  00007fff`e73f3458 Windows_UI_FileExplorer!winrt::impl::heap_implements<CModernSearchBox>::`vftable'

Looks to be a bunch of vtable pointers. Maybe this is an MSCOM issue. I also don't see anything in the push range that is at all related but weird COM bugs are often timing related. This closely resembles the issue from bug 1704500, which we're hoping we handled by switching the COM context to CLSCTX_ALL. (We could probably narrow that down but it appears to need multiple flags set.) We should try a build where ShowFilePicker and ShowFolderPicker get the same treatment.

Would you be up for trying the build from here (direct link to zip)?

Flags: needinfo?(pridefulmizuki)

I also went through that change list and there's nothing (relevant looking) in there that was in Nightly in 64 but rode to release in 67.

mozregression should be able to bisect further to the exact changeset on inbound, I think, if reporter has the time and patience to do that, and the problem is sufficiently stable.

Flags: needinfo?(gpascutto)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #21)

I also went through that change list and there's nothing (relevant looking) in there that was in Nightly in 64 but rode to release in 67.

mozregression should be able to bisect further to the exact changeset on inbound, I think, if reporter has the time and patience to do that, and the problem is sufficiently stable.

I don't think we keep inbound builds from 4 years ago. :-(

(In reply to David Parks [:handyman] from comment #20)

Would you be up for trying the build from here (direct link to zip)?

still crashed unfortunately
https://crash-stats.mozilla.org/report/index/84687eda-2cdf-4758-9ab6-e445a0220610

(In reply to Gian-Carlo Pascutto [:gcp] from comment #21)

if reporter has the time and patience to do that, and the problem is sufficiently stable.

i'd be willing to help you all get to the bottom of this

Flags: needinfo?(pridefulmizuki)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #21)

I also went through that change list and there's nothing (relevant looking) in there that was in Nightly in 64 but rode to release in 67.

mozregression should be able to bisect further to the exact changeset on inbound, I think, if reporter has the time and patience to do that, and the problem is sufficiently stable.

I managed to narrow it down a little more (forgot that oftentimes there would be more than one build in a day)
2018-09-26-14-20-55 works, 2018-09-26-22-01-46 crashes

(In reply to pridefulmizuki from comment #24)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #21)

I also went through that change list and there's nothing (relevant looking) in there that was in Nightly in 64 but rode to release in 67.

mozregression should be able to bisect further to the exact changeset on inbound, I think, if reporter has the time and patience to do that, and the problem is sufficiently stable.

I managed to narrow it down a little more (forgot that oftentimes there would be more than one build in a day)
2018-09-26-14-20-55 works, 2018-09-26-22-01-46 crashes

Thanks! This points to this range instead: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=48cc597db296ea5234a96321e2ce0c52094cc254&tochange=7ac88abc3c57832e7e271dede934f8935c7f83a0

This includes enabling the launcher process on Windows nightly, and in bug 1530539 that rode the train to 67 release, which would fit very well. So I guess my next question is - if you temporarily disable the launcher process (change browser.launcherProcess.enabled in about:config to false and restart Firefox) does that fix the problem? If so, if you re-enable the process but start Firefox with the -no-deelevate commandline flag, does that still crash?

Do you have UAC disabled and/or are you running Firefox as admin?

Flags: needinfo?(pridefulmizuki)

(In reply to :Gijs (he/him) from comment #25)

(In reply to pridefulmizuki from comment #24)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #21)

I also went through that change list and there's nothing (relevant looking) in there that was in Nightly in 64 but rode to release in 67.

mozregression should be able to bisect further to the exact changeset on inbound, I think, if reporter has the time and patience to do that, and the problem is sufficiently stable.

I managed to narrow it down a little more (forgot that oftentimes there would be more than one build in a day)
2018-09-26-14-20-55 works, 2018-09-26-22-01-46 crashes

Thanks! This points to this range instead: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=48cc597db296ea5234a96321e2ce0c52094cc254&tochange=7ac88abc3c57832e7e271dede934f8935c7f83a0

This includes enabling the launcher process on Windows nightly, and in bug 1530539 that rode the train to 67 release, which would fit very well. So I guess my next question is - if you temporarily disable the launcher process (change browser.launcherProcess.enabled in about:config to false and restart Firefox) does that fix the problem? If so, if you re-enable the process but start Firefox with the -no-deelevate commandline flag, does that still crash?

Do you have UAC disabled and/or are you running Firefox as admin?

seems you nailed it! switching that prefs to false fixed it, and leaving it as true but add the command line flag fixed it also
I'm running it as admin

Flags: needinfo?(pridefulmizuki)

:gcp, who knows about the launcher process and de-elevation, and why that might cause crashes here? I assume we should try to fix this to at least not crash...

Flags: needinfo?(gpascutto)
Flags: needinfo?(gpascutto)
Summary: Firefox crashes doing anything causing the file picker/choose save location dialog to shows up. → Running as admin causes crashes in the file picker/choose save location dialog

The last similar bug that I can find is bug 1642577, but neither Aaron nor Toshi is here so we'll have to triage this as normal.

We have a support article for running as admin (https://support.mozilla.org/en-US/kb/windows-administrator-launcher-process-error-fix) but it looks like we weren't aware of the effect on the common dialogs.

The severity field is not set for this bug.
:spohl, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(spohl.mozilla.bugs)
Severity: -- → S3
Flags: needinfo?(spohl.mozilla.bugs)
Priority: -- → P2
Whiteboard: [win:stability]

(In reply to :Gijs (he/him) from comment #27)

who knows about the launcher process and de-elevation,

*raises hand*

and why that might cause crashes here?

*lowers hand*

My best guess — which I have little confidence in — is that the launcher's taking the code-path involving creating a medium-integrity access token with which to launch the parent process, and that something somewhere in the file/folder picker is dying because it can't handle not having high integrity in the context it finds itself in.

Given the function names in the explorerframe.dll part of the callstack, it's at least vaguely plausible that the specific cause is that the default folder which the file/folder picker opens to has a path one of whose components is high-integrity and therefore unqueryable by the DLL's host process (us), and that CNavBar is dying because it doesn't know how to handle that.

(My second guess is the same, except that instead of CNavBar, it's CUniversalSearchBand trying to query some sort of search history.)

(In reply to pridefulmizuki from comment #26)

I'm running it as admin

In practice that could mean a few different things. More specifically:

  • Are you logged in as the actual "Administrator" account, as some other user with admin privileges, or as a user without admin privileges?
  • Exactly how are you starting Firefox? (e.g.: double-clicking the default shortcut, double-clicking a modified shortcut, opening it via some "Run as administrator" option, starting it from a Command Prompt which was itself started with admin privileges...)
  • When you do that, are you seeing a UAC prompt? If not, do you know why?

And when you run with -no-deelevate:

  • What directory does the file picker start in by default?
  • What are the permissions for that directory and its parent directories set to?
Flags: needinfo?(pridefulmizuki)

(In reply to Ray Kraesig [:rkraesig] from comment #31)

(In reply to pridefulmizuki from comment #26)

I'm running it as admin

In practice that could mean a few different things. More specifically:

  • Are you logged in as the actual "Administrator" account, as some other user with admin privileges, or as a user without admin privileges?
  • Exactly how are you starting Firefox? (e.g.: double-clicking the default shortcut, double-clicking a modified shortcut, opening it via some "Run as administrator" option, starting it from a Command Prompt which was itself started with admin privileges...)
  • When you do that, are you seeing a UAC prompt? If not, do you know why?

And when you run with -no-deelevate:

  • What directory does the file picker start in by default?
  • What are the permissions for that directory and its parent directories set to?

I am logged in as the actual admin account (awful idea I know but this is more a screwing around machine without important stuffs)

  • Exactly how are you starting Firefox?

(in cmd) "firefox -profile data"
and no uac cause actual admin account

  • What directory does the file picker start in by default?

Download folder of current user (which is admin), I haven't touched the security tab so they should all just be windows default
I don't think this is the issue though, cause I managed to change the default download folder to a folder on a data drive in about:config, and triggering the file picker still crashes FF

Flags: needinfo?(pridefulmizuki)

(In reply to pridefulmizuki from comment #32)

Download folder of current user (which is admin), I haven't touched the security tab so they should all just be windows default
I don't think this is the issue though, cause I managed to change the default download folder to a folder on a data drive in about:config, and triggering the file picker still crashes FF

If you're using about:config and only changed some of the browser.download.dir and/or browser.download.lastDir options, chances are you left browser.download.folderList alone (ie set to 1) which overrides it to system default. I would recommend changing the setting through the regular Firefox settings interface instead (search for "download" in the search box in the top right of the Firefox settings/preferences).

(In reply to :Gijs (he/him) from comment #33)

(In reply to pridefulmizuki from comment #32)

Download folder of current user (which is admin), I haven't touched the security tab so they should all just be windows default
I don't think this is the issue though, cause I managed to change the default download folder to a folder on a data drive in about:config, and triggering the file picker still crashes FF

If you're using about:config and only changed some of the browser.download.dir and/or browser.download.lastDir options, chances are you left browser.download.folderList alone (ie set to 1) which overrides it to system default. I would recommend changing the setting through the regular Firefox settings interface instead (search for "download" in the search box in the top right of the Firefox settings/preferences).

I do have it set to 2 (still crashes)

You need to log in before you can comment on or make changes to this bug.