Closed Bug 1275030 Opened 5 years ago Closed 5 years ago

Perma startup crash when using the skia content backend on Windows 7

Categories

(Core :: Graphics, defect)

49 Branch
x86_64
Windows 7
defect
Not set
normal

Tracking

()

RESOLVED DUPLICATE of bug 1046205

People

(Reporter: mchang, Unassigned)

Details

(Keywords: crash, regression, topcrash, Whiteboard: gfx-noted)

Crash Data

Attachments

(2 files)

I can basically perma stat up crash when enabling the skia content backend on Windows 7 only. My windows 10 machine doesn't seem to have this problem.

STR:
1) go to about:config
2) Change preference "gfx.content.azure.backends" to "skia"
3) Restart and perma-crash

This happened on nightly between 05/19/2016 and 05/20/2016. However, I can't actually bisect this locally as local builds do not reproduce the problem, only official nightly builds.

Crash report - https://crash-stats.mozilla.com/report/index/ffe89410-fae0-4733-8cfc-35c1d2160523
Keywords: regression
Whiteboard: gfx-noted
Version: unspecified → 49 Branch
I can only reproduce this with a 32 bit build.
I tried going through mozregression for both Opt / PGO builds, and even using the same profile, I could not reproduce the permacrash through mozregression.
Interestingly, if I run Nightly from a downloaded .zip file from ftp.mozilla.org, everything works. If I install nightly via the installer, then I get the crash.
I narrowed the installation range. The first bad nightly is April 27 [1]. If I download the zip [2], everything works fine, except I get a popup saying "The patch could not be applied". 

Ryan, do you know who owns the installer or what changes with the installer? Thanks!

[1] https://ftp.mozilla.org/pub/firefox/nightly/2016/04/2016-04-27-03-02-15-mozilla-central/firefox-49.0a1.en-US.win32.installer.exe
[2] https://ftp.mozilla.org/pub/firefox/nightly/2016/04/2016-04-27-03-02-15-mozilla-central/firefox-49.0a1.en-US.win32.zip
Flags: needinfo?(ryanvm)
See Also: 1274145
Rob Strong or Stephen Pohl. That said, here's a regression range between that nightly and the previous one if you want to start looking for possible culprits:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=9ce31e9f90cb&tochange=ab0044bfa1df858919797bcd6a9aef76a668cd4a

I can't say anything stands out there :\. Maybe try grabbing some of those builds from fx-team/inbound and see if you can bisect further?
Flags: needinfo?(ryanvm)
Mason, could you provide a link to the last installer that works and the first installer that doesn't?
Flags: needinfo?(mchang)
(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #6)
> Mason, could you provide a link to the last installer that works and the
> first installer that doesn't?

Last good - https://ftp.mozilla.org/pub/firefox/nightly/2016/04/2016-04-26-04-46-09-mozilla-central/firefox-49.0a1.en-US.win32.installer.exe

Last bad - https://ftp.mozilla.org/pub/firefox/nightly/2016/04/2016-04-27-03-02-15-mozilla-central/firefox-49.0a1.en-US.win32.installer.exe
Flags: needinfo?(mchang)
There don't appear to be any new files though there could be in omni.ja.

Could you try the following?

With 7-Zip extract both of those builds to separate directories.
Take the setup.exe from the extracted last good and replace the one in the extracted last bad.
Run the setup.exe in the last bad and install.
Check to see if it still crashes.

Note: the installer just copies the files it is given by the build system into the install directory and adds/modifies/deletes registry entries and shortcuts.
(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #8)
> There don't appear to be any new files though there could be in omni.ja.
> 
> Could you try the following?
> 
> With 7-Zip extract both of those builds to separate directories.
> Take the setup.exe from the extracted last good and replace the one in the
> extracted last bad.
> Run the setup.exe in the last bad and install.
> Check to see if it still crashes.
> 
> Note: the installer just copies the files it is given by the build system
> into the install directory and adds/modifies/deletes registry entries and
> shortcuts.

I tried this and it is still crashing. What registry entries are being modified?
Flags: needinfo?(robert.strong.bugs)
(In reply to Ryan VanderMeulen [:RyanVM] from comment #5)
> Rob Strong or Stephen Pohl. That said, here's a regression range between
> that nightly and the previous one if you want to start looking for possible
> culprits:
> https://hg.mozilla.org/mozilla-central/
> pushloghtml?fromchange=9ce31e9f90cb&tochange=ab0044bfa1df858919797bcd6a9aef76
> a668cd4a
> 
> I can't say anything stands out there :\. Maybe try grabbing some of those
> builds from fx-team/inbound and see if you can bisect further?

Something in this changeset seems to have caused it with installer builds - https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=52072b6bec14

There are a couple of graphics bugs in there. Bug 1267253 and 1262187 look suspicious to me.
If you tried this and it is still crashing then it isn't the installer since you used the known good installer to perform the installation.

The installer primarily sets up numerous shell integration registry entries for setting as default, protocol handlers, etc. which are used by the operating system, uninstall information, etc. the vast majority of which are not used by Firefox.

BTW: you should be able to get the same end result of a zip build just by extracting the installer and running the build from in the core directory. This might be due to the zip builds not containing the exact same files as the installer due to the way the files are packaged by the build system.

You should try running the extracted known bad from the core directory without installing to see if it also crashes.
Bug 1267253 caused it, builds here - http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-win32/1461674276/

Also still reproducing it only when running it via the installer and not the downloaded .zip file.
If I directly launch Firefox.exe from the install directory, I crash. If I directly launch firefox.exe from the extracted bad core directory, it works. If I copy the core directory over the install directory, it crashes. If I just copy the core directory to C:\Program Files\core\, it works. If I rename that core directory to Nightly, everything blows up.
After much help from Robert, it basically seems like once we run Firefox.exe from C:\Program Files (x86)\Nightly, we get this crash. Installed or run from anywhere else, everything works fine.

George, I know you were working on some naming stuff, do you have any idea why this happens?
Flags: needinfo?(robert.strong.bugs) → needinfo?(gwright)
The problem comes from using single buffering at [1]. It means we'll have a d3d11 compositor but a skia content client, and that'll cause us to use single buffering. This configuration is supposed to be supported and works EVERYWHERE other than when running from explicitly C:\Program Files (x86)\Nightly\firefox.exe. If I call the folder "Nightly" and move it to C:\, it works. Even moving it to C:\Program Files\Nightly\, everything works. So something, somewhere is basically doing something special at a hard coded path :/.


[1] https://dxr.mozilla.org/mozilla-central/source/gfx/layers/client/ContentClient.cpp?from=ContentClient.cpp#87
Attached file about:support
Doesn't appear to be GPU related as I can still reproduce with an nvidia quadro gpu instead of an intel HD GPU.
This is the #1 topcrash in Nightly 20160523030225, with 38 occurrences, which is a lot. All on Windows 7.

The earliest crash date I see is 2016-05-20. But we have build IDs going back to 20160427030215.
Keywords: crash, topcrash
(In reply to Nicholas Nethercote [:njn] from comment #18)
> This is the #1 topcrash in Nightly 20160523030225, with 38 occurrences,
> which is a lot. All on Windows 7.

That was probably me :)
 
> The earliest crash date I see is 2016-05-20. But we have build IDs going
> back to 20160427030215.

That was probably also me.
Flags: needinfo?(gwright)
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1046205
You need to log in before you can comment on or make changes to this bug.