Closed Bug 1769414 Opened 3 years ago Closed 3 years ago

Firefox Nightly 102.0a1 freezes and is unusable and unresponsive after launch, does not spawn child processes, does not display anything in content area

Categories

(Core :: DOM: Content Processes, defect, P3)

Product:
Core
Component:
DOM: Content Processes
Version:
Firefox 102
Type:
defect

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1769845

People

(Reporter: jupiters02, Unassigned)

References

Details

Attachments

(1 file)

ff.zip
222.84 KB, application/x-zip-compressed
Details
Attached file ff.zip —

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0

Steps to reproduce:

Launch Firefox Nightly 102.0a1, built after 2022-05-09.

Actual results:

After launching Firefox Nightly 102.0a1 WIn64, built after 2022-05-09 (BuildID 20220509190429) either updated or after a clean installation, Firefox is generally unusable and unresponsive, does not display anything in content area, and does not spawn child processes, as seen in the attached screenshots.

It seems to load opened tabs, but the tabs themselves are blank and unusable. It seems to load new tabs but are blank, unusable and unresponsive. I can type an address in the address bar but it never loads or displays anything in the content area.

Firefox Home (default) page homepage does not load, is also blank.

Settings do not load, nothing on the menu loads, no shortcuts work either (like Ctrl+Shift+A).

In Troubleshoot mode everything works OK, but even after removing all add-ons, it does not work in normal mode.
It doesn't even work after a clean installation with a new profile.
I have tried refreshing Firefox, deleting the profile and testing with a brand new profile, and even uninstalling and reinstalling Firefox, starting with a new profile.

Shutting down firefox also hangs, the UI is killed but the 2 processes remain running in the background for over a minute, which triggers the firefox crash reporter.

Everything worked flawlessly up till Firefox Nightly 102.0a1, built on 2022-05-09.
Every version afterwards has the same problem.
It happens on two machines, the first with Win 11 21H2, the other with Win 10 1909 x64.

Expected results:

Everything should work.

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Content Processes' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → DOM: Content Processes
Product: Firefox → Core

Hi, sorry for your trouble! Could you identify which precise build id is the first to fail? We had two nightly builds on May 09. Thanks for your support!

Flags: needinfo?(jupiters02)

(In reply to Jens Stutte [:jstutte] from comment #2)

Hi, sorry for your trouble! Could you identify which precise build id is the first to fail? We had two nightly builds on May 09. Thanks for your support!

Hi, Jens.
Thanks for the reply.

As I wrote above, Nightly 102.0a1 WIn64, BuildID 20220509190429
/pub/firefox/nightly/2022/05/2022-05-09-19-04-29-mozilla-central/
seems to be the last one to work properly for me.

I have updated Firefox,
either directly through Firefox (Help -> About -> Update),
or by downloading the win64.installer.exe from https://www.mozilla.org/en-US/firefox/all/#product-desktop-nightly or from ftp
to the latest available version of the day
but every version after BuildID 20220509190429 has the same behavior.
Uninstalling, reinstalling, deleting profiles, refreshing, installing from scratch etc do not work either.

So, the first build to fail is 20220510095538 (2022-05-10-09-55-38).
I just tested that particular build and it fails. And every version after that.
Reinstalling - overwritting the installation with the 20220509190429 build is the only solution for me for about a week now.

Thanks in advance.

Flags: needinfo?(jupiters02)

Hi Kagami, IIUC in that push range landed bug 1764771. Can you imagine some relation here?

Flags: needinfo?(krosylight)

I don't think it can break startup on a fresh profile.

Hi reporter, could you try https://mozilla.github.io/mozregression/ to catch the problematic patch? Thanks!

Flags: needinfo?(krosylight) → needinfo?(jupiters02)

(In reply to Kagami :saschanaz from comment #5)

I don't think it can break startup on a fresh profile.

Hi reporter, could you try https://mozilla.github.io/mozregression/ to catch the problematic patch? Thanks!

Hi Kagami.

Nope, no luck with mozregression either.
There is no build 102 available to select from the dropdown list, and I have tried every range date close to:
last good build: 2022-05-09
first bad build: 2022-05-10
up to from 2022-04-01 to 2022-05-16
but every build runs fine on mozregression. Loads fine and displays content fine.

On the actual installation however every build after 20220509190429 fails.
After numerous uninstallations, deletions of profile and whole folders, restarts and deletions of everything that loosely matches mozil,
nothing seems to work.
On two different machines. On Win10 x64 and Win11.

Is it just me...?
Thanks.

Flags: needinfo?(jupiters02)

Hmm, what does about:profiles say on a working build?

(In reply to Kagami :saschanaz from comment #7)

Hmm, what does about:profiles say on a working build?

https://imgur.com/a/5La2LAX

(In reply to jupiters02 from comment #8)

(In reply to Kagami :saschanaz from comment #7)

Hmm, what does about:profiles say on a working build?

https://imgur.com/a/5La2LAX

I have attached the ff.zip on the first post, that has all the troubleshooting information and screenshots
both of working builds and of non-working builds.

(In reply to jupiters02 from comment #9)

(In reply to jupiters02 from comment #8)

(In reply to Kagami :saschanaz from comment #7)

Hmm, what does about:profiles say on a working build?

https://imgur.com/a/5La2LAX

I have attached the ff.zip on the first post, that has all the troubleshooting information and screenshots
both of working builds and of non-working builds.

Thanks for clarifying. Looking at the JSON of the faulty start I see some active extensions that are not part of a vanilla installation. Did you check if the problem persists starting without them?

Flags: needinfo?(jupiters02)

(In reply to Jens Stutte [:jstutte] from comment #10)

(In reply to jupiters02 from comment #9)

(In reply to jupiters02 from comment #8)

(In reply to Kagami :saschanaz from comment #7)

Hmm, what does about:profiles say on a working build?

https://imgur.com/a/5La2LAX

I have attached the ff.zip on the first post, that has all the troubleshooting information and screenshots
both of working builds and of non-working builds.

Thanks for clarifying. Looking at the JSON of the faulty start I see some active extensions that are not part of a vanilla installation. Did you check if the problem persists starting without them?

That was one of the first things I tried, as I wrote above:
"In Troubleshoot mode everything works OK, but even after removing all add-ons, it does not work in normal mode."
" It doesn't even work after a clean installation with a new profile."
"I have tried refreshing Firefox, deleting the profile and testing with a brand new profile, and even uninstalling and reinstalling Firefox, starting with a new profile."
"Uninstalling, reinstalling, deleting profiles, refreshing, installing from scratch etc do not work either."
"After numerous uninstallations, deletions of profile and whole folders, restarts and deletions of everything that loosely matches mozil,
nothing seems to work."

Flags: needinfo?(jupiters02)

(In reply to jupiters02 from comment #11)

(In reply to Jens Stutte [:jstutte] from comment #10)

(In reply to jupiters02 from comment #9)

(In reply to jupiters02 from comment #8)

(In reply to Kagami :saschanaz from comment #7)

Hmm, what does about:profiles say on a working build?

https://imgur.com/a/5La2LAX

I have attached the ff.zip on the first post, that has all the troubleshooting information and screenshots
both of working builds and of non-working builds.

Thanks for clarifying. Looking at the JSON of the faulty start I see some active extensions that are not part of a vanilla installation. Did you check if the problem persists starting without them?

That was one of the first things I tried, as I wrote above:
"In Troubleshoot mode everything works OK, but even after removing all add-ons, it does not work in normal mode."
" It doesn't even work after a clean installation with a new profile."
"I have tried refreshing Firefox, deleting the profile and testing with a brand new profile, and even uninstalling and reinstalling Firefox, starting with a new profile."
"Uninstalling, reinstalling, deleting profiles, refreshing, installing from scratch etc do not work either."
"After numerous uninstallations, deletions of profile and whole folders, restarts and deletions of everything that loosely matches mozil,
nothing seems to work."

I have tried everything I can think of regarding the profiles and uninstallation/installation from scratch with no profiles at all.

The most odd thing is that Firefox does not spawn child processes.
As you can see in the screenshots from ProcessExplorer in the attached file,
normally the parent process spawns 6+ child processes with a single tab open.
But with 20220510095538 and newer builds, the parent process spawns just one child process,
regardless of the number of (unresponsive and empty of content) tabs.

Could you see if there is some message on the console log ? Thanks, and sorry for the repeated questions

Multiprocess Browser Console does not work.
It loads but it is empty.
https://imgur.com/a/5NZ9jxe

Web Developer Tools do not work either. So no console log from that either.
It loads the Developer Tools UI window on the bottom, but it is empty and unresponsive.
In fact, none of the Browser tools work.
https://imgur.com/a/OfAVyAu

Is there any way to see/read the console log from a file?

You might be able to see something if started from a command line, Firefox should write log messages to stdout.

(In reply to Jens Stutte [:jstutte] from comment #15)

You might be able to see something if started from a command line, Firefox should write log messages to stdout.

Would you please elaborate with the actual command line syntax on windows?

start firefox
or
.\firefox.exe &

are just launching firefox, with no log messages to stdout.
Tried with regular command prompt, terminal, terminal preview, with and without admin rights.

AFAIK Windows does not support stdout on GUI apps 😞. I'm not sure there's easy way to get it without using mach.

Looking at the attachment, I wonder this has anything to do with ESET Security since I think it can implicitly affect Firefox. Might be good to try turning it off?

Oh sorry for the hassle, I am too much used to see the output from mach run ...

(In reply to Kagami :saschanaz from comment #17)

AFAIK Windows does not support stdout on GUI apps 😞. I'm not sure there's easy way to get it without using mach.

You can launch with --attach-console, which should get you stdout at least.

Looking at the attachment, I wonder this has anything to do with ESET Security since I think it can implicitly affect Firefox. Might be good to try turning it off?

Yes, please investigate this. Otherwise, I wonder if there's an interaction with the launcher process? There is an active ticket where two installations may be getting in the way of the launcher. Is it possible that "Troubleshoot mode" also disables the launcher? I certainly see some interaction but I can't quite tell what it means at first glance.

Shutting down firefox also hangs, the UI is killed but the 2 processes remain running in the background for over a minute, which triggers the firefox crash reporter.

Do you have the crash reports from these?

Do you have the crash reports from these?

(about:crashes will show the previous reports)

OK. One thing at a time.

  1. --attach-console launches firefox, but there is no stdout output, no logging or anything, at least nothing I can find.

  2. I have found this
    https://firefox-source-docs.mozilla.org/networking/http/logging.html
    with the option
    -MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5,cookie:5 -MOZ_LOG_FILE=%TEMP%\log.txt
    I have managed to log enough, but the file is rather large, several MBs, and I am reluctant to post it on anything public.
    Any way to post it in a private way?

  3. I have thought of DEP permanent, disabled, restarted, but no change.

  4. I have thought of ESET Security, had it disabled before a few days ago,
    and after you mentioned it, this time I have disabled everything that can be disabled,
    antivirus and firewall, apart from uninstalling, and after several reboots no change.
    In any case, I am reluctant to uninstall it at this point.

  5. As for the crash reports, I have only the latest two, since I have been uninstalling and deleting everything mozilla related a lot lately.
    https://crash-stats.mozilla.org/report/index/bp-2be6208d-40af-4325-8c61-54fa30220517
    https://crash-stats.mozilla.org/report/index/bp-c62c9fca-4491-4fde-a597-a85090220517

So, as for

  1. This is actually not going to be very useful - the problem sounds like the child processes are not starting, and those are sandboxed, so they cannot access the console. (You could still get some debug output from the parent, but not everything)
  2. Those default settings log a lot of info about the network connections, but again I suspect that's probably not related to this failure, so I would ignore that for now (especially because that could have quite some private info). We may ask for another log with different settings later.

The crash reports are hangs on shutdown, but they do show that ESET is still injecting into and modifying Firefox even if you disable it.

up to from 2022-04-01 to 2022-05-16
but every build runs fine on mozregression. Loads fine and displays content fine.
On the actual installation however every build after 20220509190429 fails.

This seems like a key price of information. It may be worth trying installing Firefox in a non-default folder, i.e. not in C:\Program Files, but some random other folder, and seeing if that magically makes it work. (We've seen this before, something else on the system is looking for us in specific places...)

Installed Firefox in C:\firefox
and still the problem remains.

Have to take another backup and see If I can find time to completely uninstall Eset, just to give that a try.

OK, so I have uninstalled and deleted everything Eset Security related.
Several reboots later, I uninstalled and deleted everything Mozilla related.
Several reboots later, I installed from scratch nightly, with a brand new profile.

The problem still remains.
https://crash-stats.mozilla.org/report/index/bp-7d8ec99d-ee5e-40e0-adfe-cc41d0220518
https://crash-stats.mozilla.org/report/index/bp-a9d89b98-77ee-4d1c-bcfd-0b2310220518

I'm at a loss.
Any ideas?
Thanks.

Also tested Nightly installation on non-default folder, like C:\firefox, with no spaces in the path, just to be safe,
but still no luck, the same behavior.

I have also tested portable Nightly from portableapps.com.
The same problem.
Is it really just me that is facing this issue...?

Anyway, after some more tests, I have come up to peculiar situation and a kind of a workaround.
I have uninstalled Nightly and deleted all profiles, again...
After a brand new installation of latest Nightly from scratch, I have created a blank profile.
Launching latest Nightly with that profile, or any profile whatsoever, leaves Nightly hanging, with a parent process and a single child process.

Creating a new tab and typing about:profiles does not load, the tab is empty of content.
But, launching More Troubleshooting Information from Help Menu and selecting about:profiles works normally.
From there I can create a new profile, or select any other profile other than default, and when I click on Launch profile in new browser,
the new profile loads in a new window, and it is all working normally.
It creates a new child process which is the parent process of 6+ child processes, but it works normally.

Does that make any sense?
https://imgur.com/a/OpLEwOa
https://imgur.com/a/xgHNZmu

I have no explanation, but yes, it seems you are quite alone with this, for now. So I base the severity setting on this assumption for now.

Severity: -- → S3
Priority: -- → P3

We have one other report here (bug 1769845), and the date range matches, but we're still a bit clueless as to what is the cause.

The fact that the mozregression builds work and the normal Nightly install does not is a real nuisance here, I'm hoping the user in the other report has a steady repro of the problem, so perhaps they can nail down the change.

See Also: → 1769845

There's a way to point mozregression at your profile (instead of having it create a new one), I wonder if that would make the problem reproduce with it?

Perhaps you can try the builds linked here and see if they have differently?

https://bugzilla.mozilla.org/show_bug.cgi?id=1769845#c9

(In reply to Gian-Carlo Pascutto [:gcp] from comment #32)

Perhaps you can try the builds linked here and see if they have differently?

https://bugzilla.mozilla.org/show_bug.cgi?id=1769845#c9

Both builds work fine (292df8ed886d and 53032d712512).

So summarizing:

  • Every build from the mozregression tool works.
  • The Nightly builds between 2022-05-10 2022-05-09 broke.
  • A fresh profile doesn't help.
  • CI builds for the most likely regressing change both work, so that wasn't the cause.

I'm a bit clueless here.

Yes, in a nutshell.
I have looked at bug 1769845 (reported 3 days after mine, but never mind)
and the behavior is similar to what I am experiencing (although, I have no trouble with print preview).

I have retested all bisections with mozregression, with every related setting to every imaginable value,
testing for buildnumbers and for dates, even far broader than that one day range (up to a month range).
Every build in mozregression works normally.

The latest installed build to work normally is 20220509190429.
In every build after 20220510095538, Firefox does not start child processes,
it displays an empty interface and content window, and nothing works.
I have tested almost every build to this day and nothing works.

I have deleted all profiles, uninstalled and deleted everything mozilla related, and installed from scratch with a new profile,
dozens of times, but it does not make a difference.

CI builds 292df8ed886d and 53032d712512 work normally.

I have tested with Eset Security disabled, and even uninstalled it, but no change.
I have tested with DEP Enabled but not permanent, but no change.
I have installed Firefox on a non-standard folder, with no spaces or anything, like C:\1\firefox
and I have tested Firefox portable installations on a similar C:\2\firefoxportable\ folder.

I have also tested Dev and Beta editions, in normal installation mode and in portable mode.
They do not work either.

I have tested all of the above on three (3) different machines, with almost nothing in common in hardware, and a few in common in software.

  1. 3 year-old laptop with 8th gen Intel and Win11.
  2. 6 year-old desktop with 4th gen Intel and Win11.
  3. 10 year-old laptop with 1st gen Intel and Win10 1909 x64.

Nothing newer than 2022-05-09, on any release channel, on any machine works.
Nightly, Developer and Beta do not work.
(Fortunately, on a dev Debian VM, Nightly works normally).

During the portable editions testing, I have noticed that the version installed from the portable installer is outdated and works fine,
but as soon as I initiate an update to the newest update available on any channel, after Firefox restarts, it displays the same behavior.
The date range is similar, the problem appears after around 2022-05-10.
I have uploaded all the Troubleshooting Information of a working and a non-working Dev edition here.
You can see screenshots here.
And you can see the parent and child processes below, with info from Process Explorer.

Working Developer Version 
Version 	101.0b2
Build ID 	20220503185929

C:\2\FirefoxPortableDeveloper\App\Firefox64\firefox.exe -profile C:\2\FirefoxPortableDeveloper\Data\profile

"C:\2\FirefoxPortableDeveloper\App\Firefox64\firefox.exe" -contentproc --channel="10180.1.935353932\156742096" -parentBuildID 20220503185929 -prefsHandle 1888 -prefMapHandle 1912 -prefsLen 1706 -prefMapSize 245664 -appDir "C:\2\FirefoxPortableDeveloper\App\Firefox64\browser" - 10180 "\\.\pipe\gecko-crash-server-pipe.10180" 2072 238833c7548 gpu

"C:\2\FirefoxPortableDeveloper\App\Firefox64\firefox.exe" -contentproc --channel="10180.3.1037275207\292584539" -childID 2 -isForBrowser -prefsHandle 3896 -prefMapHandle 3892 -prefsLen 3095 -prefMapSize 245664 -jsInitHandle 1412 -jsInitLen 277204 -a11yResourceId 64 -parentBuildID 20220503185929 -appDir "C:\2\FirefoxPortableDeveloper\App\Firefox64\browser" - 10180 "\\.\pipe\gecko-crash-server-pipe.10180" 3904 238881ccb48 tab

"C:\2\FirefoxPortableDeveloper\App\Firefox64\firefox.exe" -contentproc --channel="10180.6.1026999031\1230818736" -childID 5 -isForBrowser -prefsHandle 4836 -prefMapHandle 4832 -prefsLen 8525 -prefMapSize 245664 -jsInitHandle 1412 -jsInitLen 277204 -a11yResourceId 64 -parentBuildID 20220503185929 -appDir "C:\2\FirefoxPortableDeveloper\App\Firefox64\browser" - 10180 "\\.\pipe\gecko-crash-server-pipe.10180" 4848 23885d8ef48 tab

"C:\2\FirefoxPortableDeveloper\App\Firefox64\firefox.exe" -contentproc --channel="10180.7.751642417\592590416" -childID 6 -isForBrowser -prefsHandle 5836 -prefMapHandle 3328 -prefsLen 11578 -prefMapSize 245664 -jsInitHandle 1412 -jsInitLen 277204 -a11yResourceId 64 -parentBuildID 20220503185929 -appDir "C:\2\FirefoxPortableDeveloper\App\Firefox64\browser" - 10180 "\\.\pipe\gecko-crash-server-pipe.10180" 5848 2388883ca48 tab

"C:\2\FirefoxPortableDeveloper\App\Firefox64\firefox.exe" -contentproc --channel="10180.8.4620535\1108261898" -childID 7 -isForBrowser -prefsHandle 5988 -prefMapHandle 5984 -prefsLen 11578 -prefMapSize 245664 -jsInitHandle 1412 -jsInitLen 277204 -a11yResourceId 64 -parentBuildID 20220503185929 -appDir "C:\2\FirefoxPortableDeveloper\App\Firefox64\browser" - 10180 "\\.\pipe\gecko-crash-server-pipe.10180" 5968 2388883d948 tab

"C:\2\FirefoxPortableDeveloper\App\Firefox64\firefox.exe" -contentproc --channel="10180.9.815284724\2090547818" -childID 8 -isForBrowser -prefsHandle 6000 -prefMapHandle 5996 -prefsLen 11578 -prefMapSize 245664 -jsInitHandle 1412 -jsInitLen 277204 -a11yResourceId 64 -parentBuildID 20220503185929 -appDir "C:\2\FirefoxPortableDeveloper\App\Firefox64\browser" - 10180 "\\.\pipe\gecko-crash-server-pipe.10180" 5876 23889cfab48 tab



Non-working Developer Version
Version 	101.0b9
Build ID 	20220519220322

C:\2\FirefoxPortableDeveloper\App\Firefox64\firefox.exe -profile C:\2\FirefoxPortableDeveloper\Data\profile

"C:\2\FirefoxPortableDeveloper\App\Firefox64\firefox.exe" -contentproc --channel="8736.1.1558891383\1485875967" -parentBuildID 20220519220322 -prefsHandle 2056 -prefMapHandle 2052 -prefsLen 4552 -prefMapSize 250656 -appDir "C:\2\FirefoxPortableDeveloper\App\Firefox64\browser" - 8736 "\\.\pipe\gecko-crash-server-pipe.8736" 2088 21eb16f0448 gpu

Remote Processes
Web Content	1 / 8
Extension 1
Privileged About 1
GPU	1


Preallocated 3

The difference between them is that the 3 Preallocated processes on the working version are absent in the non-working version.
So, 1 parent process and 5+ child processes for the working version with a single tab.
And 1 parent process and just 1 child process for the non-working version, with 1 single tab or 1 million tabs open, it makes no difference,
and all the tabs are empty and non-functional.

You mentioned the sandbox.
Does it have anything to do with DEP, ASLR, CFG or anything like that?

On a personal note, I have been using Firefox ever since I remember, and I have been using Nightly (or the most unstable channel) as my default browser ever since I remember.
But, unfortunately, I find it impossible to remain on a browser that is left behind in security updates for so long.

Just a pure curiosity: If you try installing https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/firefox-102.0a1.multi.win64.installer.msix, does it work? (This installer currently has no autoupdate support, so normally not recommended)

(In reply to Kagami :saschanaz from comment #36)

Just a pure curiosity: If you try installing https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/firefox-102.0a1.multi.win64.installer.msix, does it work? (This installer currently has no autoupdate support, so normally not recommended)

Trying to install msix with double click opens app installer but during installation it throws error Access Denied.
Installing the msix through powershell completes normally.
But it behaves exactly the same: Unresponsive and non fuctional interface, tabs empty of content.

I can not stress this enough:
The problem persists.
And it happens on Nightly, and on Dev and Beta channels as well.

Status: UNCONFIRMED → RESOLVED
Closed: 3 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: