Open Bug 1360392 Opened 6 years ago Updated 29 days ago

crash in EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER (Firefox for Desktop only)

Categories

(Toolkit :: Crash Reporting, defect)

10 Branch
Unspecified
All
defect

Tracking

()

Tracking Status
firefox-esr45 --- wontfix
firefox-esr52 --- wontfix
firefox53 --- wontfix
firefox54 --- wontfix
firefox55 - wontfix
firefox56 --- wontfix
firefox57 --- wontfix
firefox59 --- wontfix
firefox60 --- affected
firefox61 --- affected

People

(Reporter: skywalker333, Unassigned)

References

(Depends on 1 open bug)

Details

(4 keywords, Whiteboard: [tbbird crash])

Crash Data

+++ This bug was initially created as a clone of Bug #1245570 +++

This bug was filed from the Socorro interface and is 
report bp-17f15d85-4d9d-4c7d-a1c8-ae42a2160202.
=============================================================

We fixed one cause of this in bug 1142944, but I am still seeing reports in 45.0b2, so I guess there's something else still going on.
Firefox 	52.0.2 	3583 	8.7% 	3872
Firefox 52.1.0esr 	3344 	8.1% 	2872
Firefox 	53.0 	3100 	7.5% 	3222
Firefox 	54.0b1 	1316 	3.2% 	573
Firefox 53.0b99 	638 	1.5% 	975
Firefox 45.9.0esr 	523 	1.3% 	387
Firefox 	54.0b2 	469 	1.1% 	214
Firefox 52.0.2esr 	458 	1.1% 	515
Firefox 	54.0a2 	348 	0.8% 	108
Firefox 45.8.0esr 	346 	0.8% 	274
Firefox 	51.0.1 	235 	0.6% 	246
Firefox 	52.0 	205 	0.5% 	254
Firefox 	52.0.1 	202 	0.5% 	146
Firefox 	53.0b9 	198 	0.5% 	142
Firefox 	47.0.2 	184 	0.4% 	245
Firefox 24.6.0esr 	161 	0.4% 	120
Firefox 	55.0a1 	156 	0.4% 	107
Firefox 	53.0b8 	150 	0.4% 	87
Firefox 24.3.0esr 	146 	0.4% 	106
Firefox 52.0.1esr 	141 	0.3% 	60
Firefox 	10.0.2 	136 	0.3% 	18
Bug 1245570 
(In reply to Kevin Brosnan [:kbrosnan] from comment #13) 
> Firefox for Android does not ship ESR. This signature is not a one bug fix.
> It can occur for a variety of reasons one of the most common and
> unaddressable is the device that Firefox is running on is out of memory.

I cloned this bug as Bug 1360392 to track the issues related to the desktop version of firefox (not android) for this crash signature.
No longer depends on: 1245570
See Also: → 1245570
Summary: crash in EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER (Firefox for Android only) → crash in EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER (Firefox for Desktop only)
Severity: blocker → critical
See Also: → 1247399
See Also: 1247399
Signature report for EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER

Showing results from a month ago

Operating System
	182605 	100.0%
	
Product
Firefox 	53.0.2 	5682 	14.2% 	5428
Firefox	52.1.1esr 	3098 	7.8% 	1759
Firefox 	53.0 	1218 	3.1% 	973
Firefox	45.9.0esr 	717 	1.8% 	452
Firefox 	54.0b5 	667 	1.7% 	332
Firefox	52.1.0esr 	556 	1.4% 	342
Firefox 	54.0b6 	524 	1.3% 	339

Process Type
plugin 	773 	0.4%
content	16 	0.0%
gpu 	1 	0.0%

Uptime Range
< 1 min 	47704 	26.1%
> 1 hour 	45815 	25.1%
15-60 min 	31344 	17.2%
1-5 min 	31000 	17.0%
5-15 min 	26742 	14.6%

Architecture
	182605 	100.0%
	
Flash™ Version
	182605 	100.0%
	
Graphics Adapter
qualcomm adreno	30226 	16.6%
qualcomm tm 	30226 	16.6%
arm 	mali 	25820 	14.1%
arm 	mp 	13064 	7.2%
arm 	400 	10136 	5.6%
qualcomm 305 	8569 	4.7%
0x0000 	0x0000 	6284 	3.4%
0x8086 	0x0f31 	6274 	3.4%
0x8086 	0x22b1 	5546 	3.0%
91777f4d-b1d6-45b0-9bd6-4cd500170414 
Firefox has been shutting down at least twenty times everyday. Is there anything I can do about this infuriating problem as I'm doing some very important work, and keep losing it? 	2017-04-14 10:53:39 	en-GB 


2839ee44-be5f-4d41-ba04-ed5ee0170414 
Please improve this because im about to uninstall this 	2017-04-14 12:41:57 	en-US 
3e0fe1aa-696d-4455-a3d3-a55e60170414 
constantly crashing and its getting very annoying! 	2017-04-14 12:34:11 	en-US 


087b00e9-5ab4-4438-bb03-70e622170413 
my mozilla keeps crasing and i cant open it. 	2017-04-13 12:02:26 	None 


16e8bd71-5ab8-4c80-8c4a-eb7aa2170413 
Fx Dev on default 4 process is crashing frequently 	2017-04-13 16:45:31 	en-GB 
c426365c-9c5e-49b1-a032-5a6452170413 
I am not able to log on to the internet 	2017-04-13 17:02:45 	en-US 


8f79d0ba-d171-474e-9270-b8aa52170417 
As soon as I open Firefox it crashes. I have uninstalled and reinstalled the program already. 	2017-04-17 15:44:58 	en-US 
9adcf3e1-9273-47f4-b2f4-66d000170418 
bring back my message and reply 	2017-04-18 02:55:36 	en-US 
a005d445-0eed-4e48-9465-b82e50170418 
crashed 	2017-04-18 03:42:32 	en-GB 
08501ba0-731a-4b9d-944f-1b96b2170417 
crashes a lot 	2017-04-17 03:12:50 	en-US 


0a828698-6f66-4081-9f77-5b4002170417 
fiefox is not even opening to a tab before shutting down 	2017-04-17 14:42:32 	en-US 


607c7e2a-835d-471d-80ef-21ff60170419 
Okay, so this time I decided to closr rather than restore tabs following the crash reported a minute or so ago. Browser popped again! Coumd be related to McAfee Site Advisor (guessing at the name)? Browser has been horrifically slow at times over the past several weeks. Chrome does not seem to suffer these pto=roblems, 	2017-04-19 00:37:37 	en-US 


8f79d0ba-d171-474e-9270-b8aa52170417 
As soon as I open Firefox it crashes. I have uninstalled and reinstalled the program already. 	2017-04-17 15:44:58 	en-US 


ee468392-d824-4994-b7c2-1ef490170419 
crash on start with every program every time, every way. I'm all up to date with 4.4. may be within last hardware com. to FF compli. update. or a open inserted back door. 	2017-04-19 02:12:22 	en-US 


0a828698-6f66-4081-9f77-5b4002170417 
fiefox is not even opening to a tab before shutting down 	2017-04-17 14:42:32 	en-US 


4f404600-8cfd-4cae-85c9-28cfe0170419 
I am getting used to Firefox constantly crashing. Opera runs just fine along with any other operating system and browzers. Only Firefox do I have a crashing problem. I am tempted to find another system that will work. Too much of my time is wasted rebooting. 	2017-04-19 17:49:03 	en-US 
4a9c05e7-76fe-49c5-a548-5b54d0170420 
i want to see youtube but it doesnt let me 	2017-04-20 18:59:09 	en-US 


e989e506-f7dc-4461-bd86-13fce0170421 
browser gives up too fast on flaky tablet. need more time to type URL. 	2017-04-21 14:41:05 	en-US 
59e1a9fa-81fc-4b05-bf17-0a3840170421 
Crashes every other time I click a link. 	2017-04-21 02:18:54 	en-US 
20649e2b-dec2-4276-97ef-6a3620170421 
f 	2017-04-21 11:05:14 	None 
eb4c71de-0248-4dea-a687-7ca620170421 
Firefox crashed again. It happens all the time. 	2017-04-21 17:22:43 	None 
b6c308a8-2071-4760-a8bf-cdc0d0170422 
FIX THIS NOW!!!!!!!!!!!!!!!!!!!1 	2017-04-22 17:34:06 	en-US 


f35bf9f2-b9c8-4106-a757-bac390170421 
It keeps crashing?! 	2017-04-21 01:44:26 	en-US 
2a1446e5-935d-4915-8784-a5e840170421 
Keeps on crashing... 	2017-04-21 09:51:41 	en-US 
76679c11-26b9-481a-a354-35fb20170423 
Keeps on crashing.Slow to load etc. It was o.k. when I first used it. Bloody useless ! I was on facebook at the time 	2017-04-23 08:02:30 	en-GB 


1f70e79c-7293-44f5-9527-011950170420 
The 10th time its crashed today 	2017-04-20 23:38:00 	en-GB 
12c8a710-1a49-461c-9d44-156dd0170422 
This crush has been occured until I updated my PC to Windows 10 Creators Update from Windows 10 Anniversary Update. 	2017-04-22 06:54:38 	ja 
533cac29-7203-4127-a302-c9c500170422 
trying to get to home page yahoo.com But Firefox closes on it's on 	2017-04-22 21:32:39 	en-US 
2b1def0e-a4a5-4e19-96e9-29ba20170423 
way 	2017-04-23 06:27:32 	None 
e4975bef-ad56-4848-9a8c-b2f190170423 
What is going on with Firefox it crashes ALL the time!!!!! 	2017-04-23 00:17:58 	en-US
[Tracking Requested - why for this release]:

This crash signature is top browser crasher # 6.

EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER 

Count Win Mac Lin Inst isGC	
109 	0 	0 	0 	56 	3

Top Crashers for Firefox 55.0a1
Top 50 Crashing Signatures. 7 days ago through in a few seconds.

The report covers 2780 (48.34%) of all 5751 crashes during this period.
We would really need to be able to reproduce this crash in order to solve this issue - although there are lots of reports, without a stack it is difficult to debug.
Signature> EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER 

Top Crashers for Firefox 55.0a1
Top 50 Crashing Signatures. 7 days ago through 2 minutes ago. 

Top browser crasher #3.


Top Crashers for Firefox 54.0b
Top 50 Crashing Signatures. 7 days ago through 2 minutes ago. 

Top browser crasher #2.


Top Crashers for Firefox 54.0a2
Top 50 Crashing Signatures. 7 days ago through 3 minutes ago. 

Top browser crasher #1.


Top Crashers for Firefox 53.0.2
Top 50 Crashing Signatures. 7 days ago through 3 minutes ago. 

Top browser crasher #2.
(In reply to Marcia Knous [:marcia - use ni] from comment #6)
> We would really need to be able to reproduce this crash in order to solve
> this issue - although there are lots of reports, without a stack it is
> difficult to debug.

Skywalker333,
the signature means, that FF crashes, but the crash reporter (Breakpad [CrashPad]) isn't able to collect data of the crash. There are no crashing thread identified (look at the end of a crash report, e.g. https://crash-stats.mozilla.com/report/index/25ce4993-56d1-4341-929d-6a5c20170509, threads are missing) and no minidump header is send.
The engineers of FF need this stacks to analyze whats happened in FF before FF crash - but without data they can't analyze anything.
What they need (when all other things are missing) are detailed steps to reproduce the crash (e.g. info about which add-on crash FF with this sig).
E.g. info like in crash https://crash-stats.mozilla.com/report/index/25ce4993-56d1-4341-929d-6a5c20170509:
"In OSX from 'Apple' Icon menu of Finder switching Network 'Location' causes FireFox to crash."
Such crashes can be tried to reproduce and analyzed. Maybe you can find comments to crash reports that have such information.

Greets, Tobias.
(In reply to skywalker333 from comment #2)
> Bug 1245570 
> (In reply to Kevin Brosnan [:kbrosnan] from comment #13) 
> > Firefox for Android does not ship ESR. This signature is not a one bug fix.
> > It can occur for a variety of reasons one of the most common and
> > unaddressable is the device that Firefox is running on is out of memory.

"out of memory" (OOM) can happen on desktops, too. But there was a try (some times ago) to make a memory dump when the task runes out of memory before FF crash. Don't know what happened to this try and why breakpad can't save this info before FF crashes, but normally OOM should be this sig: https://crash-stats.mozilla.com/signature/?signature=OOM%20|%20small
See Also: → 711568
(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #8)
> E.g. info like in crash
> https://crash-stats.mozilla.com/report/index/25ce4993-56d1-4341-929d-
> 6a5c20170509:
> "In OSX from 'Apple' Icon menu of Finder switching Network 'Location' causes
> FireFox to crash."

Tracy, sorry, don't know who else to tag for reproduce (QA).

(Is flag in-qa-testsuite for this or is there any other way to ask QA for reproduce?)
Flags: needinfo?(twalker)
qawanted (along with a note in whiteboard indicating the specific need) is the keyword to use here.  I don't know who is actively triaging that keyword.  Let's see what happens.  I'll leave myself NI'd to check in later if no one pics it up.
QA Whiteboard: Need steps to reproduce this crash
Keywords: qawanted
Whiteboard: Need steps to reproduce this crash
IIRC I have been getting these signatures when running inside `firejail` without `--allow-debuggers`. So maybe there's some amount of (linux) reports caused by this.
This isn't actionable at this point, not tracking for 55.
Comments checked from 2017-05-13 15:07:26 to 2017-05-14 23:15:32 (15 pages).
https://crash-stats.mozilla.com/signature/?signature=EMPTY%3A%20no%20crashing%20thread%20identified%3B%20ERROR_NO_MINIDUMP_HEADER#comments

f36a3f43-7772-4d2f-94e5-1ade80170513 	I've been getting this error every time I shut down. I'm running Firefox in Firejail, using its default config. 	2017-05-13 15:37:37 	en-US

97b14d02-d462-450b-b959-051c20170514 	I was using the JW.org app for Daily Bible reading. 	2017-05-14 10:00:14 	en-US

818e547f-b905-4916-ba54-ec5a80170516 	it just crashed immidiately as I oppened the browser. 	2017-05-16 01:59:38 	en-US

9ee4c76a-b423-4fd7-8ad1-914870170517 	i encountered problem of opening firefox 	2017-05-17 10:07:51 	en-US

966f9a80-87aa-4263-addf-1addb0170518 	this is a xolo a500 qualcomm msm 8225 ics 4.0.4 android gsm,3g,4.5 screen,512mb ram,rooted, I just started the app. it crashes. I have enabled telemetry and other data also. 	2017-05-18 11:55:06 	None

dbae88e6-7aeb-47e4-94e0-65e770170518 	ich bekomme mehrmals die Info: Der Computer hat nicht genug Arbeitsspeicher. 	2017-05-18 10:10:48 	de (<- Msg not enough mem.)

66748124-c446-49a8-926f-b2e8b0170519 	Firefox stürzt in letzter Zeit öfters ab: bei FAZ Auto und Verkehr 	2017-05-19 12:09:47 	de

1c7eb083-7b10-4946-b015-8d2270170519 	Moving 29,000 junk mail messages to junk folder 	2017-05-19 12:25:03 	None

4511a976-5b6e-4801-bfb7-4d07d0170517 	Since thursday, my computer is so slow, it is running in the background, but I have all windows closed. Now it is telling me I am low on memory, have defrag, and took all unnecesary programs off. 	2017-05-17 21:24:22 	en-US

e21fc878-4460-45fd-96ad-dfebb0170513 	BEIM STARTEN??? 	2017-05-13 23:04:09 	de (<- At start.)

b29b6000-115c-4129-b3be-a40540170514 	I started the browser and encountered this error screen. mark. automated qa 	2017-05-14 01:25:45 	None

c4e38fdd-9eb9-42e4-a3bc-c05f20170515 	Rubenstein you've downloader2.3.8 quisiera bajarme yo tubeMate YouTube 22 23 8 	2017-05-15 18:59:17 	None

d4b3ed22-64f5-4650-9c74-366f10170516 	It has become very slow, unresponsive, laggigng and some websites don't work including youtube.com and airmp3.me 	2017-05-16 09:24:29 	en-GB

eea3397d-be6a-40d8-ab4d-1359a0170516 	its being slow and freezing and not respoinding 	2017-05-16 10:09:06 	en-US

e87c4a7f-daf6-4ad7-bb1d-a54550170518 	Arbeitspeicher ist nur zu kacke 	2017-05-18 12:08:05 	None (<- Mem problems.)

3aac0f39-4d71-4e2a-bbd8-817d00170514 	I was using the JW.org app for Daily Bible reading. 	2017-05-14 10:00:14 	en-US

4606e85e-0ee8-4be4-92ec-1738f0170514 	crashed again ... after 10 min of being open 	2017-05-14 09:36:45 	en-GB

89a3b81b-78a8-4bf0-93f3-43eae0170520 	I tried to open the link from gmail by clicking on a toast with text "the tab was added to Firefox", and there was a tab opened with video from coursera. Possibly, Firefox was closed, as I have an application manager which stops running processes after user inactivity. 	2017-05-20 03:18:25 	None

a83fced3-0e3f-4744-ba1e-904f90170520 	tried to load carsales.com.au, was part way through opening program, then firefox froz. 	2017-05-20 11:21:40 	chrome://global/locale/intl.properties

00e274f4-951d-4b65-a077-22bf40170514 	why is it every time I go onto Winged wonders of the world, which is my group, firefox shuts me down? It is so annoying! 	2017-05-14 20:05:44 	en-US

57160d63-7073-4f58-9872-b16b50170517 	I have been having problems with it saying memory low but I just closed it and restarted it and then when reopened, it crashed 	2017-05-17 15:31:46 	en-US

db47c7f5-f9c5-47a6-ab66-518f40170517 	it keeps closimg whenever i open it 	2017-05-17 13:11:08 	en-US

a2fb16f3-6f4d-490e-a221-a783d0170519 	my pc keeps crashing even though i try to restart firefox. 	2017-05-19 22:03:48 	en-US
OK, some thoughts to it...

(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #9)
> (In reply to skywalker333 from comment #2)
> > Bug 1245570 
> > (In reply to Kevin Brosnan [:kbrosnan] from comment #13) 
> > > Firefox for Android does not ship ESR. This signature is not a one bug fix.
> > > It can occur for a variety of reasons one of the most common and
> > > unaddressable is the device that Firefox is running on is out of memory.
>
> "out of memory" (OOM) can happen on desktops, too. But there was a try (some
> times ago) to make a memory dump when the task runes out of memory before FF
> crash. Don't know what happened to this try and why breakpad can't save this
> info before FF crashes, but normally OOM should be this sig:
> https://crash-stats.mozilla.com/signature/?signature=OOM%20|%20small
>
Why we have so much crashes with OOM with this sig when OOM should be a other sig?

(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #14)
> 818e547f-b905-4916-ba54-ec5a80170516 	it just crashed immidiately as I
> oppened the browser. 	2017-05-16 01:59:38 	en-US
> 
Is it possible to start/load breakpad earlier to get some more threads?

And whats about a check & msg like "Firefox needs at least xxxMB to start!" to prevent some of the crashes because OOM?
I got this type of crashes on ニコニコ動画 with flash version player.
Sometimes, just to start playing the movie, browser becomes totally unresponsive unless stop plugin-container process.
(In reply to Toshihiro Yamada from comment #16)
> I got this type of crashes on ニコニコ動画 with flash version player.
Is this a URL? Can you write the complete URL as a link like https://bugzilla.mozilla.org/?
Flags: needinfo?(yamadat501)
URL of ニコニコ動画 is here(need registration to play movies)
http://www.nicovideo.jp/

I got browser freeze on various movies with flash version.
For example:
http://www.nicovideo.jp/watch/nm15429765
http://www.nicovideo.jp/watch/nm15268636
Flags: needinfo?(yamadat501)
OK, I registered with Twitter and tried it...
By both videos I have "high memory action" in the Task Manager, but the videos don't play!
I use Firefox 55.0a1, 64-bit with E10s on Win7 and Flash 26.0.0.115 with "Block dangerous and intrusive Flash content" activated.
Toshihiro Yamada, what version of FF and Flash (info like me above) do you use?
Can anybody else test/confirm it?
Can imagine that this are damaged videos or exploits... but IMHO FF should crash with OOM if the "high memory action" is the problem...
Flags: needinfo?(yamadat501)
(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #19)
> OK, I registered with Twitter and tried it...
> By both videos I have "high memory action" in the Task Manager, but the
> videos don't play!
> I use Firefox 55.0a1, 64-bit with E10s on Win7 and Flash 26.0.0.115 with
> "Block dangerous and intrusive Flash content" activated.
> Toshihiro Yamada, what version of FF and Flash (info like me above) do you
> use?
> Can anybody else test/confirm it?
> Can imagine that this are damaged videos or exploits... but IMHO FF should
> crash with OOM if the "high memory action" is the problem...

I use latest 55.0a1 nightly on 64-bit Win10 and Flash 25.0.0.171.

I also noticed "Block dangerous and intrusive Flash content" blocks playing videos.
(In my environment, this setting works only in safe mode. But as the freeze happened in safe-mode, it is other problem...)

Recently, ニコニコ動画 provides html5 player for modern browsers. (For example: http://www.nicovideo.jp/watch/sm2416354)
In this case, video plays normally with html5 version player and the problem happens with flash version player.
Flags: needinfo?(twalker)
Wontfix for 53 and likely for 54. Is there any way we can improve the info we're getting in the crash to have better diagnosis of what's happening? 
Maire, is this something the a/v team might help investigate, since the recent spike in this signature may have something to do with video?
Flags: needinfo?(mreavy)
Flags: needinfo?(Tobias.Besemer)
I wish we could help, but there's nothing actionable for an A/V developer here. 

Ted -- Do you have any ideas how we can get better signatures from this?
Flags: needinfo?(mreavy) → needinfo?(ted)
This is effectively just "we failed to write a minidump". `ERROR_NO_MINIDUMP_HEADER` means the stackwalker couldn't read the 32-byte minidump header, which usually means the file is zero bytes in length.

> Why we have so much crashes with OOM with this sig when OOM should be a other sig?

I don't think these are necessarily all OOM crashes.

> Is it possible to start/load breakpad earlier to get some more threads?

Breakpad is initialized very early in application startup, before we spawn any other threads, AFAIK. Unfortunately writing a minidump from within a crashed process is doomed to fail sometimes.

I picked an arbitrary crash report from a recent Firefox desktop version out of the list in the query in comment 14:
https://crash-stats.mozilla.com/report/index/ca4d07ce-71dd-449c-98e2-d99de0170607#tab-metadata

It claims to have a process uptime of "2 days, 48 minutes and 16 seconds". In the metadata tab I see:
GraphicsCriticalError	

|[0][GFX1-]: [D2D1.1] 4CreateBitmap failure Size(20,24000) Code: 0x80070057 format 0 (t=3786.81) |[1][GFX1-]: [D2D1.1] 4CreateBitmap failure Size(20,24000) Code: 0x80070057 format 0 (t=6849.66) 

It looks like we failed to create a very large bitmap. Unfortunately the MozCrashReason field is just "MOZ_CRASH()", so it doesn't give us any real useful info. I wonder if we shouldn't put __FILE__ and __LINE__ in there, or at least write them out in a separate annotation field to give ourselves something to work with in these situations?

https://dxr.mozilla.org/mozilla-central/rev/5801aa478de12a62b2b2982659e787fcc4268d67/mfbt/Assertions.h#269
Flags: needinfo?(ted)
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #23)
> Breakpad is initialized very early in application startup, before we spawn
> any other threads, AFAIK. Unfortunately writing a minidump from within a
> crashed process is doomed to fail sometimes.

Could an outside process be used to generate the minidump?

What about running it as a service (in Windows)? A breakpad service.
(In reply to skywalker333 from comment #24)
> (In reply to Ted Mielczarek [:ted.mielczarek] from comment #23)
> > Breakpad is initialized very early in application startup, before we spawn
> > any other threads, AFAIK. Unfortunately writing a minidump from within a
> > crashed process is doomed to fail sometimes.
> 
> Could an outside process be used to generate the minidump?
> 
> What about running it as a service (in Windows)? A breakpad service.

Sounds like a good idea to me!
Google do it now with GoogleCrashHandler.exe and GoogleCrashHandler64.exe.


(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #21)
> Wontfix for 53 and likely for 54. Is there any way we can improve the info
> we're getting in the crash to have better diagnosis of what's happening? 

(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #19)
> Can imagine that this are damaged videos or exploits... but IMHO FF should
> crash with OOM if the "high memory action" is the problem...

FF made in the past a memory-info and send it out to mozilla for analyzes when mem was low and a OOM was near...
Maybe someone should have a look on it again because maybe this fails in some cases. Maybe because it starts "to late" and can't finish before the crash?

(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #15)
> And whats about a check & msg like "Firefox needs at least xxxMB to start!"
> to prevent some of the crashes because OOM?

Is there such a "required resources"-check & -popup integrated in FF?
I can imagine that some crashes occur because of this. E.g. users use FF version >54 on WinXP or because of any other scenario (old machines/hardware and OSes) where support was dropped in the last releases...


(In reply to Toshihiro Yamada from comment #20)
> (In reply to Tobias B. Besemer [:BesTo] (QA) from comment #19)
> > I use Firefox 55.0a1, 64-bit with E10s on Win7 and Flash 26.0.0.115 with
> > "Block dangerous and intrusive Flash content" activated.
> > Toshihiro Yamada, what version of FF and Flash (info like me above) do you
> > use?
> 
> I use latest 55.0a1 nightly on 64-bit Win10 and Flash 25.0.0.171.
> 
> I also noticed "Block dangerous and intrusive Flash content" blocks playing
> videos.
> (In my environment, this setting works only in safe mode. But as the freeze
> happened in safe-mode, it is other problem...)

Toshihiro, do you have FF installed as a 32-bit, or a 64-bit version?
Do you have multi-process-support activated in your browser-settings?
Do FF really crash with this signature, when you try to watch the videos?
Flags: needinfo?(Tobias.Besemer)
(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #25)
> Is there such a "required resources"-check & -popup integrated in FF?
> I can imagine that some crashes occur because of this. E.g. users use FF
> version >54 on WinXP or because of any other scenario (old machines/hardware
> and OSes) where support was dropped in the last releases...

Sorry! Mean: "E.g. users use FF version >52.x on WinXP..."
(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #25)
> Toshihiro, do you have FF installed as a 32-bit, or a 64-bit version?
> Do you have multi-process-support activated in your browser-settings?
> Do FF really crash with this signature, when you try to watch the videos?

64-bit and e10s enabled.
A underlying problem is that tab frequently hangs when I try to watch videos on ニコニコ動画's flash-version player.
I need to stop the tab-process manually and I get this crash signatures.

I tried to investigated the situation more during the past few days, and I found a11y is related to the problem.
As far as I tested, no more hangs after I disabled a11y.
So, I think my case can boil down to the issue of a11y like Bug 1339589.
Flags: needinfo?(yamadat501)
(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #25)
> (In reply to Tobias B. Besemer [:BesTo] (QA) from comment #15)
> > And whats about a check & msg like "Firefox needs at least xxxMB to start!"
> > to prevent some of the crashes because OOM?
> Is there such a "required resources"-check & -popup integrated in FF?
> I can imagine that some crashes occur because of this. E.g. users use FF
> version >54 on WinXP or because of any other scenario (old machines/hardware
> and OSes) where support was dropped in the last releases...

...and a msg like "Please use Firefox Version 52.x with your Hardware/OS." would be nice.

Liz, should I fill a new bug for this?
Flags: needinfo?(lhenry)
See Also: → 1372650
Flags: needinfo?(lhenry)
Given that our crash-handling code already has in-process and OOP dump generation, and we are already generating minidumps OOP for child processes, I think it makes sense to also make dump generation OOP for the parent process.

A quick path for this is creating a child process that is used to monitor crash of the parent process. We can make it a preferred dump method and fallback to in-process dump if we fail to initialize this monitoring process for any reason.

If this turns out to be working well, we can migrate dump generation of all processes to this monitoring process.

Ted, is there any work ongoing for make dump generation of the parent process OOP? If not I am going to pursue this idea.
Flags: needinfo?(ted)
I've suggested in the past that breakpad could be run as an operating system service so that it could always handle crashes, error reporting, and minidump analysis, even when the main process is hung or crashed.
(In reply to Cervantes Yu [:cyu] [:cervantes] from comment #30)
> Ted, is there any work ongoing for make dump generation of the parent
> process OOP? If not I am going to pursue this idea.

We've had bug 587729 open for a long time, and I did some exploratory work around it, but never anything that was close to usable.

(In reply to skywalker333 from comment #31)
> I've suggested in the past that breakpad could be run as an operating system
> service so that it could always handle crashes, error reporting, and
> minidump analysis, even when the main process is hung or crashed.

This is what Google does with Chrome. It's a nice idea, but it does complicate things quite a bit.
Flags: needinfo?(ted)
We also see empty or partial minidump during automated tests. This is an example: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=124159471&lineNumber=4894 where there is an incomplete minidump for when we were generating paired minidumps. Maybe the process was force-killed before the it completed writing the dump.
For me, this crash occured right after a crash with a signature:
@ shutdownhang | mozilla::SpinEventLoopUntil<T> | nsThread::Shutdown | nsThreadPool::Shutdown

See the timing of these crashes:
https://crash-stats.mozilla.com/report/index/b6660cea-9e52-4afd-8a41-7c3270180106
https://crash-stats.mozilla.com/report/index/a840196f-c55e-4746-a8d7-e89b50180106

Not sure what was I doing at that time.
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #32)
> (In reply to Cervantes Yu [:cyu] [:cervantes] from comment #30)
> > Ted, is there any work ongoing for make dump generation of the parent
> > process OOP? If not I am going to pursue this idea.
> 
> We've had bug 587729 open for a long time, and I did some exploratory work
> around it, but never anything that was close to usable.


Depends on 587729.
See Also: → 610551
I received a crash with this signature when I ran out of hard drive space.
https://crash-stats.mozilla.com/report/index/cdbe3f47-178c-4f8b-879a-096f50180417

It seems... pretty reasonable that we can't write dump information without any free disk space. Is there any way to tell how many of these could be the same situation?

This is still happening in current Firefox versions, even for completely blank profiles, with a brand new reinstallation of Firefox.

Definitely not an OOM issue for me.

I'd also like to note that Firefox crashed on me 150+ times without any crash report before it produced a ERROR_NO_MINIDUMP_HEADER crash dump once. While that might not be the case for everyone, it could mean there's thousands more crashes than reported.

If it helps, I can give you about 45 Windows Error Report ids or .wer files. Some of them sent hundreds of megabytes to microsoft servers, there's bound to be something useful there.

.wer files are not very useful unfortunately but if you could provide me with a minidump collected locally via WER that would help a lot. To do so you need to add a registry key (HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps) and then when Firefox crashes look for the minidump under C:\Users\<username>\AppData\Local\CrashDumps. You can find more info about this here: Collecting User-Mode Dumps.

Send me the minidump directly via e-mail, do not attach it to this bug as it might contain privacy-sensitive information.

Flags: needinfo?(analytiq)

It's been a while here - Gabriele, is there actionable work we can do here? It looks like things have substantially improved since 3 months ago but crash volume is still non-trivial.

Flags: needinfo?(analytiq) → needinfo?(gsvelto)

Yes, I would like to add some error reporting to the code that generates minidumps so that we can track the root causes of the failures. Unfortunately I've got my hands full ATM and I don't think I'll be able to look into it before next month. I'll file a bug that blocks this one so I don't forget.

Flags: needinfo?(gsvelto)
Depends on: 1666733

This is a signature change caused by the new stack walker. BTW a quick glance at the remaining volume of crashes on desktop points to OOMs.

Crash Signature: [@ EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER] → [@ EMPTY: no crashing thread identified; EmptyMinidump] [@ EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER]
Crash Signature: [@ EMPTY: no crashing thread identified; EmptyMinidump] [@ EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER] → [@ EMPTY: no crashing thread identified; EmptyMinidump] [@ EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER] [@ OOM | large | EMPTY: no crashing thread identified; EmptyMinidump]

Hey gsvelto, what's a good Product :: Component for this bug?

Flags: needinfo?(gsvelto)

This definitely belongs to crash reporting

Component: General → Crash Reporting
Flags: needinfo?(gsvelto)
Product: Firefox → Toolkit
Crash Signature: [@ EMPTY: no crashing thread identified; EmptyMinidump] [@ EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER] [@ OOM | large | EMPTY: no crashing thread identified; EmptyMinidump] → [@ EMPTY: no crashing thread identified; EmptyMinidump] [@ EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER] [@ EMPTY: no frame data available; EmptyMinidump] [@ EMPTY: no frame data available; MissingThreadList] [@ OOM | large | EMPTY: …
Severity: critical → S2
Duplicate of this bug: 1803899

This bug makes Firefox crash with 100% reproducibility on many websites, in particular major ones, such as https://facebook.com/ and https://www.adobe.com/

(In reply to Vincent Lefevre from comment #56)

This bug makes Firefox crash with 100% reproducibility on many websites, in particular major ones, such as https://facebook.com/ and https://www.adobe.com/

This isn't reproducible for us, and the crashes have no information... Gabriele, is there some way we can get more info from Vincent given it's crashing 100% of the time for him?

Flags: needinfo?(gsvelto)

I've just tried again the same tests but with an access to the machine via ssh, thus with an X server running on a different machine, and Firefox doesn't crash in such a case. In short, the crash occurs only on some particular hardware and only locally. So the cause of the crash could be some form of race condition, or perhaps Firefox runs different code in this case.

(In reply to Vincent Lefevre from comment #58)

I've just tried again the same tests but with an access to the machine via ssh, thus with an X server running on a different machine, and Firefox doesn't crash in such a case. In short, the crash occurs only on some particular hardware and only locally. So the cause of the crash could be some form of race condition, or perhaps Firefox runs different code in this case.

It's odd to get a crash without a minidump but it can happen if some form of external sandboxing (or kernel security module) is used, as those can prevent the crash report from accessing /proc and thus generating the minidump. Could this machine be using something like that? Additionally the crash in bug 1803899 comment 5 shows that you're using a closed-source Nvidia driver on the machine. They're often a source of problems so that might also be the culprit.

Flags: needinfo?(gsvelto)

I'm using firejail (as I've done for several years) with the firefox profile. Even if I use --ignore='include disable-proc.inc' to fully allow /proc, I get the same behavior (crash without a crash report) with both FF 107 and Nightly.

And I'm using the Nvidia driver because the nouveau driver is too broken and the developers didn't react to my bug reports. This is a possible culprit as a new version appeared yesterday fixing security issues and Debian's changelog says "Improved compatibility with recent Linux kernels". So I'm going to do a few tests, then upgrade.

BTW, among the URLs that make Firefox crash, there's https://support.mozilla.org/en-US/kb/profile-manager-create-remove-switch-firefox-profiles (but I cannot reproduce the crash if I first download the HTML file and open it locally with the "file:" URL scheme).

The buggy Nvidia driver was the cause of the crashes. And when I rebooted, the display manager couldn't even start (apparently because the X server was failing) and the Linux virtual terminals didn't work either, though I had no issues with the reboot on November 30, after the latest updates. I had to do the upgrade via ssh from my phone.

Moreover, I inspected the journalctl logs and saw lots of lines like

Dec 02 19:53:18 zira audit[1440595]: SECCOMP auid=1000 uid=1000 gid=1000 ses=2 subj=firejail-default pid=1440595 comm="CanvasRenderer" exe="/usr/lib/firefox/firefox" sig=0 arch=c000003e syscall=101 compat=0 ip=0x7f22b960f275 code=0x50000

and

Dec 02 19:53:18 zira kernel: audit: type=1326 audit(1670007198.987:31): auid=1000 uid=1000 gid=1000 ses=2 subj=firejail-default pid=1440595 comm="CanvasRenderer" exe="/usr/lib/firefox/firefox" sig=0 arch=c000003e syscall=101 compat=0 ip=0x7f22b960f275 code=0x50000

starting at this date (nothing like that before, since the beginning of my logs on 2021-04-24). I suppose that this corresponded to the crashes (or was related to them).

Now, after the upgrade, everything is fine.

bp-a67c304a-b335-4718-8a5a-858970230107 Windows 10. It had been running for two weeks, but that's not unusual.
In addons manager I had just disabled adblock plus add-on, and then in addons manager was searching for ublock origin

Just a note to say that the absence of information in the crash reports is (predictably) causing multiple issues across multiple OS and packaging types to be conflated. For example, my crashes on Linux from a Flatpak installation are linked to this thread discussing Windows 10.

I can tell you that even in the Flatpak distribution, which should have updates disabled, Firefox is still downloading something and so randomly corrupting itself. This manifests as tabs suddenly start to crash and then either cannot be reloaded or crash again after e.g. scrolling. Sometimes the browser closes completely without warning and then cannot be relaunched until the installation has been repaired.

EMPTY: no frame data available; MissingThreadList #40 crash for Thunderdbird 102.9.0, all linux

Whiteboard: Need steps to reproduce this crash → [tbbird crash]

Once again, Firefox randomly closes itself (all tabs and windows are lost) due to self-corruption and cannot be relaunched until the installations is repaired:

$ flatpak repair --user
[19/213] Verifying flathub:app/org.mozilla.firefox/x86_64/stable…
fsck content object 266baf34571baf031d5968384f75582f0c1008d90663d6bcdd53b24613acad06: Corrupted file object; checksum expected='266baf34571baf031d5968384f75582f0c1008d90663d6bcdd53b24613acad06' actual='b47e27f3e9f5ff86828ef5d6f112922e45d49ffd9f29f34fec7f97e89f85fceb', deleting object
Marking commit as partial: efba60722ae51de8ec2d8b933f76f3f850f0e38f90325dc16c48c1581c4a530c
Deleting ref flathub:app/org.mozilla.firefox/x86_64/stable due to invalid objects

(In reply to womagrid from comment #66)

Once again, Firefox randomly closes itself (all tabs and windows are lost) due to self-corruption and cannot be relaunched until the installations is repaired:

$ flatpak repair --user
[19/213] Verifying flathub:app/org.mozilla.firefox/x86_64/stable…
fsck content object 266baf34571baf031d5968384f75582f0c1008d90663d6bcdd53b24613acad06: Corrupted file object; checksum expected='266baf34571baf031d5968384f75582f0c1008d90663d6bcdd53b24613acad06' actual='b47e27f3e9f5ff86828ef5d6f112922e45d49ffd9f29f34fec7f97e89f85fceb', deleting object
Marking commit as partial: efba60722ae51de8ec2d8b933f76f3f850f0e38f90325dc16c48c1581c4a530c
Deleting ref flathub:app/org.mozilla.firefox/x86_64/stable due to invalid objects

This should probably be filed as a separate bug, and marked as blocking bug 1278719. I'm not personally familiar with flatpak but it would also be useful if you could check a directory diff between a functional install and the broken one that flatpak is "repairing" (whatever that means). "Downloading something" is not really much to go on - Firefox downloads lots of stuff while running (not least stuff you tell it to download), but off-hand I can't think of anything that would normally end up in the application directory (neither for flatpaks nor anything else). It should all go either where the user tells it to put things, or in the user's profile directory, which isn't part of the flatpak itself. So I've no idea what the thing is that we could be downloading that is "corrupting" anything.

Flags: needinfo?(womagrid)

Predictable response, but the same thing used to happen with the .deb distribution. Yes, I'm sure that you would like more information, but I simply don't have it. This has been happening for years.

Flags: needinfo?(womagrid)

(In reply to womagrid from comment #68)

Predictable response, but the same thing used to happen with the .deb distribution. Yes, I'm sure that you would like more information, but I simply don't have it. This has been happening for years.

Is it not possible for you to zip up (or tar.gz or w/e) the "working" application directory and the "broken" version of the same app, and then compare the two?

Flags: needinfo?(womagrid)

If it happens again, I will try to remember to do that. Usually I am in the middle of something critical, so the priority is to get the browser working again ASAP. Fixing it via re-installation has become habitual at this point.

Flags: needinfo?(womagrid)

God, this has been affecting me 10+ times a day for the past week. Never happened before.

I'm on Linux Flatpak, version 112.0.2, here's the latest crash report: https://crash-stats.mozilla.org/report/index/98b86f9e-75a3-4387-ab24-8d2860230502

Do not mind the incorrect version number, that's because of issue https://bugzilla.mozilla.org/show_bug.cgi?id=1653852 ...

You need to log in before you can comment on or make changes to this bug.