Closed Bug 814954 Opened 12 years ago Closed 10 years ago

crash in JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern

Categories

(Core :: JavaScript Engine, defect)

Product:
Core
Component:
JavaScript Engine
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
RESOLVED WONTFIX
Tracking Flags:
Tracking Status
firefox23 --- wontfix
firefox24 - wontfix
firefox25 - wontfix
firefox26 --- wontfix
firefox27 --- wontfix
firefox28 --- wontfix
firefox29 --- wontfix
firefox30 --- wontfix
firefox31 --- wontfix
firefox32 --- unaffected
firefox33 --- unaffected

People

(Reporter: scoobidiver, Unassigned, Mentored)

References

(Blocks 1 open bug)

Details

(Keywords: crash, steps-wanted, topcrash-win, Whiteboard: [js:inv:p2])

Crash Data

It first showed up in 19.0a1/20121104. The regression range might be (discontinuous across builds): http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=2718739a1c83&tochange=5c6b71348e20 It might be an OOM according to System Memory Use Percentage in crash reports. Signature JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern(JSC::Yarr::PatternTerm*) More Reports Search UUID b9cfc62a-c635-4610-aa10-225fb2121125 Date Processed 2012-11-25 11:46:10 Uptime 184 Install Age 14.3 hours since version was first installed. Install Time 2012-11-24 21:27:45 Product Firefox Version 20.0a1 Build ID 20121124030824 Release Channel nightly OS Windows NT OS Version 6.1.7601 Service Pack 1 Build Architecture x86 Build Architecture Info AuthenticAMD family 16 model 4 stepping 2 Crash Reason EXCEPTION_ACCESS_VIOLATION_WRITE Crash Address 0x77840004 App Notes AdapterVendorID: 0x10de, AdapterDeviceID: 0x0605, AdapterSubsysID: 00000000, AdapterDriverVersion: 9.18.13.697 D2D? D2D+ DWrite? DWrite+ D3D10 Layers? D3D10 Layers+ EMCheckCompatibility False Adapter Vendor ID 0x10de Adapter Device ID 0x0605 Total Virtual Memory 2147352576 Available Virtual Memory 198828032 System Memory Use Percentage 95 Available Page File 2866634752 Available Physical Memory 170909696 Frame Module Signature Source 0 mozjs.dll JSC::Yarr::YarrGenerator<1>::opCompileParenthesesSubpattern js/src/yarr/YarrJIT.cpp:2359 1 mozjs.dll JSC::Yarr::YarrGenerator<1>::opCompileAlternative js/src/yarr/YarrJIT.cpp:2440 2 mozjs.dll JSC::Yarr::YarrGenerator<1>::opCompileBody js/src/yarr/YarrJIT.cpp:2515 3 mozjs.dll JSC::Yarr::YarrGenerator<1>::compile js/src/yarr/YarrJIT.cpp:2639 4 mozjs.dll js::detail::RegExpCode::compile js/src/vm/RegExpObject.cpp:201 5 mozjs.dll js::RegExpShared::compile js/src/vm/RegExpObject.cpp:453 6 mozjs.dll js::RegExpCompartment::get js/src/vm/RegExpObject.cpp:582 7 mozjs.dll js::NewObjectCache::invalidateEntriesForShape js/src/jsscope.cpp:1261 8 mozjs.dll js::ExecuteRegExp js/src/builtin/RegExp.cpp:578 9 mozjs.dll ExecuteRegExp js/src/builtin/RegExp.cpp:646 10 mozjs.dll js::regexp_test js/src/builtin/RegExp.cpp:678 11 mozjs.dll js::InvokeKernel js/src/jsinterp.cpp:369 12 mozjs.dll js::Interpret js/src/jsinterp.cpp:2334 13 mozjs.dll js::RunScript js/src/jsinterp.cpp:326 14 mozjs.dll js::InvokeKernel js/src/jsinterp.cpp:384 15 mozjs.dll js::Invoke js/src/jsinterp.cpp:417 16 mozjs.dll JS_CallFunctionValue js/src/jsapi.cpp:5793 ... More reports at: https://crash-stats.mozilla.com/report/list?signature=JSC%3A%3AYarr%3A%3AYarrGenerator%3Cint%3E%3A%3AopCompileParenthesesSubpattern%28JSC%3A%3AYarr%3A%3APatternTerm*%29
Whiteboard: [js:inv:p2]
It's not a regression but a crash signature that has morphed. See the previous one: https://crash-stats.mozilla.com/report/list?signature=JSC%3A%3AYarr%3A%3AYarrGenerator%3A%3AopCompileParenthesesSubpattern%28JSC%3A%3AYarr%3A%3APatternTerm*%29
Crash Signature: [@ JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern(JSC::Yarr::PatternTerm*)] → [@ JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern(JSC::Yarr::PatternTerm*)] [@ JSC::Yarr::YarrGenerator::opCompileParenthesesSubpattern(JSC::Yarr::PatternTerm*)]
Keywords: regression
Version: 19 Branch → Trunk
Crash Signature: [@ JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern(JSC::Yarr::PatternTerm*)] [@ JSC::Yarr::YarrGenerator::opCompileParenthesesSubpattern(JSC::Yarr::PatternTerm*)] → [@ JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern(JSC::Yarr::PatternTerm*)] [@ JSC::Yarr::YarrGenerator::opCompileParenthesesSubpattern(JSC::Yarr::PatternTerm*)] [@ JSC::Yarr::YarrGenerator<int>::opCompileAlternative(JSC::Yarr::PatternAlt…
It's #52 browser crasher in 23.0, #70 in 24.0b1, #13 in 25.0a2, and #38 in 26.0a1. It's correlated to Ghostery: * 23.0: JSC::Yarr::YarrGenerator<int>::opCompileAlternative(JSC::Yarr::PatternAlternative*)|EXCEPTION_BREAKPOINT (85 crashes) 60% (51/85) vs. 1% (326/51156) firefox@ghostery.com (Ghostery, https://addons.mozilla.org/addon/9609) 1% (1/85) vs. 0% (2/51156) 2.9.4 22% (19/85) vs. 0% (185/51156) 2.9.6 29% (25/85) vs. 0% (116/51156) 5.0.0 7% (6/85) vs. 0% (13/51156) 5.0.1 * 24.0b1: JSC::Yarr::YarrGenerator<int>::opCompileAlternative(JSC::Yarr::PatternAlternative*)|EXCEPTION_BREAKPOINT (14 crashes) 36% (5/14) vs. 0% (68/22597) firefox@ghostery.com (Ghostery, https://addons.mozilla.org/addon/9609) 14% (2/14) vs. 0% (35/22597) 2.9.6 21% (3/14) vs. 0% (25/22597) 5.0.0
Summary: crash in JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern → crash in JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern mainly with Ghostery
It's now #20 browser crasher in 23.0, #5 in 24.0b1, #10 in 25.0a2, and #30 in 26.0a1.
tracking-firefox24: --- → ?
Keywords: topcrash
Crash Signature: JSC::Yarr::YarrGenerator<int>::opCompileAlternative(JSC::Yarr::PatternAlternative*) ] → JSC::Yarr::YarrGenerator<int>::opCompileAlternative(JSC::Yarr::PatternAlternative*) ] [@ mozjs.dll@0x291265 ] [@ mozjs.dll@0x28ca0e ]
Ioana, can you please see if someone on your team can reproduce this?
Keywords: qawanted, steps-wanted
QA Contact: ioana.budnar
This continues to be high-volume, being #5 in yesterday's 23.0 data. Naveed, can we get someone who is somewhat familiar with Yarr to look into this? It looks like the crash is in the Yarr code we ship, even though Ghostery is the trigger. Jorge, can we find out if a Ghostery update became available to public updates this week? If that was the case on Tuesday or Wednesday, that's the trigger for this and it may make sense to work with them to find some way of working around the problem until we can ship a fix.
Flags: needinfo?(nihsanullah)
Flags: needinfo?(jorge)
Version 5.0.1 was released on Tuesday but it was later deleted by the developer. A new version is awaiting review, but I don't know if it addresses this problem. I contacted the developer about it.
Flags: needinfo?(jorge)
5.0.3 release fixes this issue. In 5.0.1 an infinite loop occurs between abp and ghostery.
Err, forgot to mnetion, I am the dev for Ghostery.
(In reply to felix from comment #8) > 5.0.3 release fixes this issue. In 5.0.1 an infinite loop occurs between abp > and ghostery. Thanks! So is this an issue on the add-on side or is there an issue in our code behind it and you just triggered it?
Didn't manage to reproduce the crash with Firefox 23 on a Windows 7 32bit machine. I tried the next scenarios: - installed the next add-ons: Adblock Plus, AutoCopy 2, BetterPrivacy, Download Helper, Ghostery 5.0.3, Greasemonkey, IE Tab, NoScript, Stylish, TableTools - tried to send an email from a Yahoo address to a Gmail one - open >4 tabs: espn.com, facebook (kept on scrolling), punchbowl.com, treegear.com.au, a movie on youtube, drudgereport - opened and browsed through both asu.edu and http://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn.aspx - selected + middle-clicked on a bookmark - switched tabs - looked at a book preview on Amazon, then hit the down arrow multiple times quickly - opened ~10 tabs in 2 windows, containing: cnn.com, facebook, twitter, dealnews, a couple of forums, amazon, instagram, etc.
Keywords: qawanted
this has fallen out of topcrash on all channels. It appears ghostery 5.0.3 has indeed fixed this issue.
Keywords: topcrash
Given comment 12, I am removing this from our tracking lists.
tracking-firefox24: + → -
tracking-firefox25: + → -
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WORKSFORME
Uh, regardless how well Ghostery plays with AdBlock Plus, it shouldn't be able to induce a YARR crash. This is still a valid bug, from the sounds of it, if we can get steps to reproduce it. (Good that the Ghostery update doesn't seem to trigger this, of course -- but that doesn't mean there's nothing to fix on our side. After all, other extensions, or web code, could conceivably trigger the exact same issue.)
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Summary: crash in JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern mainly with Ghostery → crash in JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern with Ghostery 5.0.1
Jeff, a very valid point. We can provide 5.0.1 for testing, but I can describe how to replicate the issue. Essentially, we had an issue with removing our content policy when upgrading (using restart-less, so had to clean up), for some reason the nsiCategoryManager.deleteCategoryEntry would not remove our addon during the upgrade. So we opted for a hacky solution (pitched by Mossop) that used enumerateCategories(), copied the ones into array except ours, used deleteCategory() for content policy category and then readded (addCategoryEntry) categories copied earlier. When coupled with ABP, this procedure would eat up 100% of the CPU. As far as I understand its because ABP actually looks for the event that remove categories, and if it finds that its content policy is being removed, it will simply re-add itself. Why this would cause high CPU load is the root of the bug, potentially its because there would be registration for 2 content-policies from ABP, but thats a guess.
QA Contact: ioana.budnar
I am still seeing this issue. I am running Ghostery 5.0.4 on Firefox 24.0 (winXP/sp3).
Just crashed with Ghostery 5.0.4 and ABP 2.2.4 on Fx 25.0.1 (win7/32bit) https://crash-stats.mozilla.com/report/index/1bc30540-2ee9-4dd8-bbea-2d2da2131120
(In reply to Ezra from comment #17) > Just crashed with Ghostery 5.0.4 and ABP 2.2.4 on Fx 25.0.1 (win7/32bit) > > https://crash-stats.mozilla.com/report/index/1bc30540-2ee9-4dd8-bbea- > 2d2da2131120 Do you have steps to reproduce? What were you doing before Firefox crashed?
This crash is now showing up more as a result of fixing the crash reporting in OOM situations (previously it was lumped with all the other unknown OOMs in EMPTY DUMP). The crash is an EXCEPTION_BREAKPOINT at http://hg.mozilla.org/releases/mozilla-release/annotate/d20d499b219f/js/src/yarr/YarrJIT.cpp#l2480 which is: m_ops.append(term); and m_ops is a Vector<YarrOp, 128> I'm not actually sure *which* Vector this is: mfbt/Vector.h and js/src/yarr/wtfbridge.h and js/public/Vector.h although dxr is linking me to the wtfbridge version so that seems most likely, which points to http://mxr.mozilla.org/mozilla-central/source/js/src/yarr/wtfbridge.h#167 as the MOZ_CRASH we're hitting. How many ops is reasonable to have in a compiled regex like this? Should this code be using a fallible vector and checking/bailing out on OOM conditions?
Blocks: 943017
This is #11 in early 26.0 data
Keywords: topcrash-win
This continues to be in the top 20 on 26, module correlations for yesterday have these at the top: 91% (290/317) vs. 35% (13380/37691) dhcpcsvc.dll 82% (261/317) vs. 27% (10204/37691) dhcpcsvc6.DLL 85% (271/317) vs. 32% (12053/37691) mfplat.dll 85% (270/317) vs. 32% (12002/37691) ksuser.dll 70% (222/317) vs. 19% (7095/37691) WindowsCodecs.dll I think those are all Windows DLLs. There is no significant correlation to add-ons here, but I somewhat feel like there might be a connection to bug 856796 rising as well, both are in JSC:Yarr. Those crashes might have happened all along with EMPTY signatures and missing minidump and just be more visible now that we are able to get minidumps in many scenarios due to reserving memory for crash reporter.
Ranks #8 on Firefox 26 at 1.20%, rising 0 positions in the last week. Ranks #11 on Beta 27 at 0.78%, rising 17 positions in the last week. Ranks #20 on Aurora 28 at 0.80%, rising 22 positions in the last week. Ranks #103 on Nightly 29 at 0.11%, rising 14 positions in the last week.
status-firefox23: affected → wontfix
status-firefox24: affected → wontfix
status-firefox25: affected → wontfix
status-firefox26: --- → affected
status-firefox27: --- → affected
status-firefox28: --- → affected
status-firefox29: --- → affected
I looked into this, and it's strange. The vector on which the append fails is a wrapped MFBT Vector. It should just grow when capacity overflows. A simple test case that just contains a lot of grouped elements (as in /(a)(a)..(a)/, with the groups repeated a few dozen times) confirms that this does indeed happen. However, the reports don't look like it's an OOM crash. (In reply to Benjamin Smedberg [:bsmedberg] from comment #19) > How many ops is reasonable to have in a compiled regex like this? Should > this code be using a fallible vector and checking/bailing out on OOM > conditions? Bailing out instead of crashing would mean serious changes to Yarr. It's not meant to recover from OOM at all, so just crashing in a (real) OOM situation is the expected behavior.
Current Rank > Firefox 29: #24 @ 0.47% > Firefox 28: #9 @ 1.25% > Firefox 27: #8 @ 0.95% > Firefox 26: #9 @ 1.21%
Fx 29: #42 @ 0.28% Fx 28: #12 @ 0.92% Fx 27: #5 @ 1.45% Fx 26: #7 @ 1.44%
FWIW, I don't see a significant correlation with anything non-Windows there right now, certainly not a Ghostery one.
This is wontfix for Firefox 26 as it's EOL with tomorrow's release of Firefox 27.
status-firefox26: affected → wontfix
Summary: crash in JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern with Ghostery 5.0.1 → crash in JSC::Yarr::YarrGenerator<int>::opCompileParenthesesSubpattern
The only signature showing up in significant volume now is JSC::Yarr::YarrGenerator<int>::opCompileAlternative(JSC::Yarr::PatternAlternative*). Top URLs: 138 https://www.facebook.com/ 134 about:blank 49 http://ct1.addthis.com/static/r07/sh149.html# 14 http://templates.buscape.com/dynatemplate/msnplus_800x440.html?type=800x440 12 http://s7.addthis.com/static/r07/sh149.html# 11 https://mail.google.com/mail/u/0/?shva=1#inbox 10 https://www.facebook.com/?ref=tn_tnmn 10 http://www.youtube.com/ Top DLLs (>= 90%): 100% (796/795) vs. 75% (64627/85820) mswsock.dll 100% (795/795) vs. 76% (64969/85820) wintrust.dll 100% (795/795) vs. 73% (62889/85820) dnsapi.dll 100% (795/795) vs. 66% (56772/85820) nssckbi.dll 100% (795/795) vs. 66% (56868/85820) nssdbm3.dll 100% (795/795) vs. 66% (56870/85820) freebl3.dll 100% (795/795) vs. 66% (57012/85820) softokn3.dll 100% (795/795) vs. 67% (57397/85820) winrnr.dll 100% (794/795) vs. 74% (63184/85820) dbghelp.dll 100% (792/795) vs. 71% (61015/85820) browsercomps.dll 99% (788/795) vs. 66% (56986/85820) rsaenh.dll 99% (790/795) vs. 68% (58347/85820) rasadhlp.dll 96% (766/795) vs. 64% (54936/85820) icm32.dll 96% (764/795) vs. 69% (59487/85820) wbemprox.dll 96% (763/795) vs. 69% (59214/85820) wbemcomn.dll 95% (759/795) vs. 69% (59008/85820) wbemsvc.dll 95% (758/795) vs. 68% (58784/85820) fastprox.dll 95% (755/795) vs. 56% (47859/85820) ntmarta.dll 92% (729/795) vs. 62% (52817/85820) apphelp.dll 91% (722/795) vs. 63% (54182/85820) ntdsapi.dll 90% (717/795) vs. 36% (31171/85820) dhcpcsvc.dll Top Add-ons: 28% (221/795) vs. 14% (12311/85820) {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} (Adblock Plus, https://addons.mozilla.org/addon/1865) 6% (51/795) vs. 1% (890/85820) WebSiteRecommendation@weliketheweb.com 10% (83/795) vs. 5% (4410/85820) wrc@avast.com 6% (44/795) vs. 0% (309/85820) ffxtlbra@softonic.com Some key phrases mentioned in comments: * HTML5 video on Youtube * Printing coupons * Uploading photos * Reminderfox when tagging an element as "finished" Flagging for QAWANTED to see if we can reproduce anything in the next Beta based in the above info.
Keywords: qawanted
QA Contact: ioana.budnar
QA Contact: ioana.budnar → andrei.vaida
I was unable to reproduce the crash on the latest Beta (Build ID: 20140213172947), Windows 7 32-bit. Several attempts were made for a couple of a hours, using the crash associated add-ons and websites, but with no success.
Keywords: qawanted
Thanks for trying, Andrei. Till, does the information in comment 28 provide you any new leads?
Flags: needinfo?(till)
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #30) > Till, does the information in comment 28 provide you any new leads? Not really, sadly. It gives more info, though: this is pretty obviously triggered by regexps in chrome JS. It might well be triggered by multiple regexps, maybe even arbitrary ones. But about:blank being in the list of URLs, it can't be caused by specific regexps used by some websites. It'd be really helpful to have at least one of these regexps. Even that might be a dead end, though: maybe the crash is caused by something else and just triggered by whatever regexp is compiled next.
Flags: needinfo?(till)
QA Contact: andrei.vaida
(In reply to Till Schneidereit [:till] from comment #31) > It'd be really helpful to have at least one of these regexps. Even that > might be a dead end, though: maybe the crash is caused by something else and > just triggered by whatever regexp is compiled next. Is this information we can get from the crash reports, or elsewhere?
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #32) > Is this information we can get from the crash reports, or elsewhere? It's not readily available anywhere. :( The only way to get it would be to find a website in the crash reports that only has a small number of different regexps, so that it's likely that one of them is involved. However, given that about:blank is one of the associated URLs, it might just be the case that all crashes are actually triggered by some internal, chrome regexp.
(In reply to Till Schneidereit [:till] from comment #33) > However, given that about:blank is one of the associated URLs, it might just > be the case that all crashes are actually triggered by some internal, chrome > regexp. Would it be easier to start here, since it's an internally controlled asset, as opposed to trying to investigate an external website? How would we go about investigating this?
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #34) > Would it be easier to start here, since it's an internally controlled asset, > as opposed to trying to investigate an external website? How would we go > about investigating this? Sadly, that's not how it works in this case. I think it's highly likely that the regexp that triggers this isn't in about:blank at all. It can be anywhere in chrome code at all. I really don't know any way to investigate this further, as it seems almost impossible to create a reduced test case. Solid STR would be the only really helpful thing, I think.
Also note that what we report as the URL is IIRC just the most recently loaded URL. There might even be some tab in the background that runs JS and causes this, or some chrome code running anywhere that triggers the crash. The URL in the crash doesn't necessarily mean that the offending code was run from that URL load, it's just meant to be a helper to make it easier to find likely culprits. This is by far not exact science.
Wontfix for Firefox 28 given we are releasing tomorrow.
status-firefox27: affected → wontfix
status-firefox28: affected → wontfix
(In reply to Till Schneidereit [:till] from comment #35) > (In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #34) > > Would it be easier to start here, since it's an internally controlled asset, > > as opposed to trying to investigate an external website? How would we go > > about investigating this? > > Sadly, that's not how it works in this case. I think it's highly likely that > the regexp that triggers this isn't in about:blank at all. It can be > anywhere in chrome code at all. > > I really don't know any way to investigate this further, as it seems almost > impossible to create a reduced test case. Solid STR would be the only really > helpful thing, I think. The BPs below have no "Related Bugs" and while I was looking for a Dupe I spotted this Thread. Hope this assists with an STR. When watching Videos on YouTube I had two crashes today and each one popped up the "WinXP Error Reporter" in addition to the MCR (very unusual, not seen in over a year). I do have ABP. My Flash is the newest version. My 'Yarr BP' is bp-e2187843-0b60-4744-9b6d-0e2c82140331 and my 'non-Yarr BP' (FWIW) is bp-af2ef3a8-25ac-44d6-a36a-3292d2140331 (with both of those entirely different Signatures causing the "WinXP Error Reporter" to appear). I see no connection between the "Modules" and "Source" in those BPs but both caused the WCR to appear. References: Firefox 31.0a1 Crash Report [@ JSC::Yarr::Vector<JSC::Yarr::YarrGenerator<int>::YarrOp, int>::append<JSC::Yarr::YarrGenerator<int>::YarrOpCode>(JSC::Yarr::YarrGenerator<int>::YarrOpCode const&) ] bp-e2187843-0b60-4744-9b6d-0e2c82140331 Firefox 31.0a1 Crash Report [@ nsTArray_base<nsTArrayInfallibleAllocator, nsTArray_CopyWithMemutils>::IncrementLength(unsigned int) | `anonymous namespace''::WorkerJSRuntime::WorkerJSRuntime(JSRuntime*, mozilla::dom::workers::WorkerPrivate*) ] bp-af2ef3a8-25ac-44d6-a36a-3292d2140331 Socorro BUG Note: In my Socorro "User Comments" I wrote: "Plugin Container.exe keeps crashing (with WinXp Error Popup) at 0(URL removed) (third time in less than 4 hours). Triggers MCR only some of the time." . The Text "at 0(URL removed)" is where I type the crashing address (as reported by WCR) in the format 0x0ABC:0123 but Socorro thought it was a URL and unhelpfully removed it. :( It is the same address each time and WCR says it is caused by Plugin-Container.exe . Thanks.
This YARR crash is no longer relevant because bug 976446 replaced YARR with irregexp.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 10 years ago
Flags: needinfo?(nihsanullah)
Resolution: --- → WONTFIX
See Also: → 976446
Mentor: sayak.bugsmith
You need to log in before you can comment on or make changes to this bug.