1272230 - Crash in CCGraphBuilder::BuildGraph

Status: RESOLVED INCOMPLETE

(Core :: DOM: Workers, defect)

Description

[[:philipp]]

This bug was filed from the Socorro interface and is 
report bp-0f018c93-9c00-43b2-8f19-f9bb32160511.
=============================================================
Crashing Thread (0)
Frame 	Module 	Signature 	Source
0 	xul.dll 	CCGraphBuilder::BuildGraph(js::SliceBudget&) 	xpcom/base/nsCycleCollector.cpp:2274
1 	xul.dll 	nsCycleCollector::MarkRoots(js::SliceBudget&) 	xpcom/base/nsCycleCollector.cpp:2886
2 	xul.dll 	nsCycleCollector::Collect(ccType, js::SliceBudget&, nsICycleCollectorListener*, bool) 	xpcom/base/nsCycleCollector.cpp:3649
3 	xul.dll 	nsCycleCollector_collectSlice(js::SliceBudget&, bool) 	xpcom/base/nsCycleCollector.cpp:4153
4 	xul.dll 	NeedsGCAfterCC 	dom/base/nsJSEnvironment.cpp:295
5 	msctf.dll 	__CppXcptFilter 	
6 	xul.dll 	nsTimerImpl::Fire() 	xpcom/threads/nsTimerImpl.cpp:524
7 	xul.dll 	nsTimerEvent::Run() 	xpcom/threads/TimerThread.cpp:279
8 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp:994
9 	xul.dll 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/glue/nsThreadUtils.cpp:297
10 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp:127
11 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc:227
12 	xul.dll 	NS_DispatchToCurrentThread(already_AddRefed<nsIRunnable>&&) 	xpcom/glue/nsThreadUtils.cpp:172
13 	xul.dll 	nsAppShell::Run() 	widget/windows/nsAppShell.cpp:262
14 	xul.dll 	nsAppStartup::Run() 	toolkit/components/startup/nsAppStartup.cpp:281
15 	xul.dll 	XREMain::XRE_mainRun() 	toolkit/xre/nsAppRunner.cpp:4337
16 	xul.dll 	XREMain::XRE_main(int, char** const, nsXREAppData const*) 	toolkit/xre/nsAppRunner.cpp:4434
17 	xul.dll 	XRE_main 	toolkit/xre/nsAppRunner.cpp:4540
18 	xul.dll 	nsAString_internal::ReplacePrep(unsigned int, unsigned int, unsigned int) 	xpcom/string/nsTSubstring.cpp:186
19 	firefox.exe 	firefox.exe@0x151d3

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'CCGraphBuilder::BuildGraph':
 - nightly (version 50): 28 crashes from 2016-06-06.
 - aurora  (version 49): 112 crashes from 2016-06-07.
 - beta    (version 48): 907 crashes from 2016-06-06.
 - release (version 47): 2382 crashes from 2016-05-31.
 - esr     (version 45): 116 crashes from 2016-04-07.

Crash volume on the last weeks:
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly       4       3       3       4       4       3       3
 - aurora       25      14      10      11      13      18      16
 - beta        112     129     120     113     124     118     121
 - release     340     308     312     296     336     307     300
 - esr          20      15      11       7      15       8      14

Affected platforms: Windows, Mac OS X, Linux

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'CCGraphBuilder::BuildGraph':
 - nightly (version 51): 10 crashes from 2016-08-01.
 - aurora  (version 50): 28 crashes from 2016-08-01.
 - beta    (version 49): 1343 crashes from 2016-08-02.
 - release (version 48): 418 crashes from 2016-07-25.
 - esr     (version 45): 160 crashes from 2016-05-02.

Crash volume on the last weeks (Week N is from 08-22 to 08-28):
            W. N-1  W. N-2  W. N-3
 - nightly       1       4       0
 - aurora       10      11       0
 - beta        452     432     154
 - release     119     112      68
 - esr          17      17       9

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
           Browser   Content     Plugin
 - nightly #196      #160
 - aurora  #154      #125
 - beta    #21       #71
 - release #177      #65
 - esr     #446

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

High frequency UAF signature (especially in beta).  Almost all crashes are on Windows (all in the last week), but the bot says mac and linux too.

The small sample I looked at all had dom/workers/RuntimeService.cpp:943 (CustomGCCallback()) in stack frame 4, called from MarkRoots() in the GC code.

Andrew - who should look at this from the Worker side?  Can you look at the GC side of things, or point someone else to?  And move elsewhere if it should be in the GC component (whatever that is)

Comment 4

[Andrew McCreight [:mccr8]]

Nice find. Those certainly do look like bad crashes. I have no idea what could be causing that, though. It looks like we're crashing while building the cycle collector graph, when accessing some internal CC data structures. Workers don't even use incremental collection.

The worker parts of the stack don't look unusual to me. It doesn't seem like we're shutting down the worker or anything. Andrea, do you have any ideas?

Comment 5

[Randell Jesup [:jesup] (needinfo me)]

Also concerning: crash rate (from bot above, and from a search now) seems MUCH higher in beta than release (when you take account of population size especially).  This makes me think we're about to ship a memory-corruption bug that triggers failures during CC.  (and the rate spiked starting in August, though it wasn't low before).  Does this match?

Comment 6

[Andrew McCreight [:mccr8]]

I don't understand what you are asking.

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

My concern given the crash rates for release vs 49 that I see on crash-stats is that the crashrate per user-day is much higher in 49.  This implies to me something regressed and/or caused additional triggers; a supposition is that some form of memory trashing is happening which causes delayed crashes in GC/CC.  My question is "is this a reasonable hypothesis?" - do you concur; should we be worried about what will happen when 49 ships?

Comment 8

[Andrew McCreight [:mccr8]]

It does sound like a reasonable hypothesis. I have no idea what the problem could be, though. It is odd because unlike most CC crashes it is happening while touching an internal CC data structure, not tracing through allocated objects. The main thread isn't running while the CC is running, so it would have to be some other thread. nsCycleCollector.cpp looks like it hadn't changed in either 48 or 49, so it doesn't seem like a CC change is the root cause here.

Comment 9

[Daniel Veditz [:dveditz]]

This increased rate seems to have started with the very first 49.0b1 release. Look at the graphs tab below and use the select box to aggregate on version.

https://crash-stats.mozilla.com/signature/?date=%3E2016-08-01&date=%3C2016-08-11&signature=CCGraphBuilder%3A%3ABuildGraph#graphs

The difference is even more stark if you restrict this to the beta channel so the populations are similar: goes from < 20 a day to 50-60 a day

https://crash-stats.mozilla.com/signature/?date=%3E2016-07-21&date=%3C2016-08-09&release_channel=beta&signature=CCGraphBuilder%3A%3ABuildGraph#graphs

The reason for the crashes has changed. In the timeline above only ~10 of the 48.0b crashes were EXCEPTION_ACCESS_VIOLATION_WRITE (and all but one were 0x14) compared to 175 in the same period for 49.0b. If I've got the dates right that period is two weeks of 48 beta and only 4 days of 49 beta. These are very different crashes, and a lot more of them.

https://crash-stats.mozilla.com/signature/?date=%3E2016-07-21&date=%3C2016-08-09&release_channel=beta&reason=~EXCEPTION_ACCESS_VIOLATION_WRITE&signature=CCGraphBuilder%3A%3ABuildGraph&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_sort=version&_sort=-date&page=1#reports

Comment 10

[Liz Henry (:lizzard) (relman/hg->git project)]

Tracking for 49 since this sounds concerning. I'm not clear if it's something we should block release for.

Comment 11

[Liz Henry (:lizzard) (relman/hg->git project)]

David, FYI possible blocker for 49.

Comment 12

[Daniel Veditz [:dveditz]]

hard to tell in the trickle nightly crashes, and complicated by the occasional UAF/Write crash in earlier releases, but the earliest 49.0a1 release that crashes is build 20160509040557 bp-65784047-ca86-45a5-8667-0d3fb2160509. Maybe we can find something in the days before that
https://hg.mozilla.org/mozilla-central/pushloghtml?startdate=2016-05-04&enddate=2016-05-09+06%3A00

Comment 13

[Andrea Marchesini [:baku]]

> The worker parts of the stack don't look unusual to me. It doesn't seem like
> we're shutting down the worker or anything. Andrea, do you have any ideas?

I don't see workers commonly involved in the reports. Actually mainly, the crash happens on the main-thread.

Comment 14

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Andrea Marchesini [:baku] from comment #13)
> > The worker parts of the stack don't look unusual to me. It doesn't seem like
> > we're shutting down the worker or anything. Andrea, do you have any ideas?
> 
> I don't see workers commonly involved in the reports. Actually mainly, the
> crash happens on the main-thread.

Look for crashes with the 5e-based pattern - they all appear to be called from Worker code; there were 342 in the last week:
https://crash-stats.mozilla.com/signature/?product=Firefox&address=%3D0xffffffffe5e5e609&signature=CCGraphBuilder%3A%3ABuildGraph&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_sort=-date&page=1#reports

The nullptr-looking and random-address-looking ones I looked at (a few) were somewhere in TimerImpl, which may make sense for CC on mainthread.

Comment 15

[Andrew McCreight [:mccr8]]

Can you take a look at this some more Andrea? There are a lot of worker crashes here.

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'CCGraphBuilder::BuildGraph':
 - nightly (version 52): 6 crashes from 2016-09-19.
 - aurora  (version 51): 14 crashes from 2016-09-19.
 - beta    (version 50): 366 crashes from 2016-09-20.
 - release (version 49): 1131 crashes from 2016-09-05.
 - esr     (version 45): 234 crashes from 2016-06-01.

Crash volume on the last weeks (Week N is from 10-03 to 10-09):
            W. N-1  W. N-2
 - nightly       4       2
 - aurora       10       4
 - beta        290      76
 - release     902     229
 - esr          22      16

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
           Browser   Content     Plugin
 - nightly #189      #217
 - aurora  #364      #136
 - beta    #49       #88
 - release #59       #97
 - esr     #423

Comment 17

[Liz Henry (:lizzard) (relman/hg->git project)]

Andrew, can you help find someone to take on the investigation here?

Comment 18

[Randell Jesup [:jesup] (needinfo me)]

Note: this search: https://crash-stats.mozilla.com/signature/?date=%3C%3D2016-05-01&date=%3E%3D2016-02-01&signature=CCGraphBuilder%3A%3ABuildGraph#graphs shows it goes back to at least 37 beta, maybe earlier.  Older data (before 4/4/2016) appears to be purged

Comment 19

[Andrew Overholt [:overholt]]

I've emailed baku as he's a good person to investigate.

Comment 20

[Andrew McCreight [:mccr8]]

I'm going to reduce this to sec-high, because I don't think these data structures are very accessible to content, plus we have no idea if it can even be reproduced.

Comment 21

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Andrew McCreight [:mccr8] from comment #20)
> I'm going to reduce this to sec-high, because I don't think these data
> structures are very accessible to content, plus we have no idea if it can
> even be reproduced.

I'd be cautious here... the crashrate from this bug is ramping up (perhaps related to increased use of workers).  This is a CC issue, and a clear UAF, and not only that it's a UAF write of a pointer in SetLastChild() -- which has this lovely line: (this + 1)->mFirstChild = aLastChild;

At minimum it might be useful to have some MOZ_ASSERT()s in SetLastChild/etc and perhaps scattered in areas related to this crash - maybe even a MOZ_RELEASE_ASSERT() at least in aurora/nightly.

Since this is happening async to mainthread, content could be madly allocating to try to reallocate the freed memory.

Comment 23

[Randell Jesup [:jesup] (needinfo me)]

baku?

Comment 24

[Marcia Knous [:marcia]]

Tracking 53+ for this sec high issue.

Comment 25

[Randell Jesup [:jesup] (needinfo me)]

andrew, can you push this forward?  Thanks

Comment 26

[Andrew Overholt [:overholt]]

baku and I spoke earlier and he's going to dig in some more.

Comment 27

[Gerry Chang [:gchang]]

Track 51+ as sec-high.

Comment 28

[Andrea Marchesini [:baku]]

I and Andrew discussed this during the work-week. He will take a look because probably the issue here is about how we shutdown workers when we have GC/CC running. and the reason why we see this more often in workers compared with main-thread, is that we shutdown workers more often.

Comment 29

[Andrew McCreight [:mccr8]]

It looks to me like the dead pointer is to a CCGraphBuilder. The only place we destroy it is in nsCycleCollector::MarkRoots, and then we clear the pointer to it in nsCycleCollector. Looking at this some more, the only way I can think that we might end up with a dead pointer is that somehow we're running BuildGraph, and then somehow we reenter the CC and do some more work to finish the graph building phase, maybe via a Traverse method that is spinning the event loop, destroy the graph builder, and then we come back and resume graph building. I think by making the mScanInProgress check fatal in release builds we might be able to figure out where that reentrance happens. I filed bug 1322536 for that. I intentionally did not link it to this because I don't want to make it obvious that it links to a security issue. Anyways, depending on what the cause is, we can fix it or try to handle it correctly (presumably by not running the CC).

Comment 30

[Andrew McCreight [:mccr8]]

Bug 1322536 does seem to have made the signature go away from Nightly, but I'm not sure what it turned into, so maybe it is a fluke. The volume is quite low there.

Comment 31

[Julien Cristau [:jcristau]]

This signature is still present in 51.0b10, even though bug 1322536 made it to beta 3 weeks ago.

Comment 32

[Randell Jesup [:jesup] (needinfo me)]

very few 52/53 crashes; those that are happening are in two classes from quick check of the last week.

1: crashes in nsCycleCollector.cpp:2298 doing SetLastChild(), which are 5e5e UAFs
2: crashes in nsCycleCollector.cpp:2282, doing pi->mParticipant->Traverse(pi->mPointer, *this);, which is a random value ptr crash.

Comment 33

[Andrew McCreight [:mccr8]]

Yeah, I guess my release assert didn't really do anything. When I looked a few weeks ago I didn't see anybody hitting the asserts I added.

The Traverse one (2) is probably a different sort of memory corruption.

Comment 34

[Frederik Braun [:freddy]]

Andrew, this is an inactive sec-high. Will you continue working on this?

Comment 35

[Andrew McCreight [:mccr8]]

I still have no idea what is going wrong here, unfortunately.

Comment 36

[Jan Varga [:janv]]

Olli, do you have any ideas ?

Comment 37

[Olli Pettay [:smaug][[e-mail]]]

Let me see what happens once I finally get back to worker GC/CC scheduling, hopefully later today.
But I don't recall seeing anything suspicious there.

Comment 38

[Wayne Mery (:wsmwk)]

crash rate just doubled, perhaps starting Wed March 8.  with version 51.0.1?
https://crash-stats.mozilla.com/signature/?signature=CCGraphBuilder%3A%3ABuildGraph&date=%3E%3D2017-02-14T01%3A42%3A53.000Z&date=%3C2017-03-14T01%3A42%3A53.000Z#graphs

bp-8c1cc38c-e9e8-4175-8681-947d02170309

Comment 39

[Olli Pettay [:smaug][[e-mail]]]

(still not back to worker GC stuff)

Comment 40

[Olli Pettay [:smaug][[e-mail]]]

Hmm, I wonder. Workers have rather unique handling for event loop. Could we somehow re-enter CC there after all.

Comment 41

[Andrew McCreight [:mccr8]]

(In reply to Olli Pettay [:smaug] from comment #40)
> Hmm, I wonder. Workers have rather unique handling for event loop. Could we
> somehow re-enter CC there after all.

This also shows up on the main thread, though less often.

Comment 42

[Liz Henry (:lizzard) (relman/hg->git project)]

Noting that the crash volume is somewhat high in 52.0.1esr - 1397 crashes in the last week, which puts it at the #42 top crash for 52 esr.

Comment 43

[Stephanie Ouillon [:arroway] (needinfo me)]

Andrew, any new idea of what's going on or who could help investigate on this?

Comment 44

[Andrew McCreight [:mccr8]]

(In reply to Stephanie Ouillon [:arroway] from comment #43)
> Andrew, any new idea of what's going on or who could help investigate on
> this?

No.

Comment 45

[Wayne Mery (:wsmwk)]

In the case of Thunderbird, about half also experience  CompareCacheMatchEntry crash bug 1353702.
A quarter to a third experience also bug 1337105 / bug 1257387

Comment 46

[Bob Clary [:bc] (inactive)]

fwiw, Bughunter can reproduce this on 32bit Windows 7 and 10 with 3G Ram. The opt builds are rated by exploitable as medium. The following urls reproduced in the last day:

http://vakilspremedia.com/borosil/files/assets/basic-html/page105.html
http://dev.crv.nu/event_flyer/files/assets/basic-html/page2.html
http://rail.hobidas.com/rmn/archives/0200jr/
http://spm-verlag.de/fileadmin/user_upload/Roth%20Stadt/files/assets/basic-html/page130.html
http://dev.crv.nu/event_flyer/files/assets/basic-html/page2.html
http://rail.hobidas.com/rmn/archives/0200jr/
http://spm-verlag.de/fileadmin/user_upload/Roth%20Stadt/files/assets/basic-html/page130.html

I don't see non Windows non 32bit crashes. I see lots of other crashes and fatal assertions in debug builds and lots of oom aborts. You may need to automatically reload the url and keep trying until you get this one.

Comment 47

[Bob Clary [:bc] (inactive)]

PS. I forgot to add this which points to freed memory. Let me know if we can help otherwise.

Crash reason:  EXCEPTION_ACCESS_VIOLATION_WRITE
Crash address: 0xffffffffe5e5e609
Process uptime: not available

Thread 48 (crashed)
 0  xul.dll!CCGraphBuilder::BuildGraph(js::SliceBudget &) [nsCycleCollector.cpp:5b74bbf20e80 : 2285 + 0x6]
    eip = 0x0f3fab9f   esp = 0x165ff5ac   ebp = 0x165ff5b4   ebx = 0x00000000
    esi = 0x34c78a00   edi = 0x165ff628   eax = 0x14aafc90   ecx = 0xe5e5e5e5
    edx = 0x14aafc8c   efl = 0x00010202

Comment 48

[Nathan Froyd [:froydnj]]

Andrew, want to take a look at the failing URLs now? :)

Comment 49

[Andrew McCreight [:mccr8]]

Hmm, yeah maybe I could rig up a build that logs dtor stacks for a poor man's ASan build.

Comment 50

[Andrew McCreight [:mccr8]]

I filed bug 1367496 about a few more places we could add release checks. There's no release check in CleanupAfterCollection, so if you manage to reenter the CC while we're in that phase, you'll destroy the CC graph. I'm not sure how that would happen, but it doesn't hurt to check.

Comment 51

[Andrew McCreight [:mccr8]]

Bob, could you try a Nightly build with bug 1367496 on the test cases you found and see if that turns up the release assert I added? Unfortunately I don't have a developer box set up with Windows. Thanks.

Comment 52

[Bob Clary [:bc] (inactive)]

I ran the urls in comment 46 30+ times with a build from today and reproduced the CGGraphBuild::BuildGraph crash but nothing related to mScanInProgress. Sorry.

Comment 53

[Olli Pettay [:smaug][[e-mail]]]

30-ish times doesn't sound too much. Would it be possible to use rr here?

Comment 54

[Bob Clary [:bc] (inactive)]

I don't know what it would take to get rr on the 32bit windows vms. Do you have a link for rr I could look at?

Comment 55

[Andrew McCreight [:mccr8]]

No, you can't use rr on Windows. rr is Linux only.

Comment 56

[Liz Henry (:lizzard) (relman/hg->git project)]

Sounds like 55 is also affected then. Too late to get a fix into 53 or 54.

Comment 57

[Frederik Braun [:freddy]]

(In reply to Bob Clary [:bc:] from comment #52)
> I ran the urls in comment 46 30+ times with a build from today and
> reproduced the CGGraphBuild::BuildGraph crash but nothing related to
> mScanInProgress. Sorry.

I know it sounds weak, but if you find the time, can you retest a few times more? :/

Comment 58

[Bob Clary [:bc] (inactive)]

freddyb: Sure, what would be a good number? 100? more? It's not a manual process so I can do as many as you like. I just pull a vm / worker out of the rotation for a bit and run a command line loop with spider and log the results looking for a crash or assertion or something.

Comment 59

[Bob Clary [:bc] (inactive)]

I submitted 

http://dev.crv.nu/event_flyer/files/assets/basic-html/page2.html
http://rail.hobidas.com/rmn/archives/0200jr/
http://spm-verlag.de/fileadmin/user_upload/Roth%20Stadt/files/assets/basic-html/page130.html
http://vakilspremedia.com/borosil/files/assets/basic-html/page105.html

100 times each to Bughunter.

http://rail.hobidas.com/rmn/archives/0200jr/ was the most reliable reproducer though it was in the 20-30% range. I did not see mScanInProgress in the logs at all.

Comment 60

[Stephanie Ouillon [:arroway] (needinfo me)]

Hey Bob, since the reproducibility rate is a bit low, could you try to run the test a higher number of times, e.g 1000 times?
Is there any different URLs in the most recent reports that we could try?

Comment 61

[Bob Clary [:bc] (inactive)]

Sure. I'm off today and Bug 1379831 is kind of a blocker since it prevents me from shutting down cleanly. As soon as that is fixed, I'll hit this right away.

Comment 62

[Bob Clary [:bc] (inactive)]

If I look back at crashes with CCGraphBuilder::BuildGraph anywhere in the signature I get 171 urls. If I restrict this to signatures beginning with CCGraphBuilder::BuildGraph then I have 99 urls. Running either set 1000 times would tie up Bughunter for an appreciable time. I counted the occurrence of each url and have decided to pick the top 10 repeatable urls from the 99 urls and run them 1000 times. I'll let you know when I have results.

Comment 63

[Andrew McCreight [:mccr8]]

I'm not sure what repeating things many times is supposed to accomplish. We already know this is intermittently reproducible at some URLs. We need some better logging for UAFs for this to go anywhere. I haven't had time to put together a patch for that yet.

Comment 64

[Bob Clary [:bc] (inactive)]

Attachment: crash report ecx = 0xe5e5e5e5 http://rail.hobidas.com/rmn/archives/0200jr/

So far, Windows 7 opt 32bit with 3G ram found crashes on 
http://rail.hobidas.com/rmn/archives/0200jr/ where debug shows Assertion failure: false (Ran out of memory while building cycle collector graph) and opt (see attachment) crashes with EXCEPTION_ACCESS_VIOLATION_WRITE at crash address Crash address: 0xffffffffe5e5e609 where ecx = 0xe5e5e5e5.

Comment 65

[Bob Clary [:bc] (inactive)]

If that is the case, then I'll cancel these urls and continue with normal testing until you have something more actionable for me.

Comment 66

[Stephanie Ouillon [:arroway] (needinfo me)]

mccr8, any update regarding logging for UAFs?

Comment 67

[Andrew McCreight [:mccr8]]

(In reply to Stephanie Ouillon [:arroway] from comment #66)
> mccr8, any update regarding logging for UAFs?

If I have any updates, I'll post them in the bug.

Comment 68

[Wayne Mery (:wsmwk)]

FWIW, a user who crashes with this in verison 55 also has different crash signs in versoin 47:
* js::GCMarker::lazilyMarkChildren- bug 719114 - Crash in [@ js::GCMarker::processMarkStackTop bp-cf193757-7de5-4ee7-b3a0-89a800170913
* js::TraceEdge<T> bp-35c3398c-f10c-4894-afda-0d96b0170913
* @0x0 | js::ObjectGroup::sweep bp-4f34de4f-3160-4463-a9fe-0cd130170907

Comment 69

[Daniel Veditz [:dveditz]]

crash rate is still high but we don't seem to have the information to figure out where the corruption happened (long before the cycle collector stumbles on it)