Closed Bug 1017721 Opened 9 years ago Closed 9 years ago

Random Assertion failure: rc != 0 (destroyed timer off its target thread!) on shutdown EventTokenBucket

Categories

(Core :: Networking, defect)

All
Windows XP
defect
Not set
normal

Tracking

()

RESOLVED DUPLICATE of bug 1024765

People

(Reporter: bc, Unassigned)

References

Details

(Keywords: assertion)

Attachments

(4 files)

Attached file windbg stack
I have seen 

Assertion failure: rc != 0 (destroyed timer off its target thread!), at c:\work\
mozilla\builds\nightly\mozilla\xpcom\threads\TimerThread.cpp:279

on shutdown on Windows very often in bughunter's crash automation testing but have not been able to find reproducible steps. In just the last few weeks I have seen this assertion on over 400 urls though it has been occurring for much longer. This is not restricted to the Nightly branch. It appears to be random. I can only reproduce the assertion about 10% of the time when testing the urls where the assertion has previously been seen.

I have this captured in windbg for https://www.theclymb.com/ on the build machine for bughunter: win61i32-bld.bughunter.ateam.phx1.mozilla.com. I will leave it available for the until tomorrow when I need to create new builds.

I would appreciate some help with diagnosing this assertion. It is causing problems with regular crash testing due to its frequency. I've filed this under Firefox:General since it may be specific to the offending timer, but I don't know how to determine it.
We need to know the timer client. I'm not sure whether you can get this out of the assertion or not: at http://hg.mozilla.org/mozilla-central/annotate/5add3261493b/xpcom/threads/TimerThread.cpp#l279 the timer object has already been destroyed, so probably timer->mCallbackType and timer->mCallback are already dead.

Attachment 8415866 [details] [diff] has some code which will help make this more reproducible; we should probably make that NS_WARNING a MOZ_ASSERT in general. And when you hit that, the debugger will have the real timer data.

See bug 881413 for a version of this that we fixed.
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #1)
> We need to know the timer client. I'm not sure whether you can get this out
> of the assertion or not: at
> http://hg.mozilla.org/mozilla-central/annotate/5add3261493b/xpcom/threads/
> TimerThread.cpp#l279 the timer object has already been destroyed, so
> probably timer->mCallbackType and timer->mCallback are already dead.
> 

Yes, they are.

> Attachment 8415866 [details] [diff] has some code which will help make this
> more reproducible; we should probably make that NS_WARNING a MOZ_ASSERT in
> general. And when you hit that, the debugger will have the real timer data.

Great. I've attached the refreshed patch. Fwiw, prior to this patch when trying to reproduce this assertion the "WARNING: An event was posted to a thread that will never run it (rejected)" appears just before the assertion but can occur without the assertion being fired.

I've deployed this patch to win32 workers and see it firing the false assertion.

I've caught a different url <https://www.consignacaoalagoas.com.br/ES1/index.html> in the assertion

[2416] WARNING: An event was posted to a thread that will never run it (rejected
): file c:\work\mozilla\builds\nightly\mozilla\xpcom\threads\nsThread.cpp, line
462
Assertion failure: false, at c:\work\mozilla\builds\nightly\mozilla\xpcom\thread
s\nsThread.cpp:463

I'll attach the stack and locals next.
What's interesting here is aEvent->mTimer.mRawPtr->mCallback.i

which is a nsITimerCallback* but we really need to know the underlying type.
Flags: needinfo?(bclary)
Attached file another url locals
Are these the right ones?

          mCallbackType	0x00 ''
          mType	0x00 ''
Flags: needinfo?(bclary)
fyi, I still have this assertion in windbg and can answer any other questions about the data you may have. Just ping me on irc.
      __vtcast_mozilla::net::EventTokenBucket	class mozilla::net::EventTokenBucket

Looks like networking needs to cancel a timer before shutdown, probably the one here: http://hg.mozilla.org/mozilla-central/annotate/6a984e21c2ca/netwerk/base/src/EventTokenBucket.cpp#l306
Component: General → Networking
Product: Firefox → Core
Summary: Random Assertion failure: rc != 0 (destroyed timer off its target thread!) on shutdown → Random Assertion failure: rc != 0 (destroyed timer off its target thread!) on shutdown EventTokenBucket
Resetting the os and platform since this assertion appears on Windows XP and Windows 7 for both 32 and 64bit.

mcmanus, this bug is a major source of pain for me in crash testing. Is there anything I can do to help get it resolved?
OS: Windows 7 → Windows XP
Hardware: x86_64 → All
good first bug - I think jason should get one of the newer necko team members to work on it. I'll be happy to mentor.
Flags: needinfo?(jduell.mcbugs)
Sounds good--it's on my list.
Flags: needinfo?(jduell.mcbugs)
This and all other Assertion failure: rc != 0 failures were fixed by bug 1024765. This was the top crash in bughunter but I was unable to reproduce it after bug 1024765 was fixed and testing over 400 urls on all three branches and all three platforms.
Group: core-security
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → DUPLICATE
Group: core-security → core-security-release
Group: core-security-release
See Also: → 1729964
You need to log in before you can comment on or make changes to this bug.