Assertion failure: mTailQueue.Length(), at netwerk/base/RequestContextService.cpp:311 when visiting ebay.com

RESOLVED FIXED in Firefox 57

Status

()

Core
Networking
RESOLVED FIXED
3 months ago
2 months ago

People

(Reporter: keeler, Assigned: schien)

Tracking

unspecified
mozilla57
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox57 fixed)

Details

(Whiteboard: [necko-active])

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

str: visit ebay.com

(doesn't appear to happen every time - maybe requires clearing history/cache?)

#0  0x00007f8543a4de89 in mozilla::net::RequestContext::Notify(nsITimer*) (this=0x7f85264d7380, timer=0x7f850b4e6fe0) at /home/keeler/mozilla-unified/netwerk/base/RequestContextService.cpp:311
#1  0x00007f8543a4e10d in non-virtual thunk to mozilla::net::RequestContext::Notify(nsITimer*) () at /home/keeler/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include/mozilla/Printf.h:181
#2  0x00007f85439d9a90 in nsTimerImpl::Fire(int) (this=0x7f850b434b70, aGeneration=1) at /home/keeler/mozilla-unified/xpcom/threads/nsTimerImpl.cpp:517
#3  0x00007f85439c0850 in nsTimerEvent::Run() (this=0x7f852399f258) at /home/keeler/mozilla-unified/xpcom/threads/TimerThread.cpp:286
#4  0x00007f85439c6ea2 in nsThread::ProcessNextEvent(bool, bool*) (this=0x7f8539802690, aMayWait=<optimized out>, aResult=0x7fffffffbb67) at /home/keeler/mozilla-unified/xpcom/threads/nsThread.cpp:1039
#5  0x00007f85439c8e3f in NS_ProcessNextEvent(nsIThread*, bool) (aThread=0x7f8555b5d500 <_IO_2_1_stderr_>, aMayWait=<error reading variable: access outside bounds of object referenced via synthetic pointer>)
    at /home/keeler/mozilla-unified/xpcom/threads/nsThreadUtils.cpp:521
#6  0x00007f8543fe52ff in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) (this=0x7f853a85de40, aDelegate=0x7f8555574540) at /home/keeler/mozilla-unified/ipc/glue/MessagePump.cpp:125
#7  0x00007f8543f6b927 in MessageLoop::RunHandler() (this=0x7f8555b5d500 <_IO_2_1_stderr_>) at /home/keeler/mozilla-unified/ipc/chromium/src/base/message_loop.cc:319
#8  0x00007f8543f6b927 in MessageLoop::Run() (this=0x7f8555b5d500 <_IO_2_1_stderr_>) at /home/keeler/mozilla-unified/ipc/chromium/src/base/message_loop.cc:299
#9  0x00007f85466d9b79 in nsBaseAppShell::Run() (this=0x7f853178b580) at /home/keeler/mozilla-unified/widget/nsBaseAppShell.cpp:158
#10 0x00007f85481804de in nsAppStartup::Run() (this=0x7f8531a32150) at /home/keeler/mozilla-unified/toolkit/components/startup/nsAppStartup.cpp:288
#11 0x00007f854823e80a in XREMain::XRE_mainRun() (this=<optimized out>) at /home/keeler/mozilla-unified/toolkit/xre/nsAppRunner.cpp:4643
#12 0x00007f854823f4c2 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) (this=0x7fffffffbe30, argc=<optimized out>, argv=<optimized out>, aConfig=...)
    at /home/keeler/mozilla-unified/toolkit/xre/nsAppRunner.cpp:4807
#13 0x00007f854823ff4e in XRE_main(int, char**, mozilla::BootstrapConfig const&) (argc=1437984560, argv=0x0, aConfig=...) at /home/keeler/mozilla-unified/toolkit/xre/nsAppRunner.cpp:4902
#14 0x0000000000405bf2 in do_main(int, char**, char**) (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at /home/keeler/mozilla-unified/browser/app/nsBrowserApp.cpp:236
#15 0x0000000000405bf2 in main(int, char**, char**) (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at /home/keeler/mozilla-unified/browser/app/nsBrowserApp.cpp:309


This is in a local build from a recent pull from mozilla-central - doesn't look like this happens on Nightly (yet), so I assume this is from something that landed recently.

Comment 1

3 months ago
I also meet the crash while launching local build on Mac, with following options enabled.

ac_add_options --enable-debug
ac_add_options --disable-optimize

=====
[Parent 47182] WARNING: Could not get disk status from nsIDiskSpaceWatcher: file /Users/bentian/Workspace/central/uriloader/prefetch/nsOfflineCacheUpdateService.cpp, line 283
[Child 47189] WARNING: attempt to modify an immutable nsStandardURL: file /Users/bentian/Workspace/central/netwerk/base/nsStandardURL.cpp, line 1827

Assertion failure: mTailQueue.Length(), at /Users/bentian/Workspace/central/netwerk/base/RequestContextService.cpp:311

#01: mozilla::net::RequestContext::Notify(nsITimer*)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x341ff5]
#02: non-virtual thunk to mozilla::net::RequestContext::Notify(nsITimer*)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x34238c]
#03: nsTimerImpl::Fire(int)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x26680b]
#04: nsTimerEvent::Run()[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x23bc4d]
#05: nsThread::ProcessNextEvent(bool, bool*)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x2451f6]
#06: NS_ProcessNextEvent(nsIThread*, bool)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x248acc]
#07: bool mozilla::SpinEventLoopUntil<(mozilla::ProcessFailureBehavior)1, nsThread::Shutdown()::$_4>(nsThread::Shutdown()::$_4&&, nsIThread*)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x2446af]
#08: nsThread::Shutdown()[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x244626]
#09: ShutdownThreadEvent::Run()[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x20b065a]
#10: nsThread::ProcessNextEvent(bool, bool*)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x2451f6]
#11: NS_ProcessNextEvent(nsIThread*, bool)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x248acc]
#12: bool mozilla::SpinEventLoopUntil<(mozilla::ProcessFailureBehavior)1, nsThread::Shutdown()::$_4>(nsThread::Shutdown()::$_4&&, nsIThread*)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x2446af]
#13: nsThread::Shutdown()[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x244626]
#14: mozilla::storage::Connection::shutdownAsyncThread()[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x1718dee]
#15: decltype(*(fp).*fp0(Get<>(fp1).PassAsParameter())) mozilla::detail::RunnableMethodArguments<>::applyImpl<mozilla::storage::Connection, void (mozilla::storage::Connection::*)()>(mozilla::storage::Connection*, void (mozilla::storage::Connection::*)(), mozil[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x17206da]
#16: _ZN7mozilla6detail23RunnableMethodArgumentsIJEE5applyINS_7storage10ConnectionEMS5_FvvEEEDTcl9applyImplfp_fp0_dtdefpT10mArgumentscvNS_13IndexSequenceIJEEE_EEEPT_T0_[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x1720659]
#17: mozilla::detail::RunnableMethodImpl<RefPtr<mozilla::storage::Connection>, void (mozilla::storage::Connection::*)(), true, (mozilla::RunnableKind)0>::Run()[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x172048e]
#18: nsThread::ProcessNextEvent(bool, bool*)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x2451f6]
#19: NS_ProcessPendingEvents(nsIThread*, unsigned int)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x24297c]
#20: nsBaseAppShell::NativeEventCallback()[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x4df798e]
#21: nsAppShell::ProcessGeckoEvents(void*)[/Users/bentian/Workspace/central/obj-x86_64-apple-darwin16.7.0/dist/NightlyDebug.app/Contents/MacOS/XUL +0x4e943b7]
Hi Ben,

Do you have a reliable way to reproduce this?
Could you provide the commit id of your local build?

Thanks.
Flags: needinfo?(btian)
I did some code inspection.
The crash happened during shutdown procedure and RequestContextService will remove all the tailed request for each RequestContext. However, the tailed timer is not canceled.
https://searchfox.org/mozilla-central/rev/44c693914255638d74bcf1ec3b0bcd448dfab2fd/netwerk/base/RequestContextService.cpp#472

I'm writing a patch to fix this. It would be really help if anyone has reliable STR.

Comment 4

3 months ago
Let me re-build with debug option to reproduce. I disabled debug to workaround it.

The hg changeset I use is below. It's normal at first build and launch, but crashes after I made some change and rebuild.

changeset:   379572:b4c1ad9565ee
tag:         tip
parent:      379517:64bf417d1bdf
parent:      379571:52433c05f306
user:        Sebastian Hengst <archaeopteryx@coole-files.de>
date:        Thu Sep 07 23:59:58 2017 +0200
summary:     merge mozilla-inbound to mozilla-central. r=merge a=merge
Flags: needinfo?(btian)
Thanks to @btian. Here is the reliable STR on my mac:

[STR]
1. prepare a debug build and run firefox from terminal
2. open https://twitter.com/micahflee
3. after page loaded, use ctrl-C to abort firefox from terminal
4. run firefox from terminal again
5. click restore session 

[Expected]
https://twitter.com/micahflee is restored successfully

[Actual]
firefox crashed because of "Assertion failure: mTailQueue.Length()" in RequestContext::Notify()
Comment hidden (mozreview-request)
Assignee: nobody → schien
Blocks: 1358060
Whiteboard: [necko-active]
Comment hidden (mozreview-request)

Comment 8

3 months ago
I've seen this for a bit on Bughunter but wasn't able to get a reproducible case. The following url asserts for me on Fedora 26:

http://www.fr.de/frankfurt/bombe-in-frankfurt-die-evakuierung-im-minutenprotokoll-a-1343969

Comment 9

3 months ago
fyi, Bughunter has seen this on 906 urls so far. It is a pain since I keep hitting it while attempting to reproduce other crashes.

Comment 10

3 months ago
mozreview-review
Comment on attachment 8905808 [details]
Bug 1397880 - Cancel tailing timer during shutdown.

https://reviewboard.mozilla.org/r/177618/#review183406

Drive-by review: Looks good. Let's ship this ASAP.

::: netwerk/base/RequestContextService.cpp:480
(Diff revision 2)
>  
>  void
>  RequestContextService::Shutdown()
>  {
>    MOZ_ASSERT(NS_IsMainThread());
> +

nit: whitespace only change.
Attachment #8905808 - Flags: review+
Comment on attachment 8905808 [details]
Bug 1397880 - Cancel tailing timer during shutdown.

https://reviewboard.mozilla.org/r/177618/#review183416

lgtm
Attachment #8905808 - Flags: review?(hurley) → review+
Comment hidden (mozreview-request)

Comment 13

3 months ago
Pushed by schien@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/269426fcc3f3
Cancel tailing timer during shutdown. r=nwgh,valentin
https://hg.mozilla.org/mozilla-central/rev/269426fcc3f3
Status: NEW → RESOLVED
Last Resolved: 3 months ago
status-firefox57: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
Depends on: 1401538

Updated

2 months ago
Depends on: 1406613
Depends on: 1408411
You need to log in before you can comment on or make changes to this bug.