crash in [@ Servo_IsUnknownPropertyRecordedInUseCounter]
Categories
(Core :: CSS Parsing and Computation, defect, P4)
Tracking
()
Tracking | Status | |
---|---|---|
firefox-esr68 | --- | unaffected |
firefox71 | --- | unaffected |
firefox72 | --- | wontfix |
firefox73 | --- | fixed |
People
(Reporter: tsmith, Assigned: emilio)
References
(Blocks 1 open bug)
Details
(Keywords: crash)
Attachments
(4 files)
The fuzzers have been hitting this for a while now but it does not seem to be reproducable. Seems to be hit on fuzzing-debug and ccov builds.
First seen on m-c 20191006-3fa65bda1e50
Report from m-c 20191202-778b6b11194c
rax = 0x0000000000000060 rdx = 0x2000000000000000
rcx = 0x00002c513fae4c7d rbx = 0x0000000000000000
rsi = 0x0000000000000300 rdi = 0x00007f184d2fefa0
rbp = 0x00007ffdf67062f0 rsp = 0x00007ffdf67062f0
r8 = 0x0000000000000000 r9 = 0x00007f184c10966c
r10 = 0x000000004c100008 r11 = 0x00007f184c10965f
r12 = 0x00007f184d2b6000 r13 = 0x00007f184d2fefa0
r14 = 0x00007f1866e6e278 r15 = 0x00007f184d65d2d0
rip = 0x00007f185a06e3dc
OS|Linux|0.0.0 Linux 4.19.34-coreos #1 SMP Mon Apr 22 20:32:34 -00 2019 x86_64
CPU|amd64|family 6 model 79 stepping 1|64
GPU|||
Crash|SIGSEGV|0x7f184d2ff048|0
0|0|libxul.so|Servo_IsUnknownPropertyRecordedInUseCounter|hg:hg.mozilla.org/mozilla-central:servo/ports/geckolib/glue.rs:778b6b11194c072d2603e58118aaf6959e98902f|6777|0x8
0|1|libxul.so|mozilla::dom::Document::SetCssUseCounterBits()|hg:hg.mozilla.org/mozilla-central:dom/base/Document.cpp:778b6b11194c072d2603e58118aaf6959e98902f|14276|0xb
0|2|libxul.so|mozilla::dom::Document::ReportUseCounters()|hg:hg.mozilla.org/mozilla-central:dom/base/Document.cpp:778b6b11194c072d2603e58118aaf6959e98902f|14320|0x5
0|3|libxul.so|mozilla::dom::Document::Destroy()|hg:hg.mozilla.org/mozilla-central:dom/base/Document.cpp:778b6b11194c072d2603e58118aaf6959e98902f|10277|0x8
0|4|libxul.so|nsDocumentViewer::Destroy()|hg:hg.mozilla.org/mozilla-central:layout/base/nsDocumentViewer.cpp:778b6b11194c072d2603e58118aaf6959e98902f|1874|0x18
0|5|libxul.so|nsDocShell::Destroy()|hg:hg.mozilla.org/mozilla-central:docshell/base/nsDocShell.cpp:778b6b11194c072d2603e58118aaf6959e98902f|4684|0x14
0|6|libxul.so|nsWebBrowser::SetDocShell(nsIDocShell*)|hg:hg.mozilla.org/mozilla-central:toolkit/components/browser/nsWebBrowser.cpp:778b6b11194c072d2603e58118aaf6959e98902f|1240|0x11
0|7|libxul.so|nsWebBrowser::InternalDestroy()|hg:hg.mozilla.org/mozilla-central:toolkit/components/browser/nsWebBrowser.cpp:778b6b11194c072d2603e58118aaf6959e98902f|202|0xa
0|8|libxul.so|nsWebBrowser::Destroy()|hg:hg.mozilla.org/mozilla-central:toolkit/components/browser/nsWebBrowser.cpp:778b6b11194c072d2603e58118aaf6959e98902f|917|0x5
0|9|libxul.so|mozilla::dom::BrowserChild::DestroyWindow()|hg:hg.mozilla.org/mozilla-central:dom/ipc/BrowserChild.cpp:778b6b11194c072d2603e58118aaf6959e98902f|998|0x18
0|10|libxul.so|mozilla::dom::BrowserChild::RecvDestroy()|hg:hg.mozilla.org/mozilla-central:dom/ipc/BrowserChild.cpp:778b6b11194c072d2603e58118aaf6959e98902f|2488|0x8
0|11|libxul.so|mozilla::dom::PBrowserChild::OnMessageReceived(IPC::Message const&)|s3:gecko-generated-sources:289bc9d1a8753697164100bde482ea3411d20a8538e4fa2388a5ddea71fbb1f303b8d8725e565f8f956c951d9cbfc875ea04a058a53aaf0220296e826fef43eb/ipc/ipdl/PBrowserChild.cpp:|6326|0x9
0|12|libxul.so|mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&)|s3:gecko-generated-sources:65518ffd7b57fd1df7143167ed0bada056f671ca93262ea53afdd1cd9af4f7991c73326585ddf9dcd7dc58dce81c40fc4e3784fc15a692f8b3e7ae6e0b9e14a4/ipc/ipdl/PContentChild.cpp:|8168|0x15
0|13|libxul.so|mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:778b6b11194c072d2603e58118aaf6959e98902f|2208|0x6
0|14|libxul.so|mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:778b6b11194c072d2603e58118aaf6959e98902f|2130|0xb
0|15|libxul.so|mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:778b6b11194c072d2603e58118aaf6959e98902f|1972|0xb
0|16|libxul.so|mozilla::ipc::MessageChannel::MessageTask::Run()|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:778b6b11194c072d2603e58118aaf6959e98902f|2003|0xc
0|17|libxul.so|mozilla::SchedulerGroup::Runnable::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/SchedulerGroup.cpp:778b6b11194c072d2603e58118aaf6959e98902f|295|0x15
0|18|libxul.so|nsThread::ProcessNextEvent(bool, bool*)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThread.cpp:778b6b11194c072d2603e58118aaf6959e98902f|1250|0x15
0|19|libxul.so|NS_ProcessNextEvent(nsIThread*, bool)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.cpp:778b6b11194c072d2603e58118aaf6959e98902f|486|0x11
0|20|libxul.so|mozilla::dom::ContentChild::ProvideWindowCommon(mozilla::dom::BrowserChild*, mozIDOMWindowProxy*, bool, unsigned int, bool, bool, bool, nsIURI*, nsTSubstring<char16_t> const&, nsTSubstring<char> const&, bool, bool, nsDocShellLoadState*, bool*, mozilla::dom::BrowsingContext**)|hg:hg.mozilla.org/mozilla-central:dom/ipc/ContentChild.cpp:778b6b11194c072d2603e58118aaf6959e98902f|1254|0x77
0|21|libxul.so|mozilla::dom::BrowserChild::ProvideWindow(mozIDOMWindowProxy*, unsigned int, bool, bool, bool, nsIURI*, nsTSubstring<char16_t> const&, nsTSubstring<char> const&, bool, bool, nsDocShellLoadState*, bool*, mozilla::dom::BrowsingContext**)|hg:hg.mozilla.org/mozilla-central:dom/ipc/BrowserChild.cpp:778b6b11194c072d2603e58118aaf6959e98902f|956|0x1b
0|22|libxul.so|nsWindowWatcher::OpenWindowInternal(mozIDOMWindowProxy*, char const*, char const*, char const*, bool, bool, bool, nsIArray*, bool, bool, bool, nsDocShellLoadState*, mozilla::dom::BrowsingContext**)|hg:hg.mozilla.org/mozilla-central:toolkit/components/windowwatcher/nsWindowWatcher.cpp:778b6b11194c072d2603e58118aaf6959e98902f|807|0x3f
0|23|libxul.so|nsWindowWatcher::OpenWindow2(mozIDOMWindowProxy*, char const*, char const*, char const*, bool, bool, bool, nsISupports*, bool, bool, bool, nsDocShellLoadState*, mozilla::dom::BrowsingContext**)|hg:hg.mozilla.org/mozilla-central:toolkit/components/windowwatcher/nsWindowWatcher.cpp:778b6b11194c072d2603e58118aaf6959e98902f|381|0xa
0|24|libxul.so|nsGlobalWindowOuter::OpenInternal(nsTSubstring<char16_t> const&, nsTSubstring<char16_t> const&, nsTSubstring<char16_t> const&, bool, bool, bool, bool, bool, nsIArray*, nsISupports*, nsDocShellLoadState*, bool, mozilla::dom::BrowsingContext**)|hg:hg.mozilla.org/mozilla-central:dom/base/nsGlobalWindowOuter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|7218|0x5f
0|25|libxul.so|nsGlobalWindowOuter::OpenJS(nsTSubstring<char16_t> const&, nsTSubstring<char16_t> const&, nsTSubstring<char16_t> const&, mozilla::dom::BrowsingContext**)|hg:hg.mozilla.org/mozilla-central:dom/base/nsGlobalWindowOuter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|5767|0x18
0|26|libxul.so|nsGlobalWindowOuter::OpenOuter(nsTSubstring<char16_t> const&, nsTSubstring<char16_t> const&, nsTSubstring<char16_t> const&, mozilla::ErrorResult&)|hg:hg.mozilla.org/mozilla-central:dom/base/nsGlobalWindowOuter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|5731|0x1f
0|27|libxul.so|nsGlobalWindowInner::Open(nsTSubstring<char16_t> const&, nsTSubstring<char16_t> const&, nsTSubstring<char16_t> const&, mozilla::ErrorResult&)|hg:hg.mozilla.org/mozilla-central:dom/base/nsGlobalWindowInner.cpp:778b6b11194c072d2603e58118aaf6959e98902f|3732|0x29
0|28|libxul.so|mozilla::dom::Window_Binding::open|s3:gecko-generated-sources:5716d1d875843b868fa2c899a937865aabf3326a81fd1e81ab138d259b999f3b47fba929b13b4dd71bd23ba1a7833d42cc01def826eae3089ee297c1194aba02/dom/bindings/WindowBinding.cpp:|2623|0x2d
0|29|libxul.so|bool mozilla::dom::binding_detail::GenericMethod<mozilla::dom::binding_detail::MaybeCrossOriginObjectThisPolicy, mozilla::dom::binding_detail::ThrowExceptions>(JSContext*, unsigned int, JS::Value*)|hg:hg.mozilla.org/mozilla-central:dom/bindings/BindingUtils.cpp:778b6b11194c072d2603e58118aaf6959e98902f|3153|0x24
0|30|libxul.so|CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), js::CallReason, JS::CallArgs const&)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|456|0x15
0|31|libxul.so|js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|548|0x12
0|32|libxul.so|InternalCall|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|617|0x10
0|33|libxul.so|js::jit::DoCallFallback(JSContext*, js::jit::BaselineFrame*, js::jit::ICCall_Fallback*, unsigned int, JS::Value*, JS::MutableHandle<JS::Value>)|hg:hg.mozilla.org/mozilla-central:js/src/jit/BaselineIC.cpp:778b6b11194c072d2603e58118aaf6959e98902f|2940|0x13
0|34|||||0xd3585d8263
0|35|||||0x7f184d427c78
0|36|||||0xd3585cda64
0|37|libxul.so|EnterJit|hg:hg.mozilla.org/mozilla-central:js/src/jit/Jit.cpp:778b6b11194c072d2603e58118aaf6959e98902f|109|0x30
0|38|libxul.so|js::RunScript(JSContext*, js::RunState&)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|408|0xb
0|39|libxul.so|js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|589|0xf
0|40|libxul.so|InternalCall|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|617|0x10
0|41|libxul.so|js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:778b6b11194c072d2603e58118aaf6959e98902f|634|0x8
0|42|libxul.so|JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>)|hg:hg.mozilla.org/mozilla-central:js/src/jsapi.cpp:778b6b11194c072d2603e58118aaf6959e98902f|2752|0x1f
0|43|libxul.so|mozilla::dom::Function::Call(JSContext*, JS::Handle<JS::Value>, nsTArray<JS::Value> const&, JS::MutableHandle<JS::Value>, mozilla::ErrorResult&)|s3:gecko-generated-sources:a7c2e5fbc8d754dda6a548b7b47f3620d4c9b10ea79911d63eee97dce3943509e5dac2b094e8f79cc317b469bbff7142455a8edf1bb906af40be582eb3c8ce34/dom/bindings/FunctionBinding.cpp:|41|0x5
0|44|libxul.so|void mozilla::dom::Function::Call<nsCOMPtr<nsIGlobalObject> >(nsCOMPtr<nsIGlobalObject> const&, nsTArray<JS::Value> const&, JS::MutableHandle<JS::Value>, mozilla::ErrorResult&, char const*, mozilla::dom::CallbackObject::ExceptionHandling, JS::Realm*)|s3:gecko-generated-sources:46fb03c1a8c0e11d34f5e031c89d1a474e669ef986de20768612085e4233f6871c6674b6fc7e076da097557e955e8fa4dd7337ed126d39cecea8d5c6e05d7972/dist/include/mozilla/dom/FunctionBinding.h:|73|0x23
0|45|libxul.so|mozilla::dom::CallbackTimeoutHandler::Call(char const*)|hg:hg.mozilla.org/mozilla-central:dom/base/TimeoutHandler.cpp:778b6b11194c072d2603e58118aaf6959e98902f|167|0x27
0|46|libxul.so|nsGlobalWindowInner::RunTimeoutHandler(mozilla::dom::Timeout*, nsIScriptContext*)|hg:hg.mozilla.org/mozilla-central:dom/base/nsGlobalWindowInner.cpp:778b6b11194c072d2603e58118aaf6959e98902f|5904|0x14
0|47|libxul.so|mozilla::dom::TimeoutManager::RunTimeout(mozilla::TimeStamp const&, mozilla::TimeStamp const&, bool)|hg:hg.mozilla.org/mozilla-central:dom/base/TimeoutManager.cpp:778b6b11194c072d2603e58118aaf6959e98902f|892|0x1b
0|48|libxul.so|mozilla::dom::TimeoutExecutor::MaybeExecute()|hg:hg.mozilla.org/mozilla-central:dom/base/TimeoutExecutor.cpp:778b6b11194c072d2603e58118aaf6959e98902f|179|0x13
0|49|libxul.so|mozilla::dom::TimeoutExecutor::Notify(nsITimer*)|hg:hg.mozilla.org/mozilla-central:dom/base/TimeoutExecutor.cpp:778b6b11194c072d2603e58118aaf6959e98902f|246|0x5
0|50|libxul.so|nsTimerImpl::Fire(int)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsTimerImpl.cpp:778b6b11194c072d2603e58118aaf6959e98902f|564|0xe
0|51|libxul.so|nsTimerEvent::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TimerThread.cpp:778b6b11194c072d2603e58118aaf6959e98902f|260|0x14
0|52|libxul.so|mozilla::ThrottledEventQueue::Inner::ExecuteRunnable()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/ThrottledEventQueue.cpp:778b6b11194c072d2603e58118aaf6959e98902f|252|0x11
0|53|libxul.so|mozilla::ThrottledEventQueue::Inner::Executor::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/ThrottledEventQueue.cpp:778b6b11194c072d2603e58118aaf6959e98902f|80|0xd
0|54|libxul.so|mozilla::SchedulerGroup::Runnable::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/SchedulerGroup.cpp:778b6b11194c072d2603e58118aaf6959e98902f|295|0x15
0|55|libxul.so|nsThread::ProcessNextEvent(bool, bool*)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThread.cpp:778b6b11194c072d2603e58118aaf6959e98902f|1250|0x15
0|56|libxul.so|NS_ProcessNextEvent(nsIThread*, bool)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.cpp:778b6b11194c072d2603e58118aaf6959e98902f|486|0x11
0|57|libxul.so|mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:778b6b11194c072d2603e58118aaf6959e98902f|110|0xd
0|58|libxul.so|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:778b6b11194c072d2603e58118aaf6959e98902f|315|0x17
0|59|libxul.so|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:778b6b11194c072d2603e58118aaf6959e98902f|290|0x8
0|60|libxul.so|nsBaseAppShell::Run()|hg:hg.mozilla.org/mozilla-central:widget/nsBaseAppShell.cpp:778b6b11194c072d2603e58118aaf6959e98902f|137|0xd
0|61|libxul.so|XRE_RunAppShell()|hg:hg.mozilla.org/mozilla-central:toolkit/xre/nsEmbedFunctions.cpp:778b6b11194c072d2603e58118aaf6959e98902f|934|0x11
0|62|libxul.so|mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:778b6b11194c072d2603e58118aaf6959e98902f|238|0x5
0|63|libxul.so|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:778b6b11194c072d2603e58118aaf6959e98902f|315|0x17
0|64|libxul.so|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:778b6b11194c072d2603e58118aaf6959e98902f|290|0x8
0|65|libxul.so|XRE_InitChildProcess(int, char**, XREChildData const*)|hg:hg.mozilla.org/mozilla-central:toolkit/xre/nsEmbedFunctions.cpp:778b6b11194c072d2603e58118aaf6959e98902f|769|0x8
0|66|firefox-bin|content_process_main(mozilla::Bootstrap*, int, char**)|hg:hg.mozilla.org/mozilla-central:ipc/contentproc/plugin-container.cpp:778b6b11194c072d2603e58118aaf6959e98902f|56|0x14
0|67|firefox-bin|main|hg:hg.mozilla.org/mozilla-central:browser/app/nsBrowserApp.cpp:778b6b11194c072d2603e58118aaf6959e98902f|272|0x12
0|68|libc-2.27.so||||0x21b97
0|69|firefox-bin|MOZ_ReportCrash|hg:hg.mozilla.org/mozilla-central:mfbt/Assertions.h:778b6b11194c072d2603e58118aaf6959e98902f|203|0x5
Assignee | ||
Comment 1•5 years ago
|
||
It seems it could be a crash near null looking at the registers, can you confirm?
What is the full output, if possible? MOZ_ReportCrash
should give a more informative message...
But I have no idea how this could happen honestly, looking at the code (short of a miscompilation of course). There's nothing funky going on at the loop.
Reporter | ||
Comment 3•5 years ago
|
||
I wonder if it is a miscompile these builds both use GCC[1] and the ASan builds use Clang.
[1] https://searchfox.org/mozilla-central/source/taskcluster/ci/build/linux.yml#91
Assignee | ||
Comment 4•5 years ago
|
||
Hmmm.... could you give me a link to a build known to have triggered this?
Reporter | ||
Comment 5•5 years ago
•
|
||
This is the build used to triggered the crash.
https://firefox-ci-tc.services.mozilla.com/tasks/index/gecko.v2.mozilla-central.pushdate.2019.12.02.20191202115917.firefox/linux64-fuzzing-debug
Comment 7•4 years ago
|
||
Marking as P4 for now, given this doesn't appear to affect the builds we actually ship; but if it turns up more widely, that could change.
Assignee | ||
Comment 8•4 years ago
|
||
The code generated for the rust function is the same in GCC and Clang builds (as expected, as it is rustc generating it):
(gdb) disas Servo_IsUnknownPropertyRecordedInUseCounter
Dump of assembler code for function Servo_IsUnknownPropertyRecordedInUseCounter:
0x000000000564e3d0 <+0>: push %rbp
0x000000000564e3d1 <+1>: mov %rsp,%rbp
0x000000000564e3d4 <+4>: mov %esi,%eax
0x000000000564e3d6 <+6>: shr $0x3,%eax
0x000000000564e3d9 <+9>: and $0xfffffff8,%eax
0x000000000564e3dc <+12>: mov 0x48(%rdi,%rax,1),%rax
0x000000000564e3e1 <+17>: bt %rsi,%rax
0x000000000564e3e5 <+21>: setb %al
0x000000000564e3e8 <+24>: pop %rbp
0x000000000564e3e9 <+25>: retq
End of assembler dump.
So it might be an issue with the caller. I tried to disassemble that but I didn't manage to get useful addresses and I'm a bit confused about the crashreporter-symbols* stuff.
David, do you know how can I disassemble Document::SetCssUseCounterBits
from a build from try? I've tried to load the symbols with add-symbol-file path/to/symbols/libxul.so/<something>/libxul.so.dbg
(after unzipping the crashreporter-symbols-full and gunzipping the .dbg file), but that just shows:
(gdb) disas mozilla::dom::Document::SetCssUseCounterBits
Dump of assembler code for function mozilla::dom::Document::SetCssUseCounterBits():
0x0000000002970c2e <+0>: test %al,%al
0x0000000002970c30 <+2>: jne 0x2970c40 <mozilla::dom::Document::SetCssUseCounterBits()+18>
0x0000000002970c32 <+4>: mov %rbx,%rdi
0x0000000002970c35 <+7>: add $0x8,%rsp
0x0000000002970c39 <+11>: pop %rbx
0x0000000002970c3a <+12>: pop %rbp
0x0000000002970c3b <+13>: jmpq 0x2970c7a <_ZN7mozilla17LinkedListElementINS_15SegmentedVectorI6RefPtrINS_29WebGLExtensionInstancedArraysEELm4096ENS_17MallocAllocPolicyEE11SegmentImplILm509EEEED2Ev>
0x0000000002970c40 <+18>: lea 0x438ddf3(%rip),%rdi # 0x6cfea3a <style::properties::<impl style::gecko_properties::ComputedValues>::differing_properties+13818>
0x0000000002970c47 <+25>: lea 0x4378095(%rip),%rsi # 0x6ce8ce3 <<style::stylesheets::supports_rule::SupportsCondition as style_traits::values::ToCss>::to_css+643>
0x0000000002970c4e <+32>: mov $0x1b1,%edx
0x0000000002970c53 <+37>: callq 0x28c6af5 <_ZL26MOZ_ReportAssertionFailurePKcS0_i>
0x0000000002970c58 <+42>: lea 0x438de76(%rip),%rax # 0x6cfead5 <style::properties::<impl style::gecko_properties::ComputedValues>::differing_properties+13973>
0x0000000002970c5f <+49>: mov 0x70fb062(%rip),%rcx # 0x9a6bcc8
0x0000000002970c66 <+56>: mov %rax,(%rcx)
0x0000000002970c69 <+59>: movl $0x1b1,0x0
0x0000000002970c74 <+70>: callq 0xe102a0 <abort@plt>
0x0000000002970c79 <+75>: nop
0x0000000002970c7a <+76>: push %rbp
0x0000000002970c7b <+77>: mov %rsp,%rbp
0x0000000002970c7e <+80>: push %rbx
0x0000000002970c7f <+81>: push %rax
0x0000000002970c80 <+82>: cmpb $0x0,0x10(%rdi)
0x0000000002970c84 <+86>: jne 0x2970ca0 <mozilla::dom::Document::SetCssUseCounterBits()+114>
0x0000000002970c86 <+88>: mov %rdi,%rbx
0x0000000002970c89 <+91>: callq 0x297090e <_ZNK7mozilla17LinkedListElementINS_15SegmentedVectorI6RefPtrINS_29WebGLExtensionInstancedArraysEELm4096ENS_17MallocAllocPolicyEE11SegmentImplILm509EEEE8isInListEv>
0x0000000002970c8e <+96>: test %al,%al
0x0000000002970c90 <+98>: je 0x2970ca0 <mozilla::dom::Document::SetCssUseCounterBits()+114>
0x0000000002970c92 <+100>: mov %rbx,%rdi
0x0000000002970c95 <+103>: add $0x8,%rsp
0x0000000002970c99 <+107>: pop %rbx
0x0000000002970c9a <+108>: pop %rbp
0x0000000002970c9b <+109>: jmpq 0x2970b6e <_ZN7mozilla17LinkedListElementINS_15SegmentedVectorI6RefPtrINS_29WebGLExtensionInstancedArraysEELm4096ENS_17MallocAllocPolicyEE11SegmentImplILm509EEEE6removeEv>
0x0000000002970ca0 <+114>: add $0x8,%rsp
0x0000000002970ca4 <+118>: pop %rbx
0x0000000002970ca5 <+119>: pop %rbp
0x0000000002970ca6 <+120>: retq
0x0000000002970ca7 <+121>: push %rbp
0x0000000002970ca8 <+122>: mov %rsp,%rbp
0x0000000002970cab <+125>: push %r15
0x0000000002970cad <+127>: push %r14
0x0000000002970caf <+129>: push %r13
0x0000000002970cb1 <+131>: push %r12
0x0000000002970cb3 <+133>: push %rbx
0x0000000002970cb4 <+134>: sub $0x38,%rsp
0x0000000002970cb8 <+138>: mov %rcx,%r14
0x0000000002970cbb <+141>: mov %rdx,%r15
0x0000000002970cbe <+144>: mov %rdi,%r12
0x0000000002970cc1 <+147>: mov %fs:0x28,%rax
0x0000000002970cca <+156>: mov %rax,-0x30(%rbp)
0x0000000002970cce <+160>: movabs $0xaaaaaaaaaaaaaaaa,%rax
0x0000000002970cd8 <+170>: lea -0x48(%rbp),%rcx
--Type <RET> for more, q to quit, c to continue without paging--
0x0000000002970cdc <+174>: mov %rax,(%rcx)
0x0000000002970cdf <+177>: movb $0x0,(%rcx)
0x0000000002970ce2 <+180>: mov %rcx,-0x38(%rbp)
0x0000000002970ce6 <+184>: mov 0x88(%rdi),%rdi
0x0000000002970ced <+191>: mov %rdi,0x8(%rcx)
0x0000000002970cf1 <+195>: test %rdi,%rdi
0x0000000002970cf4 <+198>: je 0x2970d19 <mozilla::dom::Document::SetCssUseCounterBits()+235>
0x0000000002970cf6 <+200>: lea 0x50f6ad1(%rip),%rsi # 0x7a677ce
0x0000000002970cfd <+207>: lea 0x523a59a(%rip),%rdx # 0x7bab29e
0x0000000002970d04 <+214>: lea -0x48(%rbp),%rcx
0x0000000002970d08 <+218>: mov $0x18,%r8d
0x0000000002970d0e <+224>: mov $0x90,%r9d
0x0000000002970d14 <+230>: callq 0xe768a6 <_ZN14ProfilingStack14pushLabelFrameEPKcS1_PvN2JS21ProfilingCategoryPairEj>
0x0000000002970d19 <+235>: lea -0x38(%rbp),%rdi
0x0000000002970d1d <+239>: callq 0xe17c2e <_ZN7mozilla6detail19GuardObjectNotifierD2Ev>
0x0000000002970d22 <+244>: mov 0x8(%r14),%ecx
0x0000000002970d26 <+248>: cmp $0x3,%ecx
0x0000000002970d29 <+251>: jbe 0x2970e29 <mozilla::dom::Document::InlineScriptAllowedByCSP()+229>
0x0000000002970d2f <+257>: lea -0x4c(%rbp),%rbx
0x0000000002970d33 <+261>: movl $0xaaaaaaaa,(%rbx)
0x0000000002970d39 <+267>: xor %r13d,%r13d
0x0000000002970d3c <+270>: mov %r14,%rdi
0x0000000002970d3f <+273>: xor %esi,%esi
0x0000000002970d41 <+275>: callq 0x21e95d6 <_ZNK2JS6detail12CallArgsBaseINS0_10NoUsedRvalEEixEj
Which I don't think is the correct disassembly for the function.
I only speak WinDbg... but, fun tip, WinDbgX can open ELF files nowadays! (When all you have is a hammer...)
0:000> uf libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv:
00000000`01ebf082 55 push rbp
00000000`01ebf083 4889e5 mov rbp,rsp
00000000`01ebf086 4155 push r13
00000000`01ebf088 4154 push r12
00000000`01ebf08a 53 push rbx
00000000`01ebf08b 4883ec08 sub rsp,8
00000000`01ebf08f 4c8baf78060000 mov r13,qword ptr [rdi+678h]
00000000`01ebf096 4d85ed test r13,r13
00000000`01ebf099 0f84a7000000 je libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0xc4 (00000000`01ebf146) Branch
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x1d:
00000000`01ebf09f 4989fc mov r12,rdi
00000000`01ebf0a2 e833c518ff call libxul!NS_IsMainThread (00000000`0104b5da)
00000000`01ebf0a7 84c0 test al,al
00000000`01ebf0a9 7536 jne libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x5f (00000000`01ebf0e1) Branch
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x29:
00000000`01ebf0ab 488d35d66b2c04 lea rsi,[libxul!ZL20kINIParserFactoryCID+0x7f68 (00000000`06185c88)]
00000000`01ebf0b2 488d3dd8800105 lea rdi,[libxul!ZZN22nsXMLContentSerializer12IsJavaScriptEP10nsIContentP6nsAtomiRK12nsTSubstringIDsEE11kJavaScript+0x2431 (00000000`06ed7191)]
00000000`01ebf0b9 ba31020000 mov edx,231h
00000000`01ebf0be e8d71b06ff call libxul!MOZ_ReportAssertionFailure (00000000`00f20c9a)
00000000`01ebf0c3 488b05966bec06 mov rax,qword ptr [libxul!+0x47a8 (00000000`08d85c60)]
00000000`01ebf0ca 488d0d54810105 lea rcx,[libxul!ZZN22nsXMLContentSerializer12IsJavaScriptEP10nsIContentP6nsAtomiRK12nsTSubstringIDsEE11kJavaScript+0x24c5 (00000000`06ed7225)]
00000000`01ebf0d1 488908 mov qword ptr [rax],rcx
00000000`01ebf0d4 c704250000000000000000 mov dword ptr [0],0
00000000`01ebf0df 0f0b ud2
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x5f:
00000000`01ebf0e1 803d285ded0600 cmp byte ptr [libxul!ZN7mozilla11StaticPrefs39sMirror_layout_css_use_counters_enabledE (00000000`08d94e10)],0
00000000`01ebf0e8 742a je libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x92 (00000000`01ebf114) Branch
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x68:
00000000`01ebf0ea 31db xor ebx,ebx
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x6a:
00000000`01ebf0ec 89de mov esi,ebx
00000000`01ebf0ee 4c89ef mov rdi,r13
00000000`01ebf0f1 e86af28803 call libxul!Servo_IsPropertyIdRecordedInUseCounter (00000000`0574e360)
00000000`01ebf0f6 84c0 test al,al
00000000`01ebf0f8 740e je libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x86 (00000000`01ebf108) Branch
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x78:
00000000`01ebf0fa 8db350010000 lea esi,[rbx+150h]
00000000`01ebf100 4c89e7 mov rdi,r12
00000000`01ebf103 e8f2c8ffff call libxul!ZN7mozilla3dom8Document13SetUseCounterENS_10UseCounterE (00000000`01ebb9fa)
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x86:
00000000`01ebf108 48ffc3 inc rbx
00000000`01ebf10b 4881fb2e020000 cmp rbx,22Eh
00000000`01ebf112 75d8 jne libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x6a (00000000`01ebf0ec) Branch
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x92:
00000000`01ebf114 8b05f25ced06 mov eax,dword ptr [libxul!ZN7mozilla11StaticPrefs53sMirror_layout_css_use_counters_unimplemented_enabledE (00000000`08d94e0c)]
00000000`01ebf11a 85c0 test eax,eax
00000000`01ebf11c 7428 je libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0xc4 (00000000`01ebf146) Branch
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x9c:
00000000`01ebf11e 31db xor ebx,ebx
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x9e:
00000000`01ebf120 4088de mov sil,bl
00000000`01ebf123 4c89ef mov rdi,r13
00000000`01ebf126 e8a5f28803 call libxul!Servo_IsUnknownPropertyRecordedInUseCounter (00000000`0574e3d0)
00000000`01ebf12b 84c0 test al,al
00000000`01ebf12d 740e je libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0xbb (00000000`01ebf13d) Branch
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0xad:
00000000`01ebf12f 8db37e030000 lea esi,[rbx+37Eh]
00000000`01ebf135 4c89e7 mov rdi,r12
00000000`01ebf138 e8bdc8ffff call libxul!ZN7mozilla3dom8Document13SetUseCounterENS_10UseCounterE (00000000`01ebb9fa)
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0xbb:
00000000`01ebf13d 48ffc3 inc rbx
00000000`01ebf140 4883fb70 cmp rbx,70h
00000000`01ebf144 75da jne libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x9e (00000000`01ebf120) Branch
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0xc4:
00000000`01ebf146 58 pop rax
00000000`01ebf147 5b pop rbx
00000000`01ebf148 415c pop r12
00000000`01ebf14a 415d pop r13
00000000`01ebf14c 5d pop rbp
00000000`01ebf14d c3 ret
Comment 10•4 years ago
|
||
(In reply to Emilio Cobos Álvarez (:emilio) from comment #1)
There's nothing funky going on at the loop.
My reading of it is that the Document has a bogus mStyleUseCounters.
Assignee | ||
Comment 11•4 years ago
|
||
(In reply to :dmajor from comment #10)
(In reply to Emilio Cobos Álvarez (:emilio) from comment #1)
My reading of it is that the Document has a bogus mStyleUseCounters.
Hmm, that's odd, because it's a UniquePtr
owned by the document that never changes... https://searchfox.org/mozilla-central/rev/efdf9bb55789ea782ae3a431bda6be74a87b041e/dom/base/Document.cpp#1368
We've seen some oddness on other GCC builds and functions returning this POD-but-generic generic struct... https://searchfox.org/mozilla-central/source/__GENERATED__/layout/style/ServoStyleConsts.h#928 but this should be crashing much more often in that case...
So something must be messing up with that pointer or something...
Comment 12•4 years ago
|
||
(In reply to :dmajor from comment #10)
(In reply to Emilio Cobos Álvarez (:emilio) from comment #1)
There's nothing funky going on at the loop.
My reading of it is that the Document has a bogus mStyleUseCounters.
Hm or maybe not. Do we know that an id
of 0x300 is valid?
Comment 13•4 years ago
|
||
Do we know that an
id
of 0x300 is valid?
Divide by eight, that's 0x60, but the use counters object is only 0x58 bytes long.
Assignee | ||
Comment 14•4 years ago
|
||
0x60 is a pretty perfectly acceptable ID, afaict. There are more than a hundred counted unknown properties.
That object is a bitfield effectively, see https://searchfox.org/mozilla-central/rev/efdf9bb55789ea782ae3a431bda6be74a87b041e/servo/components/style/use_counters/mod.rs#56
Comment 15•4 years ago
|
||
Right, so the ID is 0x300, the 0x60 is the byte offset for the 0x300th bit (the result of the shift right by three).
Assignee | ||
Comment 16•4 years ago
|
||
Hmmm... It seems rust should've panicked in that case, right?
Comment 17•4 years ago
|
||
If I understand correctly, the total counters are 0x58 bytes, of which the first 0x48 are non_custom_properties
, leaving 0x10 == 16 bytes == 128 bits for unknown properties. That seems OK by https://searchfox.org/mozilla-central/source/__GENERATED__/layout/style/CountedUnknownProperties.h#114.
So how on earth did the loop count up to 0x300? :-o
Comment 18•4 years ago
|
||
So how on earth did the loop count up to 0x300? :-o
Here's how:
libxul!ZN7mozilla3dom8Document20SetCssUseCounterBitsEv+0x9e:
00000000`01ebf120 4088de mov sil,bl
The caller only sets the low byte of the parameter, it was actually 0x00, and the 0x3__ was previous garbage in the register, but the callee interpreted it as a full 32-bit value:
0x000000000564e3d4 <+4>: mov %esi,%eax
It is not clear to me who is in the wrong, caller or callee.
Assignee | ||
Comment 19•4 years ago
|
||
Hmm, so the rust definition of the enum has repr(u8)
, which should make the C++ definition just fine.
How is the ABI for enum classes in extern functions defined?
Assignee | ||
Comment 20•4 years ago
|
||
Seems like this could explain the heisenbugs we've seen with gcc builds such as bug 1581121 and co...
Comment 21•4 years ago
|
||
How is the ABI for enum classes in extern functions defined?
Good question, I'm not sure! (I'm not even sure who to ask, a Rust person or a C++ person...)
Assignee | ||
Comment 22•4 years ago
|
||
Ok, I can try to do some digging tomorrow. You already saved me a whole lot of time, thank you so much!
Comment 23•4 years ago
|
||
I would assume the passing of the enum class would be the same as if you passed an integer of the same size and signedness of the underlying integer for the enum class. This is kind of a mess, see the (almost four years old!) thread for trying to figure out who should be performing extension for < 32-bit arguments at:
https://groups.google.com/forum/?hl=en#!topic/x86-64-abi/E8O33onbnGQ
and the last comment on the GCC bug at:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=46942#c10
I think llvm will always pass the argument extended to 32 bits, per:
https://reviews.llvm.org/rL34623
Though the code on trunk is radically different:
https://raw.githubusercontent.com/llvm/llvm-project/master/llvm/lib/Target/X86/X86ISelLowering.cpp
// If this is an 8 or 16-bit value, it is really passed promoted to 32
// bits. Insert an assert[sz]ext to capture this, then truncate to the
// right size.
if (VA.getLocInfo() == CCValAssign::SExt)
ArgValue = DAG.getNode(ISD::AssertSext, dl, RegVT, ArgValue,
DAG.getValueType(VA.getValVT()));
else if (VA.getLocInfo() == CCValAssign::ZExt)
ArgValue = DAG.getNode(ISD::AssertZext, dl, RegVT, ArgValue,
DAG.getValueType(VA.getValVT()));
else if (VA.getLocInfo() == CCValAssign::BCvt)
ArgValue = DAG.getBitcast(VA.getValVT(), ArgValue);
I don't speak enough LLVM backend to know whether that does the same thing as the patch (I assume that it does).
A stripped-down example:
#include <stdint.h>
#include <stdlib.h>
enum class CountedProperty : uint8_t {
A, B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z,
Count,
};
extern "C" bool Servo_IsUnknownProperty(void*, CountedProperty);
extern void SetUseCounter(int);
void SetCssUseCounterBits(void* x)
{
for (size_t i = 0; i < size_t(CountedProperty::Count); ++i) {
auto id = CountedProperty(i);
if (Servo_IsUnknownProperty(x, id)) {
SetUseCounter(i + 5);
}
}
}
compiled with GCC and Clang locally appears to zero-extend id
prior to calling, though only clang actually extends from an 8-bit value; GCC apparently knows the upper bits are already zero and 32-bit MOVs id
into the right register.
So I think this is a bug somewhere in clang where the zero-extension on the caller side is getting dropped, but how I couldn't really say. :(
Assignee | ||
Comment 24•4 years ago
|
||
You mean a bug in gcc, right? We're only seeing this in gcc builds...
Comment 25•4 years ago
|
||
(In reply to Emilio Cobos Álvarez (:emilio) from comment #24)
You mean a bug in gcc, right? We're only seeing this in gcc builds...
The ccov builds are gcc, AIUI, but aren't the fuzzing builds clang? Tyson, what C/C++ compiler is being used here?
Assignee | ||
Comment 27•4 years ago
|
||
This seems to be fixed with newer versions of gcc
.
With my system gcc (gcc version 9.2.1 20190827 (Red Hat 9.2.1-1) (GCC)
), if I disassemble Document::SetCssUseCounterBits
, I get the following in the relevant section of the function (the call to Servo_IsUnknownProperty
, when compiled wirh gcc t.cc -g -Os -c -o t1.o
:
0x000000000000006b <+107>: mov r13d,0x1
0x0000000000000071 <+113>: mov esi,ebp
0x0000000000000073 <+115>: mov rdi,r12
0x0000000000000076 <+118>: call 0x7b <Document::SetCssUseCounterBits()+123>
0x000000000000007b <+123>: test al,al
0x000000000000007d <+125>: je 0x9a <Document::SetCssUseCounterBits()+154>
Note the mov
being an esi
(lower 32-bit) move, IIUC.
However from automation's gcc
, which is gcc (GCC) 7.4.0
, I get, with the same arguments:
0x000000000000006e <+110>: mov r13d,0x1
0x0000000000000074 <+116>: mov sil,bpl
0x0000000000000077 <+119>: mov rdi,r12
0x000000000000007a <+122>: call 0x7f <Document::SetCssUseCounterBits()+127>
0x000000000000007f <+127>: test al,al
0x0000000000000081 <+129>: je 0x9e <Document::SetCssUseCounterBits()+158>
Which has the problematic mov
, IIUC.
What's the usual procedure for this?
Should I try to bisect gcc to make sure what fixed is an intentional change, and report a bug otherwise?
If so, any good tool to do that or should I just check out gcc and do manual builds?
Comment 28•4 years ago
|
||
You can probably do a rough bisection using https://gcc.godbolt.org and that's probably good enough to create a gcc bug report.
Assignee | ||
Comment 29•4 years ago
|
||
It seems like it's broken with gcc 7.5 and fixed with gcc 8.1: https://godbolt.org/z/4rQFAk
It also seems it only affects -Os
, so that's nice, I guess, in the sense that it probably only affects debug builds.
Assignee | ||
Comment 30•4 years ago
|
||
Thanks jeff. I'll try to check gcc 8 manually tomorrow, and bisect to see if the fix was intentional or not.
Assignee | ||
Comment 31•4 years ago
|
||
This was fixed by https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82260. It doesn't look like the fix was intentional.
Assignee | ||
Comment 32•4 years ago
|
||
I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92821
We'll see if the GCC folks think this is a bug in GCC or LLVM, and if it's LLVM making too many assumptions about the argument being zero-extended I'll go file a bug there I guess :)
Assignee | ||
Comment 33•4 years ago
|
||
Relevant LLVM bug, as the GCC developers seem to think it is an LLVM bug: https://bugs.llvm.org/show_bug.cgi?id=44228
Assignee | ||
Comment 34•4 years ago
|
||
CC'ing :jhorak (it seems :stransky's account has been disabling for bouncing email), as IIRC Fedora ships GCC-built Firefox, and this may be relevant to them.
Comment 35•4 years ago
|
||
I'm not at all sure I understand this adequately, but according to Figure 3.1 "Scalar Types" in https://github.com/hjl-tools/x86-psABI/wiki/x86-64-psABI-1.0.pdf, enum
types are represented as a signed fourbyte
; it doesn't say anything about smaller explicitly-sized enums, so arguably even though this is a [repr(u8)]
enum, it should still be a 32-bit value for ABI purposes. (This is in the Data Representation section, not Parameter Passing, but seems like it would apply to the computation of the argument values, prior to assigning them to registers or stack locations for passing.) That could make this a GCC bug after all, IIUC.
Assignee | ||
Comment 36•4 years ago
|
||
The LLVM issue also reproduces with uint8_t
, so it's still an issue even without that.
Also I think the paragraph you quote does not apply to rust #[repr(u8)]
enums or enum classes, otherwise sizeof(EnumClassUint8)
should be 4
, which would defeat the purpose of specifying the underlying type explicitly.
Assignee | ||
Comment 37•4 years ago
|
||
Apparently this is not an issue for regular uint8_t
s because of C/C++'s integer promotion rules.
So comment 35 may seem a bit more appealing now. I mentioned that in the gcc bug, let's see what they think.
Assignee | ||
Comment 38•4 years ago
|
||
GCC8 happens not to generate the code that causes the crash, so do that for now
to unblock fuzzers from hitting this.
We still need to figure out what to do about the more general issue of course...
Updated•4 years ago
|
Comment 39•4 years ago
|
||
Pushed by ealvarez@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/ed92ceb9cb68 Make debug fuzzing builds use gcc 8. r=froydnj
Comment 40•4 years ago
|
||
bugherder |
Updated•4 years ago
|
Updated•4 years ago
|
Comment 41•4 years ago
|
||
Bugbug thinks this bug is a regression, but please revert this change in case of error.
Assignee | ||
Updated•4 years ago
|
Updated•4 years ago
|
Description
•