1434681 - Crash in std::_Atomic_compare_exchange_strong_8

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, defect)

Description

[Calixte Denizet (:calixte)]

This bug was filed from the Socorro interface and is
report bp-56b4f8fc-4079-447a-8974-91fe30180131.
=============================================================

Top 10 frames of crashing thread:

0 xul.dll std::_Atomic_compare_exchange_strong_8 vs2017_15.4.2/VC/include/xatomic.h:2041
1 xul.dll nsAttrValue::SetMiscAtomOrString dom/base/nsAttrValue.cpp:1809
2 xul.dll nsAttrValue::ToString dom/base/nsAttrValue.cpp:635
3 xul.dll DoMatch<mozilla::dom::Element const, <lambda_f72130ff95d9ff32759ffb681427322b> > layout/style/ServoBindings.cpp:924
4 xul.dll Gecko_AttrEquals layout/style/ServoBindings.cpp:1157
5 xul.dll style::gecko::wrapper::{{impl}}::attr_matches servo/components/style/gecko/wrapper.rs:1907
6 xul.dll selectors::matching::matches_simple_selector<style::gecko::wrapper::GeckoElement, closure> servo/components/selectors/matching.rs:690
7 xul.dll selectors::matching::matches_complex_selector_internal<style::gecko::wrapper::GeckoElement, closure> servo/components/selectors/matching.rs:428
8 xul.dll style::selector_map::SelectorMap<style::stylist::Rule>::get_matching_rules<style::gecko::wrapper::GeckoElement, closure> servo/components/style/selector_map.rs:229
9 xul.dll style::selector_map::SelectorMap<style::stylist::Rule>::get_all_matching_rules<style::gecko::wrapper::GeckoElement, closure> servo/components/style/selector_map.rs:205

=============================================================

There is 1 crash (from 1 installation) in nightly 60 with buildid 20180131100706. In analyzing the backtrace, the regression may have been introduced by patch [1] to fix bug 1418161.

[1] https://hg.mozilla.org/mozilla-central/rev?node=a4cb461ced3b

Comment 1

[Emilio Cobos Álvarez [:emilio]]

It has uBlock installed, so it is plausible...

Comment 2

[(Away)]

The instruction pointer is 0x000007fe`ef7a9318 but we were executing instructions that should have resided at 0x000007fe`ef3a9318 -- a difference of 0x00000000`00400000 (I found this by searching for those bytes in my own copy of xul.dll). This being a single-bit flip on a Win7 machine, I'm going to call it aging RAM.

Comment 3

[(Away)]

(To be clear, I calculated these addresses from offsets relative to the start of xul.dll; it's not just that my libxul got ASLR'ed somewhere else.)

Comment 4

[Bobby Holley (:bholley)]

That is very impressive analysis David, thank you!

For my edification - how can you tell the difference between the instructions we were executing and the instruction pointer? Are you basing it on the stack pointer, or does the minidump have some other information that you were able to use to make the comparison?

Comment 5

[(Away)]

Our minidumps save 512 bytes of memory on either side of @rip: https://searchfox.org/mozilla-central/rev/e06af9c36a73a27864302cd2f829e6200dee8541/toolkit/crashreporter/breakpad-client/windows/common/minidump_callback.cc#183-186

Amusingly, the key factor in solving this was the fact that I don't have crash access. =) The first thing I want to see in a crash is the faulting instruction, but since I can't open minidumps, my poor-man's approach is to grab libxul from archive.mo and disassemble at the module_offset from the Raw Dump json. But this time I got gibberish because the crashing address was not instruction-aligned in the real libxul. Then when you showed me the minidump and it claimed to have different bytes at the same offset, I became suspicious.

Comment 6

[Bobby Holley (:bholley)]

(In reply to David Major [:dmajor] from comment #5)
> Our minidumps save 512 bytes of memory on either side of @rip:
> https://searchfox.org/mozilla-central/rev/
> e06af9c36a73a27864302cd2f829e6200dee8541/toolkit/crashreporter/breakpad-
> client/windows/common/minidump_callback.cc#183-186

I don't understand though - if $rip had a bit clobbered, then wouldn't the minidump also contain garbage?

Comment 7

[(Away)]

(In reply to Bobby Holley (:bholley) from comment #6)
> I don't understand though - if $rip had a bit clobbered, then wouldn't the
> minidump also contain garbage?

The minidump contains what the machine saw: code from the wrong place.

Comment 8

[(Away)]

Maybe another way to say it is: $rip was correct, but when the CPU asked for the code at *$rip, the hardware returned the code at *($rip^0x400000). At least one of these wrong instructions executed without crashing, and then because of varying instruction lengths, $rip got incremented into state that no longer made sense in the original code.