1491844 - Crash in mbrtoc32

Status: RESOLVED WORKSFORME

(Firefox for Android Graveyard :: General, defect, P2)

Description

[Marcia Knous [:marcia]]

This bug was filed from the Socorro interface and is
report bp-0e1d3016-5375-4097-8403-5ac330180917.
=============================================================

Seen while looking at 63b5 crash stats: https://bit.ly/2xqhFrz. These crashes appear to have started in Beta 3. 29 crashes so far. Not sure if this is a signature change or something new. Pushlog: https://hg.mozilla.org/releases/mozilla-beta/pushloghtml

Top 2 frames of crashing thread:

0  @0xc22cd304 
1 libxul.so mbrtoc32 

=============================================================

Comment 1

[Marcia Knous [:marcia]]

APIs from 28 down to 19 are affected. There are no comments in the crash reports. Here are a few of the URLs:

*https://www.amazon.de/ 
*https://www.dallasnews.com/news/immigration/2018/09/16/ice-ordering-immigrants-appear-court-judges-expecting 
*https://www.dailymail.co.uk/tvshowbiz/article-5870281/Richard-Dreyfuss-70-afford-retire-reveals-financial-woes.html 
*http://bojackhorseman.wikia.com/wiki/INT._SUB

Comment 2

[Marcia Knous [:marcia]]

:sdaswani, can you have someone take a look at this crash and see if they can interpret whether this is a new issue or perhaps a signature change of some sort?

Comment 3

[Kevin Brosnan [Ex-Mozilla]]

This is platform code. Moving the NI to David. Worth noting that Firefox for Android 63.0b3 is the first beta since the last beta of 62.0. This is due to the developer edition only desktop beta releases. This seems to have been introduced in the nightly 63 development cycle. From the graph the first known bad build is 20180829100130 revision b75561ff5ffec3164338952adfe58620e5e3bc1d from bp-750e075d-af04-4910-a13f-e6c240180829

Comment 4

[David Bolter [:davidb] (NeedInfo me for attention)]

Jim could this be fallout from bug 1490493?

Comment 5

[(inactive) Jim Chen [:jchen] [:darchons]]

I don't think that bug is the cause.

Comment 6

[David Durst [:ddurst]]

Back to you, sir.

Comment 7

[David Bolter [:davidb] (NeedInfo me for attention)]

The stacks are not super helpful :(

Kevin is it possible to construct a pushlog range based on comment 3 (somehow try to get a narrow range based on first bad build)?

Comment 8

[Kevin Brosnan [Ex-Mozilla]]

For some reason there were no Aug 28th Android builds. So the regression range could be https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=768eef11f5ff119fc3b63221e326db16986900cb&tochange=b75561ff5ffec3164338952adfe58620e5e3bc1d This is a bit of a fuzzy guess as we can only know when the user encountered a bad build. We cannot know the previous good build that the user had.

This appears to be a JIT crash and there are several js and dom changes in that 2 day window. Maybe someone from the JS team has some ideas?

Comment 9

[Kevin Brosnan [Ex-Mozilla]]

Seeing lots of different signatures many in the JS stack though not all. Some sort of corruption?

Comment 10

[Marcia Knous [:marcia]]

While looking through crash stats today, I noticed https://crash-stats.mozilla.com/report/index/03a170d3-a438-400c-bcf3-ccc280180926 - that signature seems to have several instances of mbrtoc32 in it as well. Not sure if it is related or not, but thought I would mention it.

Comment 11

[Randell Jesup [:jesup] (needinfo me)]

SIGSEGV/SIGILL with random addresses - sec bug

Comment 12

[:sdaswani]

This seems to be a sec-high platform bug, NI'ing James and David.

Comment 13

[David Bolter [:davidb] (NeedInfo me for attention)]

Jim, James, anything actionable here?

Comment 14

[James Willcox (:snorp) ([e-mail]) (he/him)]

The mbrtoc32 symbol is just nonsense. The function offset for that is massive, so it just happens to be the closest thing we are able to resolve. We probably need to add it to the skip list. Even better would be to skip all symbols with huge function offsets (though not really sure where I'd draw the line).

The stacks are all over the place, but I found a few that are a MOZ_CRASH() from failing to read some stuff out of omnijar[0].

Many of the others are somewhere in JS, but largely the stacks make no sense at all. This one is pretty comical[1].

My recommendation is that we add mbrtoc32 to the skip list and reprocess the affected crashes. Hopefully this will result in some convergence around the crashes with non-broken stacks.

Also, I don't agree that this is a sec-high since the majority of the crashes I saw that had half-reasonable stacks appeared to be a MOZ_CRASH.

[0] https://hg.mozilla.org/releases/mozilla-beta/annotate/f76a129ff2f2739e286626024004d7785bdb4dc1/js/xpconnect/loader/URLPreloader.cpp#l386
[1] https://crash-stats.mozilla.com/report/index/76ef3376-3cfa-4986-b73e-025690181001

Comment 15

[Pascal Chevrel:pascalc (PTO until April 26)]

Marking as wontfix for 63 as the volume is low and this is not actionable until the signature is added to the skip list and we can investigate further. We don't have crashes reported for 64 nightly so I'll defer to Julien for 64 tracking.

Comment 16

[(inactive) Jim Chen [:jchen] [:darchons]]

Most of the crashes I looked at are in JITed code, so not sure there's much we can do.

> Also, I don't agree that this is a sec-high since the majority of the
> crashes I saw that had half-reasonable stacks appeared to be a MOZ_CRASH.

I think it's the ones with nonsensical stacks that make this a sec-high, since those possibly indicate some kind of stack corruption or use-after-free scenario.

Comment 17

[Ryan VanderMeulen [:RyanVM]]

I don't see much point in tracking this for 64 either if it's low-volume and looking inactionable.

Comment 18

[Liz Henry (:lizzard) (relman/hg->git project)]

The crashes were reprocessed in mid-October in bug 1496732, but the crash volume is still high on 63 release and moderately high for 64 beta. snorp, can you look again? Or would you consider this to be stalled?

Comment 19

[James Willcox (:snorp) ([e-mail]) (he/him)]

We're supposed to be skipping mbrtoc32 as of bug 1496732, so I'm not sure how we could still be getting this signature. Will?

Comment 20

[Will Kahn-Greene [:willkg] ET needinfo? me]

mbrtoc32 was added to the prefix list so it's included in the signature and signature generation will continue to the next frame.

With the crash that was in the description (bp-0e1d3016-5375-4097-8403-5ac330180917), there are only two frames and the second one is mbrtoc32, so the signature ends up as "mbrtoc32".

With the crash in comment #14, there are a bunch of frames, signature generation skips the first one, the second is "mbrtoc32", and the third is "non-virtual thunk to nsInputStreamReadyEvent::~nsInputStreamReadyEvent()" so the signature ends up as "mbrtoc32 | nsInputStreamReadyEvent::~nsInputStreamReadyEvent".

Signature generation seems like it's working as intended. If there are specific issues, let me know and I can look into those.

Comment 21

[James Willcox (:snorp) ([e-mail]) (he/him)]

In that case, the ones that only have "mbrtoc32" appear to be inactionable. We have no STR and no valid stacks.

Comment 22

[Marcia Knous [:marcia]]

Adding 66 as affected.

Comment 23

[David Bolter [:davidb] (NeedInfo me for attention)]

I think this has gone away. Please reopen if it ever shows up in Fennec 68.

Comment 24

[BugBot [:suhaib / :marco/ :calixte]]

Since the bug is closed, the stalled keyword is now meaningless.
For more information, please visit auto_nag documentation.