1328281 - Crash in shutdownhang | js::TraceLoggerThread::~TraceLoggerThread

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Marcia Knous [:marcia]]

This bug was filed from the Socorro interface and is 
report bp-5e643874-6cac-44f7-8765-9437a2170103.
=============================================================

Seen while looking at Nightly crashes: http://bit.ly/2j4DvMz. Crashes started using 20161229030206 build. Approximately 50 crashes in the last few days.

ni on Hannes since he worked in this area of code.

Comment 1

[Marcia Knous [:marcia]]

Adding some additional signatures that also started using the build in Comment 0.

Comment 2

[Marcia Knous [:marcia]]

ni on Triage owner since this has moved up to #2 overall top crash on nightly -  although volume isn't that high.

One comment mentions the crash happening after an update.

Comment 3

[Hannes Verschore [:h4writer]]

The first needinfo was correct. Not sure how to interpret the #2 overall top crash on nightly? It is something that isn't enabled by default. As result we shouldn't see it often, unless somebody of our team is trying something.

Comment 4

[Marcia Knous [:marcia]]

The top signature (shutdownhang | js::TraceLoggerThread::~TraceLoggerThread) has 163 crashes - http://bit.ly/2iYSQv5.

Comment 5

[Marcia Knous [:marcia]]

shutdownhang | vcruntime140.dll@0xc7e0 | free_impl | js::TraceLoggerThread::~TraceLoggerThread has 50 crashes

shutdownhang | vcruntime140.dll@0xc7c9 | free_impl | js::TraceLoggerThread::~TraceLoggerThread has 32

shutdownhang | free_impl | js::TraceLoggerThread::~TraceLoggerThread has 45

So between the 4 signatures we have almost 300 crashes.

Comment 6

[Hannes Verschore [:h4writer]]

I'm very suspicious about this bug. That deconstructor doesn't do anything, if people don't use tracelogger. Looking at the comments I don't think these are people that know about tracelogger nor specified specific environment flags to let it run.

To my knowledge this code hasn't changed in months. Making this even more strange.

My current theory about these crashes (non-nightly) is that we force-kill the browser after X seconds and take the stack. Now given a big enough population we will randomly sometimes be in "~TraceLoggerThread" even if takes only few ms.

Now what happened in 53a? (TraceLogger hasn't been touched).
Could it be that in January we fixed a big source of "shutdown hangs"? As a result we now by accident hit "~TraceLoggerThread" more often?

Comment 7

[Hannes Verschore [:h4writer]]

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=d7b6af32811bddcec10a47d24bd455a1ec1836fc&tochange=143bb4b9249e528e658f6ccc449991794b8675f8

Is the log on what changed in those builds. Nothing that jumps out for me.

Comment 8

[Hannes Verschore [:h4writer]]

In order to eliminate this path. Have we began to automate taking TraceLogger dumps for analysis? Starting around new year. This is the only way I can explain the increase of these signatures. Do you know of anything like that?

Comment 9

[Steve Fink [:sfink] [:s:]]

Good thought, but I don't think we have. We talked about it, and I think things are set to make the logs on demand, but I don't think it's on in any default configuration.

Shako, how often does Hasal run with TraceLogger on, and do you know if it would submit crash reports?

Comment 10

[Shako Ho]

Hi Steve,

  The TraceLogger data is trigged on demand. In general, we didn't provide this data in our report. And as I know we won't submit the crash reports.

Comment 11

[Hannes Verschore [:h4writer]]

I think this could be related to bug 1334129. I'll look at the crash-stats when that lands.

Comment 12

[Wayne Mery (:wsmwk)]

(In reply to Hannes Verschore [:h4writer] from comment #11)
> I think this could be related to bug 1334129. I'll look at the crash-stats
> when that lands.

That landed 18 days ago. 
Sort on buildid and we see there are still crashes in recent 54.0a1 like builds  20170208030203 and 20170202030211 (I didn't compare stacks)

https://crash-stats.mozilla.com/signature/?product=Firefox&signature=shutdownhang%20%7C%20vcruntime140.dll%400xc7c9%20%7C%20free_impl%20%7C%20js%3A%3ATraceLoggerThread%3A%3A~TraceLoggerThread&date=%3E%3D2017-01-23T19%3A09%3A00.000Z&date=%3C2017-02-17T19%3A09%3A00.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_sort=-build_id&_sort=-date&page=1

Comment 13

[Hannes Verschore [:h4writer]]

Bug 1334194 (nightly) and bug 1336448 (aurora/beta) are follow-up bugs that fixed another similar issue.
Looking at the stats the last crash on nighlty was on: Feb 08 2017. Which is the date before bug 1334194 landed. Confirming that it fixed the remaining issues. We can mark this as fixed.