Closed Bug 1270298 Opened 4 years ago Closed 4 years ago

OOM crash on code.org tutorial running in a VM

Categories

(Firefox :: Untriaged, defect)

46 Branch
x86_64
Windows 7
defect
Not set
critical

Tracking

()

RESOLVED WORKSFORME
Tracking Status
firefox47 --- affected
firefox48 --- affected
firefox-esr45 --- affected

People

(Reporter: philbogle, Unassigned, NeedInfo)

References

()

Details

(Keywords: crash, regression, Whiteboard: [MemShrink])

Crash Data

Attachments

(2 files)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36

Steps to reproduce:

Firefox is experiencing a browser crash (full crash, not just an error) on a Code.org tutorial that worked fine in past versions and which works on other browsers.

The exact point of the crash is somewhat unpredictable but the following sequence of steps crashes very reliably in FF 46.0 and FF 46.0.1 running on a Saucelabs Windows VM.  (It does not crash in OSX.)

1. Go to
https://studio.code.org/s/allthethings/stage/4/puzzle/4?noautoplay=true
2. Click OK
3. Drag the "Get nectar block" inside the if block and click Run.
4. Refresh the page when you get to the completion dialog.
5. Click Run again.


Actual results:

Browser crashes at some point in the repro steps above, display the "Firefox has crashed dialog".  Before the crash, sometimes errors are visible in the console, suggesting memory corruption.  At different times I have seen errors about "out of memory", "corrupted PNG", and also invalid scripts, while in fact these resources all appear to be be valid.  

I note that FF 46.0 includes security fixes to the Jit; is it possible that one of these introduce a memory bug (use after free, double free, etc.)


Expected results:

Browser should not crash.
WFM in Fx46.0 & Nightly 49.0a1 (2016-05-03) on Win10.

https://developer.mozilla.org/en-US/docs/How_to_get_a_stacktrace_for_a_bug_report
Severity: normal → critical
Flags: needinfo?(philbogle)
Keywords: crash
Please note our most reliable repro is Win7.  We have also seen repros on Win10 but not nearly as reliably.
OS: Unspecified → Windows 7
Hardware: Unspecified → x86_64
Please let me know if there is additional data I can send you from our crashes that would help you repro.
If it helps, we have a number of persistent crashes in our automated integration tests against Saucelabs which hit this every day.  They aren't restricted to this particular tutorial.
Ah, I just noticed your link about how to get a stacktrace, I will provide that.
I have generated three repros of this.  These are all on a Saucelabs VM running FF 46.0.1.  (I was not successful today in reproducing this on a Windows laptop, though I have done so in the past.)  

bp-75358f21-f4ca-48fd-9770-987682160517  	5/17/2016	9:00 PM
bp-7b137df5-1301-40e5-8cae-957392160517         5/17/2016	8:58 PM
bp-8fe6eeea-ce5f-43ae-a438-cc5432160517         5/17/2016	8:56 PM
Flags: needinfo?(philbogle)
Status: UNCONFIRMED → RESOLVED
Crash Signature: [@ OOM | unknown | js::AutoEnterOOMUnsafeRegion::crash | js::TenuringTracer::moveToTenured ]
Closed: 4 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1257387
This is for a specific case of this crash signature, so I think it makes sense to leave it unduplicated.

(In reply to philbogle from comment #3)
> Please let me know if there is additional data I can send you from our
> crashes that would help you repro.

This crash happens when the browser runs out of memory. The next step would be to look at an about:memory report. It doesn't have to be in a case when it crashed.
Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: DUPLICATE → ---
Keywords: regression
Summary: New crash in Firefox 46/Windows on code.org tutorial (Memory corruption?) → OOM crash on code.org tutorial
Whiteboard: [MemShrink]
We've had issues before where running Firefox in a VM caused huge memory usage, so maybe this is similar.
Summary: OOM crash on code.org tutorial → OOM crash on code.org tutorial running in a VM
Windows indicates that only 70% of physical memory is in use at the time crash.  The physical memory on my VM is quite small (1.5MB), to be sure.

I managed to grab a couple snapshots before the crash which I will attach.

I do feel like there could be some regression in memory usage-- this didn't happen in FF45 or in other modern browsers running on the same VM.
Attached file memory-report.json.gz
This is an initial memory usage report after loading the page (but before the crash.)
That memory report doesn't seem to have the page loaded, can you create a new one?
Flags: needinfo?(philbogle)
THis is after loading and refreshing the page.   The browser crashed in the process of generating the .gz file but did create the json file.
Flags: needinfo?(philbogle)
It's address space exhaustion, but it doesn't make much sense. We have 330 rss and 3,712 vsize.

> 150.59 MB (100.0%) ++ explicit
> ...
>   349.68 MB ── private
>   330.54 MB ── resident
>     6.44 MB ── system-heap-allocated
> 3,712.22 MB ── vsize
>     1.25 MB ── vsize-max-contiguous
I've managed to reproduce the issue on the latest Firefox release (46.0.1), but I 've not been able to do so in the latest Nightly (49.0a1, Build ID 20160520030251). This led me to believe that the issue was fixed, so after a lot of investigating and testing, I've managed to get to the following pushlog of the build that fixes this issue https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?changeset=eac275b1daa5.

Phil, could you please download and install the latest Firefox Beta (https://www.mozilla.org/en-US/firefox/channel/#beta) and tell us if it's still crashing?
Flags: needinfo?(philbogle)
The FFBeta VM on Saucelab currently fails to launch so I haven't been able to try that yet; will do it as soon as possible.
Phil, is the issue still reproducible on the latest Firefox release (47)?
Flags: needinfo?(philbogle)
Flags: needinfo?(philbogle)
I'm going to close this for now, philbogle please feel free to reopen if you can reproduce.
Status: REOPENED → RESOLVED
Closed: 4 years ago4 years ago
Resolution: --- → WORKSFORME
Crash volume for signature 'OOM | unknown | js::AutoEnterOOMUnsafeRegion::crash | js::TenuringTracer::moveToTenured':
 - nightly (version 50): 0 crash from 2016-06-06.
 - aurora  (version 49): 0 crash from 2016-06-07.
 - beta    (version 48): 648 crashes from 2016-06-06.
 - release (version 47): 7548 crashes from 2016-05-31.
 - esr     (version 45): 561 crashes from 2016-04-07.

Crash volume on the last weeks:
             Week N-1   Week N-2   Week N-3   Week N-4   Week N-5   Week N-6   Week N-7
 - nightly          0          0          0          0          0          0          0
 - aurora           0          0          0          0          0          0          0
 - beta           120         82        111         74        110         80         31
 - release       1047       1068        969       1177       1359       1197        338
 - esr             67         69         54         70         99         43         25

Affected platforms: Windows, Mac OS X
You need to log in before you can comment on or make changes to this bug.