Opening nytimes.com HLS video crashes the browser - [@ libart.so@0x31b6df ]

VERIFIED FIXED in Firefox 67

Status

()

defect
P3
normal
VERIFIED FIXED
8 months ago
3 months ago

People

(Reporter: csheany, Assigned: jhlin, NeedInfo)

Tracking

Firefox 65
Firefox 67
ARM
Android
Points:
---

Firefox Tracking Flags

(firefox64 wontfix, firefox65 wontfix, firefox66 wontfix, firefox67 verified)

Details

(Whiteboard: [geckoview:p1], crash signature)

Attachments

(3 attachments)

User Agent: Mozilla/5.0 (Android 7.1.1; Tablet; rv:65.0) Gecko/65.0 Firefox/65.0

Steps to reproduce:

1. Open nytimes.com/video


Actual results:

After a few seconds the browser crashes.


Expected results:

The browser shouldn't crash.
Status: UNCONFIRMED → NEW
Crash Signature: [@ libart.so@0x31b6df ]
Ever confirmed: true
OS: Unspecified → Android
Hardware: Unspecified → ARM
Summary: Opening nytimes.com/video crashes the browser → Opening nytimes.com/video crashes the browser - [@ libart.so@0x31b6df ]
The desktop site doesn't seem to be affected.
What does this mean from a browser standpoint?

Is there a fix?
Also,

Crash in @ java.lang.OutOfMemoryError: at java.lang.Long.valueOf(Long.java)

https://crash-stats.mozilla.org/report/index/b496a419-545d-4e78-9270-c95080181226

Crash in @ java.lang.OutOfMemoryError

https://crash-stats.mozilla.org/report/index/c998b98a-8eeb-4be4-a48f-208060181226
Liz, would you mind taking a look?
Flags: needinfo?(lhenry)
I'm not hitting the crash. Petru, want to see if it's reproducible for you?
Flags: needinfo?(lhenry) → needinfo?(petru.lingurar)
This crash is specific to a device. This crash is on the Samsung Galaxy Tab A 8.0 (2015) SM-T350.
With only this in one tab and Nightly Home in another it doesn't seem to crash but does cause Bug 1494748.
What does the first crash signature indicate?
Crash in @ java.lang.OutOfMemoryError: at java.util.concurrent.ConcurrentLinkedQueue.iterator(ConcurrentLinkedQueue.java)

https://crash-stats.mozilla.org/report/index/42776b2b-5799-4e46-9a99-aa6cb0181227
Crash in @ java.lang.OutOfMemoryError: at org.mozilla.gecko.media.GeckoHlsVideoRenderer.createInputBuffer(GeckoHlsVideoRenderer.java)

https://crash-stats.mozilla.org/report/index/a4ada562-c967-4bbe-862a-aff5e0181227
Priority: -- → P3
Crash in @ java.lang.OutOfMemoryError: at org.mozilla.gecko.mozglue.GeckoLoader.nativeRun(Native Method)

https://crash-stats.mozilla.org/report/index/f56198d9-254f-4d8a-a341-57bb90181228
Crash in @ java.lang.OutOfMemoryError

https://crash-stats.mozilla.org/report/index/460f0c0d-6eab-42d6-b8a6-68f610181228
Jan, please see comment 8
Flags: needinfo?(jh+bugzilla)
OOMs/near-OOMs/resource exhaustion can cause all sorts of weird behaviour.
Flags: needinfo?(jh+bugzilla)
Crash in @ java.lang.OutOfMemoryError

https://crash-stats.mozilla.org/report/index/178d530f-b5b5-46c3-b56c-79f7c0190102
With media.hls.enabled set to false the page didn't cause any issues.
Component: General → Audio/Video

Crash in @ java.lang.OutOfMemoryError: at com.android.okhttp.okio.Segment.<init>(Segment.java)

https://crash-stats.mozilla.org/report/index/63959b72-cbdf-41f5-9efd-0820d0190108

Would you mind investigating this with Focus+GV?

After some time about:blank opens and I wonder if there are similiarites.

Flags: needinfo?(cpeterson)

(In reply to csheany from comment #17)

With media.hls.enabled set to false the page didn't cause any issues.

Interesting! Some of your OutOfMemoryError stack traces all point to video codec code, so perhaps there is a bug in Gecko and/or the codec that causes video data to keep growing.

Can the video still play video when media.hls.enabled is set to false?

(In reply to csheany from comment #19)

Would you mind investigating this with Focus+GV?

Sorry, I don't have a Samsung Galaxy Tab A to test.

After some time about:blank opens and I wonder if there are similiarites.

Perhaps the about:blank indicates the tab (content process) crashed? There is a different bug open about about:blank opening in tabs.

Flags: needinfo?(cpeterson)
Summary: Opening nytimes.com/video crashes the browser - [@ libart.so@0x31b6df ] → Opening nytimes.com HLS video crashes the browser - [@ libart.so@0x31b6df ]
Whiteboard: [geckoview:p3]

Thank you for your response.

Yes, the videos do still play. I forgot to mention that.

I didn't necessarily mean with the samme device but the app in general.

Also, it doesn't require playing video just scroling the page or having it in the background.

Removing my NI as it is now on geckoview team's list.

Flags: needinfo?(petru.lingurar)

David, would you mind taking a look?

Flags: needinfo?(dbolter)

Andreas what is the priority for this bug? (This should affect GV as well)

Flags: needinfo?(dbolter) → needinfo?(abovens)

Alastor, would you mind taking a look?

Flags: needinfo?(alwu)

It looks like the crash happened while site was calling MediaCapabilities.decodingInfo() which would create a HLS decoder and caused the crash.

I will forward this NI to John, who know more Android video decoding than me, to see if he has any idea.

Flags: needinfo?(alwu) → needinfo?(jolin)

The OutOfMemoryError cases were fixed in bug 1517252. Mark this as dup for now. If the crash still happens when playing HLS video in 66, please reopen it.

Status: NEW → RESOLVED
Closed: 5 months ago
Flags: needinfo?(jolin)
Resolution: --- → DUPLICATE
Duplicate of bug: 1517252

Thank you both for your response.

It doesn't seem to crash but does give error messages.

John, are the "This video is currently unavailable" and "Video Data Failed to Load" errors in csheany's screenshot just a side effect of Fennec gracefully handling the OOM instead of crashing (bug 1517252)? If so, why are we OOM'ing in the first place?

Flags: needinfo?(jolin)
Whiteboard: [geckoview:p3] → [geckoview:p1]

After further testing, it still crashes.

(In reply to Chris Peterson [:cpeterson] from comment #31)

John, are the "This video is currently unavailable" and "Video Data Failed to Load" errors in csheany's screenshot just a side effect of Fennec gracefully handling the OOM instead of crashing (bug 1517252)? If so, why are we OOM'ing in the first place?

Yes, I think the screenshot shows the fix in bug 1517252 is effective.

I've managed to reproduce the OOM once by letting NYT video site play video for a couple for hours. From what I observed, it looks quite similar to what James Cheng found in bug 1424168 comment 2: the NYT video page creates and loads 10 video elements and Exoplayer allocates more than 100MB in the Java heap just to buffer contents for them. Although bug 1424168 has reduced the size significantly, it is still not enough for sites that preloads many videos.

Reopen this bug and will investigate if there is other way to further reduce the memory required for this sort of sites.

Status: RESOLVED → REOPENED
Flags: needinfo?(jolin)
Resolution: DUPLICATE → ---
Duplicate of this bug: 1517785

Each instance has an instance of Java ExoPlayer that consumes memory in the
limited JVM heap. Too many concurrent players will cause OutOfMemoryError.

Depends on D20419

Pushed by jolin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/da7bf6175838
p1: reduce ExoPlayer default buffer size. r=snorp
https://hg.mozilla.org/integration/autoland/rev/63b009fbe2bc
p2: limit the concurrently created HLSDecoder instances. r=jya
Status: REOPENED → RESOLVED
Closed: 5 months ago5 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 67

This doesn't look like a frequently occuring crash (at least not with this particular signature).
Do you think letting this go to release with 67 is OK, or do you want to try beta uplift?
Keeping in mind we only have 2 weeks of beta 66 left.

(In reply to Liz Henry (:lizzard) (use needinfo) from comment #39)

This doesn't look like a frequently occuring crash (at least not with this particular signature).
Do you think letting this go to release with 67 is OK, or do you want to try beta uplift?
Keeping in mind we only have 2 weeks of beta 66 left.

I prefer not to uplift to beta.

Flags: needinfo?(jolin)

I think the first crash is irrelevant at this point hence all the others.

The time depends on the amount of other tabs open.

In the middle of writing/ testing the browser crashed and didn't trip the reporter.

I have a feeling Bug 1508641 may be of further assistance.

It's still not a great experience.

There are a lot of empty elements.

More videos can be loaded than the minimum and I'm not sure it adapts.

The progress bar takes a while to complete. The page just feels heavy.

FWIW, Focus with WebView doesn't seem to have any issues.

Please see Comment 12

Flags: needinfo?(eitan)

If it helps, based on Comment 28 and Comment 33...

I have been testing further with Reference Browser 1.0.1910 (Build #1063145 Gecko 67.0a1-20190227104426) to see which Crash Signatures it triggers.

... how it performs and

I am not assigned to this bug and I don't think I can help. If this needs anymore attention you can needinfo the assignee.

Flags: needinfo?(eitan)

Thank you for your response.

The reason I brought this to your attention was to provide a use case for Bug 1508641 which I noticed has not been fixed yet.

Flags: needinfo?(eitan)

If there is anything in that bug that needs clarification you can needinfo me there. It seems to not be frequent anymore, and the cases where it still happens are not accessibility related.

Flags: needinfo?(eitan)

Marcia, does this site give you any problems?

Flags: needinfo?(mozillamarcia.knous)

Have you had any issues?

Flags: needinfo?(sorina.florean)

(In reply to csheany from comment #49)

Marcia, does this site give you any problems?

I will test over the weekend on all my devices. A cursory test on Pixel 3 running Android Pie showed no issues playing videos on that site.

Flags: needinfo?(mozillamarcia.knous)

(In reply to csheany from comment #50)

Have you had any issues?

Tested on the latest Nightly build following your steps and the browser did not crash. The videos played with no interruptions and no error was displayed.
Device: Nexus 9(Android 7.1.1).
Note that we don't have the Samsung Galaxy Tab A 8.0 (2015) SM-T350 device.

Flags: needinfo?(sorina.florean)

Thank you for your response.

Would you mind testing Nightly with other devices as well as Release?

Also, how much time did you allow for and how many tabs did you have open?

Were videos loaded after scrolling?

Flags: needinfo?(sorina.florean)

Tested on 67.0b3 with the following devices:

  • Xiaomi Mi4i(Android 5.0.2)
  • Samsung Galaxy Tab S3 (Android 8.0)
  • Google Pixel (Android 9)
  • Samsung Galaxy Note 8 (Android 9.0)

Didn't encounter any issues, the video played without interruptions. I had more than 7 tabs opened and also just one, watched 4-5 videos in fullscreen mode without interacting with the screen and also played them individually. In all cases, no issues were displayed.

If you have other issues, and no crash displayed, please make a video to understand the problem. Thanks!

Flags: needinfo?(sorina.florean)

Based on my last comment I will update the flag to VERIFIED.

Status: RESOLVED → VERIFIED

The problem isn't necessarily playing video but keeping the browser open.

It still crashes but not sure why.

Flags: needinfo?(sorina.florean)

(In reply to csheany from comment #56)

The problem isn't necessarily playing video but keeping the browser open.

It still crashes but not sure why.

The bug was filed for the crashes and this issue is fixed, if you find another problem, file a new bug. I asked in comment 54 to make a video with the new issue but with no answer.

Flags: needinfo?(sorina.florean)

The issue is still the same. It might be fixed for some but not all.

Even without these patches other devices might not have a problem in Release.

The only thing a video would display is the browser crashing.

Without a report attached that is not very actionable (Bug 1522107)

Given the different signatures it would have been ideal for this to be a meta and spread out but that is neither here nor there.

I'm still trying to understand the approach (Bug 1517785)

There are a few others (Bug 1517985 - Bug 1512147 - Bug 1544888)

You need to log in before you can comment on or make changes to this bug.