Open Bug 1652689 Opened 5 years ago Updated 1 year ago

Video decoding sometimes breaks in a tab context on macOS

Categories

(Core :: Audio/Video: Playback, defect)

Product:
Core
Component:
Audio/Video: Playback
Version:
78 Branch
Type:
defect

Tracking

()

Status:
UNCONFIRMED

People

(Reporter: naktinis, Unassigned)

References

Details

Attachments

(4 files)

firefox_video_tab_issue.mp4
1011.30 KB, video/mp4
Details
playback_log.txt.child-11.moz_log_DECODE_ERROR.gz
1.65 MB, application/x-gzip
Details
about_support.json
24.90 KB, application/json
Details
Screen recording demonstrating functional and non-functional tabs
5.96 MB, video/quicktime
Details

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Firefox/78.0

Steps to reproduce:

I used a Firefox tab to load a page with multiple videos, some using Media Source Extensions, some not. After using the tab for a while (haven't yet narrowed down the exact sequence of actions leading to this) I tried playing any video from any source and with any encoding.

Actual results:

No video playback is possible in that tab any longer. I open any video (e.g. publicly available test mp4 and webm videos) by typing the URL into the address bar, but playback gets stuck. I can seek manually and frames are rendered, but proper playback no longer works. I can play the same videos in a new tab without any issues.

Sometimes no warnings are logged (for example, when I navigate to a video file directly instead of loading it in a <video> tag).

Sometimes (maybe because the video source is set with javascript) I see this:
Media resource [VIDEO_URL_GOES_HERE] could not be decoded, error: Error Code: NS_ERROR_DOM_MEDIA_DECODE_ERR (0x806e0004)
Details: void mozilla::AppleVTDecoder::OnDecodeError(OSStatus): OnDecodeError:ffffcd9a Dg5kLL9

Expected results:

Video playback should work.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core

Has this issue started recently? How long does it take to occur?

If you're able to reliably reproduce the problem, could you try using mozregression-gui to help identify which version the problem started in?

Flags: needinfo?(naktinis)

It would be good if there is a reliable step-to-reproduce. Does it only occur on Mac OS? Is it reproducible if the page with multiple videos contains only non-MSE videos?

I do think it's rather recent, as in within the last couple of versions of Firefox, but that's just a guess at this point. I'm sorry I don't have a step by step reproduction, I will let you know if I manage to narrow it down (and then I could test it on ohter operating systems, and run mozregression-gui).

So far the only clue I have is that "mozilla::AppleVTDecoder::OnDecodeError" error.

There were some changes in Firefox 78 that made us more likely to surface the mozilla::AppleVTDecoder::OnDecodeError error when video playback on Apple failed (bug 1626364). Prior to those changes we'd expect certain errors to still take place, but for us to not surface any information. I.e. we're reporting better diagnostics.

It would be useful to know if you see this issue in Fx 77 when checking with mozregression. That would help us narrow down if the above changes could be related, or if they're just helping us surface diagnostics (as intended).

I was able to reproduce something similar, but not exactly as described above on an earlier verion. I've installed mozregression and downloaded Firefox 76.0a1:
https://archive.mozilla.org/pub/firefox/nightly/2020/04/2020-04-02-09-51-45-mozilla-central/firefox-76.0a1.en-US.mac.dmg

Here's what happened (reproduced 3 times already):

  • I open a website containing some videos in a new tab;
  • At first everything works fine;
  • Then I come back to the tab after a few hours;
  • I see this error: NS_ERROR_NOT_AVAILABLE: with no further details apart from the traceback;
  • The error happens while executing this statement: ctx.drawImage(video, 0, 0, canvas.width, canvas.height) where video is an HTMLVideoElement;
  • No matter how many times I reload that particular tab, the error keeps appearing;
  • If I open a new tab and copy-and-paste the same URL, this error no longer happens (at least for some time), and drawImage works perfectly;
  • I think that aforementioned mozilla::AppleVTDecoder::OnDecodeError error happened while processing the same video as this NS_ERROR_NOT_AVAILABLE: ;
  • The difference from the report above is that some other video content can still be played back in the "broken" tab.

Since you mentioned changes to surfacing errors, I thought this may be related as the error in 76.0a1 is more obscure. Please let me know what further information I can provide.

Flags: needinfo?(naktinis)

Does the original issue occur on any pages with videos? I'm wondering if the issue in general is canvas related.

Can you reproduce the error mentioned in comment 6 on a current version of Fx? If so, do you get a NS_ERROR_NOT_AVAILABLE and a mozilla::AppleVTDecoder::OnDecodeError?

In the current version of Firefox, when I see the NS_ERROR_DOM_MEDIA_DECODE_ERR (0x806e0004) Details: void mozilla::AppleVTDecoder error, the canvas drawing does in fact seem to be broken. However, the message is printed as a "warning" and does not seem to include a traceback like the one in the older version.

Also in the current version I don't see NS_ERROR_NOT_AVAILABLE in this context (or at least haven't seen it for quite some time).

One more clarification. In the recent version I also sometimes have the state where the canvas drawing is not working (with the mozilla::AppleVTDecoder::OnDecodeError being shown), but the other video playback IS working. But it still seems to be tab-level. As in reloading the tab doesn't resolve the canvas/warning issue, but opening a new tab/window does.

So all video playback getting broken and canvas drawing (or decoding for canvas drawing) getting broken may or may not be the same issue.

Could you try gathering some logs to help diagnose this? Could you execute the following from the terminal?

export MOZ_LOG=PlatformDecoderModule:5,MediaFormatReader:5.MediaDecoder:5,sync,timestamp
export MOZ_LOG_FILE=~/playback_log.txt
cd /Applications/Firefox.app/Contents/MacOS
./firefox-bin

That should setup logging and log to playback_log.txt files in your home dir. If you could reproduce the bug then zip and attach those files here it would be helpful. If the logs get too big to attach to bugzilla, or if I can help with more detailed instructions, let me know.

Severity: -- → S2
Flags: needinfo?(naktinis)
Flags: needinfo?(naktinis)

Thank you. I was able to reproduce it with the environment variables you've provided. Please find the logs for one of the failing tabs attached. At the first glance the following lines seem related:

2020-07-20 23:14:54.256307 UTC - [Child 10893: MediaPlayback #4]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::OnVideoDemuxCompleted: 1 video samples demuxed (sid:0)
2020-07-20 23:14:54.256331 UTC - [Child 10893: MediaPlayback #4]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::ScheduleUpdate: SchedulingUpdate(Video)
2020-07-20 23:14:54.256365 UTC - [Child 10893: MediaPlayback #4]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::Update: Processing update for Video
2020-07-20 23:14:54.256385 UTC - [Child 10893: MediaPlayback #4]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::Update: Update(Video) ni=1 no=1 in:0 out:0 qs=0 decoding:0 flushing:0 desc:apple software VT decoder pending:0 waiting:0 eos:0 ds:0 sid:4294967295 waitcdm:0
2020-07-20 23:14:54.256395 UTC - [Child 10893: MediaPlayback #4]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::HandleDemuxedSamples: Input:4225200000 (dts:4225200000 kf:1)
2020-07-20 23:14:54.256501 UTC - [Child 10893: MediaPDecoder #6]: D/PlatformDecoderModule AppleVTDecoder[0x11d568ee0] mp4 input sample 11e5785c0 pts 4225200000 duration 200000 us keyframe 1424 bytes
2020-07-20 23:14:54.256778 UTC - [Child 10893: MediaPDecoder #6]: D/PlatformDecoderModule AppleVTDecoder[0x11d568ee0] AppleVideoDecoder PlatformCallback status -12902 flags 2
2020-07-20 23:14:54.256848 UTC - [Child 10893: MediaPlayback #5]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::NotifyError: Video Decoding error
2020-07-20 23:14:54.256866 UTC - [Child 10893: MediaPlayback #5]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::ScheduleUpdate: SchedulingUpdate(Video)
2020-07-20 23:14:54.256884 UTC - [Child 10893: MediaPlayback #5]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::Update: Processing update for Video
2020-07-20 23:14:54.256896 UTC - [Child 10893: MediaPlayback #5]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::NotifyError: Video Decoding error
2020-07-20 23:14:54.256904 UTC - [Child 10893: MediaPlayback #5]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::ScheduleUpdate: SchedulingUpdate(Video)
2020-07-20 23:14:54.256916 UTC - [Child 10893: MediaPlayback #5]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::Update: Processing update for Video
2020-07-20 23:14:54.256927 UTC - [Child 10893: MediaPlayback #5]: D/MediaFormatReader MediaFormatReader[0x11d67d000] ::Update: Rejecting Video promise: DECODE_ERROR
2020-07-20 23:14:54.257836 UTC - [Child 10893: MediaPlayback #4]: V/MediaFormatReader MediaFormatReader[0x11d67d000] ::ResetDecode: 
2020-07-20 23:14:54.257887 UTC - [Child 10893: MediaPlayback #4]: D/MediaFormatReader MediaFormatReader[0x11d67d000] ::Reset: Reset(Video) BEGIN
2020-07-20 23:14:54.257930 UTC - [Child 10893: MediaPlayback #4]: D/MediaFormatReader MediaFormatReader[0x11d67d000] ::Reset: Reset(Video) END
2020-07-20 23:14:54.257983 UTC - [Child 10893: MediaPlayback #4]: D/MediaFormatReader MediaFormatReader[0x11d67d000] ::Shutdown:

Also, not sure how useful this is, but I also saw this output in the terminal (when running firefox from command line):

[Child 10400, MediaDecoderStateMachine #13] WARNING: Decoder=15647c800 Decode error: NS_ERROR_DOM_MEDIA_DECODE_ERR (0x806e0004) - void mozilla::AppleVTDecoder::OnDecodeError(OSStatus): OnDecodeError:ffffcd9a: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachine.cpp, line 3470

I see status -12902 coming back from the video decoder. This looks to be a parameter error[0], which doesn't help narrow down much what is going on, sadly

  • When the issue next takes place could you let me know if this file fails to play? This should take a different decode path, so it would be useful to check if this also becomes busted.
  • Could you navigate to about:support and copy the information there onto this bug?
  • Could you link the video you used to produce the above logs here? The logs for the video before the failure are Creating AppleVTDecoder for 178x100 h.264 video in case that helps.
  • Did the video that caused the failure above decode okay before the failure? I see logs that look like a video with the same dimensions played okay before the failure.
    • When it failed did it fail after a certain action? E.g. did it failed as soon as the video was opened, did it fail following seeking in the video?

[0] https://developer.apple.com/documentation/videotoolbox/1490398-error_code_constants/kvtparametererr?language=objc

Flags: needinfo?(naktinis)

When the issue next takes place could you let me know if this file fails to play? This should take a different decode path, so it would be useful to check if this also becomes busted.

I haven't seen the entire-tab-all-video-failing mode for a while now, but I almost always eventually see the AppleVTDecoder error. And I'd describe it as entire-tab-certain-videos-failing (when it happens some videos still play, but some videos that have just played fine or play fine in a new tab/window, will never play despite loading/reloading a new URL, maybe canvas-related). So as I've said, I'm not sure they're the exact same thing. I will try this file if I get into the entire-tab-all-video-failing mode again.

Could you navigate to about:support and copy the information there onto this bug?

Sure, I'll attach it as a separate file.

Could you link the video you used to produce the above logs here? The logs for the video before the failure are Creating AppleVTDecoder for 178x100 h.264 video in case that helps.

Sure. Here's one: https://vv3-1.muse.ai/w/9329b75e406507b7ba15d36bffaa840d6075b7f713244ae2598f65d47f932cb3/videos/video-xs.mp4
However, it's not specific to this video in the sense that these videos will play (and be drawn into canvas) fine in a new tab (for a while), and once the decoder issue happens once it will keep happening to all videos like this one (in that particular tab) but not to some different kinds of videos. Meanwhile some other video content in the same tab will play just fine.

Let me try to illustrate this in a different form:

  1. I open a new tab. All videos play fine, including all of these tiny ones (like the example above), drawing into canvas from video also works.
  2. I wait for some time (in some cases I don't even need to interact with that tab).
  3. I start getting AppleVTDecoder errors for some small videos (video-xs.mp4). Other (normally larger) videos play fine.
  4. Importantly, if I open that video (video-xs.mp4) in the same tab by directly navigating to it - it still fails. The native player shows a gray: "Video can't be played because the file is corrupt" error in the DOM.
  5. Also importantly, if I open any other video-xs.mp4 in that same tab - it still fails. Then I tried opening other small mp4 videos and they seem to fail as well, so it's probably not specific to this video-xs.mp4 type. Small webm videos seem to work fine.
  6. If I open any other (larger?) video (mp4 or webm, from any source) in the same tab - it works just fine.
  7. If I open any one of those video-xs.mp4 in a some other tabs (but not all other tabs) it works just fine.

Here's another important behaviour I've just noticed. Sometimes it also doesn't work in new tabs, but it still works in other previously opened tabs. So maybe it's not exactly tab-specific, but it's also not browser-instance-wide.

If I tried to summarize my best understanding to date is that at some point in some tabs low-resolution mp4/h264 videos stop working. In my tests 360p mp4 videos work, and 144p webm videos work, while 198p, 144p, 100p mp4 videos fail.

Did the video that caused the failure above decode okay before the failure? I see logs that look like a video with the same dimensions played okay before the failure.

Yes, it decoded fine before the failure. I don't think I need to take any particular action in that tab or even that window. I may be using Firefox in other windows though. It seems that I only need to wait.

Flags: needinfo?(naktinis)
Attached file about_support.json —

Here's one more observation. I kept a few of these tabs open for a while and was checking them from time to time. What happens is some of the tabs that have been working stop working, and some of those that were producing an error start to work. In these tabs I simply had low-resolution .mp4 files opened directly (not even on a website; also wasn't interacting with them other than reloading).

When the issue next takes place could you let me know if this file fails to play? This should take a different decode path, so it would be useful to check if this also becomes busted.

OK, so now I have one tab in a state where no videos are played back and even no error is shown. This happens to the webm video you asked to check as well. The behaviour looks identical to my original report and what I demonstrate in the attached screen capture. In these cases I can seek manually to get 1 or 2 frames played, but it gets stuck there.

Since in this case no error is logged, it may be a separate issue from what I detailed in comment 15. I was even able to combine them: I loaded one of those small videos into this fully broken tab and with a small video I do see an error.

To recap, it is possible that these are 2 separate issues:

A) The original issue shown in the attached screen capture:

  • Individual frames can be rendered, also can seek, but gets stuck after 1-2 frames
  • Both mp4 and webm videos fail to play
  • Does not seem to depend on video resolution
  • No error message
  • As I open new tabs some of them will have the same issue, while others will work fine

B) The issue described in comment 15:

  • Small mp4 files do not play at all (webm files don't seem to be affected), frames are not rendered
  • An error message is shown: Details: void mozilla::AppleVTDecoder::OnDecodeError(OSStatus): OnDecodeError:ffffcd9a
  • Tabs can spontaneously get in and out of this state
  • As I open new tabs some of them will have the same issue, while others will work fine (same as A)

I don't think you need to have any special type of files to reproduce this (other than resolution for issue B). E.g. I could reproduce B) by just searching for small videos, like on this website: https://file-examples.com/index.php/sample-video-files/sample-mp4-files/ (480x270 fails, while 640x360 works).

See Also: → 1655302

I'm seeing similar behavior. I have a tab where no mp4 videos play, despite showing no decoder exceptions. However, these same videos play with no issue on another tab. By the way, webm plays well on both tabs.
I am using Firefox Developer Edition 81.0b9.

I'm experiencing the same issue as seen in comment 15. One tab plays the video with no issues, while the other fails to play this same video, showing the following error message:

Media resource https://file-examples-com.github.io/uploads/2017/04/file_example_MP4_480_1_5MG.mp4 could not be decoded, error: Error Code: NS_ERROR_DOM_MEDIA_DECODE_ERR (0x806e0004)
Details: void mozilla::AppleVTDecoder::OnDecodeError(OSStatus): OnDecodeError:ffffc356

The tab in question fails to play any other mp4 videos, but plays webm, for example.

Browser: Firefox Developer Edition 82.0b4 (64-bit)
OS: macOS Big Sur 11.0 Beta (20A5374i)
Laptop: MacBook Pro (13-inch, 2017, Four Thunderbolt 3 Ports), Intel Iris Plus Graphics 650 1536 MB

Severity: S2 → S3

Any news on this? Did anyone fix the problem?

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: