Closed Bug 1547892 Opened 6 years ago Closed 6 years ago

Video does not playback properly on reload

Categories

(Core :: Audio/Video: Playback, defect, P2)

Product:
Core
Component:
Audio/Video: Playback
Platform:
All
Windows 10
Type:
defect

Tracking

()

Status:
VERIFIED FIXED
Milestone:
mozilla68
Tracking Flags:
Tracking Status
firefox68 --- verified

People

(Reporter: vchin, Assigned: cpearce)

Details

Attachments

(1 file)

Bug 1547892 - Load video element in VideoDocument slightly later during load. r?bzbarsky
47 bytes, text/x-phabricator-request
Details | Review

STR:

  1. Open a new tab to https://faraday.basschouten.com/mozilla/Pageload/Contentfullness/CNNOutput.mp4
  2. The video plays back as expected on first load
  3. Press F5 to reload
  4. Video "Plays" but the screen stays on the first frame
  5. All subsequent reloads also have this problem on Firefox. Edge does not have this issue.

I'm using Windows 10 1809 on the C630 Yoga with the 2019-04-29 nightly build.

All the videos from https://faraday.basschouten.com/mozilla/Pageload/Contentfullness/ with the exception of the reddit one have the same problem. Sometimes the video would also start playing back but stop showing frames (Tripadvisor).

Is this only on Windows for ARM?

Flags: needinfo?(vchin)

Good question. Just tried on the 2018 Reference Machine running Windows 10 Firefox 66 Release and I was also able to reproduce there. So no, not ARM specific it seems.

Flags: needinfo?(vchin)
Hardware: ARM64 → All

It seems when it autoplays on reload then the frame is stuck. If I let the video finish the autoplay and press play again, then the video plays back correctly.

Bas tried on his higher end windows laptop and didn't see this issue so it may only be a problem on lower end hardware? I installed Nightly on the 2018 Reference Machine and it still has the same problem.

Sounds like autoplay policy.
If you refresh the screen, you will need to click on play again. :alwu, what do you think?

Flags: needinfo?(alwu)

It seems when it autoplays on reload then the frame is stuck. If I let the video finish the autoplay and press play again, then the video plays back correctly.

Do you mean the video could only start playback correctly when you explicitly press play button? It won't automatically start?

Does the stuck mean that the video freezes on the first frame and the time stops moving forward? Or you just mean the video didn't start autoplay?

If you just mean that the video didn't start automatically, that is because video is blocked by our autoplay policy. You could try to set the pref media.autoplay.default=0 to see if the problem still exists. But if not, I don't think it's related with autoplay policy, because blocking autoplay should not affect the playback itself.

In addition, do you mind to record you screen to show us the issue? Because I'm not sure what is the meaning for the videos in comment0 and comment3.

Thank you.

Flags: needinfo?(alwu) → needinfo?(vchin)

This sounds like Bug 1532193, which will apparently be fixed in an imminent driver update.

Note video documents are allowed to autoplay audibly; if you open a video directly, we don't block autoplay.

Do you mean the video could only start playback correctly when you explicitly press play button? It won't automatically start?

Does the stuck mean that the video freezes on the first frame and the time stops moving forward? Or you just mean the video didn't start autoplay?

It automatically starts but remains on the first frame.

In addition, do you mind to record you screen to show us the issue? Because I'm not sure what is the meaning for the videos in comment0 and comment3.

Sorry the file was too big so I had to upload it to google drive:
https://drive.google.com/file/d/1o2U5BrGDpDX3_kU8AjCn8HF3XJgretan/view?usp=sharing

Flags: needinfo?(vchin)

This sounds like Bug 1532193, which will apparently be fixed in an imminent driver update.

Bug 1532193 seems related to Windows on ARM but I can reproduce this even on x86 hardware. The video shared above was created on the 2018 Reference Laptop.

(In reply to Vicky Chin [:vchin] from comment #10)

This sounds like Bug 1532193, which will apparently be fixed in an imminent driver update.

Bug 1532193 seems related to Windows on ARM but I can reproduce this even on x86 hardware. The video shared above was created on the 2018 Reference Laptop.

My bad. I got excited when I read comment #1 and jumped to conclusions.

I can repro on my Lenovo C630 ARM64 laptop. Here's a profile:
https://perfht.ml/2LhIsQP

If you look at the markers on the MediaPlayback* threads, you'll see we often end up discarding the video frames. The run in the above profile where we have markers for played video frames is when I CTRL+SHIFT+R reloaded.

What are the specs for the 2018 Reference Laptop?

Dell Inspiron 15 3000
Intel HD Graphics 400
15.6" HD 1366 x 768 pixels
500GB 5400RPM HDD
4GB DDR3L Memory
Intel Celeron N3060 1.6 (2M Cache, up to 2.48 GHz)

(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #11)

I can repro on my Lenovo C630 ARM64 laptop. Here's a profile:
https://perfht.ml/2LhIsQP

If you look at the profile, you see when that we're doing video decoding in the content process on the runs that fail to play, and doing video decoding on the runs that succeed in the GPU process. So we're using SW decoding for some reason some of the time?

Priority: -- → P2
Assignee: nobody → cpearce

We are dropping frames like crazy because we're using the software decoder. We're using the software decoder because we end up passing a null mKnowsCompositor into the MediaChangeMonitor when we create a VideoDocument, so when the MediaChangeMonitor tries to create a decoder it doesn't know whether the compositor supports HW decoding, and bails out and creates a software decoder here:

xul.dll!mozilla::GpuDecoderModule::CreateVideoDecoder(const mozilla::CreateDecoderParams & aParams) Line 60 C++
xul.dll!mozilla::MediaChangeMonitor::CreateDecoder(mozilla::DecoderDoctorDiagnostics * aDiagnostics) Line 451 C++
xul.dll!mozilla::MediaChangeMonitor::MediaChangeMonitor(mozilla::PlatformDecoderModule * aPDM, const mozilla::CreateDecoderParams & aParams) Line 241 C++
xul.dll!mozilla::PDMFactory::CreateDecoderWithPDM(mozilla::PlatformDecoderModule * aPDM, const mozilla::CreateDecoderParams & aParams) Line 292 C++
xul.dll!mozilla::PDMFactory::CreateDecoder(const mozilla::CreateDecoderParams & aParams) Line 227 C++
xul.dll!mozilla::MediaFormatReader::DecoderFactory::DoCreateDecoder(mozilla::MediaFormatReader::DecoderFactory::Data & aData) Line 446  C++
xul.dll!mozilla::MediaFormatReader::DecoderFactory::RunStage(mozilla::MediaFormatReader::DecoderFactory::Data & aData) Line 368 C++
xul.dll!mozilla::MediaFormatReader::DecoderFactory::RunStage::<unnamed-tag>::operator()(RefPtr<mozilla::AllocPolicy::Token> aToken) Line 346  C++
[...]

mKnowsCompositor is initialized to null here:

xul.dll!mozilla::MediaFormatReader::MediaFormatReader(mozilla::MediaFormatReaderInit & aInit, mozilla::MediaDataDemuxer * aDemuxer) Line 893  C++
xul.dll!mozilla::DecoderTraits::CreateReader(const mozilla::MediaContainerType & aType, mozilla::MediaFormatReaderInit & aInit) Line 263  C++
xul.dll!mozilla::ChannelMediaDecoder::CreateStateMachine() Line 220 C++
xul.dll!mozilla::ChannelMediaDecoder::Load(nsIChannel * aChannel, bool aStreamListener, nsIStreamListener * *) Line 258 C++
xul.dll!mozilla::dom::HTMLMediaElement::SetupDecoder<mozilla::ChannelMediaDecoder,nsIChannel *&,bool &,nsIStreamListener **&>(mozilla::ChannelMediaDecoder * aDecoder, nsIChannel * & aArgs, bool & aArgs, nsIStreamListener * * & aArgs) Line 4449 C++
xul.dll!mozilla::dom::HTMLMediaElement::InitializeDecoderForChannel(nsIChannel * aChannel, nsIStreamListener * * aListener) Line 4532 C++
xul.dll!mozilla::dom::HTMLMediaElement::LoadWithChannel(nsIChannel * aChannel, nsIStreamListener * * aListener) Line 2628 C++
xul.dll!mozilla::dom::VideoDocument::CreateSyntheticVideoDocument() Line 112  C++
xul.dll!mozilla::dom::VideoDocument::SetScriptGlobalObject(nsIScriptGlobalObject * aScriptGlobalObject) Line 77 C++
xul.dll!nsGlobalWindowOuter::SetNewDocument(mozilla::dom::Document * aDocument, nsISupports * aState, bool) Line 2193 C++
xul.dll!nsDocumentViewer::InitInternal(nsIWidget * aParentWidget, nsISupports * aState, const mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> & aBounds, bool) Line 966  C++
xul.dll!nsDocumentViewer::Init(nsIWidget * aParentWidget, const mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> & aBounds) Line 717  C++
xul.dll!nsDocShell::SetupNewViewer(nsIContentViewer * aNewViewer) Line 8373 C++
xul.dll!nsDocShell::Embed(nsIContentViewer * aContentViewer, const char * aCommand, nsISupports * aExtraInfo) Line 6340 C++
xul.dll!nsDocShell::CreateContentViewer(const nsTSubstring<char> & aContentType, nsIRequest * aRequest, nsIStreamListener * * aContentHandler) Line 8188  C++
xul.dll!nsDSURIContentListener::DoContent(const nsTSubstring<char> & aContentType, bool aRequest, nsIRequest * aContentHandler, nsIStreamListener * * aAbortProcess, bool *) Line 182 C++
xul.dll!nsDocumentOpenInfo::TryContentListener(nsIURIContentListener * aListener, nsIChannel * aChannel) Line 747 C++
xul.dll!nsDocumentOpenInfo::DispatchContent(nsIRequest * request, nsISupports * aCtxt) Line 420 C++
xul.dll!nsDocumentOpenInfo::OnStartRequest(nsIRequest * request) Line 299 C++
[...]

There it's null, as when we created the video element in the VideoDocument a little earlier the PresShell hadn't yet been setup, here:

xul.dll!mozilla::dom::VideoDocument::CreateSyntheticVideoDocument() Line 112 C++
xul.dll!mozilla::dom::VideoDocument::SetScriptGlobalObject(nsIScriptGlobalObject * aScriptGlobalObject) Line 77 C++
xul.dll!nsGlobalWindowOuter::SetNewDocument(mozilla::dom::Document * aDocument, nsISupports * aState, bool) Line 2193 C++
xul.dll!nsDocumentViewer::InitInternal(nsIWidget * aParentWidget, nsISupports * aState, const mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> & aBounds, bool) Line 966  C++
xul.dll!nsDocumentViewer::Init(nsIWidget * aParentWidget, const mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> & aBounds) Line 717  C++
xul.dll!nsDocShell::SetupNewViewer(nsIContentViewer * aNewViewer) Line 8373 C++
xul.dll!nsDocShell::Embed(nsIContentViewer * aContentViewer, const char * aCommand, nsISupports * aExtraInfo) Line 6340 C++
xul.dll!nsDocShell::CreateContentViewer(const nsTSubstring<char> & aContentType, nsIRequest * aRequest, nsIStreamListener * * aContentHandler) Line 8188  C++
xul.dll!nsDSURIContentListener::DoContent(const nsTSubstring<char> & aContentType, bool aRequest, nsIRequest * aContentHandler, nsIStreamListener * * aAbortProcess, bool *) Line 182 C++
xul.dll!nsDocumentOpenInfo::TryContentListener(nsIURIContentListener * aListener, nsIChannel * aChannel) Line 747 C++
xul.dll!nsDocumentOpenInfo::DispatchContent(nsIRequest * request, nsISupports * aCtxt) Line 420 C++
xul.dll!nsDocumentOpenInfo::OnStartRequest(nsIRequest * request) Line 299 C++
[...]

We end up creating the Presshell slightly later:

xul.dll!mozilla::PresShell::Initialize() Line 1667  C++
xul.dll!mozilla::dom::MediaDocument::StartLayout() Line 252 C++
xul.dll!mozilla::dom::MediaDocumentStreamListener::OnStartRequest(nsIRequest * request) Line 57 C++
xul.dll!nsDocumentOpenInfo::OnStartRequest(nsIRequest * request) Line 311 C++
[...]

So we can probably just re-arrange things so we don't create the video element until slightly later, after the PresShell has already been setup, so we can tell whether the compositor supports hardware accelerated video decoding. Then we'd use hardware decoding, and shouldn't drop frames.

Currently when we create the video inside a VideoDocument, the PresShell isn't
created yet. This means the video element can't access information about the
compositor, which means it doesn't know whether it can create a hardware
accelerated video decoder, and so we end up falling back to using a software
decoder.

So this patch moves the creation of the video element to slightly later in the
load of a VideoDocument, so that the PresShell is available when we create the
VideoDocument's video element. This means VideoDocuments's video decoder can be
hardware accelerated

Pushed by cpearce@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/2d94f58711b3 Load video element in VideoDocument slightly later during load. r=bzbarsky

https://hg.mozilla.org/mozilla-central/rev/2d94f58711b3

Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox68: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
Flags: qe-verify+

Hi guys, I retested this issue in Firefox Beta 68.0b14 as well as our latest Nightly and the issue still occurs on Yoga devices, I'm using the video link from the description and I cant get it to pass the first frame.

STR:

Open a new tab to https://faraday.basschouten.com/mozilla/Pageload/Contentfullness/CNNOutput.mp4

Should we reopen this issue ?

Flags: needinfo?(cpearce)

(In reply to Rares Doghi from comment #18)

Should we reopen this issue ?

I don't think reopening this issue would be helpful.

The patches that landed in this bug fixed the underlying cause of us not playing the video in the STR on the Yoga ARM64 devices at the time; we were incorrectly not using the HW decoder when opening videos directly, and the SW decoder can't decode fast enough to keep up on these videos.

But the landing of this fix also coincided with us blacklisting the HW video decoder on the Yoga devices, and we've not had a driver update for these devices, so we remain stuck using the SW decoder on the Yoga ARM64 devices, which can't keep up.

If you set media.hardware-video-decoding.force-enabled=true, you should override the blacklist, and you should find the video plays on the ARM64 Yoga.

Flags: needinfo?(chris)

Hi, This issue is verfied as fixed in Fx 68, and Yes setting media.hardware-video-decoding.force-enabled to True in about:config, the video plays just fine. Thank you Chris. I will mark this issue accordingly.

Status: RESOLVED → VERIFIED
status-firefox68: fixed → verified
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: