Closed Bug 1141139 Opened 5 years ago Closed 4 years ago

Enable low latency decoding for MFTDecoder

Categories

(Core :: Audio/Video, defect)

x86_64
Windows 7
defect
Not set

Tracking

()

RESOLVED FIXED
mozilla42
Tracking Status
firefox42 --- fixed

People

(Reporter: schoenefeld, Assigned: jya)

References

Details

(Whiteboard: gfx-noted)

Attachments

(1 file)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.76 Safari/537.36

Steps to reproduce:

From testing the Media Source Extension in firefox trunk with:
media.mediasource.enabled;true
media.mediasource.mp4.enabled;true
media.mediasource.youtubeonly;false

Playing a live h.264/mp.4 video stream using Media Source Extension (MSE), latency on Win7 and Win8 is very high (>1s). This is a known behavior of the MFT/DXVA decoder used on Windows, where it always buffers frames for 1 second before returning the first decoded frame.


Actual results:

The high latency makes use of MSE with mp4 unusable for low latency use-cases like webchat, desktop remoting...


Expected results:

Using the CODECAPI_AVLowLatencyMode flag when initializing the decoder will buffer less (0 frames on Win8, 4-10 frames on Win7), making it usable for low latency live stream decoding. 

Note that while the AVLowLatencyMode documentation says that it will only work on >=Win8 ,the flag will also have a positive effect on Windows 7, as long as SP2 is installed.

For reference: The AVLowLatencyMode flag has also been recently added to Chrome Canary build (dxva_video_decoder_accelerator.cc:887)
Component: Untriaged → Video/Audio
Product: Firefox → Core
Yes, I saw the video IMF filter sometimes buffer in excess of 20 frames before returning anything
Whiteboard: gfx-noted
Just a note in reference to comment 0. This seems to be only valid on Windows 8 and Windows 7 SP 2 doesn't exist (http://windows.microsoft.com/en-US/windows/service-packs-download#sptabs=win7).

Maybe a typo and he meant SP 1?
I am able to set CODECAPI_AVDecNumWorkerThreads, which is also reported as Windows 8 only, on Windows 7 SP 1. When setting CODECAPI_AVLowLatencyMode, I actually get an S_OK return code if I add a SetUINT32 at [1]. Also, I see from Visual Studio Concurrency Analyzer that the decoder threads are getting scheduled in short frequent bursts versus before I would see long decodes with 50-100 ms between each decode. So it seems to work on Windows 7 SP 1.

[1] http://mxr.mozilla.org/mozilla-central/source/dom/media/platforms/wmf/WMFVideoMFTManager.cpp#206
Assignee: nobody → jyavenard
Blocks: 1191138
Comment on attachment 8644756 [details] [diff] [review]
Enable low latency decoding on Windows. r=cpearce

Will do thorough testing before pushing...
Attachment #8644756 - Flags: review?(cpearce)
appears to work very well, we pass all the tests now (with a couple of adjustments on the test: e.g. properly calling endOfStream())
Attachment #8644756 - Flags: review?(cpearce) → review+
https://hg.mozilla.org/mozilla-central/rev/ac350571e7b8
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla42
Unfortunately, it appears that win7 try machines only have SP1 installed ; as such we still fail the test for bug 1180935.

Back to the drawing board.
I went by the original description mentioning SP2.

So try machines don't have any SP installed then.

I verified that with SP1 it works as expected. The decoder output a frame after 10 Frames as opposed to 25+ without
See Also: → 1183132
I believe the crashes mentioned previously have actually been fixed elsewhere in Firefox somewhere along the line, Low Latency Decoding should be given another shot.
And it would be relying on an undefined behaviour of the decoder.
Per Microsoft documentation, it can't do content with B-frame.. and yet it does (and Chrome uses it)...
Chrome apparently doesn't play nice with B Frames as per https://bugs.chromium.org/p/chromium/issues/detail?id=703019

Surely the decoder could fall back in the presence of b frames to the standard decoder, or only use low latency for live streaming.

I've heard of a few sites refraining from switching to full html5 players because of the latency differences between various web browsers, which isn't helping to kill Flash faster.
Hmmmm, that bug is more about the stream having wrong pts set. You would see the same kind of jittery playback in Firefox on macos...
You need to log in before you can comment on or make changes to this bug.