Closed Bug 1141139 Opened 6 years ago Closed 6 years ago
Enable low latency decoding for MFTDecoder
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.76 Safari/537.36 Steps to reproduce: From testing the Media Source Extension in firefox trunk with: media.mediasource.enabled;true media.mediasource.mp4.enabled;true media.mediasource.youtubeonly;false Playing a live h.264/mp.4 video stream using Media Source Extension (MSE), latency on Win7 and Win8 is very high (>1s). This is a known behavior of the MFT/DXVA decoder used on Windows, where it always buffers frames for 1 second before returning the first decoded frame. Actual results: The high latency makes use of MSE with mp4 unusable for low latency use-cases like webchat, desktop remoting... Expected results: Using the CODECAPI_AVLowLatencyMode flag when initializing the decoder will buffer less (0 frames on Win8, 4-10 frames on Win7), making it usable for low latency live stream decoding. Note that while the AVLowLatencyMode documentation says that it will only work on >=Win8 ,the flag will also have a positive effect on Windows 7, as long as SP2 is installed. For reference: The AVLowLatencyMode flag has also been recently added to Chrome Canary build (dxva_video_decoder_accelerator.cc:887)
Yes, I saw the video IMF filter sometimes buffer in excess of 20 frames before returning anything
Just a note in reference to comment 0. This seems to be only valid on Windows 8 and Windows 7 SP 2 doesn't exist (http://windows.microsoft.com/en-US/windows/service-packs-download#sptabs=win7). Maybe a typo and he meant SP 1?
I am able to set CODECAPI_AVDecNumWorkerThreads, which is also reported as Windows 8 only, on Windows 7 SP 1. When setting CODECAPI_AVLowLatencyMode, I actually get an S_OK return code if I add a SetUINT32 at . Also, I see from Visual Studio Concurrency Analyzer that the decoder threads are getting scheduled in short frequent bursts versus before I would see long decodes with 50-100 ms between each decode. So it seems to work on Windows 7 SP 1.  http://mxr.mozilla.org/mozilla-central/source/dom/media/platforms/wmf/WMFVideoMFTManager.cpp#206
Also, Chromium just recently enabled this - https://groups.google.com/a/chromium.org/forum/#!topic/feature-media-reviews/3fFal9GVhRg
Comment on attachment 8644756 [details] [diff] [review] Enable low latency decoding on Windows. r=cpearce Will do thorough testing before pushing...
Attachment #8644756 - Flags: review?(cpearce)
appears to work very well, we pass all the tests now (with a couple of adjustments on the test: e.g. properly calling endOfStream())
Attachment #8644756 - Flags: review?(cpearce) → review+
6 years ago
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Unfortunately, it appears that win7 try machines only have SP1 installed ; as such we still fail the test for bug 1180935. Back to the drawing board.
SP1 is the latest SP for Win7.
I went by the original description mentioning SP2. So try machines don't have any SP installed then. I verified that with SP1 it works as expected. The decoder output a frame after 10 Frames as opposed to 25+ without
We've had a surge of crashes in msmpeg2vdec.dll since this landed. https://crash-stats.mozilla.com/search/?product=Firefox&signature=~msmpeg2vdec&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-signature So in bug 1205083 I'm proposing we remove enabling low latency video decoding.
I believe the crashes mentioned previously have actually been fixed elsewhere in Firefox somewhere along the line, Low Latency Decoding should be given another shot.
And it would be relying on an undefined behaviour of the decoder. Per Microsoft documentation, it can't do content with B-frame.. and yet it does (and Chrome uses it)...
Chrome apparently doesn't play nice with B Frames as per https://bugs.chromium.org/p/chromium/issues/detail?id=703019 Surely the decoder could fall back in the presence of b frames to the standard decoder, or only use low latency for live streaming. I've heard of a few sites refraining from switching to full html5 players because of the latency differences between various web browsers, which isn't helping to kill Flash faster.
Hmmmm, that bug is more about the stream having wrong pts set. You would see the same kind of jittery playback in Firefox on macos...
You need to log in before you can comment on or make changes to this bug.