1283724 - HTML 5 video hangs in all versions after 44.0.2

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P3)

Description

[matscol]

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0
Build ID: 20160210153822

Steps to reproduce:

Visit sample page test.thispieisnuts.com/video4.aspx?pid=24795


Actual results:

Video hangs in FF (win 8.1) version 45 and all subsequent versions.


Expected results:

Video should run smoothly as it does in FF (win 8.1) version 44.0.2 and all prior versions.

Comment 1

[Loic]

WFM with FF47. Did you test with a fresh profile?
https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles

Comment 2

[matscol]

On Win 8.1 box:
Installed 47.0.1, created new default profile. Video hangs and stops after ~10 seconds. Clicking on different name on the right side of page jumps video to new position (properly) but video hangs almost immediately.

Re-installed 44.0.2, normal behavior resumed.

***NOTE: Video plays properly on OSX yosemite 10.10.5 / FF 46.0.1

Comment 3

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

(In reply to matscol from comment #0)
> Video hangs in FF (win 8.1) version 45 and all subsequent versions.

I can't reproduce the issue. Can you describe in more detail what you mean when you say "hangs"?

Comment 4

[matscol]

Thx for checking on it. Over the weekend I took an unused win 8.1 computer and installed all windows updates, then upgraded to FF 47.0.1. It appeared things were better but after ~30 seconds the same behavior appeared: the audio continues to run normally but the video either freezes for 1-2 seconds then resumes, or the video stops altogether. It can't be just connectivity because this never happens with FF 44.0.2. It first began with FF 45 and has continued with 46 and 47. It occurs on a separate Win 8.1 box here in the same way. Things I'd appreciate if you tried: 1. Let the video run for a while and see if it freezes; 2. Click other names on the left side of the screen and see if the video freezes after jumping to a new position. If you still can't produce the error I will wipe a HD and do a fresh win8.1 install with nothing else on the box to see if that makes a difference.

Comment 5

[matscol]

Err, RIGHT side, not left

Comment 6

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

That sounds like skip to next keyframe logic. This happens when your computer isn't keeping up. Can you paste the graphics section of about:support into this bug?

Comment 7

[matscol]

Sorry, was on the road, couldn't follow up for a while. Here it is for the box running 44.0.2 with no problems:

Adapter Description: NVIDIA GeForce 6200 TurboCache(TM)
Adapter Drivers: nvd3dumx,nvd3dum
Adapter RAM: 64
Asynchronous Pan/Zoom: none
Device ID: 0x0161
DirectWrite Enabled: false (6.3.9431.0)
Driver Date: 12-7-2012
Driver Version: 9.18.13.768
GPU #2 Active: false
GPU Accelerated Windows: 0/1 Basic (OMTC) Blocked for your graphics card because of unresolved driver issues.
Subsys ID: 02711462
Supports Hardware H264 Decoding: No; Hardware video decoding disabled or blacklisted
Vendor ID: 0x10de
WebGL Renderer: Google Inc. -- ANGLE (NVIDIA GeForce 6200 TurboCache(TM) Direct3D9Ex vs_3_0 ps_3_0)
windowLayerManagerRemote: true
AzureCanvasBackend: skia
AzureContentBackend: cairo
AzureFallbackCanvasBackend: cairo
AzureSkiaAccelerated: 0

Comment 8

[Loic]

Could you install the tool mozregression to narrow down a regression range in FF45.
See http://mozilla.github.io/mozregression/ for the FAQ.
Run the command "mozregression --good=44" (or 43) then copy here the final pushlog provided by the console.

It would help to find a potential regression, maybe related to your hardware.

Comment 9

[matscol]

Will do, though I really don't think it's hardware, as the problem is reproducible here on a newer box in the same way (I'll post that box's about:support graphics section as well.

Comment 10

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Updating your driver will help but it should also be better in Firefox 48.

Comment 11

[Justin [:JW_SoftvisionQA]]

I also can not reproduce on  	Mozilla/5.0 (Windows NT 6.3; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0

Comment 12

[matscol]

34:00.30 INFO: Last good revision: ada1afb12a16333d26974328d0f340712f354bf2
34:00.30 INFO: First bad revision: 7fbfb74b3dd32634e4cdc314ab9f48eaeaeada6a
34:00.30 INFO: Pushlog:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=ada1af
b12a16333d26974328d0f340712f354bf2&tochange=7fbfb74b3dd32634e4cdc314ab9f48eaeaea
da6a

34:00.68 INFO: Looks like the following bug has the changes which introduced the
 regression:
https://bugzilla.mozilla.org/show_bug.cgi?id=1223599
-----------------------------------------------------------------------------------------------------

Noted behavior: in the more recent versions (50, 46) that mozregression had me test, the video would play for a little while before beginning to catch (freeze for a fraction of a section then restart). In the versions right at the time it first went bad (Nov12-16 2015) the video was DOA, it wouldn't play at all. Sometimes a second try would get it to run for a second or two before freezing completely.

Comment 13

[Justin [:JW_SoftvisionQA]]

JW it looks like you fixed 1223599. Can you lend some assistance?

Comment 14

[JW Wang [:jwwang] [:jw_wang]]

Can't repro the issue on my local Win7.

What is your network speed?
Can you repro the issue on some other network like office network or public WIFI?

Comment 15

[matscol]

Tested just now at 26.3Mbps down, I don't see how that could be the problem, especially as it works so consistently in all versions prior to 45. I am baffled that no one can reproduce and concerned the problem is on my side but can't imagine what it could be. Is it possible it's a Win8.1 issue? (I find that hard to believe too, but willing to entertain any ideas). 

Have you let the video run for more than a few seconds? Have you clicked another name on the right side of the page? I just retested with mozregression (1st step only to make sure I'm not crazy): build 20151029045227 was perfect, I clicked various names, I let it run, I used the <:05 button, it all worked exactly as it should; build 20160731030203 ran for 14-15 seconds, stuttered then stopped. Clicking another name didn't jump the video to a new point as it should, the video sat frozen (after several seconds the video did jump to the new time but it stalled again almost immediately).

Comment 16

[JW Wang [:jwwang] [:jw_wang]]

Bug 1223599 removes the throttling of buffer ranges calculation which means:
For each 1MB data to download, |packet size=32KB| takes as twice CPU time to calculate buffer ranges as |packet size=64KB| does. So it should have something to do with your network configuration.

I will somewhat revert the changes in bug 1223599 and have a test build for you.

Btw, can you monitor the CPU usage while playback goes on and gets stuck?

Comment 17

[matscol]

cpu ran at between 45-50% during the entire time I ran it on build 20151029045227 with no playback issues.

cpu ran at between 35-50% with build 20160731030203. It was more erratic, but spent most of the time in the 35-40% range. The spikes did not seem timed to the moments the video froze or restarted, it was pretty random.

To answer earlier question, both boxes with this issue are desktops, so hard to move to another network. I will try tomorrow to reproduce on a laptop and if I can I will then try on a public network.

Comment 18

[JW Wang [:jwwang] [:jw_wang]]

It is interesting to observe that firefox (playing http://test.thispieisnuts.com/video4.aspx?pid=24795) creates a "Web Content" process which takes up to 60% CPU on my Linux box.

Comment 19

[JW Wang [:jwwang] [:jw_wang]]

Forget about comment 18. "Web Content" is the content process in e10s mode.

Comment 20

[JW Wang [:jwwang] [:jw_wang]]

Can you try if this build works for you?
http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-e5c59b20ae2a7b94211bd4c0931bd373e45296d1/try-win64/firefox-50.0a1.en-US.win64.zip

Comment 21

[matscol]

That works perfectly.

Comment 22

[JW Wang [:jwwang] [:jw_wang]]

Thanks for testing. I will submit a patch for review soon.

Comment 23

[matscol]

Thanks for everyone's help here. Can't say enough how much I appreciate the time you all took with it.

Comment 24

[JW Wang [:jwwang] [:jw_wang]]

Attachment: Bug 1283724 - throttle notifications from stochastic network activity to save computation of buffer ranges.

Review commit: https://reviewboard.mozilla.org/r/68814/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/68814/

Comment 25

[Jean-Yves Avenard [:jya]]

Comment on attachment 8777218 [details]
Bug 1283724 - throttle notifications from stochastic network activity to save computation of buffer ranges.

https://reviewboard.mozilla.org/r/68814/#review65892

::: dom/media/MediaDecoder.cpp:235
(Diff revision 1)
> +}
> +
>  void
>  MediaDecoder::ResourceCallback::NotifyDataArrived()
>  {
> +  // Throttle notifications to be at most once per 500ms.

could you define this in the header, so it's easer to find.

Additionally, some documentation on why we are throttling things.

BTW, the only reason we need to throttle is because we read by small block of 32kB.

I believe we should increase that instead.

Comment 26

[JW Wang [:jwwang] [:jw_wang]]

http://www.richard-slater.co.uk/archives/2009/10/23/change-your-mtu-under-vista-windows-7-or-windows-8/

Can you check the MTU value on your windows machine?

Comment 27

[JW Wang [:jwwang] [:jw_wang]]

Comment on attachment 8777218 [details]
Bug 1283724 - throttle notifications from stochastic network activity to save computation of buffer ranges.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68814/diff/1-2/

Comment 28

[matscol]

MTU value is set at 1500. I can't follow through with the rest of his instructions on that page as there aren't any sites I can think of that I've ever completely not been able to connect to.

Comment 29

[JW Wang [:jwwang] [:jw_wang]]

1500 is the default value which is fine.

Have you tried to connect your computer to some other network and see how the playback goes?

Comment 30

[matscol]

Tried today with a Win7/32bit laptop. On the local network FF44 was fine, FF48 showed same behavior as 2 Win8 desktops described above. Took the laptop to an outside wifi network, there was some lagging in both FF44 and FF48, more in 48, but impossible for me to tell what was just slow network trying to catch up. It did *feel* different from the behavior I've seen. I will try again tomorrow on a better external network and update.

If it is a local network issue, what would I look for? This network is as vanilla as it gets, nothing customized at all.

Comment 31

[JW Wang [:jwwang] [:jw_wang]]

The patch fixes the problem. However, we still want to know what computer/network combinations are more likely to encounter this problem for there might be an issue in our network stack.

Thanks for your testing and keep us updated when you have more test results.

Comment 32

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/31c69994270f
throttle notifications from stochastic network activity to save computation of buffer ranges. r=jya

Comment 33

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/31c69994270f

Comment 34

[matscol]

Win7 laptop with FF48 on a separate public network showed the same behavior. 

Also tried FF47 on Linux mint 18 on the original network. It ran OK for more than 2 minutes before starting to do the same stop-and-start that I see on the Win8 boxes. 

Just in case any of that info is helpful.

Comment 35

[Liz Henry (:lizzard) (relman/hg->git project)]

Does this need more verification?  Do we think it is a common problem and does this need uplift, or is it ok for the fix to ride with 51?

Comment 36

[Jean-Yves Avenard [:jya]]

I think this should be uplifted.

Comment 37

[JW Wang [:jwwang] [:jw_wang]]

I can't tell how frequent this issue is nor a particular way to repro/verify this problem according to  matscol's description. However, I think it is low risk to uplift this bug to Aurora and Beta. Will request uplifts.

Comment 38

[JW Wang [:jwwang] [:jw_wang]]

Comment on attachment 8777218 [details]
Bug 1283724 - throttle notifications from stochastic network activity to save computation of buffer ranges.

Approval Request Comment
[Feature/regressing bug #]:1223599
[User impact if declined]:playback might stutter under some specific network configuration/type.
[Describe test coverage new/current, TreeHerder]:TreeHerder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0f0f7d7d3be7
https://treeherder.mozilla.org/#/jobs?repo=try&revision=35cafee27d8f
[Risks and why]: Low. The change is simple.
[String/UUID change made/needed]:none

Comment 39

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Hello, could you please verify this issue is fixed as expected on a latest Nightly build? Thanks!

Comment 40

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 8777218 [details]
Bug 1283724 - throttle notifications from stochastic network activity to save computation of buffer ranges.

Regression in video playback in some scenarios since 45, has stabilized on Nightly51 for ~10 days, let's uplift to Aurora50.

Comment 41

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/523a2b258f62

Comment 42

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8777218 [details]
Bug 1283724 - throttle notifications from stochastic network activity to save computation of buffer ranges.

Fix for playback stutter, let's uplift this for beta 6.

Comment 43

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/d77542fadfe6