1142455 - problems with playing h264 videos

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P1)

Description

[wolfgangpue]

Attachment: Network requests

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0
Build ID: 20150305021524

Steps to reproduce:

I converted a video with ffmpeg to mp4 (h264 , High Profile, Level 3.1, Bitate 1500), 1280x720, AAC). 

http://5.79.73.18/video/trailer.mp4

Video is on a nginx server but the problems are the same when I put it on an apache server. Other mp4 videos play well.




Actual results:

When I start the video or use the seeker, there are a lot of partial downloads in developer console and the video loads very long or forever.

There is no problem with other major browser (Chrome, Opera, IE).

When I remove the aac stream, the video plays well.


Expected results:

There should only be one request to the video, like in other browser, so that the video can resume playback and don't load forever.

Comment 1

[Nickolay_Ponomarev]

The link doesn't load, but there are similar bugs on file: bug 1142084, bug 1145608.

Comment 2

[wolfgangpue]

There is a new url to the video: http://5.79.72.89/trailer/trailer.mp4

I think the similar bugs show the same behavior: A lot of partial request and the video doesn't play properly (needs very very long or forever to play)

Comment 3

[wolfgangpue]

Are there h264 settings which will work for firefox? All settings which I tried have no success. I also used different encoding software (ffmpeg, libav, handbrake).

Comment 4

[deheleri]

I tried many settings too.

As long as there is an audio stream in the mp4, Firefox will send hundreds of 206 partial content requests and the video often freezes and takes forever to seek. Once you remove the audio stream, it plays and seeks fine.

Tested these audio codecs in ffmpeg, with constant AND variable settings, with no success:
libfdk_aac
libvo_aacenc
aac (native experimental)
libfaac
libmp3lame

It's a terrible bug. I will keep trying other things.

Comment 5

[deheleri]

I also tried 1 audio channel (-ac 1), crossing audio channels, libvorbis audio, and several VBR settings for AAC and MP3. None of those settings help, the video always has seek issues in Firefox. Removing audio stream always fixes it no matter what the video settings are. Sucks!

Comment 6

[deheleri]

Sorry for triple post. This is my first bug report.

But the issue is completely fixed in Firefox Nightly 41.0a1 (2015-06-08)

Can any dev link to the patch/fix? Maybe there is a temporary solution we can utilize server-side until nightly is rolled out?

Comment 7

[wolfgangpue]

Good to know. With the nightly 41.0a1 my video plays without problems. Is there an information about the fix anywhere?

Comment 8

[buggy]

Still present as of firefox 39.0. The solution to this issue is to set media.fragmented-mp4.exposed to false. Some idiot changed the default setting to true without actually testing the feature first. Naturally, this change was not mentioned any of the update release notes.

Comment 9

[ChT]

So, can we expect the next version to have media.fragmented-mp4.exposed set to false by default?

Comment 10

[ChT]

I was able to confirm this using this video:
http://media.w3.org/2010/05/bunny/movie.mp4

If you manage to load the video right from the start, just seek to a location that is not buffered yet. 

The net console is flooded with partial requests.

Manually setting the media.fragmented-mp4.exposed to false fixes the problem
(tested on Win 7 , Firefox 39 )

Comment 11

[Marcin]

Problem still exists in 41.0a2. With long RTT is makes video almost unseekable.

Comment 12

[rjain2]

(In reply to buggy from comment #8)
> Still present as of firefox 39.0. The solution to this issue is to set
> media.fragmented-mp4.exposed to false. Some idiot changed the default
> setting to true without actually testing the feature first. Naturally, this
> change was not mentioned any of the update release notes.

Not quite true, if you set the media.fragmented-mp4.exposed to false, it forces the quicktime plugin to be activated first rather than using native decoder. I am on Mac OSX Yosemite (10.10.4) with Firefox 39.0.3. So this workaround is not viable.

Comment 13

[wolfgangpue]

Does someone have settings for the h264 codec which will work with the current version of firefox? This bug drives me crazy. I wanna use a html5 player instead of flash. I use handbrake for encoding and I tried different settings but without success. Also with other encoders I wasn't able to encode a video which plays well in firefox.

Comment 14

[Jean-Yves Avenard [:jya]]

Make sure you use -movflags faststart when encoding. Ffmpeg default encoding put the moov atoms (metadata) right at the end of the video. 
This requires the video to be parsed fully before decoding starts. 

Placing the moov at the beginning will allow playback to start much quicker.

Comment 15

[Jean-Yves Avenard [:jya]]

IIRC, handbrake have a similar option: web optimised needs to be checked. 

This page has useful information
:
https://www.drupal.org/node/1565532

Comment 16

[Jean-Yves Avenard [:jya]]

(In reply to ChT from comment #10)
> I was able to confirm this using this video:
> http://media.w3.org/2010/05/bunny/movie.mp4
> 
> If you manage to load the video right from the start, just seek to a
> location that is not buffered yet. 
> 
> The net console is flooded with partial requests.
> 
> Manually setting the media.fragmented-mp4.exposed to false fixes the problem
> (tested on Win 7 , Firefox 39 )

This video has the metadata encoded at the end.

There's no other way to play that video other than parsing it in its entirety.
So it needs to find the metadata first by skipping over all the atoms.

When you set media.fragmented-mp4.exposed to false, the playback is now done by an external plugin, and the http request won't show on the console: they will still be there.

Comment 18

[wolfgangpue]

(In reply to Jean-Yves Avenard [:jya] from comment #14)
> Make sure you use -movflags faststart when encoding

I used this option for ffmpeg and for handbrake too. But this isn't the problem, because the video is starting fast. You can check my video file:
http://5.79.72.89/trailer/trailer.mp4
I tested the file with qtfaststart and it tells my that the atom is at the beginning. The problem occurs only when you use seeking. Then there are a lot of partial requests.

I believe the problem is not an encoder problem of firefox. Maybe there is a network issue. Look at my network protocol:
http://5.79.72.89/trailer/requests.jpg

Video filepath was directly copied to address bar. 

- Video starts playing (5 partial requests) after 1-2 seconds
- After 3-4 seconds I use seeking and click in the middle of the progress bar.
  All requests after the 5th requests are from my seeking (video loads forever, doesn't start again, I abort loading after 60 seconds)

This happens not every time, but 5 of 10 times for sure.

Other videos play well from this server (nginx with mp4 module) with firefox, except those I encoded myself.

Comment 19

[Jean-Yves Avenard [:jya]]

As I commented in the other bug, when you seek you have to retrieve the keyframe that is located before your seeking point.

You then have to decode all frames until you reach the original seek point.

If the data isn't entirely in the cache, the retrieving each frame may result in a single networking request.

We cache the data by 32kB block, if your compressed frame is over 32kB that requires more than one fetch.

So there's nothing wrong with needing multiple networking fetch when you're seeking.

If your keyframes are say 4s appart, and you're video is at 30Hz
then in the worse case, you may need to fetch + decode 119 frames (assuming you seek to the frame right before a keyframe).

Not that it takes over 60s is a problem ; I couldn't test your link earlier the server (or connection) was down

Comment 20

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

(In reply to wolfgangpue from comment #18)
> I used this option for ffmpeg and for handbrake too. But this isn't the
> problem, because the video is starting fast. You can check my video file:
> http://5.79.72.89/trailer/trailer.mp4
> I tested the file with qtfaststart and it tells my that the atom is at the
> beginning. The problem occurs only when you use seeking. Then there are a
> lot of partial requests.

In theory there is nothing wrong with a lot of little requests because we're requesting only the video parts of the data and skipping over the audio parts. However in practice the propagation delay may make it worse than being more aggressive about what we read.

You can try rolling your own build and tweaking some of the numbers. Making the block sizes and the total media cache size bigger would probably improve things. My instinct would be to simply make all the numbers 8 times bigger to compensate for improved Internet speeds, bigger videos and more memory since the code was initially written.

You probably need to tweak the block size http://mxr.mozilla.org/mozilla-central/source/dom/media/MediaCache.h#188 and possibly the about:config value of media.cache_size which is set http://mxr.mozilla.org/mozilla-central/source/modules/libpref/init/all.js#272

Comment 21

[wolfgangpue]

(In reply to Jean-Yves Avenard [:jya] from comment #19)
> As I commented in the other bug, when you seek you have to retrieve the
> keyframe that is located before your seeking point.

I don't think the problem is related with the number of keyframes. I use keyframes every 50 frames (25fps). I increased the number of keyframes but that did not help.

My Handbrake encoding command:
HandBrakeCLI --input big_buck_bunny_1080p_surround.avi --output big_bunny.mp4 --encoder x264 --x264-preset medium --x264-profile high --encopts ref=3:bframes=3:qpmin=0:keyint=50:keyint_min=5:8x8dct=1:vbv_bufsize=3000:vbv_maxrate=1500 --vb 1500 --maxWidth 1280 --format mp4 --rate 25 --optimize --aencoder faac --audio 1 --ab 64 --mixdown stereo --arate 48



I believe there is a problem with the forecast, which byte range firefox needs to request.

I made a little expirement:

My video(http://5.79.72.89/trailer/trailer.mp4) has 596 seconds and has a size of 117157246 bytes. (constant rate)
When I seek to second 131 after the video starts playing the correct byte range would be 25751005 - 117157246

Google Chrome request following byte range: 24712886 - 117157246 (very close) and starts playing again immediately. There is no additional request.

Firefox 40.0.2 request 8716288 - 117157246 (far away) and needs 36 requets to come to 27557888 - 117157246 (see screenshot: http://5.79.72.89/trailer/firefox.png)
After Firefox reached the correct byte range it displays me a frame from second 39 and starts playing audio only. The video is still frozen.

Comment 22

[Jean-Yves Avenard [:jya]]

I can't download that video to test it, and check where the keyframe before 131s is located.
Place it somewhere it can be accessed thanks

Comment 23

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Robert - what was the value in MediaCache that you said could be tuned for this?

Comment 24

[wolfgangpue]

(In reply to Jean-Yves Avenard [:jya] from comment #22)
> I can't download that video to test it

Here is another link: http://www.puerstner.net/trailer_problem.mp4

I tested this file again: Firefox 40.0.2 (problem still exists)

Firefox Nightly 43.0a1 (2015-08-24): Frozen problem is gone, but often after seeking the video starts playing but with shuttering for 5-6 seconds and in the network interface there are a lot of small requests.

Comment 25

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #23)
> Robert - what was the value in MediaCache that you said could be tuned for
> this?

SEEK_VS_READ_THRESHOLD, currently set to 32K.

We should probably increase this a lot. If we assume a 100Mbit connection, and assume reissuing an HTTP seek causes a delay of 200ms, then in that 200ms we could have simply read ahead 2MB. So increasing SEEK_VS_READ_THRESHOLD to 1MB sounds reasonable.

Comment 26

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Tweak MediaCache parameters

Comment 27

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 8653828 [details] [diff] [review]
Tweak MediaCache parameters

Review of attachment 8653828 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with the BLOCK_SIZE change removed and the requested comment.

::: dom/media/MediaCache.h
@@ +184,5 @@
>  class MediaCacheStream {
>  public:
>    enum {
>      // This needs to be a power of two
> +    BLOCK_SIZE = 128 * 1024

Put this in a separate patch if we need to do this --- and explain why!

::: dom/media/MediaResource.h
@@ +24,5 @@
>  
>  // For HTTP seeking, if number of bytes needing to be
>  // seeked forward is less than this value then a read is
>  // done rather than a byte range request.
> +static const int64_t SEEK_VS_READ_THRESHOLD = 1*1024*1024;

Please add a comment similar to mine explaining why we're using this value.

Comment 28

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Tweak MediaCache parameters

Comment 29

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Tweak MediaCache parameters

Carrying r+

Comment 30

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ed4810142476

Comment 31

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/db8aa4fdcba48cc0a9bf0e7a4ea6ebad9c6459f9
Bug 1142455 - Tweak MediaCache parameters; r=roc

Comment 32

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/db8aa4fdcba4

Comment 33

[Ryan VanderMeulen [:RyanVM]]

Backed out for making bug 1179547 nearly permafail on OSX 10.6.
https://hg.mozilla.org/integration/mozilla-inbound/rev/fef376f03884

Comment 34

[Ryan VanderMeulen [:RyanVM]]

Merge of the backout:
https://hg.mozilla.org/mozilla-central/rev/fef376f03884

Comment 35

[Chris Pearce [:cpearce (Not reading bugmail)]]

Putting on my cowboy hat and shotgunning different values of this parameter:

SEEK_VS_READ_THRESHOLD 512KB:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f67f80e291ae

SEEK_VS_READ_THRESHOLD 256KB:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0ec55420ff0a

SEEK_VS_READ_THRESHOLD 128KB:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=41ad39639968

SEEK_VS_READ_THRESHOLD 64KB:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f61c2178c75e

Comment 36

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

An update: Tried on the cowboy hat, and it's getting confusing.

64KB, timeouts in 2 'test_fragment_play' mochitests out of 7 on 10.6 debug, similar to results with 64KB-512KB in tries from comment 35:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7ffd10d59b28

32KB, no timeouts:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=971d2daf6877

48KB, no timeouts:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=277dfadb4eb4

1MB with no parallel tests (was 2 by default) and with NSPR logging, timeout in 1 'test_streams_gc' mochitest (first time I see that failure, not sure if it's significant) out of 8:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f5da96d980c8

1MB with parallel tests and with NSPR logging, no timeouts in 6 tests!
https://treeherder.mozilla.org/#/jobs?repo=try&revision=184d0cbd505b
(ship it?!)
It's either a fluke (e.g., a better machine in the try arena), or logging is influencing timings in some strange and beneficial ways.

I'm currently setting up a Mac OS X 10.6 VM, to try and reproduce these tests locally.

Comment 37

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

JW let me know that bug 1179547 fixed this intermittent failure on 9 Sep, which explains why Chris' tests before that failed (comment 35) and mine after the fix worked (comment 36).

So it was probably just bad luck that this patch here was caught in the storm, and it should be safe to push again.

Recent try showing no intermittent timeouts:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=259fc625dbdb
(The visible failures are from pre-existing intermittent bug 1072313)

Comment 38

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/606e12ccc17c

Comment 39

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/606e12ccc17c

Comment 40

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Comment on attachment 8653833 [details] [diff] [review]
Tweak MediaCache parameters

Approval Request Comment
[Feature/regressing bug #]: facebook video playback
[User impact if declined]: some videos are horrible to play
[Describe test coverage new/current, TreeHerder]: been in nightly/aurora over a month
[Risks and why]: may affect seeking performance on super slow connections that aren't really suited to video anyway; pretty low risk of breaking things that aren't already terrible
[String/UUID change made/needed]: none

Comment 41

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8653833 [details] [diff] [review]
Tweak MediaCache parameters

Sure, taking it. Should be in 42 beta 5.

Comment 42

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-beta/rev/37f08780dae6