656040 - Some WebM videos are viewed with the wrong aspect ratio

Status: RESOLVED INCOMPLETE

(Core :: Audio/Video, defect)

Description

[Laurent Pointecouteau]

User-Agent:       Mozilla/5.0 (X11; Linux i686; rv:2.0.1) Gecko/20110502 Firefox/4.0.1
Build Identifier: Mozilla/5.0 (X11; Linux i686; rv:2.0.1) Gecko/20110502 Firefox/4.0.1

The example link showcases a WebM video (encoded by me) in 4:3 instead of widescreen. The video plays correctly when downloaded and opened with VLC or mplayer

Reproducible: Always

Steps to Reproduce:
1. Open http://gazooze.com/productions/?prod=42
2. Click on the Play button

Actual Results:  
The video shows up with deformed height

Expected Results:  
The video plays with actual file height

I've used XMedia Recode to crop and convert these videos. They were originally in *mp4 with black borders at the top and bottom.

Comment 1

[Alice0775 White]

Confirmed on
http://hg.mozilla.org/mozilla-central/rev/e0f6db50231f
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0a1) Gecko/20110510 Firefox/6.0a1 ID:20110510030615

Comment 2

[Alice0775 White]

Opera 11.10: It is played back with correct aspect ratio.
Google Chrome 11.0.696.65 ; Wrong aspect ratio, It becomes flat.

Comment 3

[Alice0775 White]

Regression window(cached m-c hourly):
Works:
http://hg.mozilla.org/mozilla-central/rev/733c4bc77ab5
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b11pre) Gecko/20110127 Firefox/4.0b11pre ID:20110127221516
Fails:
http://hg.mozilla.org/mozilla-central/rev/0e50480c408f
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b11pre) Gecko/20110127 Firefox/4.0b11pre ID:20110127235726
Pushlog:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=733c4bc77ab5&tochange=0e50480c408f
Suspected Bug:
408332b0a834	Chris Pearce — Bug 626979 - Handle WebM frame size changes. r=kinetik a=blocking2.0

Comment 4

[Chris Pearce [:cpearce (Not reading bugmail)]]

That WebM file has the following Track element for its video track:

Video (head size: 9 bytes, data: 22 bytes, pos: 360L, '0x168L')
        Pixel Width (head size: 2 bytes, data: 2 bytes, pos: 425L, '0x1a9L') : 720
        Pixel Height (head size: 2 bytes, data: 2 bytes, pos: 429L, '0x1adL') : 304
        Display Width (head size: 3 bytes, data: 2 bytes, pos: 433L, '0x1b1L') : 541
        Display Height (head size: 3 bytes, data: 2 bytes, pos: 438L, '0x1b6L') : 304
        Display Unit (head size: 3 bytes, data: 1 bytes, pos: 443L, '0x1bbL') : 3

According to http://www.matroska.org/technical/specs/index.html#DisplayUnit and http://lists.matroska.org/pipermail/matroska-devel/2010-July/003704.html a DisplayUnit of 3 means that the DisplayWidth and DisplayHeight should be use to calculate the aspect ratio, but not actually used as the size to scale the picture to. Hopefully I'm reading those correctly.

So in nsWebMReader::ReadMetadata() we should only set mInfo.mDisplay to displaySize if the video track either doesn't have a DisplayUnit element, or if it has a DisplayUnit element with a value of 0. Otherwise it should be (params.width,params.height).

Matthew: do you think that sounds correct?

Comment 5

[Matthew Gregan [:kinetik]]

Yeah, nestegg needs to check DisplayUnit before handing the display width/height out.  It's a simple fix, but this might be a good time to audit nestegg for other cases where it's assuming default values.  Shall I take this bug, or are you happy to fix this?

Comment 6

[Chris Pearce [:cpearce (Not reading bugmail)]]

I've got a patch which fixes this, so may as well just file a follow up to audit the other defaults so that fixing this isn't blocked.

Comment 7

[Chris Pearce [:cpearce (Not reading bugmail)]]

Oops, I just noticed in the "Muxer Guidelines" http://www.webmproject.org/code/specs/container/#muxer_guidelines that "WebM files must only support pixels for the DisplayUnit element".

That file has a DisplayUnit of 3 (for aspect ratio, not pixels). So that file isn't actually a valid WebM file.

I'll email the author of XMedia Recode and alert him that it's outputting non-conformant files.

Comment 8

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Please also contact Chrome and Opera and tell them they need to be rejecting this invalid files.

Comment 9

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(And of course, if they're not willing to change, then we need to get the WebM spec changed to allow such files.)

Comment 10

[Matthew Gregan [:kinetik]]

Reopening, because we need to either reject or handle this in some way.  Our current behaviour (assuming DisplayUnit is pixels) is not correct.

Comment 11

[Chris Pearce [:cpearce (Not reading bugmail)]]

Some comparison with other players...

Opera render the video at about 720x406 - looks correct aspec ratio, I guess they/GStreamer handle DisplayUnit.
VLC 720x410 - looks correct aspect ratio.
Firefox trunk - 720x540 - obviously incorrect aspect ratio
Chrome and IE render at 720x304 - the "picture size", obviously incorrect.

It's fairly trivial to support a DisplayUnit of "aspect ratio" (as it appears Opera and VLC are doing). Supporting a DisplayUnit of "cm" and "inches" (the other non pixel values) doesn't sound fun though.

Comment 12

[Robert O'Callahan (:roc) (email my personal email if necessary)]

You could treat cm and inches the way CSS does, which would simply mean one inch maps to 96px, one cm = 96/2.54px. That is probably the right thing to do for the Web, and should be very easy to implement.

So it's really a question of whether we want the spec to change here or not.

Comment 13

[Mathieu Pellerin]

I'm also seeing this issue with webm-converted videos. 

Firefox renders a 320x240px webm video into a 567x240px frame. Using the same source video, the theora version will render just fine. VLC plays the webm video with the correct ratio.

Hope a quick fix to this problem can be found in time for firefox 6.0 (assuming it's way too late to make it in 5.0), non-ratio videos really affects the html5 video experience.

I can provide problematic videos if needed.

Comment 14

[Mathieu Pellerin]

BTW, webm video encoded using ffmpeg using a web front end. I don't think the right behavior is to reject these files, feels to me like you'd invalidate a bunch of content already out there and be (sometimes) incompatible with well-known encoders ala ffmpeg.

Comment 15

[Mathieu Pellerin]

Attachment: webm video radio, vlc vs firefox

Comment 16

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Whatever we do here, you should switch to an encoder that produces valid Webm output.

Comment 17

[Mathieu Pellerin]

roc, I`m open to suggestions here. I am simply not aware of any cross-platform command line video converter other than ffmpeg. Do you know of such encoder? (BTW, I`ll forward this thread to the ffmpeg developers in case they want to fix this problem on their side too.)

That being said, it seems producing a practical video decoder that`ll be used by hundreds of thousands of viewers in 2011 is all about fault tolerance :)

Comment 18

[Mathieu Pellerin]

BTW, for the record, Miro Video Converter also result in wrong ratio webm videos when played with firefox.

Comment 19

[Timothy B. Terriberry (:derf)]

(In reply to comment #13)
> I'm also seeing this issue with webm-converted videos. 
> 
> Firefox renders a 320x240px webm video into a 567x240px frame. Using the
> same source video, the theora version will render just fine. VLC plays the
> webm video with the correct ratio.

This sounds more like bug 661456 (using the exact same dimensions as the test case there, even). That bug is all ours, as the files there are valid. Check the DisplayUnit FFmpeg is producing in your setup (with, e.g., mkvinfo) and make sure you're complaining about the right bug. FFmpeg may very well be doing the correct thing.

Comment 20

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(In reply to comment #17)
> That being said, it seems producing a practical video decoder that`ll be
> used by hundreds of thousands of viewers in 2011 is all about fault
> tolerance :)

We're trying not to slide right down to the bottom of the slippery slope right away :-).

Comment 21

[Mathieu Pellerin]

Timothy was right. Since bug 661456 was applied to the trunk, the ratio of my ffmpeg encoded videos are back to normal. 

Apologies for what now appears to be unrelated noise from my part into this bug :)

Comment 22

[u279076]

Is this bug WORKSFORME? FIXED? DUPE of bug 661456?

Comment 23

[(mostly gone) XtC4UaLL [:xtc4uall]]

Still relevant?

Comment 24

[u279076]

Resolving this as INCOMPLETE since this bug has been inactive for a couple years. Please file a new bug report if you have issues with aspect ratio on WebM videos in the latest Firefox version.