Webm video with mixed video dimensions is not displayed correctly with hardware acceleration

RESOLVED FIXED in Firefox 65

Status

()

defect
P5
normal
RESOLVED FIXED
9 months ago
a month ago

People

(Reporter: holger, Assigned: jya)

Tracking

61 Branch
mozilla65
x86
Windows 10
Points:
---

Firefox Tracking Flags

(firefox65 fixed)

Details

Attachments

(3 attachments)

(Reporter)

Description

9 months ago
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.84 Safari/537.36

Steps to reproduce:

Playback of the test video below with mixed video sizes (webcam: 640x360, video 960x540, screenshare 1904x1088) with hardware acceleration.

Test video:
https://cdn.ubivent.com/web/test/firefox/videotest.html






Actual results:

The video is displayed in the maximum dimensions found. Wenn a smaller segment is played, the excess area is green. 

Screenshots:
https://cdn.ubivent.com/web/test/firefox/firefox_video_hardware_acceleration_largest_segment.jpg
https://cdn.ubivent.com/web/test/firefox/firefox_video_hardware_acceleration_small_segment.jpg
https://cdn.ubivent.com/web/test/firefox/firefox_video_hardware_acceleration_smallest_segment.jpg

Firefox Nightly and Beta have same results.


Expected results:

The video element should have adjusted its size to the current video segment and there should be no green area around the video.

If hardware acceleration is turned off, the issue does not occur. 

Screenshot without acceleration:
https://cdn.ubivent.com/web/test/firefox/firefox_video_smallest_segment.jpg

Chrome with hardware acceleration works as expected. 

User Agent:
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0

Hardware information:
GPU #1
Active: Yes
Description: Intel(R) HD Graphics 620
Vendor ID: 0x8086
Device ID: 0x5916
Driver Version: 23.20.16.4973
Driver Date: 2-28-2018
Drivers: igdumdim64 igd10iumd64 igd10iumd64 igd12umd64 igdumdim32 igd10iumd32 igd10iumd32 igd12umd32
Subsys ID: 224517aa
RAM: Unknown

More information in about:support attachment.
(Reporter)

Comment 1

9 months ago
I changed the original test Video, as the audio was broken. The screenshots are from the original video.
(Reporter)

Updated

9 months ago
Component: Untriaged → Audio/Video
OS: Unspecified → Windows 10
Product: Firefox → Core
Hardware: Unspecified → x86

Comment 2

9 months ago
i am having the same issue. 

the provided test video shows this:
https://www.seuffert.biz/tmp/ff-webm-fail.jpg

GPU #1
Active	Yes
Description	Intel(R) UHD Graphics 620
Vendor ID	0x8086
Device ID	0x5917
Driver Version	24.20.100.6194
Driver Date	6-20-2018
Drivers	igdumdim64 igd10iumd64 igd10iumd64 igd12umd64 igdumdim32 igd10iumd32 igd10iumd32 igd12umd32
Subsys ID	225b17aa
RAM	Unknown

Firefox 62.0b15
Windows_NT 10.0

The issue does not occur If i switch to the second GPU (NVIDIA GeForce MX150).
(Assignee)

Comment 3

9 months ago
technically, those webm are invalid...
(Assignee)

Comment 4

9 months ago
We need something similar to the H264Converter for VP9/VP8, also required for webrtc....
(Assignee)

Updated

9 months ago
Assignee: nobody → jyavenard
(Reporter)

Comment 5

9 months ago
The source of the webm file is a MediaSource stream where the segments are merged together into a webm file. The same issue also appears in the original MediaSource stream. The idea was that a single webm is easier to test than a MediaSource stream.
(Assignee)

Comment 6

9 months ago
That doesn't make for a valid webm...

If you want to use MediaSource and adding a new webm segment containing a different resolution then you *must* insert an init segment first (https://w3c.github.io/media-source/webm-byte-stream-format.html#webm-init-segments)

If getting the situation you describe with MSE, you must not be properly adding the init segment. Which would still be an invalid use of MSE.

Comment 7

9 months ago
another observation: the issue only happens if the video starts with larger dimensions and changes to smaller dimensions.
Priority: -- → P3
Component: Audio/Video → Audio/Video: Playback
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
(Assignee)

Comment 8

6 months ago
Bug 1497951 will allow such broken streams to play
See Also: → 1497951
(Assignee)

Updated

6 months ago
Priority: P3 → P5
(Assignee)

Updated

6 months ago
Status: ASSIGNED → RESOLVED
Last Resolved: 6 months ago
Resolution: --- → INVALID
(Assignee)

Comment 9

6 months ago
So bug 1497951 doesn't solve this particular problem seems to do with the Intel VP8 decoder itself as it returns a frame smaller than what the VP8 bytestream indicate.

Would you have the segment that is 960x540 on its own by any chance?
Status: RESOLVED → REOPENED
Flags: needinfo?(holger)
Resolution: INVALID → ---
(Reporter)

Comment 10

6 months ago
I have included all the segments of the demo video. The [n]-ue.webm are the original files, 0-ue.webm being the init segment. The [n].webm files have the same initial init segment prepended.

The issue itself is apparent in segments 5 and 11.

Segments 1 to 4 are 960x540
Semgments 6 to 7 are  640x360
Segments 8 to 10 are 960x540
Semgments 12 to 13 are  640x360

https://cdn.ubivent.com/web/test/firefox/segments.zip
Flags: needinfo?(holger)
(Assignee)

Comment 11

5 months ago
And use it to determine if a frame is a keyframe, its size and display size.
(Assignee)

Comment 12

5 months ago
We also set the TrackInfoSharedPtr to each decoded sample so that on decoder that can be recycled (android) the frames are displayed with the right size.

Depends on D11588
(Assignee)

Comment 13

5 months ago
:bryce could you please action this review please?
Flags: needinfo?(bvandyk)
(Assignee)

Updated

5 months ago
Flags: needinfo?(bvandyk)

Comment 14

5 months ago
Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/76b88f20b082
P1. Implement VP8/VP9 frame header parser. r=TD-Linux
https://hg.mozilla.org/integration/autoland/rev/6b87d74e97c9
P2. Use new VPx frame parser to detect content change. r=padenot

Comment 15

5 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/76b88f20b082
https://hg.mozilla.org/mozilla-central/rev/6b87d74e97c9
Status: REOPENED → RESOLVED
Last Resolved: 6 months ago5 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
(Reporter)

Comment 16

5 months ago
I have tested it in the Nightly and the sample video mixed video sizes now works. Thank you!

Updated

a month ago
Depends on: 1537675
You need to log in before you can comment on or make changes to this bug.