Videos encoded with a 5.2+ h264 profile stay black (audio only)
Categories
(Core :: Audio/Video: Playback, defect, P3)
Tracking
()
People
(Reporter: Harest, Unassigned)
References
()
Details
Attachments
(2 files)
level metadata modification.png
|
||
Comment 1•6 years ago
|
||
|
||
Comment 3•6 years ago
|
||
|
||
Comment 4•6 years ago
|
||
|
|
||
Comment 6•6 years ago
|
||
|
|
||
Comment 9•6 years ago
|
||
|
||
Comment 10•6 years ago
|
||
|
Reporter | |
Comment 11•6 years ago
|
||
|
|
||
Comment 12•6 years ago
|
||
|
||
Comment 13•6 years ago
|
||
|
||
Comment 16•5 years ago
|
||
(In reply to Nils Ohlmeier [:drno] from comment #13)
Sounds like this is a rare edge case of content creators using non-ideal
H264 profiles causing problem on Windows 7.
idk if it is that rare, i see it on twitter and reddit etc. maybe services that auto-transcode from gif to other formats.
|
||
Comment 17•5 years ago
|
||
It's not rare anymore as Twitter seems to migrate all its MP4-encoded GIFs to level 5.2 profiles instead of just leaving them at 5.1. That's why you see more and more threads on Reddit and elsewhere with no valid solution as of yet, which isn't "disable hardware acceleration" or "upgrade to Windows 10".
And again, I have to read a valid explanation to why they are transitioning to level 5.2. Nobody has been able to answer yet (their MP4 videos, on the other hand, don't have this problem, only their transcoded GIFs).
|
||
Comment 18•5 years ago
|
||
I want to chime in noting that this is not just "rare", it is basically half of the animated content I see on Twitter. I'm not counting, exactly, but if someone asked, I would say about half of the animated content on Twitter doesn't play for me right now on my Windows 7. A presidential candidate just tweeted an animation that doesn't play. That's not an "edge case".
|
|
||
Comment 19•5 years ago
|
||
Ok, clearly this needs more attention. jya, bryce, how hard would it be to use software decoding for h264 on Windows 7 and 8.1? Do we have a path we could reuse?
|
|
||
Comment 20•5 years ago
|
||
As I said in my bug, epic and marathon can both show the vids properly on the same machine that f2f cannot.
The more alarming aspect of the bug is the continuous writes to mediacapabilities - that seems like a bug bug not a Windows 7 can't render this vid bug.
|
|
||
Comment 21•5 years ago
|
||
(In reply to Paul Adenot (:padenot) from comment #19)
Ok, clearly this needs more attention. jya, bryce, how hard would it be to use software decoding for h264 on Windows 7 and 8.1? Do we have a path we could reuse?
The solution isn't technical. We can only use the h264 decoder provided by the OS framework
|
|
||
Comment 22•5 years ago
|
||
(In reply to westsonoma from comment #20)
As I said in my bug, epic and marathon can both show the vids properly on the same machine that f2f cannot.
The more alarming aspect of the bug is the continuous writes to mediacapabilities - that seems like a bug bug not a Windows 7 can't render this vid bug.
Firefox doesn't ship with its own H264 decoder, it is limited to what the windows framework can do.
https://docs.microsoft.com/en-us/windows/win32/medfound/h-264-video-decoder
Windows 7 decoder is also limited to 1920x1088, though with recent WMF updates it can do more.
|
|
||
Comment 23•5 years ago
|
||
(In reply to Jean-Yves Avenard [:jya] from comment #22)
(In reply to westsonoma from comment #20)
As I said in my bug, epic and marathon can both show the vids properly on the same machine that f2f cannot.
The more alarming aspect of the bug is the continuous writes to mediacapabilities - that seems like a bug bug not a Windows 7 can't render this vid bug.Firefox doesn't ship with its own H264 decoder, it is limited to what the windows framework can do.
https://docs.microsoft.com/en-us/windows/win32/medfound/h-264-video-decoder
Windows 7 decoder is also limited to 1920x1088, though with recent WMF updates it can do more.
OK, thanks for explaining that. Epic is Chromium based and none of the other chromium based browsers rendered the vid.
I emailed the epic CEO and will follow up if he replies.
But... That still does not explain why ff writes to mediacapabioities continuously when an "unrenderable" vid is on the screen.
is it a fight loop of something trying and failing to render the vid? And why write to mediacapabilities shouldn't that be a read-only ish db?
Either way it tends to not down the system while doing absolutely nothing as far as the user can see.
|
|
||
Comment 24•5 years ago
|
||
ni? :achronop to investigate on why we continuously write in the DB
|
||
Comment 25•5 years ago
|
||
(In reply to Tomas Bucher from comment #10)
The videos from web.poecdn.com are not encoded optimally. They are encoded
in H.264 level 5.2, which is not supported on Windows 7 with hardware
acceleration, see
https://docs.microsoft.com/windows/desktop/medfound/h-264-video-decoder. It
should work on Windows 8.1 and later, though.The videos could be encoded just with level 4.2 that is sufficient for
1920x1080@60fps, i.e. the params the web.poecdn.com videos are encoded with
(see https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels) and would then
work in Firefox on Windows 7 as well.So essentially three possible outcomes to see the videos on Windows OS:
- stay on Windows 7 and disable default hardware acceleration support in
Firefox- upgrade to Windows 10 and keep default hardware acceleration support in
Firefox- complain to content creators at web.poecdn.com and have them decrease
H.264 level for their 1080p content (the fact that they use Mainconcept
encoder does not play any role in this) and keep default hardware
acceleration support in FirefoxAlternatively. Mozilla could try to switch to software rendering on Windows
7 and 8 for H.264 level 5.2 automatically, but not sure how feasible that is.
I have this issue but with twitter GIFs. They just stay black whenever I click on them and this might be the issue to this. I hope that this is fixed soon.
|
||
Comment 26•5 years ago
|
||
this is getting ridiculous. firefox need to have it's own decoder otherwise windows 7 to 8.1 users that not willing to upgrade to 10 will just have to switch to chrome.
this issue use to be rare but not anymore, when I browse it is on a level of half the content I have to constantly switch back between chrome and firefox. originally I stick to firefox due to the customization but now its with quantum its becoming more like chrome. if can't do it like chrome then its soon time for us to switch.
I have nothing further to add to :jya's comments above, except to reiterate that this is a licensing issue, not a technical one. We would like to support as broad a range of configurations as we can, but because we rely on the platform we are running on, we cannot support more than that platform does. We cannot ship our own decoders to these platforms due to licensing issues. So pressure needs to be applied to sites to host compatible content, or to platforms to provide better decoders.
If you have examples of media that doesn't work, that may be useful for us to verify that the bug is the one we're discussing. However, if it is, the limitations discussed above remain.
|
|
||
Comment 28•5 years ago
|
||
(In reply to Bryce Seager van Dyk (:bryce) from comment #27)
I have nothing further to add to :jya's comments above, except to reiterate that this is a licensing issue, not a technical one. We would like to support as broad a range of configurations as we can, but because we rely on the platform we are running on, we cannot support more than that platform does. We cannot ship our own decoders to these platforms due to licensing issues. So pressure needs to be applied to sites to host compatible content, or to platforms to provide better decoders.
If you have examples of media that doesn't work, that may be useful for us to verify that the bug is the one we're discussing. However, if it is, the limitations discussed above remain.
https://twitter.com/i/status/1187300381077860357
https://twitter.com/i/status/1187250649571807232
https://twitter.com/i/status/1187230620167983105
|
|
Reporter | |
Comment 29•5 years ago
|
||
(In reply to westsonoma from comment #28)
https://twitter.com/i/status/1187300381077860357
https://twitter.com/i/status/1187250649571807232
https://twitter.com/i/status/1187230620167983105
It is indeed the same issue. Checking with ffprobe -show_streams on the first video, the level is 5.2.
|
||
Comment 30•5 years ago
|
||
The use of mediacapabilities database, in this case, is odd indeed. Writing in the mediacapabilities database should happen at the EOS which, in a file on a loop, would happen at the end of each loop. However, since here we have a decoder error I don't know why we get an EOS.
It would help if you can obtain logs while reproducing the error. You can find the instructions bellow.
In a shell, execute:
export MOZ_LOG=timestamp,sync,MediaDecoder:5,MediaFormatReader:5,MediaCapabilities:5
export MOZ_LOG_FILE=/path/to/temp/dir/firefox.log
cd /path/to/firefox/binary
./firefox
note that it will write several log files there, most of them with size 0 but one file will contain all the logs from your run. Also, please mention the link that you use to create the logs.
How did you realize that the mediacapabilities database is being written? Can you please copy here the content of your database. You can do that from about:support page. Under the media section, there is a button that enumerates the database. Copy and paste the results here
|
|
||
Comment 32•5 years ago
|
||
(In reply to Bryce Seager van Dyk (:bryce) from comment #27)
because we rely on the platform we are running on, we cannot support more than that platform does.
Firefox has been been superior than internet explorer for the longest time in almost every way, more addons, more features, more customization. firefox definitely can do more than what platform can.
also, firefox doesn't even need to ship with any decoder. simply implement a feature to let user browse/choose codec file to use to decode as they browse the web and that shouldnt even conflict with licensing/legal issue.
|
|
||
Comment 33•5 years ago
|
||
add to previous post, simply allow end user to choose between platform decoder or 3rd party decoder that they can download off the internet, problem solved no?
|
||
Comment 34•5 years ago
|
||
Another recent source of this problem is Periscope. An increasing number of broadcasters are no longer watchable on Windows 7 because the streams are marked AVC level 5.2. This is experienced trying Firefox or Chrome (Periscope has a further bug that prevents using it in IE11). It also appears to be true on Windows 8. AVC 5.2 streams play fine in Firefox on Windows 10, as expected. It's interesting that the previous problem examples were Twitter media as Periscope is now also a Twitter company.
My Windows 7 PC plays 5.2 and HVEC content just fine in MPC-HC and VLC. From what I can find, if Firefox could/would use DirectShow for H.264 and H.265, then it will automatically pick up the installed LAV filters (from K-Lite) and work fine. I found google search references to Firefox options for enabling/disabling Media Foundation and DirectShow, but I do not see any such options in my Firefox 70. Maybe they never existed in released product, or just no longer exist?
I have sent a help message to Periscope identifying this oversight on their part. But who knows what they will do. And more examples will occur as HVEC and higher AVC levels continue to expand. As the limiting to 5.1 in Media Foundation is an artificial one, it's a bit irritating.
|
|
||
Comment 35•5 years ago
|
||
Alex sorry I had not been on bugzilla since I posted and had not seen your needinfo request.
I lost that machine during the power outages here in Northern California during the fires.
I have not been able to reproduce it on recent builds of nightly. Iam using an old "Velociraptor" disk so it is super loud when writing/seeking, that is why I first noticed the hammering when I encountered those videos.
I will try to test some old versions using mozregression just to see if I can reproduce it.
But at this point, it is not happening, possibly due to baby yoda coming into existence and making the world a better place, but I can't be sure.
|
|
||
Comment 37•5 years ago
|
||
(In reply to westsonoma from comment #35)
Alex sorry I had not been on bugzilla since I posted and had not seen your needinfo request.
I lost that machine during the power outages here in Northern California during the fires.
I have not been able to reproduce it on recent builds of nightly. Iam using an old "Velociraptor" disk so it is super loud when writing/seeking, that is why I first noticed the hammering when I encountered those videos.I will try to test some old versions using mozregression just to see if I can reproduce it.
But at this point, it is not happening, possibly due to baby yoda coming into existence and making the world a better place, but I can't be sure.
when it was window xp, it didnt have media foundation back then and i believe firefox at the time in about:config has something media.directshow.enabled to true. nowadays window with media foundation by default so im guessing directshow setting in about:config is removed or disabled by default and just uses media foundation which is the platform decoder with "media.windows-media-foundation.enabled".
the newer quantum probably had this media.windows-media-foundation.enabled built in and doesnt even show in about:config. when I tried adding "media.windows-media-foundation.enabled" to false and "media.directshow.enabled" to true as new boolean into about:config and installed k-lite codec, firefox still couldnt play those videos and its likely those settings don't do anything as it is a removed feature.
maybe firefox can add those back.
|
||
Comment 38•4 years ago
|
||
Hey I'm a windows 7 holdout that has been experiencing this issue. (black screen in window, stuck on a frame, media wont play, etc). I could download the media, browse to the folder, and open it with VLC, then delete the file, but that much effort is generally not worth the ~5 second clips that people post.
most of this technical stuff is over my head, but i did find a reasonable workaround that is better than just downloading the media as i was doing. its by no means a fix, but its increased my ease of browsing for the moment. here it is:
There is a Firefox extension called "Open in VLC media player" here:
https://addons.mozilla.org/en-US/firefox/addon/open-in-vlc/
this simple and elegant solution just adds a right click context menu option to open the media with VLC.
the problematic media still will not play natively in the browser (which would be preferred), and its still just stuck on a black screen, etc. but now when i run into these, i can quickly right click, open in VLC, and have VLC instantly pop open and play it. VLC also has an option to automatically quit when at the end of the playlist, so once the video is over, it just closes itself. this is only one click more than just hitting the play button on the media if it worked, or two more clicks if it were stuff that autoplays. not too bad, and not much mouse movement required... no downloading the media and having it clutter up folders or the desktop either.
its not perfect, but its something. i just wanted to share this workaround for anyone whose been frustrated with the problem and followed the trail here. maybe this can give the devs some out-of-the-box ideas for a solution, too? maybe there is some way to display media through VLC, but still inside firefox, instead of in VLCs own window? or maybe a copy of this extension but it automatically launches the "open in VLC" context menu option without me even having to right click and select it?
|
||
Comment 40•4 years ago
|
||
I think this issue can be resolved if Firefox could change the H.264 level metadata from 5.2 to 5.1 in the video stream before passing it to the MS decoder. For example, ffmpeg can do it using this command
ffmpeg -i in.mp4 -map 0 -c copy -bsf:v h264_metadata=level=5.1 out.mp4
I tested it with the reporter's video on Windows 7. The original video couldn't be played, but after changing the level metadata with the above command, it plays just fine.
Sadly, we can't re-encode the videos for the same reason we can't decode them (licensing limitations and the lack of support from Window's codecs). I think to robustly handle changing the level metadata, we may need to re-encode. This is because the levels specify constraints on the decoder. In some cases these constraints are incorrect in that a lower level could be used by the encoding application. However, I don't think we can blanket downgrade the level -- this would essentially mean distrusting the metadata given to us. This could lead to issues in cases where the decoders are legitimately unable to support the data being given to them.
|
|
||
Comment 42•4 years ago
|
||
|
|
||
Comment 43•4 years ago
|
||
I'm not suggesting re-encoding. I guess I didn't explain it well enough. What ffmpeg does is it changes only one byte in the header that specifies the level (level_idc) from 0x34 (52 dec) to 0x33 (51 dec). See https://doc-kurento.readthedocs.io/en/stable/knowledge/h264.html I attached a screenshot that demonstrates it with https://web.poecdn.com/public/news/2018-05-02/VLS_news.mp4 as an example.
I agree that it's a hacky solution, but it's simple enough and it fixes the problem. Considering that people above and in other bugs mentioned that they are switching to other browsers because of it, I think it's serious enough to consider. You can apply it on Win 7 and 8/8.1 only, so it won't affect other OS. In the worst case, if you change the level and it turns out that it was a real level 5.2 video, the decoder simply won't be able to play it, like it's already doing now. But in 99% of cases where the video doesn't require level 5.2, it will play.
What I'm saying is you can't safely change that byte without sometimes needing to re-encode. That byte is being used to specify constraints, and in some cases it's over constraining the file such that a lower level of constraints may also be applicable (as above). However, my concern stems from the case where it's not over constrained, in which case you now have a level which is wrong and which would require a re-encode to have the media comply with the constraints.
In the worst case, if you change the level and it turns out that it was a real level 5.2 video, the decoder simply won't be able to play it, like it's already doing now.
It's not clear to me that the decoder will always gracefully reject the media as it does now. If you're manipulating the h264 stream being fed into the decoder such that it may contain incorrect metadata it seems like there could be more significant downsides such as crashing the process the decoder runs in.
If we had a way to verify that the level specified in the sps was over constrained it would make sense to mutate it. But blanket rewriting level information seems like it would cause it's own set of problems.
I'm reaching out to some of the sites mentioned above to see if they can detect cases where the constraints could be relaxed to provide better decoder support.
I've spoken to some folks from Twitter about their use case and they've said that in the past some videos have been over constrained. These videos aren't re-encoded, so will still present with the problem. However, this should not be happening for recent videos -- the constraints/levels should be set at lower levels if possible.
They also mentioned that 5.2 is still used in the case of high frame rate/slow motion videos.
So if you're running into this problem with recent videos, please link them on this bug. If they're recent and slow mo there's a chance the 5.2 level is correct. However, if not, it's unexpected and I can reach out to see if something unexpected is happening.
|
|
||
Comment 47•4 years ago
|
||
The problem is not limited to Twitter. I experienced it on vimeo.com and some other sites I don't remember.
I understand your concern about possible crashes, but I think it's a really something very unlikely to happen. As you know, Windows 7 doesn't receive updates anymore, so the decoder won't crash in the future if it's not crashing right now. I can confirm that it's not crashing because I have a 8K 24 fps H.264 video marked as level 5.1 (so it's the opposite case where the level should've been set higher) and the decoder just doesn't play it, no crashes. Windows 8.1 still receives updates, so it's somewhat of a risk as you said, but it also doesn't crash right now.
If you have examples of videos with the issue on Vimeo, I can reach out to them. I appreciate the frustration with these sites not working, but if they are specifying incorrect constraints, the appropriate fix is for them to specify correct ones.
If the user agent starts making guesses at what the constraints should and mutating them without verification there's the crashing problem mentioned above -- I remain concerned about this, as while a single case may not crash, it's hard to know all new cases are safe (it gets more fun once gfx drivers are in the mix). This extends in to the more severe case of security issues and that it's difficult to know if the decoders are robust and safe if we intentionally feed them malformed inputs. Will the decoder always detect this safely? Or could it overflow for some input before it stops?
It's also a concern from a web standards point of view where such behaviour could end up hiding bugs in sites. In extreme scenarios it ends up causing more of the problem we want to avoid -- sites intentionally over constraining because they know user agents will try and mitigate it internally.
|
|
||
Comment 49•4 years ago
|
||
Any 4K video on vimeo.com has the issue. For example https://vimeo.com/186240055 and https://vimeo.com/151292804
Thanks! I've reached out to Vimeo now as well.
I've spoken to Vimeo about this and because they use 5.2 for their higher frame rate 4k content as well as some of the bigger frame 4k formats (as it's necessary in both cases), it sounds like it's easier for them the use the 5.2 level blanket across 4k content.
Understandably, sites hosting large amount of video content have a non-trivial task if they wish to reprocess existing videos. It would be ideal if they could make the distinction for videos going forward even if old videos are not reprocessed, but that will be a matter of those sites priorities and getting engineering time for the task.
I appreciate this is a frustrating issue, it is also frustrating for Mozilla/Gecko dev perspective. I'm happy to continue reaching out to sites within the context of this bug. It appears we've seen some improvement since this issue was first raised in cases such as 1080p videos unnecessarily using a 5.2 level on some sites. However, this becomes unavoidable at 4K with framerates over 30 (and higher res 4k content), and it's then up to sites to make the distinction for those videos that can use lower constraints.
For the reference of anyone arriving at this bug later, annex A.3 of the H264 spec details examples of different maximum values for different levels, including the frame rates supported at certain standard resolution, and is useful reading in this context.
|
|
Reporter | |
Comment 52•4 years ago
|
||
While i can understand it with the democratisation of 4k, in the case of Twitter it was kinda lame of them to use such profile level for videos that are always displayed with an atrocious quality (360p). Good to know they improved on this issue. In my case back then Twitter was the most annoying on this issue.
I myself am not anymore on a Win7 config. I saw on one of the threads stated as duplicate someone didn't have the issue in several browsers. I'm not sure if it was accurate or not and how others browsers are managing this issue.
Regarding the videos i put in the OP, they now mostly host their videos (even the short ones) on Youtube so it fixes the issue directly.
Thanks to anyone that looked / is still looking into this issue.
| Comment hidden (advocacy) |
|
||
Comment 54•2 years ago
|
||
Hello. Seems like I have this bug too - on a genuine updated Windows 7 Professional SP1 x64. And I have it second time, as it already appeared once several years ago, and gone after a certain time without any effort. Maybe Microsoft updated my video drivers that time, but now there are no any Windows 7 drivers at Microsoft servers, and no any other dummy OS that I would trust.
My issue is reproduced in safe mode, and my Firefox x64 installation is quite fresh - as my Windows and Radeon R5 video drivers installations are too. Radeon drivers are installed extremely correct via Device Manager from a corresponding file path on original CD disk, and I cant reinstall them, as long as I cant afford working without hardware acceleration, as my CPU is not that fast.
Description
•