Slow calls to ImageBridgeParent::RecvUpdate with GV on FireTv stick 4k
Categories
(Core :: Graphics: Layers, defect, P2)
Tracking
()
People
(Reporter: whimboo, Unassigned)
References
Details
(Whiteboard: [gvtv:p1])
Attachments
(2 files)
I'm running a 4k@60fps VP9 video via geckoview_example on my Fire TV stick 4k and got the following profile created:
As Chris pointed there are a lot of slow calls to ImageBridgeParent::RecvUpdate().
Given that there is no bug about that yet, I was requested to file one.
|
Reporter | |
Comment 1•6 years ago
|
||
Note that while playing the video which is about 20s in length, we spent about 5s in the above method. As it looks like mainly waiting all the time in poll() from libc.
The above profile has been done with a Nightly build of geckoview_example from April 10th.
|
||
Updated•6 years ago
|
|
||
Comment 2•6 years ago
|
||
We need to figure out what's calling poll() in this situation. Calling poll() here is somewhat surprising.
It might be possible to figure out what's going on with a debugger.
|
||
Comment 3•6 years ago
|
||
I've recorded systrace report when playing 4K@60FPS video (https://www.youtube.com/watch?v=Jsjtt5dWDYU) on my Stick to see if it shows what's going on in the Compositor thread. It looks like the updateTexImage() call (indirectly from [1]) takes 14-15ms to complete, and duraing the call, it sleeps for 12-13ms. However, this is probably Mediatek specific because the record on Pixel 2 shows the same call takes only 200-300ms and usually no sleeping state during the execution.
For reference, I will attach the reports to this bug.
The profile in comment 0 shows updateTexImage spends majority of its time in bindTextureImageLocked(), and that function calls some GLES and MTK functions to do its work. My guess is that it converts the frame data from YUV to RGB for the texture, probably through its MDP module[2].
[1] https://searchfox.org/mozilla-central/source/gfx/layers/opengl/TextureHostOGL.cpp#541
[2] https://www.kernel.org/doc/Documentation/devicetree/bindings/media/mediatek-mdp.txt
|
|
||
Comment 4•6 years ago
|
||
|
|
||
Comment 5•6 years ago
|
||
|
|
||
Comment 6•6 years ago
|
||
It would be very helpful if someone from Amazon or Mediatek could confirm the updateTexImage() issue or give some advice.
Chris, do you know who might be able to help pass the question?
|
||
Updated•6 years ago
|
|
|
||
Comment 7•6 years ago
|
||
I asked stpeter for an Amazon contact. I'll follow up with stpeter again.
|
||
Comment 8•6 years ago
|
||
Trying to figure out whether this is an issue on other devices we care about...
Here's a profile of GVE playing YouTube on my Samsung Galaxy S6: https://perfht.ml/30oWiEq
Profiling https://www.youtube.com/watch?v=xVKGstMFFFc with resolution set to 1080p from YouTube's hamburger menu, in fullscreen mode.
Playback is basically flawless, only a couple of dropped frames show up in the profile. This is a 3.5 year old phone, which was high end when I bought it.
It would be good to get a profile of this on a mid range and low end device.
|
|
||
Comment 9•6 years ago
|
||
A few other profiles on the Moto G5. It looks like this doesn't have HW VP9 decoding.
Copy/paste from the #perf discussion tonight:
jesup [5:19 PM]
same URl, but looks like the comments(?) overloaded the phone and video froze. https://perfht.ml/2VGXiFj
Trying another channel
15fps video @1080p: https://perfht.ml/2VIQdUW (smooth)
Matt [5:23 PM]
I can't see any obvious slow updateTexImage time in either of those
jesup [5:23 PM]
720p@30 (smooth): https://perfht.ml/2VL9p4c
1080p@60 (appeared smooth; with video stats on I saw some dropped frames): https://perfht.ml/2VJ0CQ9
Yeah, I am seeing stutters at 1080p@60 (landscape mode)
jesup crashes; 1:30am....
segment with clear freeze: https://perfht.ml/2VKiWs8 (1080p@60)
Matt [5:31 PM]
Unsure what happened with that freeze!
Not compositor-related though
Clearly a lot of issues, but it appears that the slow updateTexImage one might be Amazon specific
|
||
Comment 10•6 years ago
|
||
Correct - G5 has no VP9 HW. That's why youtube doesn't work at any reasonable resolution. So why are they sending VP9?
Note that I think twitch (profiles in the comment 9) is IIRC H.264, so the issues in those profiles would be different.
|
|
||
Comment 11•6 years ago
|
||
YouTube told us that they don't honour the results of the MediaCapabilities API, as it's "too slow". So I expect we'd need to change the YouTube extensions to MediaSource.isTypeSupported() to report we don't support high res VP9 on devices without a HW decoder.
Also, based on looking at the profiles, it looks like we're not using libvpx on Android. Using that instead of the MediaCodec software decoder may help, especially given many modernish phones have a good core count.
|
|
||
Comment 12•6 years ago
|
||
Yeah; the G5 has 8 cores, believe it or not.
|
||
Comment 13•6 years ago
|
||
(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #11)
YouTube told us that they don't honour the results of the MediaCapabilities API, as it's "too slow". So I expect we'd need to change the YouTube extensions to MediaSource.isTypeSupported() to report we don't support high res VP9 on devices without a HW decoder.
YT certainly uses MediaCapabilities. They were the first to do so and the main instigators behind the spec.
There's a difference in behaviour only for the live stream where they first rely on their custom parameters inside IsTypeSupported.
What are the steps to reproduce the problem?
Are you manually selecting the resolution or letting it choose automatically?
YT only uses Media Capabilities to determine the resolution set for auto. On Android we will report that it's not smooth for any resolutions greater than 480p if there are no HW decoders. But you can still manually select the resolution.
The G5 is a Qualcomm Snapdragon 430, which indeed doesn't do VP9. Which is surprising as my understanding was that HW VP9 was compulsory to bear the android logo since Android 6.
In any case, I don't think we want to go back to the behaviour of disabling codecs just because it's not perfectly smooth, as under default user conditions it will be.
Also, based on looking at the profiles, it looks like we're not using libvpx on Android. Using that instead of the MediaCodec software decoder may help, especially given many modernish phones have a good core count.
I'd be more than surprised if MediaCodec didn't use libvpx internally. I don't think using our own will make much of a difference.
libvpx does very poorly with multiple-core. Test shows that there's virtually no improvement over 2 cores.
|
|
||
Comment 14•6 years ago
|
||
Additional note: if we were to disable VP9, then the condition for the bug wouldn't even exist: YT will not serve > 1080p and 60fps if you're using H264.
|
|
||
Comment 15•6 years ago
|
||
(In reply to Jean-Yves Avenard [:jya] from comment #13)
YT certainly uses MediaCapabilities. They were the first to do so and the main instigators behind the spec.
There's a difference in behaviour only for the live stream where they first rely on their custom parameters inside IsTypeSupported.
Steven told us on the youtube-discuss mailing list on 5 April that they only use MediaCapabilites for "disabling features at the moment, not enabling them.".
So the question then is, whether they assume VP9 is available and allow it to be disabled by MediaCapabilities.
Also, based on looking at the profiles, it looks like we're not using libvpx on Android. Using that instead of the MediaCodec software decoder may help, especially given many modernish phones have a good core count.
I'd be more than surprised if MediaCodec didn't use libvpx internally. I don't think using our own will make much of a difference.
libvpx does very poorly with multiple-core. Test shows that there's virtually no improvement over 2 cores.
I meant to say ffvpx rather than libvpx here.
|
|
||
Comment 16•6 years ago
|
||
(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #15)
Steven told us on the youtube-discuss mailing list on 5 April that they only use MediaCapabilites for "disabling features at the moment, not enabling them.".
Only for the youtube.com/live
For standard playback, it's very much in use. I've discussed about the YT developer many times over it, including reporting errors where the MC handling was inconsistent.
MC will dictate the maximum resolution that is used when things are left on "Auto"
So the question then is, whether they assume VP9 is available and allow it to be disabled by MediaCapabilities.
If you say you do VP9 at any resolutions, they will use VP9 by default for the entire video. With the caveat mentioned above: it restricts the automatic change of a resolution to the one listed as being "smooth".
Note that this is now changing now that we've added the SourceBuffer.ChangeType API (https://developer.mozilla.org/en-US/docs/Web/API/SourceBuffer/changeType)
They will now mix codec, but AFAIK for now this is only used to play some porting with AV1 and other with H264.
They could obviously do something like use VP9 for all resolutions reported as smooth, and make a union with all the h264 content reported as smooth. Don't know if this will ever happen however.
I meant to say ffvpx rather than libvpx here.
That was backed out due to the increase in size on the exe.
It's over 1.2MB in size.
|
|
||
Comment 17•6 years ago
|
||
(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #11)
Also, based on looking at the profiles, it looks like we're not using libvpx on Android. Using that instead of the MediaCodec software decoder may help, especially given many modernish phones have a good core count.
Using built-in libvpx instead of MediaCodec through Android decoder module might not get significantly better performance but it definitely saves some JNI/Java overhead, extra process resources, IPC calls, and buffer copying.
|
||
Comment 18•3 years ago
|
||
Description
•