1074635 - don't rerender the page for every frame in a GuM tab video stream

Status: NEW

(Core :: Graphics: Layers, defect)

Description

[Brad Lassey [:blassey] (use needinfo?)]

Right now we rerender a portion of the page for every frame in a tab video stream, which obviously isn't optimal for getting high frame rates with constrained CPUs. Ideally we would pull the already composited frames off the GPU (perhaps even directly streaming them to a hardware encoder). This is far outside my expertise, so that's where I'll stop the speculation on how to do this.

Comment 1

[James Willcox (:snorp) ([e-mail]) (he/him)]

I see a couple good options here:

1) Use ImageReader[0] to create a secondary Surface which we composite into. Any time we composite into the screen, we would also composite onto the Surface provided by ImageReader. ImageReader would provide callbacks when data is available in the form of RGBA or RGB565 (I think), which we could then feed to whatever encoder. Hopefully this would be faster than a glReadPixels() type of thing, which is hopelessly slow (150ms per read last I checked). Needs investigation.

2) Same as above, but the second Surface is hooked up directly to MediaCodec in order to use hardware encoding. It's possible that the readback situation is better here, as the encoder should be able to consume the underlying gralloc surfaces. I don't think this will fit in very well with the current webrtc framework, but I could be wrong there.

Any other option would involve some type of repainting still, which I think is always going to be terrible, unless we somehow have two persistent layer managers.

Comment 2

[James Willcox (:snorp) ([e-mail]) (he/him)]

[0] https://developer.android.com/reference/android/media/ImageReader.html

Comment 3

[PRANAY]

from where should i start..!!

Comment 4

[Aryan Agal]

Hi! I'm new here, just got my firefox build done (took me 6 hours almost :/)
I'd love to work on this issue!

Comment 5

[Sneha k]

Hey! Could I take up this bug? Can you guide me on how to fix it?

Comment 6

[Jonah Ho (AAstar)]

Is this bug report still valid? It was filed 7 years ago and current Firefox uses WebRender instead of Layers.

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

Clear a needinfo that is pending on an inactive user.

Inactive users most likely will not respond; if the missing information is essential and cannot be collected another way, the bug maybe should be closed as INCOMPLETE.

For more information, please visit auto_nag documentation.