554270 - [e10s] Make image memory buffer fast API rendering for IPC(and common) plugins on linux with ImageSurface rendering path

Status: RESOLVED WONTFIX

(Core Graveyard :: Plug-ins, defect)

Description

[Oleg Romashin (:romaxa)]

Currently we are operating with plugins rendering only via X Windowless API, and sending to plugin process XWindow ID.

That approach does not work very well, because when we are rendering to local memory buffer (gfxImageSurface rendering) then we should create Temp XSurface, give it to plugin, plugin will do XShmPutImage to surface, then we get it back with X(Shm)GetImage, to temp memory buffer, and finally composite buffer to target gfxImageSurface.

Comment 1

[Oleg Romashin (:romaxa)]

We should remove existing Image expose API first, and then replace it with better version

Comment 2

[Oleg Romashin (:romaxa)]

Attachment: Windowless ImageSurface rendering API + IPC implementation

Comment 3

[Oleg Romashin (:romaxa)]

Attachment: Test plugin for local rendering API test

Comment 4

[Doug Turner (:dougt)]

Comment on attachment 435627 [details] [diff] [review]
Windowless ImageSurface rendering API + IPC implementation

maybe a stopgap until opengl is available from the NPAPI?

Comment 5

[Oleg Romashin (:romaxa)]

(In reply to comment #4)
> (From update of attachment 435627 [details] [diff] [review])
> maybe a stopgap until opengl is available from the NPAPI?

OpenGL is not available for all mobile platforms, also OpenGL is not working very well and fast for some of them... :(

Comment 6

[Oleg Romashin (:romaxa)]

Attachment: NON-IPC, default part

Make NPImageExpose and NPP_DrawImage API public.
Probably we need some description proposal in plugins wiki.

Comment 7

[Oleg Romashin (:romaxa)]

Attachment: IPC part for ImageExpose NPP_DrawImage API

Comment 8

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Oleg, just to let you know, stuart is working on an alternate proposal, and I was advised not to review this until there's a chance to compare stuart's and this.  Looking forward to an ass-kicking final product.

Comment 9

[Oleg Romashin (:romaxa)]

Stuart proposal is related to X surface wrapping with GL buffer for default NPP_HandleEvent...
This API only for rendering into gfxImageSurface target buffer...
We basically don't want to create/handle and use XSurfaces when painting into imageSurface (target layout surface - see Qt raster backend).

Comment 10

[Oleg Romashin (:romaxa)]

Attachment: Updated patch according to Plugins:NPP_DrawImageNPAPI

I have posted description for new NPP_DrawImage API
https://wiki.mozilla.org/Plugins:NPP_DrawImageNPAPI
This patch implementing non-IPC part for new update API

Comment 11

[Oleg Romashin (:romaxa)]

Attachment: IPC version for https://wiki.mozilla.org/Plugins:NPP_DrawImageNPAPI

IPC version

Comment 12

[Doug Turner (:dougt)]

Comment on attachment 437920 [details] [diff] [review]
Updated patch according to Plugins:NPP_DrawImageNPAPI

karlt loves these reviews.

Comment 13

[Oleg Romashin (:romaxa)]

Comment on attachment 437920 [details] [diff] [review]
Updated patch according to Plugins:NPP_DrawImageNPAPI

need new implementation with updated NPAPI

Comment 14

[Oleg Romashin (:romaxa)]

Attachment: Here is small patch which is implementing old maemo 5 ImageExpose API by using new gfxSharedImageSurface structure...

Comment 15

[Oleg Romashin (:romaxa)]

Attachment: New async rendering patch based on gfxSharedImageSurface and double buffering

Comment 16

[Blake Kaplan (:mrbkap) (inactive)]

Hey John, do we still care about this?

Comment 17

[John Schoenick [:johns]]

(In reply to Blake Kaplan (:mrbkap) from comment #16)
> Hey John, do we still care about this?

This would be something to ask graphics people, I'm not sure what the potential costs of the current way windowless plugins render with e10s are

Comment 18

[Karl Tomlinson (:karlt)]

Depending on Pixmaps for the windowless drawing API is a limitation, but I don't think Mozilla has any plans or wishes to add new plug-in APIs.