1206583 - [GoFlip] move GPU work to AP CPU on FxOS 2.2

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: General, defect)

Description

[Qu Yi]

During prototyping of non-touch support for Browser application we have got some technical limitations in the Firefox OS which don’t allow us to rework this application for non-touch UI without having deep modifications in the platform.

Initially the following approach was taken during the non-touch Browser prototyping:
•	a transparent surface (canvas) is drawn over the existing Browser app, which is activated upon the Browser app's launch;
•	a custom cursor is drawn on that surface and controlled by the cursor keys;
•	cursor position ([x,y] coordinates pair) is sent to the iframe (web-page content) to emulate a 'click';
•	if there is a link or button within these coordinates on a displayed page, an appropriate action is performed (link is opened, form submitted, etc.) 

The Browser window is implemented using the <iframe mozbrowser> tag, which is well described in MDN: https://developer.mozilla.org/en-US/docs/Web/API/Using_the_Browser_API.  

Although, it has some limitations, mainly caused by security reasons:
1)	The parent window cannot have access to the iframe's content, if it is from another domain. The Browser app is local and the web content is in a different domain.
It means that:
•	Browser app cannot get height, width of the displayed page;
•	Browser app cannot scroll the displayed page or zoom it.

2)	The parent window cannot modify the iframe's content.
It means that:
•	Browser app cannot get an element under cursor, determine whether it is a link, or image, or text field, etc.;
•	As a consequence, links cannot be highlighted;
•	No text fields modifications are possible;
•	No text on the page can be selected, because the focus in always kept on the canvas with the drawn cursor.

Considering written above, we can conclude that it is not possible to make a non-touch version of the Browser application without deep modifications on the platform level. This problem is also actively being discussed in Mozilla and there are some bugs about it (Bug 693515 is one of them).

Comment 1

[Qu Yi]

We are do the demo to move GPU work to AP CPU on Go Flip, 8909 3.2inch HVGA. It’s part of evaluation work for Go Flip next generation. 

We tried to set layers.acceleration.disabled and gfx.canvas.azure.accelerated, but after boot animation, the screen turn black.
QCT feedback below, so we want to know if possible to replace GPU work on FF2.2. Platform is QCT8909.
  Regarding disabling GPU, Mozilla mentioned that some of them have spent time porting firefox OS on the raspberry pi and as they had issues with opengl, they made BasicCompositor work with b2g (for instance there is this bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1082890 ).  This essentially does software composition. Looks like this work was done on Mozilla’s master branch so you would need to work with Mozilla to get it to work on v2.2 branch.

Comment 2

[Wesly Huang (EPM)]

Hi Michael:

This is the issue I've mailed you last week when you were in workshop, I suggest partner to move it to bugzilla for efficiency (pls ignore comment#0 and start w/ comment#1). Would you kindly advise here?

Thanks!

Hi Michael(In reply to Qu Yi from comment #1)
> We are do the demo to move GPU work to AP CPU on Go Flip, 8909 3.2inch HVGA.
> It’s part of evaluation work for Go Flip next generation. 
> 
> We tried to set layers.acceleration.disabled and
> gfx.canvas.azure.accelerated, but after boot animation, the screen turn
> black.
> QCT feedback below, so we want to know if possible to replace GPU work on
> FF2.2. Platform is QCT8909.
>   Regarding disabling GPU, Mozilla mentioned that some of them have spent
> time porting firefox OS on the raspberry pi and as they had issues with
> opengl, they made BasicCompositor work with b2g (for instance there is this
> bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1082890 ).  This
> essentially does software composition. Looks like this work was done on
> Mozilla’s master branch so you would need to work with Mozilla to get it to
> work on v2.2 branch.

Comment 3

[Jin ZHANG]

Attachment: device_reboot_after_disable_GPU.log

turned below prefs off

layers.acceleration.disabled
gfx.canvas.azure.accelerated

device reboot 

12-21 05:32:49.429 I/Gonk    (  213): Error mmap'ing framebuffer
12-21 05:32:49.429 E/Gonk    (  213): Failed to mmap fb(?!?), aborting ...
12-21 05:32:49.429 I/Gecko   (  213): [Parent 213] ###!!! ABORT: Can't open GL context and can't fall back on /dev/graphics/fb0 ...: file ../../../../../../../../gecko/widget/gonk/nsWindow.cpp, line 589
12-21 05:32:49.439 E/Gecko   (  213): mozalloc_abort: [Parent 213] ###!!! ABORT: Can't open GL context and can't fall back on /dev/graphics/fb0 ...: file ../../../../../../../../gecko/widget/gonk/nsWindow.cpp, line 589

Comment 4

[Jin ZHANG]

Attachment: device_reboot_after_disable_GPU_fxos2.0.log

update log name for fxos2.0

when 

12-21 05:32:49.429 I/Gonk    (  213): Error mmap'ing framebuffer
12-21 05:32:49.429 E/Gonk    (  213): Failed to mmap fb(?!?), aborting ...
12-21 05:32:49.429 I/Gecko   (  213): [Parent 213] ###!!! ABORT: Can't open GL context and can't fall back on /dev/graphics/fb0 ...: file ../../../../../../../../gecko/widget/gonk/nsWindow.cpp, line 589
12-21 05:32:49.439 E/Gecko   (  213): mozalloc_abort: [Parent 213] ###!!! ABORT: Can't open GL context and can't fall back on /dev/graphics/fb0 ...: file ../../../../../../../../gecko/widget/gonk/nsWindow.cpp, line 589

Comment 5

[Jin ZHANG]

Attachment: device_reboot_after_disable_GPU_fxos2.2.log

on fxos, only got some:

03-10 21:57:03.339 W/qdhwcomposer(  258): setGPUHint Warning: EGL current display is NULL

and device reboot, found in nsWindow.cpp::GetlayerManger, codes about fall back to software rendering removed.

Comment 6

[Jin ZHANG]

on fxos 2.2

(In reply to Jin Zhang(T2M) from comment #5)
> Created attachment 8664079 [details]
> device_reboot_after_disable_GPU_fxos2.2.log
> 
> on fxos, only got some:
> 
> 03-10 21:57:03.339 W/qdhwcomposer(  258): setGPUHint Warning: EGL current
> display is NULL
> 
> and device reboot, found in nsWindow.cpp::GetlayerManger, codes about fall
> back to software rendering removed.

Comment 7

[Wesly Huang (EPM)]

Hi Vincent:

As discussed, started from comment#3 today are related to the mail we ever had in August end, so after aligned w/ Ken he recommend you to support on this, would you kindly help check and advise?

Thank you! 


ps: pls ignore comment#0, it's irrelevant to this issue.

Comment 8

[Vincent Liu[:vliu]]

(In reply to Wesly Huang (TAM) from comment #7)
> Hi Vincent:
> 
> As discussed, started from comment#3 today are related to the mail we ever
> had in August end, so after aligned w/ Ken he recommend you to support on
> this, would you kindly help check and advise?
> 
> Thank you! 
> 
> 
> ps: pls ignore comment#0, it's irrelevant to this issue.

I also got black screen on Flame device, which runs on master. I think it could be the logic Gecko has in [1].

[1]:https://dxr.mozilla.org/mozilla-central/source/widget/gonk/HwcComposer2D.cpp?offset=1300#

Ideally, HWC acts as default compositor. Once it fails to composite, Gecko would fall back to GPU instead.

Comment 9

[Vincent Liu[:vliu]]

Sorry the wrong link in Comment 8. Post again here.

[1]: https://dxr.mozilla.org/mozilla-central/source/widget/gonk/HwcComposer2D.cpp#891

Comment 10

[Wesly Huang (EPM)]

Per discussion today w/ Vincent and Morris, need more time to understand the the design logic behind the pref. Vincent is helping check and we'll try to update again tomorrow.

Comment 11

[Vincent Liu[:vliu]]

Hi Sotaro,

This bug is a requirement from partner. The main target is removing GPU work to CPU. I'd tried to suggest modifying two preference values layers.acceleration.disabled and gfx.canvas.azure.accelerated. After modifying these values, the screen turn black. It also turns black on my Flame device. 

The changes for these preference values are

layers.acceleration.disabled -> true
gfx.canvas.azure.accelerated -> false

In HwcComposer2D::TryHwComposition(), I saw all possible composition were considered. 

    1. GPU Composition
    2. BLIT Composition
    3. Full OVERLAY Composition
    4. Partial OVERLAY Composition (GPU + OVERLAY) */ 

I think it shouldn't be the problem here. Do you know what the possible problem is? Thanks

Comment 12

[Sotaro Ikeda [:sotaro]]

vliu, can you explain about comment 0? It seems not related to disabling GPU.

Comment 13

[Sotaro Ikeda [:sotaro]]

(In reply to Vincent Liu[:vliu] from comment #11)
> 
> I think it shouldn't be the problem here. Do you know what the possible
> problem is? Thanks

I am going to check.

Comment 14

[Sotaro Ikeda [:sotaro]]

Hmm, GrallocTextureHostOGL's implementation is heavily depends on OpenGL.

Comment 15

[Sotaro Ikeda [:sotaro]]

By the following settings(disable gralloc), I confirmed the fxos boot on master flame-kk.
  - layers.acceleration.disabled -> true
  - gfx.canvas.azure.accelerated -> false
  - layers.gralloc.disable       -> true

But it had the following problems.
 - colors are reverted
 - video playback / camera preview causes system reboot.
    (They are heavily depends on gralloc implementation).

Comment 16

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #12)
> vliu, can you explain about comment 0? It seems not related to disabling GPU.

vliu, additional questions.
- Is the target device has gralloc?
- Is the target device has hardware composer?

Comment 17

[Sotaro Ikeda [:sotaro]]

Created Bug 1210182 to support basic compositor with gralloc.

Comment 18

[Vincent Liu[:vliu]]

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #16)
> (In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #12)
> > vliu, can you explain about comment 0? It seems not related to disabling GPU.
> 
> vliu, additional questions.
> - Is the target device has gralloc?
> - Is the target device has hardware composer?

Hi Sotaro,

Thanks for your great help on this bug. Actually I don't have any information about GoFlip. That's why I also tried it on Flame. ni? Vance to have some detailed information for your questions.

Comment 19

[Vincent Liu[:vliu]]

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #12)
> vliu, can you explain about comment 0? It seems not related to disabling GPU.

Hi Vance,

Sotaro also had this question on Comment 12. Could you please have a reply for him? Thanks.

Comment 20

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #15)
> 
> But it had the following problems.
>  - colors are reverted

It is caused by cairo's implementation. It seems to support only BGR. GfxFormatToCairoFormat() says it assign only BGR.
 https://dxr.mozilla.org/mozilla-central/source/gfx/2d/HelpersCairo.h#

Comment 21

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #20)
> (In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #15)
> > 
> > But it had the following problems.
> >  - colors are reverted
> 
> It is caused by cairo's implementation. It seems to support only BGR.
> GfxFormatToCairoFormat() says it assign only BGR.
>  https://dxr.mozilla.org/mozilla-central/source/gfx/2d/HelpersCairo.h#

https://dxr.mozilla.org/mozilla-central/source/gfx/2d/HelpersCairo.h#144

Comment 22

[Sotaro Ikeda [:sotaro]]

(In reply to Vincent Liu[:vliu] from comment #19)
> (In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #12)
> > vliu, can you explain about comment 0? It seems not related to disabling GPU.
> 
> Hi Vance,
> 
> Sotaro also had this question on Comment 12. Could you please have a reply
> for him? Thanks.

I also wanted to know the following question.
 - Which color format is supported as frame buffer?

Comment 23

[Sotaro Ikeda [:sotaro]]

> 
> I also wanted to know the following question.
>  - Which color format is supported as frame buffer?

It might be better to choose RGB565 as frame buffer's color format, if possible.

Comment 24

[Wesly Huang (EPM)]

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #12)
> vliu, can you explain about comment 0? It seems not related to disabling GPU.

Hi Sotaro, pls just ignore comment#0 (and start from comment#1) as it's irrelavnet to this issue, thanks.

Comment 25

[Wesly Huang (EPM)]

Hi Jin:

Would you help answer Sotaro's question in comment#16 & #22 below? Thanks!

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #16)
> vliu, additional questions.
> - Is the target device has gralloc?
> - Is the target device has hardware composer?

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #22)
> I also wanted to know the following question.
>  - Which color format is supported as frame buffer?

Comment 26

[Jin ZHANG]

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #16)
> vliu, additional questions.
> - Is the target device has gralloc?
YES on goflip.
> - Is the target device has hardware composer?
YES on goflip.

Currently we use flame(msm8610)/go_flip(msm8909) devices to test how to disable GPU, but the target chip may not have gralloc or hardware composer. Not sure as I haven't got code yet.

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #22)
> I also wanted to know the following question.
>  - Which color format is supported as frame buffer?

RGB565.

And Sotaro, how do I set 
      - layers.gralloc.disable       -> true
on fxos2.2? Can't find this pref in gecko/b2g/app/b2g.js

Comment 27

[Sotaro Ikeda [:sotaro]]

Hi Jin Zhan,

You can just add the following to b2g.js. But just changing the pref did not work on v2.2 flame-kk. b2g process crash during booting. I am going to investigate it.
 > pref("layers.gralloc.disable", true);

Comment 28

[Sotaro Ikeda [:sotaro]]

Attachment: temporary patch for b2gv2.2 - Enable BasicCompositor

Confirmed that BasicCompositor working by applying the patch on b2gv2.2 flame-kk.

Comment 29

[BMO Automation]

Firefox OS is not being worked on