[GoFlip] move GPU work to AP CPU on FxOS 2.2

RESOLVED WONTFIX

Status

defect
--
blocker
RESOLVED WONTFIX
4 years ago
5 months ago

People

(Reporter: yi.qu, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments, 1 obsolete attachment)

(Reporter)

Description

4 years ago
During prototyping of non-touch support for Browser application we have got some technical limitations in the Firefox OS which don’t allow us to rework this application for non-touch UI without having deep modifications in the platform.

Initially the following approach was taken during the non-touch Browser prototyping:
•	a transparent surface (canvas) is drawn over the existing Browser app, which is activated upon the Browser app's launch;
•	a custom cursor is drawn on that surface and controlled by the cursor keys;
•	cursor position ([x,y] coordinates pair) is sent to the iframe (web-page content) to emulate a 'click';
•	if there is a link or button within these coordinates on a displayed page, an appropriate action is performed (link is opened, form submitted, etc.) 

The Browser window is implemented using the <iframe mozbrowser> tag, which is well described in MDN: https://developer.mozilla.org/en-US/docs/Web/API/Using_the_Browser_API.  

Although, it has some limitations, mainly caused by security reasons:
1)	The parent window cannot have access to the iframe's content, if it is from another domain. The Browser app is local and the web content is in a different domain.
It means that:
•	Browser app cannot get height, width of the displayed page;
•	Browser app cannot scroll the displayed page or zoom it.

2)	The parent window cannot modify the iframe's content.
It means that:
•	Browser app cannot get an element under cursor, determine whether it is a link, or image, or text field, etc.;
•	As a consequence, links cannot be highlighted;
•	No text fields modifications are possible;
•	No text on the page can be selected, because the focus in always kept on the canvas with the drawn cursor.

Considering written above, we can conclude that it is not possible to make a non-touch version of the Browser application without deep modifications on the platform level. This problem is also actively being discussed in Mozilla and there are some bugs about it (Bug 693515 is one of them).
(Reporter)

Comment 1

4 years ago
We are do the demo to move GPU work to AP CPU on Go Flip, 8909 3.2inch HVGA. It’s part of evaluation work for Go Flip next generation. 

We tried to set layers.acceleration.disabled and gfx.canvas.azure.accelerated, but after boot animation, the screen turn black.
QCT feedback below, so we want to know if possible to replace GPU work on FF2.2. Platform is QCT8909.
  Regarding disabling GPU, Mozilla mentioned that some of them have spent time porting firefox OS on the raspberry pi and as they had issues with opengl, they made BasicCompositor work with b2g (for instance there is this bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1082890 ).  This essentially does software composition. Looks like this work was done on Mozilla’s master branch so you would need to work with Mozilla to get it to work on v2.2 branch.

Updated

4 years ago
Summary: [FFOS2.0] [GoFlip]non-touch support for Browser application in Gecko layer → [GoFlip] move GPU work to AP CPU on FxOS 2.2

Updated

4 years ago
Blocks: 1197103

Updated

4 years ago
Blocks: 1191703
No longer blocks: 1197103

Comment 2

4 years ago
Hi Michael:

This is the issue I've mailed you last week when you were in workshop, I suggest partner to move it to bugzilla for efficiency (pls ignore comment#0 and start w/ comment#1). Would you kindly advise here?

Thanks!

Hi Michael(In reply to Qu Yi from comment #1)
> We are do the demo to move GPU work to AP CPU on Go Flip, 8909 3.2inch HVGA.
> It’s part of evaluation work for Go Flip next generation. 
> 
> We tried to set layers.acceleration.disabled and
> gfx.canvas.azure.accelerated, but after boot animation, the screen turn
> black.
> QCT feedback below, so we want to know if possible to replace GPU work on
> FF2.2. Platform is QCT8909.
>   Regarding disabling GPU, Mozilla mentioned that some of them have spent
> time porting firefox OS on the raspberry pi and as they had issues with
> opengl, they made BasicCompositor work with b2g (for instance there is this
> bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1082890 ).  This
> essentially does software composition. Looks like this work was done on
> Mozilla’s master branch so you would need to work with Mozilla to get it to
> work on v2.2 branch.
Flags: needinfo?(mwu)

Comment 3

4 years ago
Posted file device_reboot_after_disable_GPU.log (obsolete) —
turned below prefs off

layers.acceleration.disabled
gfx.canvas.azure.accelerated

device reboot 

12-21 05:32:49.429 I/Gonk    (  213): Error mmap'ing framebuffer
12-21 05:32:49.429 E/Gonk    (  213): Failed to mmap fb(?!?), aborting ...
12-21 05:32:49.429 I/Gecko   (  213): [Parent 213] ###!!! ABORT: Can't open GL context and can't fall back on /dev/graphics/fb0 ...: file ../../../../../../../../gecko/widget/gonk/nsWindow.cpp, line 589
12-21 05:32:49.439 E/Gecko   (  213): mozalloc_abort: [Parent 213] ###!!! ABORT: Can't open GL context and can't fall back on /dev/graphics/fb0 ...: file ../../../../../../../../gecko/widget/gonk/nsWindow.cpp, line 589

Comment 4

4 years ago
update log name for fxos2.0

when 

12-21 05:32:49.429 I/Gonk    (  213): Error mmap'ing framebuffer
12-21 05:32:49.429 E/Gonk    (  213): Failed to mmap fb(?!?), aborting ...
12-21 05:32:49.429 I/Gecko   (  213): [Parent 213] ###!!! ABORT: Can't open GL context and can't fall back on /dev/graphics/fb0 ...: file ../../../../../../../../gecko/widget/gonk/nsWindow.cpp, line 589
12-21 05:32:49.439 E/Gecko   (  213): mozalloc_abort: [Parent 213] ###!!! ABORT: Can't open GL context and can't fall back on /dev/graphics/fb0 ...: file ../../../../../../../../gecko/widget/gonk/nsWindow.cpp, line 589
Attachment #8664071 - Attachment is obsolete: true

Comment 5

4 years ago
on fxos, only got some:

03-10 21:57:03.339 W/qdhwcomposer(  258): setGPUHint Warning: EGL current display is NULL

and device reboot, found in nsWindow.cpp::GetlayerManger, codes about fall back to software rendering removed.

Comment 6

4 years ago
on fxos 2.2

(In reply to Jin Zhang(T2M) from comment #5)
> Created attachment 8664079 [details]
> device_reboot_after_disable_GPU_fxos2.2.log
> 
> on fxos, only got some:
> 
> 03-10 21:57:03.339 W/qdhwcomposer(  258): setGPUHint Warning: EGL current
> display is NULL
> 
> and device reboot, found in nsWindow.cpp::GetlayerManger, codes about fall
> back to software rendering removed.

Comment 7

4 years ago
Hi Vincent:

As discussed, started from comment#3 today are related to the mail we ever had in August end, so after aligned w/ Ken he recommend you to support on this, would you kindly help check and advise?

Thank you! 


ps: pls ignore comment#0, it's irrelevant to this issue.
Flags: needinfo?(vliu)
(In reply to Wesly Huang (TAM) from comment #7)
> Hi Vincent:
> 
> As discussed, started from comment#3 today are related to the mail we ever
> had in August end, so after aligned w/ Ken he recommend you to support on
> this, would you kindly help check and advise?
> 
> Thank you! 
> 
> 
> ps: pls ignore comment#0, it's irrelevant to this issue.

I also got black screen on Flame device, which runs on master. I think it could be the logic Gecko has in [1].

[1]:https://dxr.mozilla.org/mozilla-central/source/widget/gonk/HwcComposer2D.cpp?offset=1300#

Ideally, HWC acts as default compositor. Once it fails to composite, Gecko would fall back to GPU instead.
Flags: needinfo?(vliu)
Per discussion today w/ Vincent and Morris, need more time to understand the the design logic behind the pref. Vincent is helping check and we'll try to update again tomorrow.
Hi Sotaro,

This bug is a requirement from partner. The main target is removing GPU work to CPU. I'd tried to suggest modifying two preference values layers.acceleration.disabled and gfx.canvas.azure.accelerated. After modifying these values, the screen turn black. It also turns black on my Flame device. 

The changes for these preference values are

layers.acceleration.disabled -> true
gfx.canvas.azure.accelerated -> false

In HwcComposer2D::TryHwComposition(), I saw all possible composition were considered. 

    1. GPU Composition
    2. BLIT Composition
    3. Full OVERLAY Composition
    4. Partial OVERLAY Composition (GPU + OVERLAY) */ 

I think it shouldn't be the problem here. Do you know what the possible problem is? Thanks
Flags: needinfo?(sotaro.ikeda.g)
vliu, can you explain about comment 0? It seems not related to disabling GPU.
Flags: needinfo?(sotaro.ikeda.g) → needinfo?(vliu)
(In reply to Vincent Liu[:vliu] from comment #11)
> 
> I think it shouldn't be the problem here. Do you know what the possible
> problem is? Thanks

I am going to check.
Flags: needinfo?(sotaro.ikeda.g)
Hmm, GrallocTextureHostOGL's implementation is heavily depends on OpenGL.
Flags: needinfo?(sotaro.ikeda.g)
By the following settings(disable gralloc), I confirmed the fxos boot on master flame-kk.
  - layers.acceleration.disabled -> true
  - gfx.canvas.azure.accelerated -> false
  - layers.gralloc.disable       -> true

But it had the following problems.
 - colors are reverted
 - video playback / camera preview causes system reboot.
    (They are heavily depends on gralloc implementation).
Flags: needinfo?(vliu)
(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #12)
> vliu, can you explain about comment 0? It seems not related to disabling GPU.

vliu, additional questions.
- Is the target device has gralloc?
- Is the target device has hardware composer?
Flags: needinfo?(vliu)
Created Bug 1210182 to support basic compositor with gralloc.
(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #16)
> (In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #12)
> > vliu, can you explain about comment 0? It seems not related to disabling GPU.
> 
> vliu, additional questions.
> - Is the target device has gralloc?
> - Is the target device has hardware composer?

Hi Sotaro,

Thanks for your great help on this bug. Actually I don't have any information about GoFlip. That's why I also tried it on Flame. ni? Vance to have some detailed information for your questions.
Flags: needinfo?(vliu) → needinfo?(vchen)
(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #12)
> vliu, can you explain about comment 0? It seems not related to disabling GPU.

Hi Vance,

Sotaro also had this question on Comment 12. Could you please have a reply for him? Thanks.
(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #15)
> 
> But it had the following problems.
>  - colors are reverted

It is caused by cairo's implementation. It seems to support only BGR. GfxFormatToCairoFormat() says it assign only BGR.
 https://dxr.mozilla.org/mozilla-central/source/gfx/2d/HelpersCairo.h#
(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #20)
> (In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #15)
> > 
> > But it had the following problems.
> >  - colors are reverted
> 
> It is caused by cairo's implementation. It seems to support only BGR.
> GfxFormatToCairoFormat() says it assign only BGR.
>  https://dxr.mozilla.org/mozilla-central/source/gfx/2d/HelpersCairo.h#

https://dxr.mozilla.org/mozilla-central/source/gfx/2d/HelpersCairo.h#144
Depends on: 1210514
(In reply to Vincent Liu[:vliu] from comment #19)
> (In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct] from comment #12)
> > vliu, can you explain about comment 0? It seems not related to disabling GPU.
> 
> Hi Vance,
> 
> Sotaro also had this question on Comment 12. Could you please have a reply
> for him? Thanks.

I also wanted to know the following question.
 - Which color format is supported as frame buffer?
> 
> I also wanted to know the following question.
>  - Which color format is supported as frame buffer?

It might be better to choose RGB565 as frame buffer's color format, if possible.
(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #12)
> vliu, can you explain about comment 0? It seems not related to disabling GPU.

Hi Sotaro, pls just ignore comment#0 (and start from comment#1) as it's irrelavnet to this issue, thanks.
Hi Jin:

Would you help answer Sotaro's question in comment#16 & #22 below? Thanks!

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #16)
> vliu, additional questions.
> - Is the target device has gralloc?
> - Is the target device has hardware composer?

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #22)
> I also wanted to know the following question.
>  - Which color format is supported as frame buffer?
Flags: needinfo?(vchen)
Flags: needinfo?(mwu)
Flags: needinfo?(jz.zhangjin)

Comment 26

4 years ago
(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #16)
> vliu, additional questions.
> - Is the target device has gralloc?
YES on goflip.
> - Is the target device has hardware composer?
YES on goflip.

Currently we use flame(msm8610)/go_flip(msm8909) devices to test how to disable GPU, but the target chip may not have gralloc or hardware composer. Not sure as I haven't got code yet.

(In reply to Sotaro Ikeda [:sotaro PTO 5/Oct - 14/Oct intermittent during PTO] from comment #22)
> I also wanted to know the following question.
>  - Which color format is supported as frame buffer?

RGB565.

And Sotaro, how do I set 
      - layers.gralloc.disable       -> true
on fxos2.2? Can't find this pref in gecko/b2g/app/b2g.js
Flags: needinfo?(jz.zhangjin) → needinfo?(sotaro.ikeda.g)
Hi Jin Zhan,

You can just add the following to b2g.js. But just changing the pref did not work on v2.2 flame-kk. b2g process crash during booting. I am going to investigate it.
 > pref("layers.gralloc.disable", true);
Flags: needinfo?(sotaro.ikeda.g)
Confirmed that BasicCompositor working by applying the patch on b2gv2.2 flame-kk.
Depends on: 1215027

Updated

4 years ago
Flags: needinfo?(jz.zhangjin)

Comment 29

a year ago
Firefox OS is not being worked on
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → WONTFIX

Updated

5 months ago
Flags: needinfo?(jz.zhangjin)
You need to log in before you can comment on or make changes to this bug.