Closed Bug 951531 Opened 8 years ago Closed 8 years ago

framebuffer to use RGB565 16bit

Categories

(Firefox OS Graveyard :: General, defect)

x86
macOS
defect
Not set
normal

Tracking

(blocking-b2g:1.3T+, b2g-v1.3T fixed)

RESOLVED FIXED
blocking-b2g 1.3T+
Tracking Status
b2g-v1.3T --- fixed

People

(Reporter: jcheng, Assigned: james.zhang)

References

Details

(Whiteboard: [POVB][~300KB])

this bug is to investigate the memory saving to have framebuffer to use only RGB565 16bits
hi CJ, i believe Thomas talked to you on this one, can you have someone looking into this? Thanks
Blocks: 128RAM
Flags: needinfo?(cku)
Whiteboard: [tarako]
Thomas will discuss further with partner
Flags: needinfo?(ttsai)
Whiteboard: [tarako] → [tarako][POVB]
estimation:
The size of display buffer should be RGB888, by changing to RGB565, 1 byte reduce for each pixel
480(w) * 320(h) * 2 (double buffer) ~= 0.3 MB
Flags: needinfo?(cku)
9-patch image might be another way to reduce memory footprint.

By reducing display buffer from 888 to 565, the final rendering result will look pretty bad, especially meet gradient image(banding visual side effect)
spreadtrum has confirmed that kernel framebuffer is still 24bits color depth.
Flags: needinfo?(ttsai)
Can you try use rgb16 in b2g.js to reduce memory ?
pref("gfx.android.rgb16.force", true);
Flags: needinfo?(ttsai)
Since gecko only processes 24bit graphic buffer, no plan to reduce graphic buffer to 16bit in gecko.
Status: NEW → RESOLVED
Closed: 8 years ago
Flags: needinfo?(ttsai)
Resolution: --- → WONTFIX
Thomas, can you help me understand what this means? Why can we not use RGB565 buffers? We can certainly render to them.
Lets close this once we have a clear explanation here.
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
Tarako's LCD and framebuffer is 16-bit now.
Let's shrink gecko/gaia png resource, thanks.

I/[Gralloc](   82): using (fd=13)
I/[Gralloc](   82): id           = sc8810fb
I/[Gralloc](   82): xres         = 320 px
I/[Gralloc](   82): yres         = 480 px
I/[Gralloc](   82): xres_virtual = 320 px
I/[Gralloc](   82): yres_virtual = 960 px
I/[Gralloc](   82): bpp          = 16
I/[Gralloc](   82): r            = 11:5
I/[Gralloc](   82): g            =  5:6
I/[Gralloc](   82): b            =  0:5
I/[Gralloc](   82): width        = 45 mm (180.622223 dpi)
I/[Gralloc](   82): height       = 75 mm (162.559998 dpi)
In particular comment 6 remains unanswered. That pref should make use use 16-bit offscreen surfaces. I know we have done fallback rendering with RGB565 (no GL), since Android requires /dev/graphics/fb0 to be 16-bit. I am not sure we have ever used a GL framebuffer that is RGB565 but I am not sure why that shouldn't work in theory.
James, please see my post in the meta bug. PNG does not support RGB565 (16-bit). It only supports 8/8/8 (per spec). Dithering a PNG and then encoding it as 8/8/8 with the bottom 3/2/3 bits always 0 reduces the size by less than 1%. Trying to redo gecko/gaia png resources is pointless.

Using RGB565 surfaces is a different story. That can save memory, but it will only work for the frame buffer and surfaces without alpha.
Random thoughts:

*if* the resource usage of the PNGs is starting to be an issue, we can look into using compressed textures for Gecko's resources.  This would be a fairly lengthy change, and the on-disk size of the images will likely increase slightly.  However, in-memory size will go down.  But there are a couple of caveats:

- to get a software buffer out from them will require decoding by loading the texture into a GL texture and then drawing a quad.  (If we can get away with ETC images, then we can do a software decoder -- e.g. an ETC1 decoder is at http://code.google.com/p/rg-etc1/.)

- we'll have to teach anything that uses the images (via imglib) to never decode, but instead pass the raw data over that we can then load into a GL texture directly.  This fast path will only help if we are using SkiaGL for content rendering, as if we're doing software rendering, we'll have to decode to a software buffer.

This is not a trivial amount of work.
(In reply to Andreas Gal :gal from comment #11)
> In particular comment 6 remains unanswered. That pref should make use use
> 16-bit offscreen surfaces. I know we have done fallback rendering with
> RGB565 (no GL), since Android requires /dev/graphics/fb0 to be 16-bit. I am
> not sure we have ever used a GL framebuffer that is RGB565 but I am not sure
> why that shouldn't work in theory.

I get about ~1MB win in the parent process with this pref on. There's the usual noise in memory measurements but also a 600k improvement in gfx-surface-image.
(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #13)
> Random thoughts:
> 
> *if* the resource usage of the PNGs is starting to be an issue, we can look
> into using compressed textures for Gecko's resources.  This would be a
> fairly lengthy change, and the on-disk size of the images will likely
> increase slightly.  However, in-memory size will go down.  But there are a
> couple of caveats:
> 
> - to get a software buffer out from them will require decoding by loading
> the texture into a GL texture and then drawing a quad.  (If we can get away
> with ETC images, then we can do a software decoder -- e.g. an ETC1 decoder
> is at http://code.google.com/p/rg-etc1/.)
> 
> - we'll have to teach anything that uses the images (via imglib) to never
> decode, but instead pass the raw data over that we can then load into a GL
> texture directly.  This fast path will only help if we are using SkiaGL for
> content rendering, as if we're doing software rendering, we'll have to
> decode to a software buffer.
> 
> This is not a trivial amount of work.
I think the compressed texture may not compatible with gralloc buffer, since you have to assign special internalformat flag when call glTexImage2D to use it, but with gralloc buffer, we use glEGLImageTargetTexImage2DOES where the format is determined by the EGLImage. As a result, we can not use compressed texture(image layers) in B2G if we use gralloc buffer directly(which should always better than or at least the same with normal texture uploading since you will have 1 buffer in main memory and 1 same size buffer in GPU memory).
(In reply to Andreas Gal :gal from comment #8)
> Thomas, can you help me understand what this means? Why can we not use
> RGB565 buffers? We can certainly render to them.

I fail to see a reason that we can't render onto 565 target.

First of all, we need to get correct color depth, 16bits, on #106
http://dxr.mozilla.org/mozilla-central/source/gfx/thebes/gfxAndroidPlatform.cpp#106
We had verified that correct color depth value return in the latest img we got from James. "gfx.android.rgb16.force" is not really need here. 

Gecko will choose kEGLConfigAttribsRGB1 accordingly 
http://dxr.mozilla.org/mozilla-central/source/gfx/gl/GLContextProviderEGL.cpp#689

From the title of this bug, I think this bug should have been fixed by partner.
https://bugzilla.mozilla.org/show_bug.cgi?id=944457#c86
(In reply to James Zhang from comment #10)
> Tarako's LCD and framebuffer is 16-bit now.
> Let's shrink gecko/gaia png resource, thanks.
Unless we choose another kind of source image format, such as carry ETC1 in app package or using nine-patch image format, to replace png, down sample png is not really useful, I think Andreas had pointed out the reason.
I added a log in kernel and found that the size of framebuffer allocated is different. 

32bit: 
[    0.121651] sprdfb got 1228800 bytes mem at 0xc4c00000, [320*480@32]

16bit: 
[    0.120914] sprdfb got  614400 bytes mem at 0xc7300000, [320*480@16]

BTW, Tarako already used 16bit framebuffer now.
Additional information to comment 18. Now Tarako's kernel already use 16bit framebuffer. 
To measure the difference of memory usage for 16bit and 32bit framebuffer, I added a log and built another kernel which use 32bit framebuffer.
Whiteboard: [tarako][POVB] → [tarako-p2][POVB]
Fixed on my side.
Assignee: nobody → james.zhang
Status: REOPENED → RESOLVED
Closed: 8 years ago8 years ago
Resolution: --- → FIXED
Whiteboard: [tarako-p2][POVB] → [tarako-p2][POVB][~300KB]
1.3T+ and [POVB] for tracking purpose
blocking-b2g: --- → 1.3T+
Whiteboard: [tarako-p2][POVB][~300KB] → [POVB][~300KB]
Marking 1.3t fixed per comment 20
You need to log in before you can comment on or make changes to this bug.