Open Bug 651060 Opened 9 years ago Updated 7 months ago

Decode JPEGs to YCbCr and do RGB conversion on the GPU

Categories

(Core :: Graphics, defect)

defect
Not set

Tracking

()

People

(Reporter: cjones, Assigned: ctai)

References

Details

(Keywords: mobile, perf, Whiteboard: [MemShrink:P2])

Rob notes that

For JPEG images, we could also decode to YCbCr format (i.e. skip the final
YCbCr-to-RGB conversion in the imglib decoder), and do the final YCbCr
conversion using the GPU at draw time. That would save half the memory since
JPEG subsamples Cb/Cr. We already have all the layer-side support for this.
Keywords: mobile, perf
Color correction makes this more difficult because we would need to do color correction on draw instead of decode. Mobile doesn't use color correction so it's not a problem there.

I think we would also probably only want to do this for large images, but that would need to be measured.

The memory win is probably the most appealing part of this, so it might even be interesting to investigate doing this even without using the GPU, at the same time I wouldn't rush to work on this without more evidence that it will provide big wins.
As somebody mentioned on irc some time ago, it should be possible to first do "decode -> YCbCr-to-RGB -> color correction -> RGB-to-YCbCr" (which can be possibly optimized to less stages), and then "YCbCr-to-RGB" when drawing (using NEON or GPU). This way we still have color correction and memory saving.
Depends on: 750172
No longer depends on: 750172
(In reply to Siarhei Siamashka from comment #2)
> As somebody mentioned on irc some time ago, it should be possible to first
> do "decode -> YCbCr-to-RGB -> color correction -> RGB-to-YCbCr" (which can
> be possibly optimized to less stages), and then "YCbCr-to-RGB" when drawing
> (using NEON or GPU). This way we still have color correction and memory
> saving.

Or just do color correction at GPU-conversion-time, as a suffix to YCbCr-to-RGB.

There's no reason we have to do this at draw time, either. Bug 750994 is an optimization to do GPU-size texture conversions at upload time.
Not to suggest that this isn't a worthwhile cause, but I don't think this should block bug 683284.  Bug 683284 comment 1 says "I'd like this bug to only address bugs where our behaviour is a clearly wrong compared to other browsers."
No longer blocks: image-suck
Whiteboard: [MemShrink]
Whiteboard: [MemShrink] → [MemShrink:P2]
IE11 does this: https://blogs.msdn.com/b/ie/archive/2013/09/12/using-hardware-to-decode-and-load-jpg-images-up-to-45-faster-in-internet-explorer-11.aspx

High quality image downsampling also can be done in subsampled YCbCr space (just treat each channel separately, maybe doing just luma and not bothering with chroma).
I would like to do some experiments on this bug for evaluating the benefit and cost on Firefox OS.
Assignee: nobody → ctai
You need to log in before you can comment on or make changes to this bug.