Closed
Bug 1041112
Opened 10 years ago
Closed 10 years ago
Optimize larger image and canvas memory usage
Categories
(Firefox OS Graveyard :: Gaia::Homescreen, defect)
Tracking
(blocking-b2g:2.0+, b2g-v2.0 fixed, b2g-v2.1 fixed)
People
(Reporter: kgrandon, Assigned: khuey)
References
Details
(Keywords: memory-footprint, perf, Whiteboard: [MemShrink][systemsfe])
Attachments
(1 file, 2 obsolete files)
We would like to use larger images in the gaia homescreen on memory constrained devices. Due to bug 1029902 we had to reduce the size for 256mb devices. On a memory constrained flame for example we had to reduce image size from 126px to 64px due to memory usage. The original 126px accounted for three columns, and display density. This is likely a meta bug.
Reporter | ||
Comment 1•10 years ago
|
||
Benoit - Is there anything we can do here or are there any open bugs that might help this?
Flags: needinfo?(bgirard)
Comment 2•10 years ago
|
||
I really think we shouldn't trigger accelerated canvas unless the canvas is attached to a document. There are many more cases where it's slower and causes memory issues than there are advantages for using gpu acceleration for off-screen canvases.
Comment 4•10 years ago
|
||
(In reply to Chris Lord [:cwiiis] from comment #2) > I really think we shouldn't trigger accelerated canvas unless the canvas is > attached to a document. There are many more cases where it's slower and > causes memory issues than there are advantages for using gpu acceleration > for off-screen canvases. The problem is you might do a drawing command to it and later attach it to the screen and drive it. Ideally what we need is the ability to transfer from software to hardware (and back would be nice to). Then if the canvas is being driven by rAF and/or updating at 60 FPS then we switch to the right backend for it. For gaia we should just be adding the moz use software flag (I don't recall the name but it's there).
Flags: needinfo?(bgirard)
Comment 5•10 years ago
|
||
(In reply to Benoit Girard (:BenWa) from comment #4) > The problem is you might do a drawing command to it and later attach it to > the screen and drive it. > > Ideally what we need is the ability to transfer from software to hardware > (and back would be nice to). Then if the canvas is being driven by rAF > and/or updating at 60 FPS then we switch to the right backend for it. While yes, we need the ability to switch, I don't think people commonly create canvases, draw to them, then attach them to a document and continue to draw to them - at least, not in situations where not having hardware acceleration would be bad (I can imagine doing that for generated images, but in those cases, the updates are infrequent and likely faster in software). That said, it's conjecture, numbers would be nice. > For gaia we should just be adding the moz use software flag (I don't recall > the name but it's there). Ah, well, we should definitely be using this then...
Assignee | ||
Comment 7•10 years ago
|
||
I'll test using that on the homescreen later and see what it does.
Comment 8•10 years ago
|
||
(In reply to Benoit Girard (:BenWa) from comment #4) > (In reply to Chris Lord [:cwiiis] from comment #2) > > I really think we shouldn't trigger accelerated canvas unless the canvas is > > attached to a document. There are many more cases where it's slower and > > causes memory issues than there are advantages for using gpu acceleration > > for off-screen canvases. > > The problem is you might do a drawing command to it and later attach it to > the screen and drive it. > > Ideally what we need is the ability to transfer from software to hardware We don't have this ability now, but it would be easy to do. There isn't very much state held in the actual DrawTarget. > (and back would be nice to). We do have this ability already (going from hardware -> software) > Then if the canvas is being driven by rAF > and/or updating at 60 FPS then we switch to the right backend for it. I don't see how the update frequency or rAF usage could determine what backend we use. It really just depends on the type of workload. If you are doing a bunch of getImageData/putImageData, you want software. If it's a bunch of drawImage(), you want hardware. If it's in between, well, we'd have to figure something out. > > For gaia we should just be adding the moz use software flag (I don't recall > the name but it's there). The long term plan was to use this flag for the initial state only, and then switch to hardware/software later as defined by the heuristic. We could also consider a canvas not attached to a document as implicitly having this flag. If you attach and start doing drawImage() frequently we could then upgrade it.
Comment 9•10 years ago
|
||
Doing this heuristic-based thing would be a great project for an intern, I think. Others simply haven't had time to look into it.
Assignee | ||
Comment 10•10 years ago
|
||
Assignee | ||
Comment 11•10 years ago
|
||
Assignee | ||
Comment 12•10 years ago
|
||
Unfortunately setting willReadFrequently: true seems to just move the unaccounted memory from one spot (the GL stuff) to another (something in libpng). heap-unclassified does drop about 2 MB, but the amount of memory reported under imagelib goes up 1 MB.
Assignee | ||
Comment 13•10 years ago
|
||
Actually, that appears to be an artifact of not having discarded images. If I force that things look a lot better.
Assignee | ||
Comment 14•10 years ago
|
||
Attachment #8459236 -
Attachment is obsolete: true
Attachment #8459237 -
Attachment is obsolete: true
Assignee | ||
Comment 15•10 years ago
|
||
A bit of testing shows that setting willReadFrequently: true has a pretty similar effect to Vivien's patch from bug 1029902 once images are discarded.
Assignee | ||
Updated•10 years ago
|
Attachment #8459242 -
Flags: feedback?(kgrandon)
Attachment #8459242 -
Flags: feedback?(21)
Reporter | ||
Comment 16•10 years ago
|
||
Comment on attachment 8459242 [details] [review] PR to test willReadFrequently I'll leave an R+ in case you want to land this. It sounds like it should fix the first-time load of the homescreen, but would not impact subsequent loads where we do not leverage canvas? I wonder if this shouldn't be it's own bug.
Attachment #8459242 -
Flags: feedback?(kgrandon) → review+
Assignee | ||
Comment 17•10 years ago
|
||
I didn't take detailed measurements but it looked like it was helping both first run and second run pretty substantially.
Updated•10 years ago
|
Blocks: CAF-v2.0-FC-metabug
blocking-b2g: --- → 2.0?
Updated•10 years ago
|
Updated•10 years ago
|
Whiteboard: [MemShrink] → [MemShrink][systemsfe]
Assignee | ||
Comment 18•10 years ago
|
||
I took 16 sets of measurements: http://people.mozilla.org/~khuey/new-measurements.tgz The 4 configuration tested were: 'baseline': tip of gaia/gecko 'big-icons' tip of gaia/gecko with kgrandon's changes to icon sizes backed out 'no-hw-accel': big-icons + willReadFrequently 'both': baseline + willReadFrequently With two variables: first run vs. second run --minimize vs. not Take aways: The amount of memory used by images seems to be highly variable, probably based on what has been discarded/redecoded whenever the about:memory report runs. In some cases we appear to have decoded substantially more images than the 47 icons the homescreen has in an engineering build. If you compare baseline (which is just the smaller icons) vs. no-hw-accel (which is just the willReadFrequently), heap-unclassified drops by a couple MB in every comparison. In the unminimized cases the memory used by images is higher. The heap-unclassified in the baseline second-run case is in the PNG decoder, while in the first-run case it's in the SkiaGL stuff. With willReadFrequently neither of these sets of stacks appear in the DMD log. In the 1st run no-minimize case the smaller icons vs. willReadFrequently is effectively a wash: -0.17 MB (100.0%) -- explicit ├──-4.13 MB (2402.90%) ── heap-unclassified ├───4.11 MB (-2391.22%) -- images/content/raster In the 2nd run no-minimze case the heap-unclassified is close to a wash, and the image memory has increased by 11 MB. This makes little sense, because the theoretical maximum image size is 182 px * 182 px * 4 bytes per px^2 * 47. And we see that there are 80-something images floating around. I don't have an explanation for this.
Assignee | ||
Comment 19•10 years ago
|
||
So I took another measurement of the 2nd run no-minimize case and this time the image memory has increased by close to 6 MB which is pretty close to (182^2 - 64^2) * 4 * 47.
Comment 20•10 years ago
|
||
Great information, so I'm starting to get lost a bit :) So far, do we have a combination of settings and approaches that would let us use large icons with comparable memory use of the current baseline (small icons), or at least, get us to the "acceptable" amount of memory used?
Assignee | ||
Updated•10 years ago
|
Assignee | ||
Comment 21•10 years ago
|
||
(In reply to Milan Sreckovic [:milan] from comment #20) > Great information, so I'm starting to get lost a bit :) So far, do we have > a combination of settings and approaches that would let us use large icons > with comparable memory use of the current baseline (small icons), or at > least, get us to the "acceptable" amount of memory used? Not comparable. But we can use the large icons for 5-6 MB more when we're painting the homescreen. Whether or not that's acceptable I don't know.
Comment 22•10 years ago
|
||
master: https://github.com/mozilla-b2g/gaia/commit/09b9aff6f288584e6be5eaaae504e395451da447
Status: NEW → RESOLVED
Closed: 10 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → 2.1 S1 (1aug)
Updated•10 years ago
|
blocking-b2g: 2.0? → 2.0+
Comment 23•10 years ago
|
||
v2.0: https://github.com/mozilla-b2g/gaia/commit/83fc38ade5f3a1303340c315ef26acb2026f2de6
status-b2g-v2.0:
--- → fixed
status-b2g-v2.1:
--- → fixed
Comment 24•10 years ago
|
||
Comment on attachment 8459242 [details] [review] PR to test willReadFrequently I guess this does not need my feedback anymore.
Attachment #8459242 -
Flags: feedback?(21)
You need to log in
before you can comment on or make changes to this bug.
Description
•