1038884 - Homescreen uses a lot ION memory in 2.0 compared to v1.3

Status: RESOLVED FIXED

(Core :: Graphics: Layers, defect, P1)

Description

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

+++ This bug was initially created as a clone of Bug #1029902 +++

Bug 1029902 is tracking additional USS memory regression for homescreen issue compared to v1.3


This bug is for tracking additional ION memory regression issue (12MB) compared to v1.3 for homescreen process 

More details and logs is in bug 1029902 comment 34

Comment 1

[Eric Rahm [:erahm]]

If I'm reading those logs right, homescreen in 2.0 is using a total of 12MB ION memory. What was it using in 1.3?

Comment 2

[Milan Sreckovic [:milan] (needinfo for best results)]

According to the original report, 1.3 was using 8MB, 2.0 is using 12MB more, for a total of 20MB.

Comment 3

[Eric Rahm [:erahm]]

(In reply to Milan Sreckovic [:milan] from comment #2)
> According to the original report, 1.3 was using 8MB, 2.0 is using 12MB more,
> for a total of 20MB.

Those are system totals AFAICT. If I break out just Homescreen entries, I see 12MB total usage.

Comment 4

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

(In reply to Eric Rahm [:erahm] from comment #3)
> (In reply to Milan Sreckovic [:milan] from comment #2)
> > According to the original report, 1.3 was using 8MB, 2.0 is using 12MB more,
> > for a total of 20MB.
> 
> Those are system totals AFAICT. If I break out just Homescreen entries, I
> see 12MB total usage.

Can you please compare betwen v1.3 data from bug 1029902 Comment 15 and v2.0 data from bug 1029902 comment 34.


v1.3 horizontal homescreen is using very less ion memory.

Comment 5

[Eric Rahm [:erahm]]

This bug is for just Homescreen, but those values are for total system memory. The system process may have increased usage as well.

Comment 6

[Benoit Girard (:BenWa)]

Aren't KGSL and Ion closely related? I'm not sure how this bug differs from bug 1030608.

Comment 7

[Milan Sreckovic [:milan] (needinfo for best results)]

Is it possible that all the icon processing, that uses canvas, ends up using SkiaGL and the extra memory is the cache?  Is there less memory used if we disable SkiaGL - pref("gfx.canvas.azure.accelerated", false);

Comment 8

[Mason Chang [Inactive] [:mchang]]

Triage: From your analysis, do you know if bug 1030608 will cover everything in this bug? If so, can you please mark this bug as dupe 1030608. Thanks!

Comment 9

[Milan Sreckovic [:milan] (needinfo for best results)]

Why don't we fix bug 1030608 and then see.

Comment 10

[Eric Rahm [:erahm]]

Tapas, can you attach an ION memory report for 1.3? That should help give some insight into whether the kgsl increase maps to the ION increase.

Comment 11

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

(In reply to Eric Rahm [:erahm] from comment #10)
> Tapas, can you attach an ION memory report for 1.3? 

please compare between v1.3 data from bug 1029902 Comment 15 and v2.0 data from bug 1029902 comment 34.
This should also tell why ION memory is different from kgsl page_alloc. 

> That should help give
> some insight into whether the kgsl increase maps to the ION increase.

ION memory usage does not map to kgsl page_alloc( bug 1030608 comment 20) . 


More about ION implementation is in : http://lwn.net/Articles/480055/

Comment 12

[Chris Lord [:cwiiis]]

(In reply to Milan Sreckovic [:milan] from comment #7)
> Is it possible that all the icon processing, that uses canvas, ends up using
> SkiaGL and the extra memory is the cache?  Is there less memory used if we
> disable SkiaGL - pref("gfx.canvas.azure.accelerated", false);

I think we just get the canvas data uri and get rid of it after generation, so probably  not... Worth noting that my suggested change of only using gl-canvas for canvas elements attached to a document would have fixed this though (ftr, I still think this is a good idea)

Comment 13

[:gerard-majax]

Tampas, I've had a look at several of the bugs you linked about measuring this ION memory usage, but I could not find a proper set of steps to perform and get figures we can definitively compare.

I had a look on my Flame, running v122 base image. What I can observe at least for sure is:
 - v1.3 horizontal homescreen, 7.1MB ION use
 - v2.1 horizontal homescreen, 8.5MB ION use
 - v2.1 vertical homescreen, 11.5MB ION use

I did focus on the output of |/sys/kernel/debug/ion/iommu| to get those figures. It mixes all processes, but on all my runs, I could see no variation on b2g process and mdss_fb0 process. All of the changes are Homescreen-related.

So my figures would point to a regression but that may bve spread on gecko and the homescreen itself.

Comment 14

[Jon Hylands [:jhylands]]

Benoit,

So https://bugzilla.mozilla.org/show_bug.cgi?id=1030608#c20 says that kgsl memory is associated with the graphics driver. https://bugzilla.mozilla.org/show_bug.cgi?id=1029902#c34 also talks about kgsl memory and ion memory as apparently different things.

I think this answers your question in comment 6, correct?

Comment 15

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

(In reply to Alexandre LISSY :gerard-majax from comment #13)
> Tampas, I've had a look at several of the bugs you linked about measuring
> this ION memory usage, but I could not find a proper set of steps to perform
> and get figures we can definitively compare.
> 
> I had a look on my Flame, running v122 base image. What I can observe at
> least for sure is:
>  - v1.3 horizontal homescreen, 7.1MB ION use

Sorry, I am seeing it only ~3MB ION which is much less than 7.1 MB ION. 
Full output of |adb shell cat /sys/kernel/debug/ion/iommu| from bug 1029902 Comment 15 v1.3 logs:

adb shell b2g-procrank && adb shell cat /sys/kernel/debug/ion/iommu && adb shell cat /sys/class/kgsl/kgsl/page_alloc
APPLICATION        PID      Vss      Rss      Pss      Uss  cmdline
b2g                279   80584K   68460K   56510K   49996K  /system/b2g/b2g
Homescreen         996   31348K   28256K   17642K   12232K  /system/b2g/plugin-container
(Nuwa)             948   20440K   20436K    9343K    3848K  /system/b2g/plugin-container
(Preallocated a   1087   17264K   17260K    8306K    4324K  /system/b2g/plugin-container
                                           ------   ------  ------
                                          115311K   90656K  TOTAL
          client              pid             size
----------------------------------------------------
fd900000.qcom,mdss_mdp                1          1536000
     adsprpc-smd                1             4096
----------------------------------------------------
orphaned allocations (info is from last known client):
      Homescreen              996          1290240 0 1
      Homescreen              996          1290240 0 1
      Homescreen              996           245760 0 1
      Homescreen              996             8192 0 1
      Homescreen              996             8192 0 1
        mdss_fb0              368          1536000 0 1
             b2g              279          1536000 0 1
             b2g              279          1536000 0 1
      Homescreen              996            20480 0 1
      Homescreen              996           245760 0 1
      Homescreen              996            20480 0 1
----------------------------------------------------
  total orphaned          7737344
          total           9277440
----------------------------------------------------
Cached Pools:
0 order 9 highmem pages in pool = 0 total
0 order 9 lowmem pages in pool = 0 total
0 order 8 highmem pages in pool = 0 total
0 order 8 lowmem pages in pool = 0 total
0 order 4 highmem pages in pool = 0 total
0 order 4 lowmem pages in pool = 0 total
0 order 0 highmem pages in pool = 0 total
1160 order 0 lowmem pages in pool = 488000 total
Uncached Pools:
0 order 9 highmem pages in pool = 0 total
0 order 9 lowmem pages in pool = 0 total
0 order 8 highmem pages in pool = 0 total
0 order 8 lowmem pages in pool = 0 total
0 order 4 highmem pages in pool = 0 total
0 order 4 lowmem pages in pool = 0 total
0 order 0 highmem pages in pool = 0 total
0 order 0 lowmem pages in pool = 0 total
Total bytes in pool: 488000
3293184


if you still see 7.1 MB ion allocation in v1.3 then please point me to your gaia/gecko revision number in [1] and also full output of |adb shell cat /sys/kernel/debug/ion/iommu| on your 512MB flame (msm8610) running v1.3 JB based FFOS.

[1] https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gecko/
    https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gaia



>  - v2.1 horizontal homescreen, 8.5MB ION use
>  - v2.1 vertical homescreen, 11.5MB ION use
> 

I collected logs only for v2.0 vertical homescreen in bug 1029902 comments 34

I can see following from attached logs for v2.0 vertical homescreen in bug 1029902 comments 34:

tkundu@tkundu-linux:/net/cros-bs-p4/local/mnt/workspace/tkundu/kk36t14th$ adb shell b2g-procrank && adb shell cat /sys/kernel/debug/ion/heaps/system && adb shell cat /sys/class/kgsl/kgsl/page_alloc
APPLICATION        PID       Vss      Rss      Pss      Uss  cmdline
b2g                230   237112K   78820K   61309K   52872K  /system/b2g/b2g
Homescreen        1093   125824K   48320K   32865K   26168K  /system/b2g/plugin-container
Built-in Keyboa   1069    70156K   21700K   10659K    6536K  /system/b2g/plugin-container
(Preallocated a   1098    60180K   18848K    8837K    5196K  /system/b2g/plugin-container
(Nuwa)             992    53940K   20368K    7001K    2144K  /system/b2g/plugin-container
                                            ------   ------  ------
                                           146005K  113160K  TOTAL
          client              pid             size
----------------------------------------------------
     mediaserver              225           434176
     mediaserver              225           434176
       voc_cal-0               31             4096
   voip_client-0               31             8192
   voip_client-1               31             4096
   adsprpc-smd-0                1             8192
fd900000.qcom,mdss_mdp-0                1          1536000
----------------------------------------------------
orphaned allocations (info is from last known client):
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
      Homescreen             1093           294912 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
      Homescreen             1093           294912 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
             b2g              230          1536000 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
      Homescreen             1093           294912 0 1
             b2g              230          1536000 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
        mdss_fb0              463          1536000 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
             b2g              230             4096 0 1
             b2g              230          1474560 0 1
      Homescreen             1093           294912 0 1
             b2g              230             4096 0 1
      Homescreen             1093          1474560 0 1
             b2g              230             4096 0 1
      Homescreen             1093           294912 0 1
----------------------------------------------------
  total orphaned         16252928
          total          18247680
----------------------------------------------------
0 order 9 highmem pages in uncached pool = 0 total
0 order 9 lowmem pages in uncached pool = 0 total
0 order 8 highmem pages in uncached pool = 0 total
0 order 8 lowmem pages in uncached pool = 0 total
0 order 4 highmem pages in uncached pool = 0 total
0 order 4 lowmem pages in uncached pool = 0 total
0 order 0 highmem pages in uncached pool = 0 total
0 order 0 lowmem pages in uncached pool = 0 total
0 order 9 highmem pages in cached pool = 0 total
0 order 9 lowmem pages in cached pool = 0 total
0 order 8 highmem pages in cached pool = 0 total
1 order 8 lowmem pages in cached pool = 1048576 total
0 order 4 highmem pages in cached pool = 0 total
32 order 4 lowmem pages in cached pool = 2097152 total
0 order 0 highmem pages in cached pool = 0 total
109 order 0 lowmem pages in cached pool = 446464 total
6762496


So the total ION allocation from homescreen is >10MB

Comment 16

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

So ION regression is still 7MB in 2.0 (vertical homescreen) compared to v1.3 .

I can also see that fix from bug 1039883 Comment 11 is clearing all homescreen ion allocations if we put homescreen in background. This will save 12MB ion allocations if we put homescreen in background. 


But if homescreen is in foreground then we will still see 7MB additional ION memory usage (compared to v1.3) which will kill background apps more frequently in v1.3. 

My guess is that all of these ION allocations is coming from Tile layers of homescreen app and we may want to do some optimization there.

Comment 17

[:gerard-majax]

Tapas, thanks, you are right I was mislead regarding the reading of the values: I mixed up between the total and only homescreen.

So this is the value I'm getting on v1.3:
> [Homescreen] Process.pid Homescreen.960 -- 2014-07-18 10:25:41.140886
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 2
> [Homescreen] --->>> Total: 3194880
> [Homescreen] --->>> Elems: 2
> [Homescreen] --->>>

Also, I have the very same result with either tiling enabled or not, there is no difference.

Comment 18

[:gerard-majax]

Running old homescreen on top of master from yesterday:

> [Homescreen] Process.pid Homescreen.961 -- 2014-07-18 10:30:58.841424
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 4
> [Homescreen] --->>> Total: 2887680
> [Homescreen] --->>> Elems: 4
> [Homescreen] --->>>

I made sure that tiling was enabled.

Comment 19

[:gerard-majax]

(In reply to Alexandre LISSY :gerard-majax from comment #18)
> Running old homescreen on top of master from yesterday:
> 
> > [Homescreen] Process.pid Homescreen.961 -- 2014-07-18 10:30:58.841424
> > [Homescreen] --->>>
> > [Homescreen] --->>> Count: 4
> > [Homescreen] --->>> Total: 2887680
> > [Homescreen] --->>> Elems: 4
> > [Homescreen] --->>>
> 
> I made sure that tiling was enabled.

And this is after just locking/unlocking the screen:

> [Homescreen] Process.pid Homescreen.961 -- 2014-07-18 10:32:14.520301
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 7
> [Homescreen] --->>> Total: 4362240
> [Homescreen] --->>> Elems: 7
> [Homescreen] --->>>

Comment 20

[:gerard-majax]

On the same system, after doing a reset-gaia:

> [Homescreen] Process.pid Homescreen.953 -- 2014-07-18 10:37:20.461062
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 20
> [Homescreen] --->>> Total: 5898240
> [Homescreen] --->>> Elems: 20
> [Homescreen] --->>>

In all case, I panned on the homescreen then sat on the main page loaded with most icons. It looks like the vertical homescreen in my master consumes more tiles but that those are smaller. That still makes a total of :
 - 5.8MB ION memory for Vertical Homescreen on master,
 - 4.3MB ION memory for old Homescreen on master,
 - 2.8MB ION memory for old Homescreen on v1.3 from v122 base image

Comment 21

[:gerard-majax]

> [Homescreen] Process.pid Homescreen.961 -- 2014-07-18 10:48:28.578345
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 22
> [Homescreen] --->>> Total: 6488064
> [Homescreen] --->>> Elems: 22
> [Homescreen] --->>>

That is with the very same build but after pushing the proper fonts.

Comment 22

[:gerard-majax]

> [Homescreen] Process.pid Homescreen.945 -- 2014-07-18 11:20:42.724199
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 28
> [Homescreen] --->>> Total: 8257536
> [Homescreen] --->>> Elems: 28
> [Homescreen] --->>>

And this is with simple tiling enabled.

Comment 23

[:gerard-majax]

Attachment: Screenshot with tiling enabled, and new fonts

I feel like there are some useless tiles on this screenshot.

Comment 24

[Kevin Grandon :kgrandon]

Thanks for the investigation here Alex. I'm not sure what we can do from the homescreen side of things regarding tiling, so moving to graphics for now until we have more info from grapics guys. If there is any action items for homescreen specifically we'll take this back or file a new bug.

Comment 25

[:gerard-majax]

Attachment: Screenshot with tiling enabled, and old fonts

Comment 26

[:gerard-majax]

Attachment: Layers dump with the old fonts

Comment 27

[:gerard-majax]

Attachment: Layers dump with the new fonts

Comment 28

[Nicolas Silva [:nical]]

(In reply to Alexandre LISSY :gerard-majax from comment #23)
> Created attachment 8458552 [details]
> Screenshot with tiling enabled, and new fonts
> 
> I feel like there are some useless tiles on this screenshot.

I think that the thinner lines are not showing tile boundaries but region boundaries within tiles: there are empty parts in some of the tiles so we don't paint them and we composite subregions of some of the tiles separately to not composite the empty subregions. AFAICT this is not showing more tiles or more memory, although it does show more draw calls (which is not the problem at hand).

Comment 29

[:gerard-majax]

Attachment: settings_oldfonts.png

Comment 30

[:gerard-majax]

Attachment: settings_newfonts.png

Comment 31

[Gregor Wagner [:gwagner]]

Assigning to Alex since he is investigating. Please re-assign if there is a better owner.

Comment 32

[:gerard-majax]

I hacked libgralloc to show the ion allocation, mapping, unmapping and deallocations.

Allocated buffers:
> alex@portable-alex:~/codaz/Mozilla/b2g/devices/Flame/B2G$ grep -c Allocated ion-master-*.log
> ion-master-homescreen.log:44
> ion-master-verticalhome.log:60

Freeing buffers:
> alex@portable-alex:~/codaz/Mozilla/b2g/devices/Flame/B2G$ grep -c Freeing ion-master-*.log
> ion-master-homescreen.log:40
> ion-master-verticalhome.log:44

So with the old homescreen running on top of master, I get a net delta of 4 ion buffers, while I have 16 for the vertical homescreen.

Comment 33

[:gerard-majax]

Impact of Tiling:

Vertical homescreen, without tiling:
> [Homescreen] Process.pid Homescreen.908 -- 2014-07-18 16:12:19.256119
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 2
> [Homescreen] --->>> Total: 4177920
> [Homescreen] --->>> Elems: 2
> [Homescreen] --->>>

Vertical homescreen, with tiling:
> [Homescreen] Process.pid Homescreen.928 -- 2014-07-18 16:14:38.365620
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 18
> [Homescreen] --->>> Total: 5308416
> [Homescreen] --->>> Elems: 18
> [Homescreen] --->>>

In both case, rebooting, swipping twice down and up. Tiling would account for 1.2MB of ION memory use.

Comment 34

[:gerard-majax]

Impact of Tiling:

Old homescreen, without tiling:
> [Homescreen] Process.pid Homescreen.905 -- 2014-07-18 16:19:42.013106
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 4
> [Homescreen] --->>> Total: 2887680
> [Homescreen] --->>> Elems: 4
> [Homescreen] --->>>

Old homescreen, with tiling:
> [Homescreen] Process.pid Homescreen.1220 -- 2014-07-18 16:17:30.686925
> [Homescreen] --->>>
> [Homescreen] --->>> Count: 7
> [Homescreen] --->>> Total: 4362240
> [Homescreen] --->>> Elems: 7
> [Homescreen] --->>>

Again, swipping twice right and left, and staying on the page with the most icons.

Comment 35

[:gerard-majax]

From comment 33 and comment 34, we can see that tiling would account for 1.40MB with vertical homescreen and 1.07MB for the old homescreen of ION memory use in master.

Comment 36

[Kevin Grandon :kgrandon]

Sounds good, thanks for the investigation here. Seems like we should get some graphics guys involved as well. There's already a ni? on Benoit it seems.

Comment 37

[Milan Sreckovic [:milan] (needinfo for best results)]

If it's tiling, then it's probably bug 1039883.

Comment 38

[:gerard-majax]

(In reply to Milan Sreckovic [:milan] from comment #37)
> If it's tiling, then it's probably bug 1039883.

My figures are when homescreen is visible. The description of bug 1039883 talks about background, so I'm unsure :(

Comment 39

[Benoit Girard (:BenWa)]

It's not bug 1039883.

Note that there is overhead to using tiling. I have bug 1036654 to measure what the overhead is and it agrees with the numbers we have collected. It's a trade off between using a surface that's larger but already allocated  and one that can be resized for free as you grow vs. doing a specific allocation for the size you need and having to do a lot of expensive work when the layer changes size in the slightly.

I wouldn't aim to get this regression down to zero. We can do it but then we're going to lose most of the performance advantage of tiling. You certainly should play around with the tile size for the device:
layers.tile-*

Having the number evenly divide the screen width/height should help. Making that number too low will make allocations and drawing a layer expensive.

Comment 40

[:gerard-majax]

I'd like to make use of the libgralloc ION debugging to be able to live spy the use of resources. It is my understanding that the following happens:
 - allocation/freeing of ION memory is performed by the B2G main process
 - mapping/unmapping of ION memory is performed by the B2G app child process

My goal would then be to spy the logcat output for those message and be able to properly track the allocation and use of ION memory on each homescreen we have: this would be more resilient than the first hack I did, which relies on the aggregated output of debugfs taken at some point.

Comment 41

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

(In reply to Alexandre LISSY :gerard-majax from comment #40)
> I'd like to make use of the libgralloc ION debugging to be able to live spy
> the use of resources. It is my understanding that the following happens:
>  - allocation/freeing of ION memory is performed by the B2G main process
>  - mapping/unmapping of ION memory is performed by the B2G app child process
> 
> My goal would then be to spy the logcat output for those message and be able
> to properly track the allocation and use of ION memory on each homescreen we
> have: this would be more resilient than the first hack I did, which relies
> on the aggregated output of debugfs taken at some point.

ION driver provides debug visibility through debugfs. It organizes debug information under /sys/kernel/debug/ion, with bookkeeping information in stored files associated with heaps and clients identified by symbolic names or PIDs. 

ION allocation tutorial: http://lwn.net/Articles/480055/

For example: I check /sys/kernel/debug/ion/iommu for JB based FFOS v1.3 and /sys/kernel/debug/ion/heaps/system for KK based FFOS 2.0. 

it can help you to see actual ION allocation from homescreen process during run time. But we still need to find out the place in gecko layer tree code where these allocation/mapping/unmapping occurs for homescreen process.

I tried with tile-size but didn't see big gain.

Comment 42

[:gerard-majax]

Attachment: ion_fromstart.png

Comment 43

[:gerard-majax]

Attachment: spy_ion.py

This python code will spy on the ION debugging messages spawns in logcat (libgralloc rebuilt for this)

Comment 44

[:gerard-majax]

Attachment: anal_ion.py

This python code will analyze the file produced by spy_ion.py and produce plots.

Comment 45

[:gerard-majax]

Attachment: ion_fromstart_v1.4_20140721000201.png

This is the result on a B2G v1.4, buildid 20140721000201

Comment 46

[:gerard-majax]

Attachment: ion_fromstart_v2.0_20140721000201.png

Result with B2G v2.0 buildid 20140721000201

Comment 47

[:gerard-majax]

Attachment: ion_fromstart_v2.1_20140721062116.png

Result with B2G v2.1 buildid 20140721062116

Comment 48

[:gerard-majax]

For attachment 8459626 [details], attachment 8459614 [details] and attachment 8459610 [details], the steps performed were:
 - flash eng gecko/gaia build from pvtbuilds
 - reset /data/local/ and /data/b2g/
 - go through FTU
 - stop b2g
 - start spying
 - start b2g
 - unlock device, unlock second SIM card
 - swipe the full homescreen, showing everything at least once
 - stop spying

For some reason it seems we can see a big increase in mapped ION memory used by Homescreen on v2.0, but homescreen from v2.1 exposes a much better status. Kevin, any thought?

Comment 49

[Milan Sreckovic [:milan] (needinfo for best results)]

By now, both 2.0 and 2.1 should have patch from bug 1039883, which I would expect would have an effect.  Are all the icon changes propagated to 2.0?

Comment 50

[Kevin Grandon :kgrandon]

Hmm, no idea why 2.0 would be that different from 2.1. As far as homescreen code is concerned, almost everything that would have an impact has been uplifted.

Comment 51

[Milan Sreckovic [:milan] (needinfo for best results)]

Could be a bug in 2.1 :) - we seem to be getting stuck in low res tiles for at least parts of the homescreen, wonder if it's making a difference (bug 1041608)

Comment 52

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

(In reply to Alexandre LISSY :gerard-majax from comment #48)
> For attachment 8459626 [details], attachment 8459614 [details] and
> attachment 8459610 [details], the steps performed were:
>  - flash eng gecko/gaia build from pvtbuilds
>  - reset /data/local/ and /data/b2g/
>  - go through FTU
>  - stop b2g
>  - start spying
>  - start b2g
>  - unlock device, unlock second SIM card
>  - swipe the full homescreen, showing everything at least once
>  - stop spying
> 
> For some reason it seems we can see a big increase in mapped ION memory used
> by Homescreen on v2.0, but homescreen from v2.1 exposes a much better
> status. Kevin, any thought?

Could you please run same automation for v1.3 too ? I am just curious to know the diff between v1.3 and v2.0 . Thanks for your nice analysis.

Comment 53

[:gerard-majax]

This was next on my checklist but we don't have prebuilt gecko/gaia available for v1.3, so it will take some more time and that's why I first focused on those :)

Comment 54

[:gerard-majax]

(In reply to Milan Sreckovic [:milan] from comment #49)
> By now, both 2.0 and 2.1 should have patch from bug 1039883, which I would
> expect would have an effect.  Are all the icon changes propagated to 2.0?

It may not be the case: I just saw a v2.0 Flame build from today, and it still makes use of the three icons layout.

Comment 55

[Kevin Grandon :kgrandon]

(In reply to Alexandre LISSY :gerard-majax from comment #54)
> (In reply to Milan Sreckovic [:milan] from comment #49)
> > By now, both 2.0 and 2.1 should have patch from bug 1039883, which I would
> > expect would have an effect.  Are all the icon changes propagated to 2.0?
> 
> It may not be the case: I just saw a v2.0 Flame build from today, and it
> still makes use of the three icons layout.

The icon layout is only changed for devices running < 512mb memory. I'm guessing that the flame you saw it on was either not artificially memory constrained, or it has not updated to a recent build? It could also be possible that we did not properly migrate 256mb devices that have already had 2.0 applied to them to change the column layout.

Comment 56

[:gerard-majax]

(In reply to Kevin Grandon :kgrandon from comment #55)
> (In reply to Alexandre LISSY :gerard-majax from comment #54)
> > (In reply to Milan Sreckovic [:milan] from comment #49)
> > > By now, both 2.0 and 2.1 should have patch from bug 1039883, which I would
> > > expect would have an effect.  Are all the icon changes propagated to 2.0?
> > 
> > It may not be the case: I just saw a v2.0 Flame build from today, and it
> > still makes use of the three icons layout.
> 
> The icon layout is only changed for devices running < 512mb memory. I'm
> guessing that the flame you saw it on was either not artificially memory
> constrained, or it has not updated to a recent build? It could also be
> possible that we did not properly migrate 256mb devices that have already
> had 2.0 applied to them to change the column layout.

I just rechecked on my Flame: layout is 4 icons. Regarding memory, my kernel is booted with "mem=343m" and:

> root@flame:/ # dmesg  | grep 'Memory:'
> <6>[    0.000000] Memory: 136MB 120MB = 256MB total
> <5>[    0.000000] Memory: 226052k/226052k available, 37116k reserved, 0K highmem

And from /proc/meminfo,
> MemTotal:         242920 kB

Comment 57

[:gerard-majax]

Attachment: ion_fromstart_v1.3_20140722151916.png

Result with B2G v1.3 buildid 20140722151916

Comment 58

[:gerard-majax]

Attachment: ion_fromstart_v2.1_20140722153704_prod.png

Results for B2G v2.1, buildid 20140722153704 with a Gaia built with PRODUCTION=1

Comment 59

[:gerard-majax]

Attachment: ion_fromstart_v2.1_20140722153704_nonprod.png

Results for B2G v2.1, buildid 20140722153704 with a Gaia built with PRODUCTION=0

Comment 60

[:gerard-majax]

Tapas, would you mind checking attachement 8460198 for the same process applied to a non user build of B2G v1.3 ?

Also, attachment 8460218 [details] and attachment 8460220 [details] documents the comparison on Gaia's PRODUCTION build variable. One of the main difference between both is the number of icons available on the homescreen.

In both case, the used layout was 4 icons.

Comment 61

[:gerard-majax]

Attachment: ion_fromstart_v2.1_20140722153704_prod-lotsoficons-1G+tiles128x128.png

B2G v2.1 buildid 20140722153704 PRODUCTION=1, with quite a lot of icons.
Tiles 128x128

Comment 62

[:gerard-majax]

Attachment: ion_fromstart_v2.1_20140722153704_prod-lotsoficons-1G+tiles120x160.png

B2G v2.1 buildid 20140722153704 PRODUCTION=1, with quite a lot of icons.
Tiles 120x160

Comment 63

[:gerard-majax]

Attachment: ion_fromstart_v2.1_20140722153704_prod-lotsoficons-1G+tiles256x256.png

B2G v2.1 buildid 20140722153704 PRODUCTION=1, with quite a lot of icons.
Tiles 256x256

Comment 64

[:gerard-majax]

The attachements:
 - attachment 8460294 [details]
 - attachment 8460295 [details]
 - attachment 8460296 [details]

Exposes results on the very same build with the device non limited in memory, several dozens of apps installed from the marketplace. The only varying parameter was the size of tiles. Lower/Higher sizes seems to result in higher ION memory use:
 - 48x96 pushes to 10-11MB
 - 72x72 pushes to ~9.9MB
 - 512x512 pushes back to 11-12MB

Comment 65

[Chris Lord [:cwiiis]]

So I'll defer to the better judgement of gfx team, but I think it's out of the question to use tile sizes as small as 48x96 or 72x72. This would have a significant performance impact due to extra allocations and texture binds.

512x512 is probably too much for the average phone because of the possible tile-wastage (maximum wasted space in a tile is s*(t-1), where s and t are the two axes, with s being the axis with the shorter tile-length).

I think that 256x256 is a fair default size, what would be interesting to see is tile-sizes that are multiples of the screen width/height - for example, on the Flame, perhaps a tile-size of 240x214 might save us some memory. Similarly, on a Hamachi, a tile-size of 160x120 perhaps (a smaller screen means we can likely get away with a smaller tile-size).

All of this, ideally, should also be done in tandem with performance tests too - there may be an impact for using tile-sizes that aren't powers of two (performance and/or memory impact, depending on chipset).

Comment 66

[Milan Sreckovic [:milan] (needinfo for best results)]

I'd be worried about making the tiles non-square in 2.0 - just not enough time to deal with any possible issues related to that.

Comment 67

[Lucas Adamski [:ladamski]]

I can confirm that I still see a 3 column layout on a 2.0 7/21 build configured to 273MB.  If I flash a fresh copy of gecko and gaia (blow away the profile), I get the 4 column layout.  So it does seem to be an issue with existing profiles.. would it make sense to force that regardless of profile config?

(In reply to Kevin Grandon :kgrandon from comment #55)
> The icon layout is only changed for devices running < 512mb memory. I'm
> guessing that the flame you saw it on was either not artificially memory
> constrained, or it has not updated to a recent build? It could also be
> possible that we did not properly migrate 256mb devices that have already
> had 2.0 applied to them to change the column layout.

Comment 68

[:gerard-majax]

(In reply to Milan Sreckovic [:milan] from comment #66)
> I'd be worried about making the tiles non-square in 2.0 - just not enough
> time to deal with any possible issues related to that.

Sure, I was just taking the opportunity to get a picture of the impact.

So, so far, my latest measures do not show such a big delta on 2.0 vertical now that a lot of things have been uplifted: if I refer to comment 2, the increment was 12MB for vertical homescreen, and a total of 20MB ION memory used.

After speaking with :nical and as documented earlier by tweaking tiles sizes, the residual impact can be easily explained by the tiling being much more used.

Comment 69

[Nicolas Silva [:nical]]

clearing the needinfo flag, Chris explains the tiling situation pretty well. Tiling does use more memory in the common case if the layer size is not a multiple of the tile size. There is little to be done about that except tweaking the tile size, which can hurt performance, so we need to be careful about proving that the performance does not regress when modifying it the tile size for the sake of memory.

One thing that we could check is how many tiles have a back buffer, on static content: A tile's back buffer is created lazily when we invalidate an existing tile so if the layer is drawn once and never invalidated, tiles should not have a back-buffer. If there's something causing us to partially draw some content and redraw it the next frame, it will make us create back buffers which effectively doubles the tile's memory footprint, although the back buffer should be discarded after a second or so.

It shouldn't be too hard to hack together a way to see if a tile has one or two buffers.

Comment 70

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

Attachment: ion logs

(In reply to Alexandre LISSY :gerard-majax from comment #60)
> Tapas, would you mind checking attachement 8460198 for the same process
> applied to a non user build of B2G v1.3 ?
> 

Sorry. my main focus is on v2.0 now. 

I saw that multiple bugs related to homescreen memory usage got fixed and I re-measured homescreen ION memory usage today. 

I checked with following gaia/gecko 

https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gaia/commit/?h=mozilla/v2.0&id=bf3fb88039843359d0a5759b2c0fb787abae544f

https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gecko/commit/?h=mozilla/v2.0&id=e46acc521bf8e25787d9ed3937b0213774355c2d

Homescreen is still using 12MB ION memory in v2.0 but it was using 3MB ION in v1.3 (Comment 15)

Comment 71

[Andreas Gal :gal]

Can we time out backbuffers?

Comment 72

[:gerard-majax]

Attachment: anal_ion.py

Reworked plotting code.

Comment 73

[:gerard-majax]

Attachment: ion_tiles.png

> python anal_ion.py --output ion_tiles.png --process Homescreen ion_fromstart_v2.1_20140722153704_prod-lotsoficons-1G+tiles*.json

Comparison of tiles on the same plot.

Comment 74

[:gerard-majax]

Attachment: ion_v1.3-v1.4-v2.0-v2.1.png

> python anal_ion.py --output ion_v1.3-v1.4-v2.0-v2.1.png --process Homescreen ion_fromstart_v1.3_20140722151916.json ion_fromstart_v1.4_20140721000201.json ion_fromstart_v2.0_20140721000201.json ion_fromstart_v2.1_20140721040210.json ion_fromstart_v2.1_20140721062116.json

Comparison of v1.3-v2.1 on the same plot.

Comment 75

[:gerard-majax]

Attachment: ion_v2.0-v2.1_2014072400.png

Results with B2G v2.0 (20140724000201) and v2.1 (20140723160203).

Comment 76

[:gerard-majax]

Attachment: ion_verticalhome_v2.0-v2.1_2014072400.png

Results with Gecko 32 (20140724000201) and Gaia from v2.0 (20140724000201), but changing the homescreen with the v2.1 one. One can see that we have the very same behavior for both and a slight difference. When we compare this to attachment 8461450 [details], I feel like this points to some improvement in master on Gecko side rather than Homescreen side.

Comment 77

[:gerard-majax]

Sotaro, Milan, Benoit, see comment 76. Would there be any pending 2.0 uplift that could explain this ?

Comment 78

[:gerard-majax]

Attachment: ion_fromstart_master-eng_layout4_2014-0607.png

Comparing from early may to yesterday. Build of may 3rd was using the old homescreen. All the series before (including) july 7th seems quite grouped together, while series after are also grouped together. Looks like some improvements happened between july 7th and july 14th.

Comment 79

[:gerard-majax]

Attachment: anal_ion.py

Improved color mapping

Comment 80

[:gerard-majax]

Attachment: ion_fromstart_master-eng_layout4_2014-0607.png

Same as previous, but with the new color mapping:

Comparing from early may to yesterday. Build of may 3rd was using the old homescreen. All the series before (including) july 7th seems quite grouped together, while series after are also grouped together. Looks like some improvements happened between july 7th and july 14th.

Comment 81

[Milan Sreckovic [:milan] (needinfo for best results)]

(In reply to Tapas Kumar Kundu from comment #70) 
> https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gaia/commit/?h=mozilla/
> v2.0&id=bf3fb88039843359d0a5759b2c0fb787abae544f
> 
> https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gecko/commit/?h=mozilla/
> v2.0&id=e46acc521bf8e25787d9ed3937b0213774355c2d
> 
> Homescreen is still using 12MB ION memory in v2.0 but it was using 3MB ION
> in v1.3 (Comment 15)

On a 256mb device?  Because the icons are resized if the device is smaller than 512mb, and should take less memory in that case.

(In reply to Alexandre LISSY :gerard-majax from comment #77)
> Sotaro, Milan, Benoit, see comment 76. Would there be any pending 2.0 uplift
> that could explain this ?

Nothing obvious on the graphics side, but there may be some Gaia patches that haven't shown up yet?

Comment 82

[Sotaro Ikeda [:sotaro]]

(In reply to Alexandre LISSY :gerard-majax from comment #77)
> Sotaro, Milan, Benoit, see comment 76. Would there be any pending 2.0 uplift
> that could explain this ?

Sorry, I do not know it around gfx layers.

Comment 83

[Sotaro Ikeda [:sotaro]]

TextureClientPool caches at most 50 TextureClients. It might be too big for low memory device.

http://mxr.mozilla.org/mozilla-central/source/gfx/layers/client/TextureClientPool.h#97

Comment 84

[Milan Sreckovic [:milan] (needinfo for best results)]

The texture pool should get emptied as we switch away from the homescreen, but the peak memory usage would be higher, you're right.  With 256x256 tiles, on a HVGA 320x480 device, the display port should not need more than 6x2=12 tiles (9 displayed), on Flame at 480x854, no more than 10x2=20 (15 displayed) tiles.

That probably means we should have at least 21=12+9 on HVGA and 35=20+15 on Flame, but do we expect some tiles to be locked up and unavailable, so we need the extras?

Comment 85

[Benoit Girard (:BenWa)]

I finally managed to get a working v1.3 build using hamachi. This was not possible for the flame because the flame never supported v1.3.

v1.3 homescreen horizontal was not optimized. We didn't have will-change at that time so we had no way to request active layers for the homescreen. This resulted in very jerky panning. This was why the ION memory allocation was so low because we constructed the layers during the start of pan. When you start to pan on the homescreen in v1.3 you see the memory get allocate during the pan and release after the pan.

Comparing panning hamachi 1.3 vs. comparing hamachi master panning we get a more appropriate comparison. We got from 3.0 MB -> 5.3 MB (hamachi has a 320x480 screen).

Thus I think the comparison is not meaningful because we never optimized the 1.3 rendering. If we compare the peak memory usage the numbers are closer. The new homescreen requires more memory but that's because it can scroll faster and thus requires more buffering.

Comment 86

[Benoit Girard (:BenWa)]

(In reply to Alexandre LISSY :gerard-majax from comment #77)
> Sotaro, Milan, Benoit, see comment 76. Would there be any pending 2.0 uplift
> that could explain this ?

I don't know. This doesn't pertain to this bug so we should investigate in another bug to focus on the original issue here.

Comment 87

[Milan Sreckovic [:milan] (needinfo for best results)]

(In reply to Sotaro Ikeda [:sotaro PTO July/25 - Aug/3] from comment #83)
> TextureClientPool caches at most 50 TextureClients. It might be too big for
> low memory device.
> 
> http://mxr.mozilla.org/mozilla-central/source/gfx/layers/client/
> TextureClientPool.h#97

In case we want to play with this - see bug 1043603

Comment 88

[:gerard-majax]

Attachment: ion_fromstart_master-eng_layout4_2014-07_07-14.png

ION memory usage between july 7th to july 14th.

Comment 89

[:gerard-majax]

So, according to attachment 8462460 [details], there is a big change between july 7th and july 9th. I'm bisecting this: a7777778f7566c2a9c8f44e1c43d287ac4d00071..df658e23cce1f941a81fde56b61987c712b7861f

Comment 90

[:gerard-majax]

Attachment: ion_fromstart_master-eng_layout4_bisect.png

Result of bisecting between a7777778f7566c2a9c8f44e1c43d287ac4d00071 and df658e23cce1f941a81fde56b61987c712b7861f

Comment 91

[:gerard-majax]

Okay I don't understand. Commit 029c0ab5282982915f71192361d845aef5031ec1 is the last one I can get exposing the issue.

With commit 5f4f51977f5fc939ed2f0b9fb8513d9c114f459c, everything is okay. On my builds, only the Gecko commit checkout was changed, Gaia was still the same. However, this range of commit is just:

> commit 5f4f51977f5fc939ed2f0b9fb8513d9c114f459c
> Author: B2G Bumper Bot <[e-mail]>
> Date:   Fri Jul 4 13:01:39 2014 -0700
> 
>     Bumping manifests a=b2g-bump
> 
> commit 918a73e2e3dddfc10c2fec05814ae8abf7053d6f
> Author: B2G Bumper Bot <[e-mail]>
> Date:   Fri Jul 4 12:55:26 2014 -0700
> 
>     Bumping gaia.json for 1 gaia revision(s) a=gaia-bump
>     
>     ========
>     
>     https://hg.mozilla.org/integration/gaia-central/rev/b6a92543f10c
>     Author: Dale Harvey <[e-mail]>
>     Desc: Bug 1033049 - Ensure notice isnt shown when < 3 chars typed. r=kgrandon
> 
> commit 126ee8dcc136e8e04a67d5ac2c055423a781f512
> Author: B2G Bumper Bot <[e-mail]>
> Date:   Fri Jul 4 09:46:44 2014 -0700
> 
>     Bumping manifests a=b2g-bump
>

So I can't tell why the ION memory usage suddenly drops.

Comment 92

[:gerard-majax]

Okay, it seems that the regression range I used is completely wrong: for example, a772941928f65820500248fa558a8460990d1342 is a commit from may 21th.

Comment 93

[:gerard-majax]

Comment on attachment 8462538 [details]
ion_fromstart_master-eng_layout4_bisect.png

Obsolete because of bogus bisect range.

Comment 94

[:gerard-majax]

Attachment: ion_fromstart_v2.0-v2.1.png

Comparison of the latest B2G v2.0 and v2.1.

Comment 95

[:gerard-majax]

Attachment: ion_fromstart_master-eng_gaia.png

Some of my previous assumptions were wrong. After checking for nothing in Gecko, I gave a try again to checking Gaia side. This plot shows the value from the pvtbuild (20140707040201) and from my very own Gaia's tree in non production mode.

One can notice that the extremas of the reproduction range (20140707040201-20140707160202) does reflect the same use of ION memory: 93daa354671a698634a3dc661c8c9dcb7d824c31..1dc9e53393ae4680a174dffa44a958ec564ebbe8.

Comment 96

[:gerard-majax]

Attachment: ion_fromstart_master-eng_gaia_validate.png

After bisecting Gaia-side, I could document the ION memory use improvement as coming from https://github.com/mozilla-b2g/gaia/commit/ac391fa9f733ddbc7a9106460d11ccd0e9095199 ; the attached plot shows that the very commit before (d8aa2da456308e1df3d43b6a3268a8933ef6bf3d) exposes a higher level of ION memory use than the Gaia built with this commit.

Comment 97

[:gerard-majax]

Tapas, please have a look at comment 94 and comment 96.

Comment 98

[Kevin Grandon :kgrandon]

(In reply to Alexandre LISSY :gerard-majax from comment #96)
> Created attachment 8463122 [details]
> ion_fromstart_master-eng_gaia_validate.png
> 
> After bisecting Gaia-side, I could document the ION memory use improvement
> as coming from
> https://github.com/mozilla-b2g/gaia/commit/
> ac391fa9f733ddbc7a9106460d11ccd0e9095199 ; the attached plot shows that the
> very commit before (d8aa2da456308e1df3d43b6a3268a8933ef6bf3d) exposes a
> higher level of ION memory use than the Gaia built with this commit.

This is interesting. AFAIK - this is just changing the height of some items - so my guess is it's changing the overall display port size. This particular patch would shrink the height of each row by about ~8px. It looks like less height of the content means less ION memory - I wonder if we are using a different number of tiles before/after this patch.

Comment 99

[:gerard-majax]

Attachment: ion_fromstart_v2.0-v2.1_20140726.png

Comparing B2G v2.0 (20140726000201) and v2.1 (20140726040205) from pvtbuilds, eng.

One can notice two big jumps on the v2.0 plot:
 - the first one, around x=50,
 - the second one, around x=210

Looking at the raw dumps, we can see:
> 53 {"pcomm": "Homescreen", "pid": 1780, "tid": 1787, "date": "2014-07-26 20:20:23.686000", "tcomm": "BufferMgrChild", "payload": {"buffer": 2955354112, "size": 1597440, "fd": 125, "op": "map"}}

So the very first action of Homescreen is to map 1560KB.

Then later,
> 215 {"pcomm": "Homescreen", "pid": 1780, "tid": 1787, "date": "2014-07-26 20:20:24.296000", "tcomm": "BufferMgrChild", "payload": {"buffer": 2944901120, "size": 1597440, "fd": 172, "op": "map"}}

We have another 1560KB getting mapped. From a rought quantitative overview, it's 3MB and it's very close to the delta that we are observing between v2.0 and v2.1.

Comment 100

[:gerard-majax]

By carefuly reading attachment 8461548 [details], one can see that the high ION start memory usage (1560KB) described in comment 99 has been mitigated between buildid 20140617040203 and 20140624040203.

This means I need to bisect gecko between commit 82463f11e1934a43c52d0bfa6ddea8da7a74711f (20140617040203) and 0c3673b6ff7488f5aa2156c6fa5522d4d0e2e975 (20140624040203).

Comment 101

[:gerard-majax]

Attachment: ion_fromstart_master-eng_201406_17-24.png

Doing a first bisect using pvtbuilds. We can see that master-eng 20140617040203 exposes this issue, while 20140617160234 and above do not expose the extra allocation.

Comment 102

[:gerard-majax]

(In reply to Alexandre LISSY :gerard-majax from comment #101)
> Created attachment 8463154 [details]
> ion_fromstart_master-eng_201406_17-24.png
> 
> Doing a first bisect using pvtbuilds. We can see that master-eng
> 20140617040203 exposes this issue, while 20140617160234 and above do not
> expose the extra allocation.

So the range of commits to bisect is: 82463f11e1934a43c52d0bfa6ddea8da7a74711f..6f80c55849c36f46f48941f735c8228b7e2f36d0

> $ git log --oneline 82463f11e1934a43c52d0bfa6ddea8da7a74711f..6f80c55849c36f46f48941f735c8228b7e2f36d0 | wc -l
> 214

Comment 103

[:gerard-majax]

Attachment: ion_fromstart_bisect_201406_17-24_final.png

Result of git bisect 82463f11e1934a43c52d0bfa6ddea8da7a74711f..6f80c55849c36f46f48941f735c8228b7e2f36d0:

Commit 57bf9f52dde5d129ab363264f55c39ebe7a9a664 is the last one exposing the big (1560KB) alloc/mapping on homescreen start. With the landing of commit 99cc55b9f2209280e75147d3d54796cc1c524ec5 this is never reproduced after.

So it would mean that it is bug 1024148 that is helping to save 3MB.

Comment 104

[:gerard-majax]

Attachment: ion_fromstart_b2g32_bug1024148.png

This charts shows the impact of bug 1024148 on the current b2g32 tree. It has been produced by checking out latest b2g32-v2.0 tree (745b486db495248e4d4503039e374cb8d5bb244f). On top of this, I added bug 1024148 by cherry-picking both patches (57bf9f52dde5d129ab363264f55c39ebe7a9a664, 99cc55b9f2209280e75147d3d54796cc1c524ec5).

We can see that the two 1560KB alloc on homescreen startup are gone after including the patches of bug 1024148.

Comment 105

[Benoit Girard (:BenWa)]

This is interesting and useful but it distracts from the original bug report. This bug is now tracking too many unrelated things. See comment 86. This bug should focus on the original report which is specifically about changes between 1.3 and 2.0.

Patches that landed in 2.1 can't be related to a regression between 1.3 and 2.0.

Comment 106

[Benoit Girard (:BenWa)]

Requested for bug 1024148 to block v2.0.

Comment 107

[Jeff Muizelaar [:jrmuizel]]

(In reply to Benoit Girard (:BenWa) from comment #105)
> This is interesting and useful but it distracts from the original bug
> report. This bug is now tracking too many unrelated things. See comment 86.
> This bug should focus on the original report which is specifically about
> changes between 1.3 and 2.0.
> 
> Patches that landed in 2.1 can't be related to a regression between 1.3 and
> 2.0.

The patch on bug 1044241 should produce an improvement here. I'd be interested in seeing the mearesurement results.

Comment 108

[:gerard-majax]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #107)
> (In reply to Benoit Girard (:BenWa) from comment #105)
> > This is interesting and useful but it distracts from the original bug
> > report. This bug is now tracking too many unrelated things. See comment 86.
> > This bug should focus on the original report which is specifically about
> > changes between 1.3 and 2.0.
> > 
> > Patches that landed in 2.1 can't be related to a regression between 1.3 and
> > 2.0.
> 
> The patch on bug 1044241 should produce an improvement here. I'd be
> interested in seeing the mearesurement results.

I gave it a try, and I could not see an impact at all on just loading homescreen and swiping it up and down.

Comment 109

[Jeff Muizelaar [:jrmuizel]]

(In reply to Alexandre LISSY :gerard-majax from comment #108)
> (In reply to Jeff Muizelaar [:jrmuizel] from comment #107)
> > (In reply to Benoit Girard (:BenWa) from comment #105)
> > > This is interesting and useful but it distracts from the original bug
> > > report. This bug is now tracking too many unrelated things. See comment 86.
> > > This bug should focus on the original report which is specifically about
> > > changes between 1.3 and 2.0.
> > > 
> > > Patches that landed in 2.1 can't be related to a regression between 1.3 and
> > > 2.0.
> > 
> > The patch on bug 1044241 should produce an improvement here. I'd be
> > interested in seeing the mearesurement results.
> 
> I gave it a try, and I could not see an impact at all on just loading
> homescreen and swiping it up and down.

If you wait a couple of seconds does the memory usage go down?

Comment 110

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #107)
> (In reply to Benoit Girard (:BenWa) from comment #105)
> > This is interesting and useful but it distracts from the original bug
> > report. This bug is now tracking too many unrelated things. See comment 86.
> > This bug should focus on the original report which is specifically about
> > changes between 1.3 and 2.0.
> > 
> > Patches that landed in 2.1 can't be related to a regression between 1.3 and
> > 2.0.
> 
> The patch on bug 1044241 should produce an improvement here. I'd be
> interested in seeing the mearesurement results.

As I mentioned in bug 1044241 Comment 4 , I am waiting to see a new patch for v2.0. If you can provide us  WIP patch for v2.0 then we can try that to see how much ION memory homescreen is still using with this patch.

Comment 111

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

Oh I see that someone has already tested this in Comment 108. 
@Jeff : You can ignore my Comment 110.

Comment 112

[Jeff Muizelaar [:jrmuizel]]

I see an improvement of about 3.4 MB with the patch from bug 1044241 on a 1G flame.

before:
          client              pid             size
----------------------------------------------------
     adsprpc-smd                1             8192
fd900000.qcom,mdss_mdp                1          1658880
----------------------------------------------------
orphaned allocations (info is from last known client):
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
        mdss_fb0            19675          1658880 0 1
             b2g            20053          1658880 0 1
             b2g            20053          1658880 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
 Built-in Keyboa            20208            61440 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
      Homescreen            20169           294912 0 1
----------------------------------------------------
  total orphaned         15065088
          total          16732160
----------------------------------------------------
Cached Pools:
0 order 9 highmem pages in pool = 0 total
0 order 9 lowmem pages in pool = 0 total
0 order 8 highmem pages in pool = 0 total
0 order 8 lowmem pages in pool = 0 total
0 order 4 highmem pages in pool = 0 total
0 order 4 lowmem pages in pool = 0 total
361 order 0 highmem pages in pool = 169000 total
3366 order 0 lowmem pages in pool = d26000 total
Uncached Pools:
0 order 9 highmem pages in pool = 0 total
0 order 9 lowmem pages in pool = 0 total
0 order 8 highmem pages in pool = 0 total
0 order 8 lowmem pages in pool = 0 total
0 order 4 highmem pages in pool = 0 total
0 order 4 lowmem pages in pool = 0 total
0 order 0 highmem pages in pool = 0 total
0 order 0 lowmem pages in pool = 0 total
Total bytes in pool: e8f000

after:

          client              pid             size
----------------------------------------------------
     adsprpc-smd                1             8192
fd900000.qcom,mdss_mdp                1          1658880
----------------------------------------------------
orphaned allocations (info is from last known client):
             b2g            20412          1658880 0 1
             b2g            20412          1658880 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
 Built-in Keyboa            20597            61440 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
 Built-in Keyboa            20597            61440 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
        mdss_fb0            20418          1658880 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
      Homescreen            20548           294912 0 1
----------------------------------------------------
  total orphaned         11587584
          total          13254656
----------------------------------------------------
Cached Pools:
0 order 9 highmem pages in pool = 0 total
0 order 9 lowmem pages in pool = 0 total
0 order 8 highmem pages in pool = 0 total
0 order 8 lowmem pages in pool = 0 total
0 order 4 highmem pages in pool = 0 total
0 order 4 lowmem pages in pool = 0 total
549 order 0 highmem pages in pool = 225000 total
4027 order 0 lowmem pages in pool = fbb000 total
Uncached Pools:
0 order 9 highmem pages in pool = 0 total
0 order 9 lowmem pages in pool = 0 total
0 order 8 highmem pages in pool = 0 total
0 order 8 lowmem pages in pool = 0 total
0 order 4 highmem pages in pool = 0 total
0 order 4 lowmem pages in pool = 0 total
0 order 0 highmem pages in pool = 0 total
0 order 0 lowmem pages in pool = 0 total
Total bytes in pool: 11e0000

Comment 113

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

Attachment: ion memory usage after homescreen scroll in FFOS 2.0 running on KK 256MB device

I am seeing that homescreen is using only 4128768 bytes (3.9375 MB) with fix from bug 1044241 in our FFOS 2.0 [1] KK 256MB device. 

Thanks to all for good work here. 

In v1.3, home screen uses 3129344 bytes (2.984375 MB) on 256MB JB device . 

So ION memory regression is only 1MB between v1.3 and v2.0 now. 


[1] https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gecko/commit/?h=mozilla/v2.0&id=902adb7850609db2fb371a2c2c5375e36668b457
    https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gaia/commit/?h=mozilla/v2.0&id=9bb159afe7b4f506a7abebb7090ee8980bc6a31d

Comment 114

[Gregor Wagner [:gwagner]]

Thanks Tapas!

We are down to 1MB instead of 7MB now. Can we call this done? We can still improve it going forward but do we need to block the release on it?

Comment 115

[Milan Sreckovic [:milan] (needinfo for best results)]

I'd really like to get the "official" measurement when running "3 larger icons across", rather than "4 smaller icons across", as we know there is UX preference for 3 across.

Comment 116

[Kevin Grandon :kgrandon]

Just making a note that with the default engineering build of 47 apps for three columns we render icons outside of the 2.7x displayport, and for four columns, we render less than that.

Any test we do should compare the two with "lots of icons", enough to fill the displayport I assume.

I've also filed bug 1046158 to spin a homescreen patch to change the number of default columns.

Comment 117

[Milan Sreckovic [:milan] (needinfo for best results)]

For a 320x480 screen, the break even should be at most 37.  For the Flame, the break even should be at most 45.  I don't know how the section separators get placed and how much space they take, so the numbers are probably lower.  Make sure we have 100, and we're safe for sure.

Comment 118

[Kevin Grandon :kgrandon]

That makes sense, for comment 116 we were seeing this with a Flame.

Comment 119

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

(In reply to Gregor Wagner [:gwagner] from comment #114)
> Thanks Tapas!
> 
> We are down to 1MB instead of 7MB now. Can we call this done? We can still
> improve it going forward but do we need to block the release on it?

I also agree. But I see that final patch from bug 1044241 has not landed yet. We can mark this bug as resolved once that patch lands. Thanks a lot for your help here.

Comment 120

[bhavana bajaj [:bajaj]]

(In reply to Tapas Kumar Kundu from comment #119)
> (In reply to Gregor Wagner [:gwagner] from comment #114)
> > Thanks Tapas!
> > 
> > We are down to 1MB instead of 7MB now. Can we call this done? We can still
> > improve it going forward but do we need to block the release on it?
> 
> I also agree. But I see that final patch from bug 1044241 has not landed
> yet. We can mark this bug as resolved once that patch lands. Thanks a lot
> for your help here.

given the needed patch has landed, marking this resolved fixed.

Comment 121

[Brogan Zumwalt [Inactive]]

Not enough information to create valid test case. Unable to create new case.

Comment 122

[Brogan Zumwalt [Inactive]]

Steps given are not functioning correctly for verification steps. Command "adb shell cat /sys/kernel/debug/ion/iommu" gives error when used on both user and eng builds. Unable to verify with current steps.