Increasing memory due to RSS feeds in bookmarks folder

RESOLVED WORKSFORME

Status

()

Firefox
General
RESOLVED WORKSFORME
6 years ago
4 months ago

People

(Reporter: andysmozbug, Unassigned)

Tracking

(Depends on: 2 bugs)

8 Branch
x86
Windows 7
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [MemShrink:P2][see comment 14])

Attachments

(10 attachments)

(Reporter)

Description

6 years ago
User Agent: Mozilla/5.0 (Windows NT 6.0; rv:8.0) Gecko/20100101 Firefox/8.0
Build ID: 20110928060149

Steps to reproduce:

[MemShrink]
Restarted FF7 in safe mode with two tabs open (about:memory and https://wiki.mozilla.org/Performance/MemShrink )
My setup has around eight RSS feeds in the bookmark toolbar including the following feed locations:
http://www.bbc.co.uk/go/rss/int/news/-/news/
http://bishophill.squarespace.com/blog/rss.xml
http://www2.politicalbetting.com/index.php/feed/
http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/front_page/rss.xml
http://www.spectator.co.uk/coffeehouse/index.txml
http://feeds2.feedburner.com/guidofawkes
http://www.iaindale.com/feed
http://feeds.feedburner.com/arseblognews?format=xml
http://noconsensus.wordpress.com/feed/




Actual results:

With the RSS feeds, after 12 hours resident memory grew from 80MB to 394, and then reduced to 192MB after hitting minimise in about:memory

Repeating the test after deleting the RSS from the toolbar, after eight hours resident memory grew from 79 to 104, and did not shrink after hitting minimise 

In FF8, on a different machine, similar results to the first test with the RSS feeds present
(Reporter)

Updated

6 years ago
Whiteboard: [memshrink]
(Reporter)

Updated

6 years ago
OS: Windows Vista → Windows 7
(Reporter)

Comment 1

6 years ago
For clarity, these tests were overnight so during these tests FF was not being used
Can you provide/attach before/after about:memory Output with "after" = kept running Firefox X Hours?
(Reporter)

Comment 3

6 years ago
The following test was carried out running FF8.  Two tabs open:  about:memory?verbose and https://wiki.mozilla.org/Performance/MemShrink

This test was on Windows Vista, though I get similar results on a Windows 7 machine running FF7

The first set of results were from about 10 minutes after restarting FF in safe mode.  During that time resident memory had grown rapidly from 60MB to 80MB then stabilised.  The second results were 8 hours later after hitting minimise.

10 minutes after starting:
Main Process

Explicit Allocations
36,035,796 B (100.0%) -- explicit
├──16,097,800 B (44.67%) -- heap-unclassified
├──13,901,450 B (38.58%) -- js
│  ├───6,883,138 B (19.10%) -- compartment([System Principal], 0x5a7a000)
│  │   ├──3,969,024 B (11.01%) -- gc-heap
│  │   │  ├──2,099,576 B (05.83%) -- objects
│  │   │  ├──1,356,280 B (03.76%) -- shapes
│  │   │  ├────314,656 B (00.87%) -- arena-unused
│  │   │  ├────173,264 B (00.48%) -- strings
│  │   │  ├─────15,504 B (00.04%) -- arena-headers
│  │   │  ├──────9,600 B (00.03%) -- arena-padding
│  │   │  └────────144 B (00.00%) -- xml
│  │   ├──1,455,360 B (04.04%) -- scripts
│  │   ├────464,664 B (01.29%) -- object-slots
│  │   ├────342,268 B (00.95%) -- property-tables
│  │   ├────306,654 B (00.85%) -- string-chars
│  │   ├────214,096 B (00.59%) -- tjit-data
│  │   │    ├──118,896 B (00.33%) -- trace-monitor
│  │   │    ├───74,000 B (00.21%) -- allocators-reserve
│  │   │    └───21,200 B (00.06%) -- allocators-main
│  │   └────131,072 B (00.36%) -- mjit-code
│  ├───4,380,444 B (12.16%) -- gc-heap-chunk-dirty-unused
│  ├─────990,640 B (02.75%) -- compartment(atoms)
│  │     ├──564,656 B (01.57%) -- string-chars
│  │     └──425,984 B (01.18%) -- gc-heap
│  │        ├──415,808 B (01.15%) -- strings
│  │        ├────7,632 B (00.02%) -- arena-unused
│  │        ├────1,664 B (00.00%) -- arena-headers
│  │        └──────880 B (00.00%) -- arena-padding
│  ├─────804,586 B (02.23%) -- compartment(https://wiki.mozilla.org/Performance/MemShrink)
│  │     ├──487,424 B (01.35%) -- gc-heap
│  │     │  ├──209,280 B (00.58%) -- objects
│  │     │  ├──185,480 B (00.51%) -- shapes
│  │     │  ├───84,152 B (00.23%) -- arena-unused
│  │     │  ├────5,568 B (00.02%) -- strings
│  │     │  ├────1,904 B (00.01%) -- arena-headers
│  │     │  └────1,040 B (00.00%) -- arena-padding
│  │     ├──187,074 B (00.52%) -- scripts
│  │     ├───65,536 B (00.18%) -- mjit-code
│  │     ├───28,720 B (00.08%) -- object-slots
│  │     ├───27,968 B (00.08%) -- property-tables
│  │     └────7,864 B (00.02%) -- string-chars
│  ├─────403,456 B (01.12%) -- runtime
│  │     ├──262,144 B (00.73%) -- atoms-table
│  │     └──141,312 B (00.39%) -- runtime-object
│  ├─────262,144 B (00.73%) -- stack
│  ├─────153,828 B (00.43%) -- gc-heap-chunk-admin
│  ├──────23,214 B (00.06%) -- compartment(moz-nullprincipal:{e6336f61-dfb6-4524-bebb-bd2f187e1679})
│  │      ├──20,480 B (00.06%) -- gc-heap
│  │      │  ├──14,800 B (00.04%) -- arena-unused
│  │      │  ├───3,352 B (00.01%) -- objects
│  │      │  ├───2,200 B (00.01%) -- shapes
│  │      │  ├──────80 B (00.00%) -- arena-headers
│  │      │  └──────48 B (00.00%) -- arena-padding
│  │      ├───2,448 B (00.01%) -- object-slots
│  │      ├─────168 B (00.00%) -- property-tables
│  │      └─────118 B (00.00%) -- scripts
│  └───────────0 B (00.00%) -- gc-heap-chunk-clean-unused
├───3,669,152 B (10.18%) -- storage
│   └──3,669,152 B (10.18%) -- sqlite
│      ├──1,328,800 B (03.69%) -- urlclassifier3.sqlite
│      │  ├──1,244,744 B (03.45%) -- cache-used
│      │  ├─────81,392 B (00.23%) -- stmt-used
│      │  └──────2,664 B (00.01%) -- schema-used
│      ├────966,736 B (02.68%) -- cookies.sqlite
│      │    ├──954,736 B (02.65%) -- cache-used
│      │    ├───10,160 B (00.03%) -- stmt-used
│      │    └────1,840 B (00.01%) -- schema-used
│      ├────660,336 B (01.83%) -- places.sqlite
│      │    ├──576,664 B (01.60%) -- cache-used [4]
│      │    ├───47,872 B (00.13%) -- schema-used [4]
│      │    └───35,800 B (00.10%) -- stmt-used [4]
│      ├────619,816 B (01.72%) -- other
│      ├─────48,896 B (00.14%) -- downloads.sqlite
│      │     ├──41,128 B (00.11%) -- cache-used
│      │     ├───5,984 B (00.02%) -- stmt-used
│      │     └───1,784 B (00.00%) -- schema-used
│      ├─────16,872 B (00.05%) -- content-prefs.sqlite
│      │     ├───8,128 B (00.02%) -- stmt-used
│      │     ├───6,344 B (00.02%) -- cache-used
│      │     └───2,400 B (00.01%) -- schema-used
│      ├─────12,152 B (00.03%) -- signons.sqlite
│      │     ├───7,480 B (00.02%) -- cache-used
│      │     ├───2,824 B (00.01%) -- schema-used
│      │     └───1,848 B (00.01%) -- stmt-used
│      ├─────11,024 B (00.03%) -- permissions.sqlite
│      │     ├───5,744 B (00.02%) -- stmt-used
│      │     ├───4,016 B (00.01%) -- cache-used
│      │     └───1,264 B (00.00%) -- schema-used
│      └──────4,520 B (00.01%) -- formhistory.sqlite
│             ├──2,856 B (00.01%) -- cache-used
│             ├──1,664 B (00.00%) -- schema-used
│             └──────0 B (00.00%) -- stmt-used
├───1,054,154 B (02.93%) -- layout
│   ├────716,998 B (01.99%) -- arenas
│   └────337,156 B (00.94%) -- styledata
├─────911,384 B (02.53%) -- xpti-working-set
├─────228,053 B (00.63%) -- images
│     ├──151,284 B (00.42%) -- chrome
│     │  ├──151,284 B (00.42%) -- used
│     │  │  ├──151,284 B (00.42%) -- uncompressed-nonheap
│     │  │  ├────────0 B (00.00%) -- raw
│     │  │  └────────0 B (00.00%) -- uncompressed-heap
│     │  └────────0 B (00.00%) -- unused
│     │           ├──0 B (00.00%) -- raw
│     │           ├──0 B (00.00%) -- uncompressed-heap
│     │           └──0 B (00.00%) -- uncompressed-nonheap
│     └───76,769 B (00.21%) -- content
│         ├──76,769 B (00.21%) -- used
│         │  ├──72,529 B (00.20%) -- raw
│         │  ├───4,240 B (00.01%) -- uncompressed-nonheap
│         │  └───────0 B (00.00%) -- uncompressed-heap
│         └───────0 B (00.00%) -- unused
│                 ├──0 B (00.00%) -- raw
│                 ├──0 B (00.00%) -- uncompressed-heap
│                 └──0 B (00.00%) -- uncompressed-nonheap
├─────171,575 B (00.48%) -- dom
└───────2,228 B (00.01%) -- cycle-collector

Other Measurements
          0 B -- gfx-d2d-surfacecache
          0 B -- gfx-d2d-surfacevram
      4,880 B -- gfx-surface-image
    157,008 B -- gfx-surface-win32
 35,018,064 B -- heap-allocated
 40,665,088 B -- heap-committed
  3,330,048 B -- heap-dirty
  7,972,702 B -- heap-unallocated
            2 -- js-compartments-system
            2 -- js-compartments-user
  9,437,184 B -- js-gc-heap
    421,240 B -- js-gc-heap-arena-unused
          0 B -- js-gc-heap-chunk-clean-unused
  4,380,444 B -- js-gc-heap-chunk-dirty-unused
       50.88% -- js-gc-heap-unused-fraction
 60,141,568 B -- private
 79,167,488 B -- resident
242,143,232 B -- vsize 




After 8 hours the hitting minimise a few times:
Main Process

Explicit Allocations
104,814,644 B (100.0%) -- explicit
├───42,414,320 B (40.47%) -- js
│   ├──31,702,128 B (30.25%) -- gc-heap-chunk-dirty-unused
│   ├───7,587,016 B (07.24%) -- compartment([System Principal], 0x5a7a000)
│   │   ├──4,485,120 B (04.28%) -- gc-heap
│   │   │  ├──1,881,192 B (01.79%) -- objects
│   │   │  ├──1,338,160 B (01.28%) -- shapes
│   │   │  ├──1,186,816 B (01.13%) -- arena-unused
│   │   │  ├─────50,288 B (00.05%) -- strings
│   │   │  ├─────17,520 B (00.02%) -- arena-headers
│   │   │  ├─────11,000 B (00.01%) -- arena-padding
│   │   │  └────────144 B (00.00%) -- xml
│   │   ├──1,477,192 B (01.41%) -- scripts
│   │   ├────458,752 B (00.44%) -- mjit-code
│   │   ├────455,496 B (00.43%) -- object-slots
│   │   ├────363,164 B (00.35%) -- property-tables
│   │   ├────214,096 B (00.20%) -- tjit-data
│   │   │    ├──118,896 B (00.11%) -- trace-monitor
│   │   │    ├───74,000 B (00.07%) -- allocators-reserve
│   │   │    └───21,200 B (00.02%) -- allocators-main
│   │   └────133,196 B (00.13%) -- string-chars
│   ├───1,016,464 B (00.97%) -- compartment(atoms)
│   │   ├────578,192 B (00.55%) -- string-chars
│   │   └────438,272 B (00.42%) -- gc-heap
│   │        ├──429,200 B (00.41%) -- strings
│   │        ├────6,464 B (00.01%) -- arena-unused
│   │        ├────1,712 B (00.00%) -- arena-headers
│   │        └──────896 B (00.00%) -- arena-padding
│   ├─────804,586 B (00.77%) -- compartment(https://wiki.mozilla.org/Performance/MemShrink)
│   │     ├──487,424 B (00.47%) -- gc-heap
│   │     │  ├──209,280 B (00.20%) -- objects
│   │     │  ├──185,480 B (00.18%) -- shapes
│   │     │  ├───84,152 B (00.08%) -- arena-unused
│   │     │  ├────5,568 B (00.01%) -- strings
│   │     │  ├────1,904 B (00.00%) -- arena-headers
│   │     │  └────1,040 B (00.00%) -- arena-padding
│   │     ├──187,074 B (00.18%) -- scripts
│   │     ├───65,536 B (00.06%) -- mjit-code
│   │     ├───28,720 B (00.03%) -- object-slots
│   │     ├───27,968 B (00.03%) -- property-tables
│   │     └────7,864 B (00.01%) -- string-chars
│   ├─────615,312 B (00.59%) -- gc-heap-chunk-admin
│   ├─────403,456 B (00.38%) -- runtime
│   │     ├──262,144 B (00.25%) -- atoms-table
│   │     └──141,312 B (00.13%) -- runtime-object
│   ├─────262,144 B (00.25%) -- stack
│   ├──────23,214 B (00.02%) -- compartment(moz-nullprincipal:{e6336f61-dfb6-4524-bebb-bd2f187e1679})
│   │      ├──20,480 B (00.02%) -- gc-heap
│   │      │  ├──14,800 B (00.01%) -- arena-unused
│   │      │  ├───3,352 B (00.00%) -- objects
│   │      │  ├───2,200 B (00.00%) -- shapes
│   │      │  ├──────80 B (00.00%) -- arena-headers
│   │      │  └──────48 B (00.00%) -- arena-padding
│   │      ├───2,448 B (00.00%) -- object-slots
│   │      ├─────168 B (00.00%) -- property-tables
│   │      └─────118 B (00.00%) -- scripts
│   └───────────0 B (00.00%) -- gc-heap-chunk-clean-unused
├───35,484,865 B (33.85%) -- heap-unclassified
├───24,619,824 B (23.49%) -- storage
│   └──24,619,824 B (23.49%) -- sqlite
│      ├──14,396,912 B (13.74%) -- places.sqlite
│      │  ├──14,156,200 B (13.51%) -- cache-used [4]
│      │  ├─────192,224 B (00.18%) -- stmt-used [4]
│      │  └──────48,488 B (00.05%) -- schema-used [4]
│      ├───8,548,592 B (08.16%) -- urlclassifier3.sqlite
│      │   ├──8,464,536 B (08.08%) -- cache-used
│      │   ├─────81,392 B (00.08%) -- stmt-used
│      │   └──────2,664 B (00.00%) -- schema-used
│      ├─────966,736 B (00.92%) -- cookies.sqlite
│      │     ├──954,736 B (00.91%) -- cache-used
│      │     ├───10,160 B (00.01%) -- stmt-used
│      │     └────1,840 B (00.00%) -- schema-used
│      ├─────614,120 B (00.59%) -- other
│      ├──────48,896 B (00.05%) -- downloads.sqlite
│      │      ├──41,128 B (00.04%) -- cache-used
│      │      ├───5,984 B (00.01%) -- stmt-used
│      │      └───1,784 B (00.00%) -- schema-used
│      ├──────16,872 B (00.02%) -- content-prefs.sqlite
│      │      ├───8,128 B (00.01%) -- stmt-used
│      │      ├───6,344 B (00.01%) -- cache-used
│      │      └───2,400 B (00.00%) -- schema-used
│      ├──────12,152 B (00.01%) -- signons.sqlite
│      │      ├───7,480 B (00.01%) -- cache-used
│      │      ├───2,824 B (00.00%) -- schema-used
│      │      └───1,848 B (00.00%) -- stmt-used
│      ├──────11,024 B (00.01%) -- permissions.sqlite
│      │      ├───5,744 B (00.01%) -- stmt-used
│      │      ├───4,016 B (00.00%) -- cache-used
│      │      └───1,264 B (00.00%) -- schema-used
│      └───────4,520 B (00.00%) -- formhistory.sqlite
│              ├──2,856 B (00.00%) -- cache-used
│              ├──1,664 B (00.00%) -- schema-used
│              └──────0 B (00.00%) -- stmt-used
├──────972,455 B (00.93%) -- layout
│      ├──678,083 B (00.65%) -- arenas
│      └──294,372 B (00.28%) -- styledata
├──────911,384 B (00.87%) -- xpti-working-set
├──────228,053 B (00.22%) -- images
│      ├──151,284 B (00.14%) -- chrome
│      │  ├──151,284 B (00.14%) -- used
│      │  │  ├──151,284 B (00.14%) -- uncompressed-nonheap
│      │  │  ├────────0 B (00.00%) -- raw
│      │  │  └────────0 B (00.00%) -- uncompressed-heap
│      │  └────────0 B (00.00%) -- unused
│      │           ├──0 B (00.00%) -- raw
│      │           ├──0 B (00.00%) -- uncompressed-heap
│      │           └──0 B (00.00%) -- uncompressed-nonheap
│      └───76,769 B (00.07%) -- content
│          ├──76,769 B (00.07%) -- used
│          │  ├──72,529 B (00.07%) -- raw
│          │  ├───4,240 B (00.00%) -- uncompressed-nonheap
│          │  └───────0 B (00.00%) -- uncompressed-heap
│          └───────0 B (00.00%) -- unused
│                  ├──0 B (00.00%) -- raw
│                  ├──0 B (00.00%) -- uncompressed-heap
│                  └──0 B (00.00%) -- uncompressed-nonheap
├──────171,235 B (00.16%) -- dom
└───────12,508 B (00.01%) -- cycle-collector

Other Measurements
          0 B -- gfx-d2d-surfacecache
          0 B -- gfx-d2d-surfacevram
      4,880 B -- gfx-surface-image
    157,008 B -- gfx-surface-win32
103,469,232 B -- heap-allocated
112,480,256 B -- heap-committed
  3,637,248 B -- heap-dirty
111,488,142 B -- heap-unallocated
            2 -- js-compartments-system
            2 -- js-compartments-user
 37,748,736 B -- js-gc-heap
  1,292,232 B -- js-gc-heap-arena-unused
          0 B -- js-gc-heap-chunk-clean-unused
 31,702,128 B -- js-gc-heap-chunk-dirty-unused
       87.40% -- js-gc-heap-unused-fraction
132,763,648 B -- private
133,341,184 B -- resident
417,583,104 B -- vsize
so, the interesting increases are heap-unclassified and gc-heap-chunk-dirty-unused, it's possible something is not freed up in livemarks reloads, those involve loadgroups, feedparser and a bunch of other stuff.
Nick, can you point your dark matter detector at this case and see what's hiding in heap-unclassified here?

This doesn't seem like something lots of people would run into though, since we've mostly hidden the bookmarks folder feed feature...
Assignee: nobody → nnethercote
Whiteboard: [memshrink] → [MemShrink:P3]
Depends on: 563700
Depends on: 670596

Updated

6 years ago
No longer depends on: 670596

Updated

6 years ago
Blocks: 670596
(In reply to Johnny Stenback (:jst, jst@mozilla.com) from comment #5)
> This doesn't seem like something lots of people would run into though, since
> we've mostly hidden the bookmarks folder feed feature...

Right, but default bookmarks, added to each profile, include a live bookmark (the Latest News one), so this may potentially affect many.
(In reply to Marco Bonardo [:mak] from comment #7)
> 
> Right, but default bookmarks, added to each profile, include a live bookmark
> (the Latest News one), so this may potentially affect many.

Indeed, I think the "Latest News" shouldn't be on by default.  Should I file a separate bug about that?
(In reply to Nicholas Nethercote [:njn] (on vacation Sep 12--Oct 17) from comment #8)
> Indeed, I think the "Latest News" shouldn't be on by default.  Should I file
> a separate bug about that?

you're free to do that, but while it's trivial to remove it for new users, it may be not to remove it for old users (and most likely even not desirable in case they use it)
(Reporter)

Comment 10

6 years ago
I believe this bug could effect large numbers of users

The FF8 URL row has an RSS icon between reload and stop, and the default behaviour on a page that supports RSS is to add a subscription to the bookmarks toolbar.  The same applies if a user clicks on an RSS icon on a web page (e.g. top right within the header on page http://www.bbc.co.uk/news/   )

If the bookmark toolbar is hidden, users may have multiple subscriptions without realising it

The alternative option when clicking on an RSS icon is to subscribe into the bookmarks folders.  I have not checked yet whether this would result in the same memory growth
Blocks: 563700
Depends on: 676724
No longer depends on: 563700
Created attachment 569293 [details]
DMD output

I created a new profile with the usual "Latest Headlines" feed plus the eight from comment 1, started up, and viewed about:memory a few times.  The memory usage went up and down a bit as the minutes passed, and heap-unclassified varied in the range 30% to 50%, going down up and down in a fashion that looked like GC was involved.  Which is surprising, because the JS engine is extremely well covered by memory reporters.

Firing up DMD, I saw a number of unfamiliar entries, all relating to xpconnect native thingies.  (I also saw a big nsStringBuffer entry that lacked sufficient context to tell where it came from, but I bet it's related.)  Looks like adding memory reporters for them will be useful in this case, and hopefully others.
(In reply to Nicholas Nethercote [:njn] from comment #11)
> (I also saw a big nsStringBuffer entry that
> lacked sufficient context to tell where it came from, but I bet it's
> related.)

hm, to update feeds we have to parse the rss contents that may be this big string entry, this is done by the feedProcessor component, and considering its code, I'd not be surprised if it'd leak. Btw, hard to tell off-hand, I'll take a look there to check for evident offenders. That code should likely be deCOMtamined too ( I should have a patch somewhere on disk that does that).
This bug may explain why I often see a lot of unclassified memory compared to others, I have quite some feeds.
(In reply to Nicholas Nethercote [:njn] from comment #8)
> 
> Indeed, I think the "Latest News" shouldn't be on by default.  Should I file
> a separate bug about that?

Filed as bug 697038.
I spun off bug 697041 for adding memory reporters to xpconnect, which will give us measurements of the memory usage.  This bug should now be about whether we change anything relating to the implementation of RSS feeds.
No longer blocks: 563700
Whiteboard: [MemShrink:P3] → [MemShrink:P3][see comment 14]
njn, could we possibly get DMD to dump another 2-3 stack frames?  Right now string memory allocations are completely opaque because the callstack from string Assign to the malloc is already 7 frames or so deep, so we have no idea who's calling Assign.  Similar for the xpconnect gunk (including the string allocations via ReadableToJsval!); some of it I'd love to know who the callers are.
(In reply to Boris Zbarsky (:bz) from comment #15)
> njn, could we possibly get DMD to dump another 2-3 stack frames?

I did this in bug 697041.
If I should trust tree-management numbers after bug 696163 landed (read this as Trace-malloc maxheap improvements of up to 30%) I should think we actually have some more issues than just the need to measure xpconnect.
Something starting from feedprocessor is clearly leaking or acting weird.
I don't see any noticeable changes to tracemalloc maxheap numbers at graphs-new.mozilla.org.  (The MacOSX 10.5.2 do vary hugely, though.)  Which platform was the claimed improvement on?

I'd be delighted but highly surprised if this improvement was real;  I've done a lot of memory profiling and I've seen nothing that could explain such an improvement.
looks like it is sticking the values to low peaks that were already present in the past, really, like if something could or not be executed.
Wow, those graphs are impressive... but I see no change on Linux for either "allocs" or "maxHeap", nor for Mac on "maxHeap".  And I don't know how to interpret the bi-modality of those graphs in general.

Still, this can only be a good thing.  How can we get people with existing profiles to disable the Latest Headlines feed?
(In reply to Nicholas Nethercote [:njn] from comment #21)
> Still, this can only be a good thing.  How can we get people with existing
> profiles to disable the Latest Headlines feed?

That's not the point, if there's a problem we should not try to workaround it
Whiteboard: [MemShrink:P3][see comment 14] → [MemShrink][see comment 14]
Whiteboard: [MemShrink][see comment 14] → [MemShrink:P1][see comment 14]
Whiteboard: [MemShrink:P1][see comment 14] → [MemShrink:P2][see comment 14]
Assignee: n.nethercote → nobody

Comment 24

5 years ago
Just want to add that I am experiencing exact the same problem under Mac OS 10.8 and Firefox 16.0.2.
I have subscribed to a very big RSS feed with ~ 3400 items. After some minutes browsing and looking from time to time into the feed, Firefox' memory consumption increases rapidly.
(Reporter)

Comment 25

5 years ago
Created attachment 688196 [details]
Results immediately after FF restarted with add-ons disabled
(Reporter)

Comment 26

5 years ago
Created attachment 688197 [details]
Results 10 minutes after restart
(Reporter)

Comment 27

5 years ago
Created attachment 688198 [details]
about:memory after 10 hours of no activity
(Reporter)

Comment 28

5 years ago
I have restested using FF18 beta.  Now very little of the memory growth is in heap-unclassified.  
For this process, FF18 was restarted with add-ons disabled.  There are 14 live bookmarks in the bookmark toolbar.
Result 1 - https://bugzilla.mozilla.org/attachment.cgi?id=688196 is immediately after restarting FF
Result 2 - https://bugzilla.mozilla.org/attachment.cgi?id=688197 is after waiting 10 minutes, checking that the bookmarks were loaded, then hitting minimise three times
Result 3 - https://bugzilla.mozilla.org/attachment.cgi?id=688198 is after waiting a further 10 hours.  During that time I refreshed the bookmarks a couple of times but no other activity.  I then waited five minutes after the latest resfresh and hit minimise three times.
I have filed bug 817548 about nsLivemarkService generating a large compartment, would be really interesting to figure out what's leaking, not much for livemarks (small userbase) but more globally could be a win re-applicable to other components.
Status: UNCONFIRMED → NEW
Depends on: 817548
Ever confirmed: true
(Reporter)

Comment 30

5 years ago
I'm delighted this is being picked up, but I question whether live bookmarks has such a small userbase, for two reasons:
1) RSS Feed Reader is one of the features highlighted under the heading "Browsing Made Easy" on http://www.mozilla.org/en-US/firefox/features/
2) Any user who clicks on an RSS feed icon on a web page will automatically create a live bookmark, even though the bookmark toolbar may be hidden.  It is possible that users will have tried this several times creating multiple hidden bookmarks and assumed it did not work
(In reply to andysmozbug from comment #30)
> 2) Any user who clicks on an RSS feed icon on a web page will automatically
> create a live bookmark, even though the bookmark toolbar may be hidden.  It
> is possible that users will have tried this several times creating multiple
> hidden bookmarks and assumed it did not work

This is not a big deal, until the user accesses the first livemark, we don't do anything. So having hidden livemarks is a no-op. Using them is the only way to activate the service.
Created attachment 688508 [details]
analysis of what happened between 10 minutes and 10 hours

Thanks for the useful data, Andy!

I did some analysis of the "after 10 minutes" and "after 10 hours"
compartments.  Full details are in the attached file, but the notable thing is
that four compartments increased by more than 1 MiB(!) during the 9 hours and
50 minutes of idleness.

- nsLivemarkService.js (+19,010,976): Change is mostly due to more objects
  (function and non-function) and shapes (which correlate with objects).

- about:blank (+3,035,720): Change is due to more non-function objects and
  the CCW table, which increased to 1 MiB.  (Note that FF18 didn't
  distinguish CCW *objects*, whereas the trunk now does, so many of the new
  objects are probably CCW objects.)

- XPCOMUtils.jsm (+1,900,912): Change is mostly due to more non-function
  objects, and bit due to more strings.

- WindowDraggingUtils.jsm (+1,040,384): Change is almost entirely due to the
  CCW *table* growing to 1 MiB.  However, there are very few CCW *objects*
  (because there are very few objects of any kind).  So I'd guess that the
  CCW table grew and then all the objects in it died and it never shrunk.
  I don't know how this would happen during 10 hours of idleness, though.

I guess the obvious next question is whether the latter three increases occur
when livemarks aren't used.
> I guess the obvious next question is whether the latter three increases occur
> when livemarks aren't used.

I'll start one shortly and report back tomorrow.
(Reporter)

Comment 34

5 years ago
Just to clarify, as stated earlier the 10 hours of 'idleness' did include refreshing the bookmarks a couple of times, either by clicking once on the entry in the live bookmark to activate the pull-down, or using right-click - Reload Live Bookmark.  However I did not move away from the about:memory page.
I''m happy to do any further tests you suggest.  As you appear to be doing a test with no live bookmarks, how about with bookmarks but no refresh?
(In reply to Nicholas Nethercote [:njn] from comment #32)
> - nsLivemarkService.js (+19,010,976): Change is mostly due to more objects
>   (function and non-function) and shapes (which correlate with objects).

We cache objects (one per entry) but after the first caching shouldn't really grow that much, especially if you didn't touch browser at all during that time.
I did a run with a fresh trunk build:
- Started with a new profile.  (No livemarks!)
- After 10 minutes idling, dumped memory reports with "kill -34 <pid>"
- After 12 hours ideling, dumped memory reports with "kill -35 <pid>" (this 
  triggers a "minimize-memory-use" event first)

Findings:

- "explicit" went from 47.6 MB to 52.4 MB.

- "js-non-window" went from 21.9 MB to 22.8 MB.  Part of that was compartment
  increases, but pretty much all for reasonable-looking things like
  nsPlacesExpiration.js, nsUpdateService.js.  Certainly no changes over 1 MiB.
  
- "heap-unclassified" went from 10.0 MB to 11.4 MB.  Workers may be a factor 
  in this -- osfile_async_worker.js and PageThumbsWorker.js both don't get 
  reported at all (see bug 813867).
  
- "storage/prefixset" went from 0.7 MB to 1.5 MB.  Seems reasonable -- it's 
  all safe-browsing stuff.
  
- "storage/sqlite/{places.sqlite,other}" increased quite a bit, from this:

  ├──2,415,912 B (05.07%) -- sqlite
  │  ├────880,120 B (01.85%) ── other
  │  ├────452,112 B (00.95%) -- places.sqlite
  │  │    ├──365,376 B (00.77%) ── cache-used [2]
  │  │    ├───65,392 B (00.14%) ── schema-used [2]
  │  │    └───21,344 B (00.04%) ── stmt-used [2]

  to this:

  ├──4,178,120 B (07.97%) -- sqlite
  │  ├──1,734,104 B (03.31%) -- places.sqlite
  │  │  ├──1,487,376 B (02.84%) ── cache-used [2] 
  │  │  ├────180,448 B (00.34%) ── stmt-used [2] 
  │  │  └─────66,280 B (00.13%) ── schema-used [2] 
  │  ├──1,120,136 B (02.14%) ── other
  
  This is the thing that surprised me the most.  What places.sqlite activity 
  happens when we're idle?

Anyway, none of the stuff in comment 32 showed up.
Created attachment 688724 [details]
njn results after 10 minutes of idling (JSON)
Created attachment 688725 [details]
njn results after 12 hours of idling and minimize-memory-usage (JSON)
>   This is the thing that surprised me the most.  What places.sqlite activity 
>   happens when we're idle?

maintenance, frecency and other non-critical updates
(Reporter)

Comment 40

5 years ago
I re-ran this test, to make sure idle overnight really was idle.
There are three results, all following restart with no add-ons.  14 Live Bookmarks in the bookmark toolbar
1) Refresh all the live bookmarks by clicking on them once to ensure they are loaded, then wait 10 minutes and minimise three times
2) 8 hours with absolutely no activity, then hit minimise three times
3) Immediately after results 2, refresh the bookmarks one time again, wait ten more minutes and hit miminise three times

Summary  findings:
In the 8 hours wtih no activity, explicit grew 3.4MB.  This is in line with the results in comment 32
Refreshing the bookmarks just once after the long period of inactivity caused explicit to grow another 3.6MB and not shrink
(Reporter)

Comment 41

5 years ago
Created attachment 690221 [details]
10 minutes after restart and live bookmarks refreshed once
(Reporter)

Comment 42

5 years ago
Created attachment 690222 [details]
After completely idle for 8 hours
(Reporter)

Comment 43

5 years ago
Created attachment 690223 [details]
Refresh bookmarks once after 8 hour idle
This problem seems to have been fixed around Beta release 51.  Memory no longer grows continuously during the session. 
I can't be sure exactly when this was fixed.  There were some serious CPU problems during that r51 beta and when these were fixed the memory issues around the RSS bookmarks have been resolved.  The CPU problem may have masked the RSS problem being fixed earlier.
Per comment 44 it sounds like this has been fixed. I'll note that we no longer have UI to subscribe to RSS feeds specified in <link rel> entries in websites, you have to go directly to feed, so if this turns out to still be an issue it should be no more than a MemShrink:P3.
Status: NEW → RESOLVED
Last Resolved: 4 months ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.