Closed Bug 692748 Opened 13 years ago Closed 7 years ago

Increasing memory due to RSS feeds in bookmarks folder

Categories

(Firefox :: General, defect)

Product:
Firefox
Component:
General
Version:
8 Branch
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: andysmozbug, Unassigned)

References

Details

(Whiteboard: [MemShrink:P2][see comment 14])

Attachments

(10 files)

DMD output
7.29 KB, text/plain
Details
Results immediately after FF restarted with add-ons disabled
141.14 KB, text/plain
Details
Results 10 minutes after restart
135.77 KB, text/plain
Details
about:memory after 10 hours of no activity
143.25 KB, text/plain
Details
analysis of what happened between 10 minutes and 10 hours
6.50 KB, text/plain
Details
njn results after 10 minutes of idling (JSON)
805.30 KB, text/plain
Details
njn results after 12 hours of idling and minimize-memory-usage (JSON)
834.95 KB, application/json
Details
10 minutes after restart and live bookmarks refreshed once
67.45 KB, text/plain
Details
After completely idle for 8 hours
140.03 KB, text/plain
Details
Refresh bookmarks once after 8 hour idle
69.72 KB, text/plain
Details
User Agent: Mozilla/5.0 (Windows NT 6.0; rv:8.0) Gecko/20100101 Firefox/8.0 Build ID: 20110928060149 Steps to reproduce: [MemShrink] Restarted FF7 in safe mode with two tabs open (about:memory and https://wiki.mozilla.org/Performance/MemShrink ) My setup has around eight RSS feeds in the bookmark toolbar including the following feed locations: http://www.bbc.co.uk/go/rss/int/news/-/news/ http://bishophill.squarespace.com/blog/rss.xml http://www2.politicalbetting.com/index.php/feed/ http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/front_page/rss.xml http://www.spectator.co.uk/coffeehouse/index.txml http://feeds2.feedburner.com/guidofawkes http://www.iaindale.com/feed http://feeds.feedburner.com/arseblognews?format=xml http://noconsensus.wordpress.com/feed/ Actual results: With the RSS feeds, after 12 hours resident memory grew from 80MB to 394, and then reduced to 192MB after hitting minimise in about:memory Repeating the test after deleting the RSS from the toolbar, after eight hours resident memory grew from 79 to 104, and did not shrink after hitting minimise In FF8, on a different machine, similar results to the first test with the RSS feeds present
Whiteboard: [memshrink]
OS: Windows Vista → Windows 7
For clarity, these tests were overnight so during these tests FF was not being used
Can you provide/attach before/after about:memory Output with "after" = kept running Firefox X Hours?
The following test was carried out running FF8. Two tabs open: about:memory?verbose and https://wiki.mozilla.org/Performance/MemShrink This test was on Windows Vista, though I get similar results on a Windows 7 machine running FF7 The first set of results were from about 10 minutes after restarting FF in safe mode. During that time resident memory had grown rapidly from 60MB to 80MB then stabilised. The second results were 8 hours later after hitting minimise. 10 minutes after starting: Main Process Explicit Allocations 36,035,796 B (100.0%) -- explicit ├──16,097,800 B (44.67%) -- heap-unclassified ├──13,901,450 B (38.58%) -- js │ ├───6,883,138 B (19.10%) -- compartment([System Principal], 0x5a7a000) │ │ ├──3,969,024 B (11.01%) -- gc-heap │ │ │ ├──2,099,576 B (05.83%) -- objects │ │ │ ├──1,356,280 B (03.76%) -- shapes │ │ │ ├────314,656 B (00.87%) -- arena-unused │ │ │ ├────173,264 B (00.48%) -- strings │ │ │ ├─────15,504 B (00.04%) -- arena-headers │ │ │ ├──────9,600 B (00.03%) -- arena-padding │ │ │ └────────144 B (00.00%) -- xml │ │ ├──1,455,360 B (04.04%) -- scripts │ │ ├────464,664 B (01.29%) -- object-slots │ │ ├────342,268 B (00.95%) -- property-tables │ │ ├────306,654 B (00.85%) -- string-chars │ │ ├────214,096 B (00.59%) -- tjit-data │ │ │ ├──118,896 B (00.33%) -- trace-monitor │ │ │ ├───74,000 B (00.21%) -- allocators-reserve │ │ │ └───21,200 B (00.06%) -- allocators-main │ │ └────131,072 B (00.36%) -- mjit-code │ ├───4,380,444 B (12.16%) -- gc-heap-chunk-dirty-unused │ ├─────990,640 B (02.75%) -- compartment(atoms) │ │ ├──564,656 B (01.57%) -- string-chars │ │ └──425,984 B (01.18%) -- gc-heap │ │ ├──415,808 B (01.15%) -- strings │ │ ├────7,632 B (00.02%) -- arena-unused │ │ ├────1,664 B (00.00%) -- arena-headers │ │ └──────880 B (00.00%) -- arena-padding │ ├─────804,586 B (02.23%) -- compartment(https://wiki.mozilla.org/Performance/MemShrink) │ │ ├──487,424 B (01.35%) -- gc-heap │ │ │ ├──209,280 B (00.58%) -- objects │ │ │ ├──185,480 B (00.51%) -- shapes │ │ │ ├───84,152 B (00.23%) -- arena-unused │ │ │ ├────5,568 B (00.02%) -- strings │ │ │ ├────1,904 B (00.01%) -- arena-headers │ │ │ └────1,040 B (00.00%) -- arena-padding │ │ ├──187,074 B (00.52%) -- scripts │ │ ├───65,536 B (00.18%) -- mjit-code │ │ ├───28,720 B (00.08%) -- object-slots │ │ ├───27,968 B (00.08%) -- property-tables │ │ └────7,864 B (00.02%) -- string-chars │ ├─────403,456 B (01.12%) -- runtime │ │ ├──262,144 B (00.73%) -- atoms-table │ │ └──141,312 B (00.39%) -- runtime-object │ ├─────262,144 B (00.73%) -- stack │ ├─────153,828 B (00.43%) -- gc-heap-chunk-admin │ ├──────23,214 B (00.06%) -- compartment(moz-nullprincipal:{e6336f61-dfb6-4524-bebb-bd2f187e1679}) │ │ ├──20,480 B (00.06%) -- gc-heap │ │ │ ├──14,800 B (00.04%) -- arena-unused │ │ │ ├───3,352 B (00.01%) -- objects │ │ │ ├───2,200 B (00.01%) -- shapes │ │ │ ├──────80 B (00.00%) -- arena-headers │ │ │ └──────48 B (00.00%) -- arena-padding │ │ ├───2,448 B (00.01%) -- object-slots │ │ ├─────168 B (00.00%) -- property-tables │ │ └─────118 B (00.00%) -- scripts │ └───────────0 B (00.00%) -- gc-heap-chunk-clean-unused ├───3,669,152 B (10.18%) -- storage │ └──3,669,152 B (10.18%) -- sqlite │ ├──1,328,800 B (03.69%) -- urlclassifier3.sqlite │ │ ├──1,244,744 B (03.45%) -- cache-used │ │ ├─────81,392 B (00.23%) -- stmt-used │ │ └──────2,664 B (00.01%) -- schema-used │ ├────966,736 B (02.68%) -- cookies.sqlite │ │ ├──954,736 B (02.65%) -- cache-used │ │ ├───10,160 B (00.03%) -- stmt-used │ │ └────1,840 B (00.01%) -- schema-used │ ├────660,336 B (01.83%) -- places.sqlite │ │ ├──576,664 B (01.60%) -- cache-used [4] │ │ ├───47,872 B (00.13%) -- schema-used [4] │ │ └───35,800 B (00.10%) -- stmt-used [4] │ ├────619,816 B (01.72%) -- other │ ├─────48,896 B (00.14%) -- downloads.sqlite │ │ ├──41,128 B (00.11%) -- cache-used │ │ ├───5,984 B (00.02%) -- stmt-used │ │ └───1,784 B (00.00%) -- schema-used │ ├─────16,872 B (00.05%) -- content-prefs.sqlite │ │ ├───8,128 B (00.02%) -- stmt-used │ │ ├───6,344 B (00.02%) -- cache-used │ │ └───2,400 B (00.01%) -- schema-used │ ├─────12,152 B (00.03%) -- signons.sqlite │ │ ├───7,480 B (00.02%) -- cache-used │ │ ├───2,824 B (00.01%) -- schema-used │ │ └───1,848 B (00.01%) -- stmt-used │ ├─────11,024 B (00.03%) -- permissions.sqlite │ │ ├───5,744 B (00.02%) -- stmt-used │ │ ├───4,016 B (00.01%) -- cache-used │ │ └───1,264 B (00.00%) -- schema-used │ └──────4,520 B (00.01%) -- formhistory.sqlite │ ├──2,856 B (00.01%) -- cache-used │ ├──1,664 B (00.00%) -- schema-used │ └──────0 B (00.00%) -- stmt-used ├───1,054,154 B (02.93%) -- layout │ ├────716,998 B (01.99%) -- arenas │ └────337,156 B (00.94%) -- styledata ├─────911,384 B (02.53%) -- xpti-working-set ├─────228,053 B (00.63%) -- images │ ├──151,284 B (00.42%) -- chrome │ │ ├──151,284 B (00.42%) -- used │ │ │ ├──151,284 B (00.42%) -- uncompressed-nonheap │ │ │ ├────────0 B (00.00%) -- raw │ │ │ └────────0 B (00.00%) -- uncompressed-heap │ │ └────────0 B (00.00%) -- unused │ │ ├──0 B (00.00%) -- raw │ │ ├──0 B (00.00%) -- uncompressed-heap │ │ └──0 B (00.00%) -- uncompressed-nonheap │ └───76,769 B (00.21%) -- content │ ├──76,769 B (00.21%) -- used │ │ ├──72,529 B (00.20%) -- raw │ │ ├───4,240 B (00.01%) -- uncompressed-nonheap │ │ └───────0 B (00.00%) -- uncompressed-heap │ └───────0 B (00.00%) -- unused │ ├──0 B (00.00%) -- raw │ ├──0 B (00.00%) -- uncompressed-heap │ └──0 B (00.00%) -- uncompressed-nonheap ├─────171,575 B (00.48%) -- dom └───────2,228 B (00.01%) -- cycle-collector Other Measurements 0 B -- gfx-d2d-surfacecache 0 B -- gfx-d2d-surfacevram 4,880 B -- gfx-surface-image 157,008 B -- gfx-surface-win32 35,018,064 B -- heap-allocated 40,665,088 B -- heap-committed 3,330,048 B -- heap-dirty 7,972,702 B -- heap-unallocated 2 -- js-compartments-system 2 -- js-compartments-user 9,437,184 B -- js-gc-heap 421,240 B -- js-gc-heap-arena-unused 0 B -- js-gc-heap-chunk-clean-unused 4,380,444 B -- js-gc-heap-chunk-dirty-unused 50.88% -- js-gc-heap-unused-fraction 60,141,568 B -- private 79,167,488 B -- resident 242,143,232 B -- vsize After 8 hours the hitting minimise a few times: Main Process Explicit Allocations 104,814,644 B (100.0%) -- explicit ├───42,414,320 B (40.47%) -- js │ ├──31,702,128 B (30.25%) -- gc-heap-chunk-dirty-unused │ ├───7,587,016 B (07.24%) -- compartment([System Principal], 0x5a7a000) │ │ ├──4,485,120 B (04.28%) -- gc-heap │ │ │ ├──1,881,192 B (01.79%) -- objects │ │ │ ├──1,338,160 B (01.28%) -- shapes │ │ │ ├──1,186,816 B (01.13%) -- arena-unused │ │ │ ├─────50,288 B (00.05%) -- strings │ │ │ ├─────17,520 B (00.02%) -- arena-headers │ │ │ ├─────11,000 B (00.01%) -- arena-padding │ │ │ └────────144 B (00.00%) -- xml │ │ ├──1,477,192 B (01.41%) -- scripts │ │ ├────458,752 B (00.44%) -- mjit-code │ │ ├────455,496 B (00.43%) -- object-slots │ │ ├────363,164 B (00.35%) -- property-tables │ │ ├────214,096 B (00.20%) -- tjit-data │ │ │ ├──118,896 B (00.11%) -- trace-monitor │ │ │ ├───74,000 B (00.07%) -- allocators-reserve │ │ │ └───21,200 B (00.02%) -- allocators-main │ │ └────133,196 B (00.13%) -- string-chars │ ├───1,016,464 B (00.97%) -- compartment(atoms) │ │ ├────578,192 B (00.55%) -- string-chars │ │ └────438,272 B (00.42%) -- gc-heap │ │ ├──429,200 B (00.41%) -- strings │ │ ├────6,464 B (00.01%) -- arena-unused │ │ ├────1,712 B (00.00%) -- arena-headers │ │ └──────896 B (00.00%) -- arena-padding │ ├─────804,586 B (00.77%) -- compartment(https://wiki.mozilla.org/Performance/MemShrink) │ │ ├──487,424 B (00.47%) -- gc-heap │ │ │ ├──209,280 B (00.20%) -- objects │ │ │ ├──185,480 B (00.18%) -- shapes │ │ │ ├───84,152 B (00.08%) -- arena-unused │ │ │ ├────5,568 B (00.01%) -- strings │ │ │ ├────1,904 B (00.00%) -- arena-headers │ │ │ └────1,040 B (00.00%) -- arena-padding │ │ ├──187,074 B (00.18%) -- scripts │ │ ├───65,536 B (00.06%) -- mjit-code │ │ ├───28,720 B (00.03%) -- object-slots │ │ ├───27,968 B (00.03%) -- property-tables │ │ └────7,864 B (00.01%) -- string-chars │ ├─────615,312 B (00.59%) -- gc-heap-chunk-admin │ ├─────403,456 B (00.38%) -- runtime │ │ ├──262,144 B (00.25%) -- atoms-table │ │ └──141,312 B (00.13%) -- runtime-object │ ├─────262,144 B (00.25%) -- stack │ ├──────23,214 B (00.02%) -- compartment(moz-nullprincipal:{e6336f61-dfb6-4524-bebb-bd2f187e1679}) │ │ ├──20,480 B (00.02%) -- gc-heap │ │ │ ├──14,800 B (00.01%) -- arena-unused │ │ │ ├───3,352 B (00.00%) -- objects │ │ │ ├───2,200 B (00.00%) -- shapes │ │ │ ├──────80 B (00.00%) -- arena-headers │ │ │ └──────48 B (00.00%) -- arena-padding │ │ ├───2,448 B (00.00%) -- object-slots │ │ ├─────168 B (00.00%) -- property-tables │ │ └─────118 B (00.00%) -- scripts │ └───────────0 B (00.00%) -- gc-heap-chunk-clean-unused ├───35,484,865 B (33.85%) -- heap-unclassified ├───24,619,824 B (23.49%) -- storage │ └──24,619,824 B (23.49%) -- sqlite │ ├──14,396,912 B (13.74%) -- places.sqlite │ │ ├──14,156,200 B (13.51%) -- cache-used [4] │ │ ├─────192,224 B (00.18%) -- stmt-used [4] │ │ └──────48,488 B (00.05%) -- schema-used [4] │ ├───8,548,592 B (08.16%) -- urlclassifier3.sqlite │ │ ├──8,464,536 B (08.08%) -- cache-used │ │ ├─────81,392 B (00.08%) -- stmt-used │ │ └──────2,664 B (00.00%) -- schema-used │ ├─────966,736 B (00.92%) -- cookies.sqlite │ │ ├──954,736 B (00.91%) -- cache-used │ │ ├───10,160 B (00.01%) -- stmt-used │ │ └────1,840 B (00.00%) -- schema-used │ ├─────614,120 B (00.59%) -- other │ ├──────48,896 B (00.05%) -- downloads.sqlite │ │ ├──41,128 B (00.04%) -- cache-used │ │ ├───5,984 B (00.01%) -- stmt-used │ │ └───1,784 B (00.00%) -- schema-used │ ├──────16,872 B (00.02%) -- content-prefs.sqlite │ │ ├───8,128 B (00.01%) -- stmt-used │ │ ├───6,344 B (00.01%) -- cache-used │ │ └───2,400 B (00.00%) -- schema-used │ ├──────12,152 B (00.01%) -- signons.sqlite │ │ ├───7,480 B (00.01%) -- cache-used │ │ ├───2,824 B (00.00%) -- schema-used │ │ └───1,848 B (00.00%) -- stmt-used │ ├──────11,024 B (00.01%) -- permissions.sqlite │ │ ├───5,744 B (00.01%) -- stmt-used │ │ ├───4,016 B (00.00%) -- cache-used │ │ └───1,264 B (00.00%) -- schema-used │ └───────4,520 B (00.00%) -- formhistory.sqlite │ ├──2,856 B (00.00%) -- cache-used │ ├──1,664 B (00.00%) -- schema-used │ └──────0 B (00.00%) -- stmt-used ├──────972,455 B (00.93%) -- layout │ ├──678,083 B (00.65%) -- arenas │ └──294,372 B (00.28%) -- styledata ├──────911,384 B (00.87%) -- xpti-working-set ├──────228,053 B (00.22%) -- images │ ├──151,284 B (00.14%) -- chrome │ │ ├──151,284 B (00.14%) -- used │ │ │ ├──151,284 B (00.14%) -- uncompressed-nonheap │ │ │ ├────────0 B (00.00%) -- raw │ │ │ └────────0 B (00.00%) -- uncompressed-heap │ │ └────────0 B (00.00%) -- unused │ │ ├──0 B (00.00%) -- raw │ │ ├──0 B (00.00%) -- uncompressed-heap │ │ └──0 B (00.00%) -- uncompressed-nonheap │ └───76,769 B (00.07%) -- content │ ├──76,769 B (00.07%) -- used │ │ ├──72,529 B (00.07%) -- raw │ │ ├───4,240 B (00.00%) -- uncompressed-nonheap │ │ └───────0 B (00.00%) -- uncompressed-heap │ └───────0 B (00.00%) -- unused │ ├──0 B (00.00%) -- raw │ ├──0 B (00.00%) -- uncompressed-heap │ └──0 B (00.00%) -- uncompressed-nonheap ├──────171,235 B (00.16%) -- dom └───────12,508 B (00.01%) -- cycle-collector Other Measurements 0 B -- gfx-d2d-surfacecache 0 B -- gfx-d2d-surfacevram 4,880 B -- gfx-surface-image 157,008 B -- gfx-surface-win32 103,469,232 B -- heap-allocated 112,480,256 B -- heap-committed 3,637,248 B -- heap-dirty 111,488,142 B -- heap-unallocated 2 -- js-compartments-system 2 -- js-compartments-user 37,748,736 B -- js-gc-heap 1,292,232 B -- js-gc-heap-arena-unused 0 B -- js-gc-heap-chunk-clean-unused 31,702,128 B -- js-gc-heap-chunk-dirty-unused 87.40% -- js-gc-heap-unused-fraction 132,763,648 B -- private 133,341,184 B -- resident 417,583,104 B -- vsize
so, the interesting increases are heap-unclassified and gc-heap-chunk-dirty-unused, it's possible something is not freed up in livemarks reloads, those involve loadgroups, feedparser and a bunch of other stuff.
Nick, can you point your dark matter detector at this case and see what's hiding in heap-unclassified here? This doesn't seem like something lots of people would run into though, since we've mostly hidden the bookmarks folder feed feature...
Assignee: nobody → nnethercote
Whiteboard: [memshrink] → [MemShrink:P3]
No longer depends on: 670596
Blocks: 670596
(In reply to Johnny Stenback (:jst, jst@mozilla.com) from comment #5) > This doesn't seem like something lots of people would run into though, since > we've mostly hidden the bookmarks folder feed feature... Right, but default bookmarks, added to each profile, include a live bookmark (the Latest News one), so this may potentially affect many.
(In reply to Marco Bonardo [:mak] from comment #7) > > Right, but default bookmarks, added to each profile, include a live bookmark > (the Latest News one), so this may potentially affect many. Indeed, I think the "Latest News" shouldn't be on by default. Should I file a separate bug about that?
(In reply to Nicholas Nethercote [:njn] (on vacation Sep 12--Oct 17) from comment #8) > Indeed, I think the "Latest News" shouldn't be on by default. Should I file > a separate bug about that? you're free to do that, but while it's trivial to remove it for new users, it may be not to remove it for old users (and most likely even not desirable in case they use it)
I believe this bug could effect large numbers of users The FF8 URL row has an RSS icon between reload and stop, and the default behaviour on a page that supports RSS is to add a subscription to the bookmarks toolbar. The same applies if a user clicks on an RSS icon on a web page (e.g. top right within the header on page http://www.bbc.co.uk/news/ ) If the bookmark toolbar is hidden, users may have multiple subscriptions without realising it The alternative option when clicking on an RSS icon is to subscribe into the bookmarks folders. I have not checked yet whether this would result in the same memory growth
Blocks: DarkMatter
Depends on: DMD
No longer depends on: DarkMatter
Attached file DMD output —
I created a new profile with the usual "Latest Headlines" feed plus the eight from comment 1, started up, and viewed about:memory a few times. The memory usage went up and down a bit as the minutes passed, and heap-unclassified varied in the range 30% to 50%, going down up and down in a fashion that looked like GC was involved. Which is surprising, because the JS engine is extremely well covered by memory reporters. Firing up DMD, I saw a number of unfamiliar entries, all relating to xpconnect native thingies. (I also saw a big nsStringBuffer entry that lacked sufficient context to tell where it came from, but I bet it's related.) Looks like adding memory reporters for them will be useful in this case, and hopefully others.
(In reply to Nicholas Nethercote [:njn] from comment #11) > (I also saw a big nsStringBuffer entry that > lacked sufficient context to tell where it came from, but I bet it's > related.) hm, to update feeds we have to parse the rss contents that may be this big string entry, this is done by the feedProcessor component, and considering its code, I'd not be surprised if it'd leak. Btw, hard to tell off-hand, I'll take a look there to check for evident offenders. That code should likely be deCOMtamined too ( I should have a patch somewhere on disk that does that). This bug may explain why I often see a lot of unclassified memory compared to others, I have quite some feeds.
(In reply to Nicholas Nethercote [:njn] from comment #8) > > Indeed, I think the "Latest News" shouldn't be on by default. Should I file > a separate bug about that? Filed as bug 697038.
I spun off bug 697041 for adding memory reporters to xpconnect, which will give us measurements of the memory usage. This bug should now be about whether we change anything relating to the implementation of RSS feeds.
No longer blocks: DarkMatter
Whiteboard: [MemShrink:P3] → [MemShrink:P3][see comment 14]
njn, could we possibly get DMD to dump another 2-3 stack frames? Right now string memory allocations are completely opaque because the callstack from string Assign to the malloc is already 7 frames or so deep, so we have no idea who's calling Assign. Similar for the xpconnect gunk (including the string allocations via ReadableToJsval!); some of it I'd love to know who the callers are.
(In reply to Boris Zbarsky (:bz) from comment #15) > njn, could we possibly get DMD to dump another 2-3 stack frames? I did this in bug 697041.
Depends on: 697332
If I should trust tree-management numbers after bug 696163 landed (read this as Trace-malloc maxheap improvements of up to 30%) I should think we actually have some more issues than just the need to measure xpconnect. Something starting from feedprocessor is clearly leaking or acting weird.
I don't see any noticeable changes to tracemalloc maxheap numbers at graphs-new.mozilla.org. (The MacOSX 10.5.2 do vary hugely, though.) Which platform was the claimed improvement on? I'd be delighted but highly surprised if this improvement was real; I've done a lot of memory profiling and I've seen nothing that could explain such an improvement.
looks like it is sticking the values to low peaks that were already present in the past, really, like if something could or not be executed.
Wow, those graphs are impressive... but I see no change on Linux for either "allocs" or "maxHeap", nor for Mac on "maxHeap". And I don't know how to interpret the bi-modality of those graphs in general. Still, this can only be a good thing. How can we get people with existing profiles to disable the Latest Headlines feed?
(In reply to Nicholas Nethercote [:njn] from comment #21) > Still, this can only be a good thing. How can we get people with existing > profiles to disable the Latest Headlines feed? That's not the point, if there's a problem we should not try to workaround it
Sure, let's do both.
Whiteboard: [MemShrink:P3][see comment 14] → [MemShrink][see comment 14]
Whiteboard: [MemShrink][see comment 14] → [MemShrink:P1][see comment 14]
Whiteboard: [MemShrink:P1][see comment 14] → [MemShrink:P2][see comment 14]
Assignee: n.nethercote → nobody
Just want to add that I am experiencing exact the same problem under Mac OS 10.8 and Firefox 16.0.2. I have subscribed to a very big RSS feed with ~ 3400 items. After some minutes browsing and looking from time to time into the feed, Firefox' memory consumption increases rapidly.
I have restested using FF18 beta. Now very little of the memory growth is in heap-unclassified. For this process, FF18 was restarted with add-ons disabled. There are 14 live bookmarks in the bookmark toolbar. Result 1 - https://bugzilla.mozilla.org/attachment.cgi?id=688196 is immediately after restarting FF Result 2 - https://bugzilla.mozilla.org/attachment.cgi?id=688197 is after waiting 10 minutes, checking that the bookmarks were loaded, then hitting minimise three times Result 3 - https://bugzilla.mozilla.org/attachment.cgi?id=688198 is after waiting a further 10 hours. During that time I refreshed the bookmarks a couple of times but no other activity. I then waited five minutes after the latest resfresh and hit minimise three times.
I have filed bug 817548 about nsLivemarkService generating a large compartment, would be really interesting to figure out what's leaking, not much for livemarks (small userbase) but more globally could be a win re-applicable to other components.
Status: UNCONFIRMED → NEW
Depends on: 817548
Ever confirmed: true
I'm delighted this is being picked up, but I question whether live bookmarks has such a small userbase, for two reasons: 1) RSS Feed Reader is one of the features highlighted under the heading "Browsing Made Easy" on http://www.mozilla.org/en-US/firefox/features/ 2) Any user who clicks on an RSS feed icon on a web page will automatically create a live bookmark, even though the bookmark toolbar may be hidden. It is possible that users will have tried this several times creating multiple hidden bookmarks and assumed it did not work
(In reply to andysmozbug from comment #30) > 2) Any user who clicks on an RSS feed icon on a web page will automatically > create a live bookmark, even though the bookmark toolbar may be hidden. It > is possible that users will have tried this several times creating multiple > hidden bookmarks and assumed it did not work This is not a big deal, until the user accesses the first livemark, we don't do anything. So having hidden livemarks is a no-op. Using them is the only way to activate the service.
Thanks for the useful data, Andy! I did some analysis of the "after 10 minutes" and "after 10 hours" compartments. Full details are in the attached file, but the notable thing is that four compartments increased by more than 1 MiB(!) during the 9 hours and 50 minutes of idleness. - nsLivemarkService.js (+19,010,976): Change is mostly due to more objects (function and non-function) and shapes (which correlate with objects). - about:blank (+3,035,720): Change is due to more non-function objects and the CCW table, which increased to 1 MiB. (Note that FF18 didn't distinguish CCW *objects*, whereas the trunk now does, so many of the new objects are probably CCW objects.) - XPCOMUtils.jsm (+1,900,912): Change is mostly due to more non-function objects, and bit due to more strings. - WindowDraggingUtils.jsm (+1,040,384): Change is almost entirely due to the CCW *table* growing to 1 MiB. However, there are very few CCW *objects* (because there are very few objects of any kind). So I'd guess that the CCW table grew and then all the objects in it died and it never shrunk. I don't know how this would happen during 10 hours of idleness, though. I guess the obvious next question is whether the latter three increases occur when livemarks aren't used.
> I guess the obvious next question is whether the latter three increases occur > when livemarks aren't used. I'll start one shortly and report back tomorrow.
Just to clarify, as stated earlier the 10 hours of 'idleness' did include refreshing the bookmarks a couple of times, either by clicking once on the entry in the live bookmark to activate the pull-down, or using right-click - Reload Live Bookmark. However I did not move away from the about:memory page. I''m happy to do any further tests you suggest. As you appear to be doing a test with no live bookmarks, how about with bookmarks but no refresh?
(In reply to Nicholas Nethercote [:njn] from comment #32) > - nsLivemarkService.js (+19,010,976): Change is mostly due to more objects > (function and non-function) and shapes (which correlate with objects). We cache objects (one per entry) but after the first caching shouldn't really grow that much, especially if you didn't touch browser at all during that time.
I did a run with a fresh trunk build: - Started with a new profile. (No livemarks!) - After 10 minutes idling, dumped memory reports with "kill -34 <pid>" - After 12 hours ideling, dumped memory reports with "kill -35 <pid>" (this triggers a "minimize-memory-use" event first) Findings: - "explicit" went from 47.6 MB to 52.4 MB. - "js-non-window" went from 21.9 MB to 22.8 MB. Part of that was compartment increases, but pretty much all for reasonable-looking things like nsPlacesExpiration.js, nsUpdateService.js. Certainly no changes over 1 MiB. - "heap-unclassified" went from 10.0 MB to 11.4 MB. Workers may be a factor in this -- osfile_async_worker.js and PageThumbsWorker.js both don't get reported at all (see bug 813867). - "storage/prefixset" went from 0.7 MB to 1.5 MB. Seems reasonable -- it's all safe-browsing stuff. - "storage/sqlite/{places.sqlite,other}" increased quite a bit, from this: ├──2,415,912 B (05.07%) -- sqlite │ ├────880,120 B (01.85%) ── other │ ├────452,112 B (00.95%) -- places.sqlite │ │ ├──365,376 B (00.77%) ── cache-used [2] │ │ ├───65,392 B (00.14%) ── schema-used [2] │ │ └───21,344 B (00.04%) ── stmt-used [2] to this: ├──4,178,120 B (07.97%) -- sqlite │ ├──1,734,104 B (03.31%) -- places.sqlite │ │ ├──1,487,376 B (02.84%) ── cache-used [2] │ │ ├────180,448 B (00.34%) ── stmt-used [2] │ │ └─────66,280 B (00.13%) ── schema-used [2] │ ├──1,120,136 B (02.14%) ── other This is the thing that surprised me the most. What places.sqlite activity happens when we're idle? Anyway, none of the stuff in comment 32 showed up.
> This is the thing that surprised me the most. What places.sqlite activity > happens when we're idle? maintenance, frecency and other non-critical updates
I re-ran this test, to make sure idle overnight really was idle. There are three results, all following restart with no add-ons. 14 Live Bookmarks in the bookmark toolbar 1) Refresh all the live bookmarks by clicking on them once to ensure they are loaded, then wait 10 minutes and minimise three times 2) 8 hours with absolutely no activity, then hit minimise three times 3) Immediately after results 2, refresh the bookmarks one time again, wait ten more minutes and hit miminise three times Summary findings: In the 8 hours wtih no activity, explicit grew 3.4MB. This is in line with the results in comment 32 Refreshing the bookmarks just once after the long period of inactivity caused explicit to grow another 3.6MB and not shrink
This problem seems to have been fixed around Beta release 51. Memory no longer grows continuously during the session. I can't be sure exactly when this was fixed. There were some serious CPU problems during that r51 beta and when these were fixed the memory issues around the RSS bookmarks have been resolved. The CPU problem may have masked the RSS problem being fixed earlier.
Per comment 44 it sounds like this has been fixed. I'll note that we no longer have UI to subscribe to RSS feeds specified in <link rel> entries in websites, you have to go directly to feed, so if this turns out to still be an issue it should be no more than a MemShrink:P3.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: