1688315 - 1.1GB used by icu in parent process

Status: NEW

(Core :: Internationalization, defect, P3)

Description

[Timothy Nikkel (:tnikkel)]

Attachment: memory-report.json.gz

You'll also find 1.7GB heap unclassified in the parent process in the memory report, that has been around for a while and I think some work related to fonts that hasn't yet made it to beta will likely fix that. But icu being high is new. It was a few hundred MBs yesterday, and now is 1.1GB.

Anything I can do to investigate further?

Comment 1

[Timothy Nikkel (:tnikkel)]

Attachment: about support

Comment 2

[Zibi Braniecki [:zbraniecki][:gandalf]]

Jonathan - does it seem plausible that it's font related?

Comment 3

[Jonathan Kew [:jfkthame]]

I can't think of any reason the large ICU usage would be font-related. The font code does use some Unicode character property lookups, which are ICU-backed, but that shouldn't involve such huge allocations (and I don't recall any recent changes that would have inflated it).

Comment 4

[Zibi Braniecki [:zbraniecki][:gandalf]]

Timothy - can you advise what should we do about this bug? Is it something reproducible? Intermittent?

Comment 5

[Zibi Braniecki [:zbraniecki][:gandalf]]

Andre - does that look familiar to you? I know in bug 1686052 you're about to update us to ICU 68, but I'm not sure if it'll fix this.

Comment 6

[Timothy Nikkel (:tnikkel)]

It's the first time I've ever noticed ICU in my about:memory (which I check frequently due to some heap-unclassified problems I've been having). So either: 1) it's a one off and it'll never happen again, 2) it's triggered by some site or something I did that was unusual in this session of Firefox, although I can't think of anything out of the ordinary that I did, 3) there was some regression that caused it to show up for the first time and it'll happen again (I haven't quit the session yet in case there was something I could do to investigate). When I restart my Firefox session I will see if it comes back and we can start to get a better answer to this.

I don't know what kinds of things are measured in icu, I thought I would file the bug and see if anyone else had any ideas on what to do, or if other people were seeing the same thing they could chime in.

Comment 7

[Timothy Nikkel (:tnikkel)]

Can the ICU memory reporter be broken down to display what contributes to it? That might help.

Comment 8

[André Bargull [:anba]]

No, it doesn't look familiar to me. Maybe it was caused by this recently fixed memory leak: https://phabricator.services.mozilla.com/D102256? Apart from that one, other obvious memory leaks when calling ICU only occur under error conditions:

• Key::EncodeLocaleString doesn't close the UCollator when keyBuffer.SetLength fails.
• nsCollation::EnsureCollator doesn't close the UCollator when one of the ucol_setAttribute calls fails.

The memory leak fixed in https://phabricator.services.mozilla.com/D102256 can be triggered when viewing the history repeatedly. Each leaked UDateFormat can leak up to 90 kb, for the history view up to five UDateFormat are created (last six months, excluding the current month, so "December 2020" to "August 2020"), so we have 5 * 90 kb = 450 kb. Leaking 1.1 gb means that the history was opened at least (1.1 * 1024**2) / 450 = ~2563 times. Assuming the history navigator creates the date/time strings exactly once when opening the history. If they're repeatedly created, for example when searching for a specific entry and the view is live updated, it should be easier to accumulate a few megabytes.

---

The ICU memory reporter works by wrapping malloc and free using the built-in ICU memory allocator interface. So whenever ICU needs to allocate some memory, the memory allocation goes through our ICU memory reporter where we accumulate the amount of requested memory. And later on, when ICU frees the memory, the memory reporter subtracts the free'ed memory from the overall allocated memory.

So, in order to create a more fine-grained memory overview, we'd need to wrap each and every call to ICU to check how much memory was allocated. So maybe something along the lines of:

enum class ICUMemoryId { /* ... */ };

template <typename F, typename... Args>
auto CallICU(ICUMemoryId id, F f, Args&&... args) {
  ICUReporter reporter = GetICUReporter();
  auto before = reporter.MemoryAllocated();
  auto res = f(std::forward<Args>(args)...);
  auto after = reporter.MemoryAllocated();
  TrackMemory(id, after - before);
  return res;
}

UDateFormat* df = CallICU(
  // Identifier to record the allocation happens in DateTimeFormat::FormatDateTime.
  ICUMemoryId::Locale_DateTimeFormat_FormatDateTime,
  udat_open, ...);

(This is ignoring that ICUReporter is currently not visible outside of "XPCOMInit.cpp" and the SpiderMonkey won't be able to use that approach, because ICUReporter uses classes like nsIMemoryReporter which aren't accessible in SpiderMonkey.)

Comment 9

[Zibi Braniecki [:zbraniecki][:gandalf]]

Timothy: can you try to reproduce in the current nightly?

Comment 10

[Timothy Nikkel (:tnikkel)]

(In reply to André Bargull [:anba] from comment #8)

The memory leak fixed in https://phabricator.services.mozilla.com/D102256 can be triggered when viewing the history repeatedly. Each leaked UDateFormat can leak up to 90 kb, for the history view up to five UDateFormat are created (last six months, excluding the current month, so "December 2020" to "August 2020"), so we have 5 * 90 kb = 450 kb. Leaking 1.1 gb means that the history was opened at least (1.1 * 1024**2) / 450 = ~2563 times. Assuming the history navigator creates the date/time strings exactly once when opening the history. If they're repeatedly created, for example when searching for a specific entry and the view is live updated, it should be easier to accumulate a few megabytes.

I don't use history nearly that much.

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #9)

Timothy: can you try to reproduce in the current nightly?

Since this happened on my main profile I'm always trying to reproduce. I run beta, and I just updated to the latest, so it's basically nightly from a day ago now. I hope that is good enough. After a few days I'll report what I find.