Closed Bug 1322027 Opened 3 years ago Closed 3 years ago

Update jemalloc 4 to version 4.4.0

Categories

(Core :: Memory Allocator, defect)

defect
Not set

Tracking

()

RESOLVED FIXED
mozilla53
Tracking Status
firefox53 --- fixed

People

(Reporter: cpeterson, Assigned: RyanVM)

References

Details

Attachments

(1 file)

https://github.com/jemalloc/jemalloc/releases/tag/4.4.0

* 4.4.0 (2016-12-03)

New features:
  - Add configure support for *-*-linux-android. (@cferris1000, @jasone)
  - Add the --disable-syscall configure option, for use on systems that place
    security-motivated limitations on syscall(2). (@jasone)
  - Add support for Debian GNU/kFreeBSD. (@thesam)

Optimizations:
  - Add extent serial numbers and use them where appropriate as a sort key that
    is higher priority than address, so that the allocation policy prefers
    older extents. This tends to improve locality (decrease fragmentation) when
    memory grows downward. (@jasone)
  - Refactor madvise(2) configuration so that MADV_FREE is detected and
    utilized on Linux 4.5 and newer. (@jasone)
  - Mark partially purged arena chunks as non-huge-page. This improves
    interaction with Linux's transparent huge page functionality. (@jasone)

Bug fixes:
  - Fix size class computations for edge conditions involving extremely large
    allocations. This regression was first released in 4.0.0. (@jasone,
    @ingvarha)
  - Remove overly restrictive assertions related to the cactive statistic. This
    regression was first released in 4.1.0. (@jasone)
  - Implement a more reliable detection scheme for os_unfair_lock on macOS.
    (@jszakmeister)
FWIW, I'd been playing around with 4.4.0 on Try for a bit leading up to the release and while everything works OK for the most part, Linux32 TC PGO builds appear to permacrash during startup cache precompilation. Annoyingly, it doesn't reproduce on buildbot builds nor on subsequent test runs if I hack the harness to skip that step. It was a regression from https://github.com/jemalloc/jemalloc/commit/e98a620c59ac20b13e2de796164cc67f050ed2bf, but I'm not sure how much help we're going to get from upstream without any real diagnostic information.
Blocks: 1200951
Here's the patch, anyway.

The Try run below shows the Linux32 TC PGO issue I mentioned before. TBH, I'm not sure it *needs* to block this from landing since jemalloc4 is off by default anyway. But it'll certainly end up blocking enabling assuming we still support Linux32 by that point.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e46c89db123cde5c262b8a7a0d758aba7503be6d
Any thoughts on comment 2, Mike? I'm out of useful ideas for debugging at this point and I don't think we're going to get far with upstream without useful STR.
Flags: needinfo?(mh+mozilla)
I'm not committing to owning this bug given the possibly non-trivial issues that need answers still.
Assignee: ryanvm → nobody
Status: ASSIGNED → NEW
Attachment #8817465 - Flags: review?(mh+mozilla)
Comment on attachment 8817465 [details] [diff] [review]
update jemalloc 4 to version 4.4.0

Review of attachment 8817465 [details] [diff] [review]:
-----------------------------------------------------------------

It would be worth trying to force disable the new hugepage thing on Linux. Try adding je_cv_thp=no the same way some ac_cv_* are added in build/autoconf/jemalloc.m4
Attachment #8817465 - Flags: review?(mh+mozilla) → review+
Flags: needinfo?(mh+mozilla)
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e0ee0cfe7c87
Update jemalloc 4 to version 4.4.0. r=glandium
(In reply to Mike Hommey [:glandium] from comment #5)
> It would be worth trying to force disable the new hugepage thing on Linux.
> Try adding je_cv_thp=no the same way some ac_cv_* are added in
> build/autoconf/jemalloc.m4

I did a Try push with this change and it came back green, so it was included in the push in comment 6. However, I also did another Try push without it after realizing that we'd switched to gcc 4.9 since the last time I'd run a Try push. That was also turned out green.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e3b5767404350aabacaf6f9f3db5c0cb887ee5a9

So I'm going to revert the je_cv_thp=no change and chalk this up to some kind of compiler bug. I can still file an upstream report if you feel strongly about it, but it still is going to be a pretty low-quality one.
Assignee: nobody → ryanvm
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/80af97ee4861
Don't disable hugepage support since it no longer causes PGO issues.
https://hg.mozilla.org/mozilla-central/rev/e0ee0cfe7c87
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
Blocks: 1343432
You need to log in before you can comment on or make changes to this bug.