839548 - Excessive memory while loading PDF thumbnail view

Status: RESOLVED DUPLICATE of bug 878397

(Firefox :: PDF Viewer, defect, P3)

Description

[u279076]

> Originally report in bug 820379 comment 7

When I open a PDF (see URL) and turn on the thumbnail pane, I noticed the memory usage climbs to ~3.3 GB. I'm not sure if this is the only use case where PDF.js is using excessive memory. It might be prudent to do some exploratory analysis to find out. In some cases (ex. bug 820379) this results in stability and performance issues.

Comment 1

[u279076]

The million dollar question of course is whether this should block enabling of PDF.js in Firefox 19? or whether we should ship with this known issue and hotfix disable it if feedback indicates it's a prevalent issue.

Comment 2

[Andrew McCreight [:mccr8] (PTO until 12-2)]

The page appears to be a series of scanned high resolution images.

While loading, about:memory for the relevant tab looks like this:
│  ├──1,515.25 MB (67.32%) -- top( ... )/active/window( ... )
│  │  ├──1,511.84 MB (67.17%) -- js/compartment( ... )
│  │  │  ├──1,501.60 MB (66.72%) -- objects-extra
│  │  │  │  ├──1,501.44 MB (66.71%) ── elements
│  │  │  │  └──────0.16 MB (00.01%) ── slots
│  │  │  └─────10.24 MB (00.45%) ++ (8 tiny)

As you can see it is almost entirely JS elements.

After hitting minimize memory usage the memory goes down to a more reasonable quantity:

│  ├───77.84 MB (11.23%) -- top( ... )/active/window( ... )
│  │   ├──74.44 MB (10.74%) -- js/compartment( ... )
│  │   │  ├──66.20 MB (09.55%) -- objects-extra
│  │   │  │  ├──66.04 MB (09.53%) ── elements
│  │   │  │  └───0.16 MB (00.02%) ── slots
│  │   │  └───8.24 MB (01.19%) ++ (6 tiny)
│  │   └───3.41 MB (00.49%) ++ (4 tiny)

I don't know if this is a matter of pdf.js allocating too many things, the GC not running enough, or some combination of both.

Comment 3

[Andrew McCreight [:mccr8] (PTO until 12-2)]

Oh, and this was without the thumbnail view.

When I scrolled through the document again, the memory when up to about 1.5 gigs again.  After waiting a few seconds, it dropped back down to 77mb.  So it is more than just the initial loading that is causing heavy memory usage, I guess, and the GC does seem to take care of the memory within, say, a minute.

Comment 4

[Alex Keybl [:akeybl]]

(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #0)
> When I open a PDF (see URL) and turn on the thumbnail pane, I noticed the
> memory usage climbs to ~3.3 GB. I'm not sure if this is the only use case
> where PDF.js is using excessive memory. It might be prudent to do some
> exploratory analysis to find out. In some cases (ex. bug 820379) this
> results in stability and performance issues.

We should track PDF.js OOM crashes (and we are), but not temporary/excessive memory usage.

Comment 5

[u279076]

FWIW, this would cause an OOM situation on PCs with less than 4GB (fairly common AFAIK).

Comment 6

[Nicholas Nethercote [inactive]]

Taking the MemShrink tag off because we have a tag on 839631.

Comment 7

[u279076]

Dave, could this be covered by a Mozmill Endurance test?

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #7)
> Dave, could this be covered by a Mozmill Endurance test?

Yes, we could definitely add a few PDF.js related endurance tests (opening/closing a PDF, entering/leaving thumbnail view, etc).

Comment 9

[u279076]

Thanks Dave, I've filed bug 851507 to track development of an Endurance test once *this* bug is resolved.

Comment 10

[Nicholas Nethercote [inactive]]

After investigation, I've found that this has the same root cause as bug 878397 -- we're creating lots of these enormous mask arrays. Just like in that bug, we see "explicit" measurements that are much higher than the "resident" measurements.

2,639.78 MB (100.0%) -- explicit
├──2,482.55 MB (94.04%) -- window-objects
│  ├──2,459.42 MB (93.17%) -- top(file:///home/njn/37355_1_A.pdf, id=8)
│  │  ├──2,458.45 MB (93.13%) -- active/window(file:///home/njn/37355_1_A.pdf)
│  │  │  ├──2,457.40 MB (93.09%) -- js-compartment(resource://pdf.js/web/viewer.html, file:///home/njn/37355_1_A.pdf)
│  │  │  │  ├──2,456.43 MB (93.05%) -- objects
│  │  │  │  │  ├──2,456.01 MB (93.04%) -- malloc-heap
│  │  │  │  │  │  ├──2,455.96 MB (93.04%) ── elements/non-asm.js
│  │  │  │  │  │  └──────0.05 MB (00.00%) ── slots
...
  733.56 MB ── resident
  721.81 MB ── resident-unique
3,393.72 MB ── vsize

Comment 11

[u279076]

Marking this fixed by bug 878397 and flagging for verification.

Dave, now that this is fixed can we get traction on bug 851507?

Comment 12

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #11)
> Marking this fixed by bug 878397 and flagging for verification.
> 
> Dave, now that this is fixed can we get traction on bug 851507?

Bug 851507 is still blocked by bug 830379.