740171 - Investigate pdf.js memory usage and find ways to reduce it

Status: RESOLVED DUPLICATE of bug 881974

(Firefox :: PDF Viewer, enhancement)

Description

[Marco Castelluccio [:marco]]

pdf.js uses a lot of memory when opening big PDF files compared to other readers.

Sometimes I see high heap-unclassified numbers:
' 845.98 MB (100.0%) -- explicit
' ├──316.54 MB (37.42%) ── heap-unclassified

Comment 1

[Marco Castelluccio [:marco]]

Attachment: PDF file

For example for this small PDF file, Firefox takes 50 MB.

Comment 2

[Nicholas Nethercote [inactive]]

I'll look at the heap-unclassified and then decide what to do with this.

Comment 3

[Nicholas Nethercote [inactive]]

I haven't run DMD yet but I was just browsing back and forth through http://njn.valgrind.org/pubs/phd2004.pdf and I got this:

1,230,914,043 B (100.0%) -- explicit
├────946,764,228 B (76.92%) ── heap-unclassified

Yikes.

Comment 4

[Brendan Dahl [:bdahl]]

We're aware of one bug where some data was being sent from the worker to the main thread causing a big duplication.  We're also refactoring our API so all the data only lives in the worker thread.

Comment 5

[Nicholas Nethercote [inactive]]

> I haven't run DMD yet but I was just browsing back and forth through
> http://njn.valgrind.org/pubs/phd2004.pdf and I got this:
> 
> 1,230,914,043 B (100.0%) -- explicit
> ├────946,764,228 B (76.92%) ── heap-unclassified

I just tried again and I couldn't get "explicit" above ~600MB and "heap-unclassified" was in the 20--25% range.

Marco, can you still reproduce this?

Comment 6

[Marco Castelluccio [:marco]]

(In reply to Nicholas Nethercote [:njn] from comment #5)
> I just tried again and I couldn't get "explicit" above ~600MB and
> "heap-unclassified" was in the 20--25% range.
> 
> Marco, can you still reproduce this?

I get ~60 MB for the pdf file I've attached and ~150 MB for your file.

Comment 7

[Nicholas Nethercote [inactive]]

> I get ~60 MB for the pdf file I've attached and ~150 MB for your file.

How does that compare with what you originally got?  i.e. can we close this bug?

Comment 8

[Marco Castelluccio [:marco]]

(In reply to Nicholas Nethercote [:njn] from comment #7)
> How does that compare with what you originally got?  i.e. can we close this
> bug?

I originally got ~50 MB for the pdf file I've attached, so now it's using more memory than before (and it's just 2 pages long).

I still get really high memory usage with other pdf files compared to what I get with other readers, but not as much as before (i.e. not ~800 MBs, but ~400 MBs, when other readers take ~10 MBs). And heap-unclassified is now low (most of the memory is used by the worker and by the compartment for the PDF, so probably there's still that duplication that Brendan was talking about).

However I'm now testing with Linux, I don't know how Windows numbers look like.

Comment 9

[Nicholas Nethercote [inactive]]

(In reply to Nicholas Nethercote [:njn] from comment #5)
> > I haven't run DMD yet but I was just browsing back and forth through
> > http://njn.valgrind.org/pubs/phd2004.pdf and I got this:
> > 
> > 1,230,914,043 B (100.0%) -- explicit
> > ├────946,764,228 B (76.92%) ── heap-unclassified
> 
> I just tried again and I couldn't get "explicit" above ~600MB and
> "heap-unclassified" was in the 20--25% range.

Now I get ~400MB for "explicit" and 11% "heap-unclassified".  Good enough, I think.

Comment 10

[Marco Castelluccio [:marco]]

(In reply to Nicholas Nethercote [:njn] from comment #9)
> (In reply to Nicholas Nethercote [:njn] from comment #5)
> > > I haven't run DMD yet but I was just browsing back and forth through
> > > http://njn.valgrind.org/pubs/phd2004.pdf and I got this:
> > > 
> > > 1,230,914,043 B (100.0%) -- explicit
> > > ├────946,764,228 B (76.92%) ── heap-unclassified
> > 
> > I just tried again and I couldn't get "explicit" above ~600MB and
> > "heap-unclassified" was in the 20--25% range.
> 
> Now I get ~400MB for "explicit" and 11% "heap-unclassified".  Good enough, I
> think.

Do you get 400MB using another PDF reader?
I think 400MB is a lot of memory, above all if we want pdf.js on smartphones.

Comment 11

[Nicholas Nethercote [inactive]]

The bug has no clear end-point.  Please change the title so it has one.

Comment 12

[Andrew McCreight [:mccr8]]

meta bugs don't really need MemShrink tracking.  Add the tag to any blocking bugs that seem relevant to MemShrink.

Comment 13

[Marco Castelluccio [:marco]]

(In reply to Andrew McCreight [:mccr8] from comment #12)
> meta bugs don't really need MemShrink tracking.  Add the tag to any blocking
> bugs that seem relevant to MemShrink.

This bug is about investigating pdf.js high memory usage, in my opinion it should be MemShrink tagged until that investigation is carried through.
I mean, this is completely useless as a tracking bug if no investigation is done.

I don't know how to do this investigation, this is why I filed the bug. If you think pdf.js memory usage can't be improved, please close it.

Comment 14

[Justin Lebar (not reading bugmail)]

FWIW I'm OK having the [MemShrink] tag on this even though it's a metabug, if Marco thinks that would help him get the attention he needs to identify the issues with PDF.js that need to be fixed.

Comment 15

[Marco Castelluccio [:marco]]

(In reply to Justin Lebar [:jlebar] from comment #14)
> FWIW I'm OK having the [MemShrink] tag on this even though it's a metabug,
> if Marco thinks that would help him get the attention he needs to identify
> the issues with PDF.js that need to be fixed.

It's entirely up to you.

Comment 16

[Marco Castelluccio [:marco]]

Looks like there's bug 881974 now.

Comment 17

[Nicholas Nethercote [inactive]]

> Looks like there's bug 881974 now.

Sorry I forgot about this one!