Last Comment Bug 740171 - Investigate pdf.js memory usage and find ways to reduce it
: Investigate pdf.js memory usage and find ways to reduce it
Status: RESOLVED DUPLICATE of bug 881974
[pdfjs-c-feature]
: meta
Product: Firefox
Classification: Client Software
Component: PDF Viewer (show other bugs)
: 13 Branch
: x86_64 Windows 7
: -- enhancement with 3 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
: Brendan Dahl [:bdahl]
Mentors:
Depends on: 839548
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-28 15:07 PDT by Marco Castelluccio [:marco]
Modified: 2013-06-14 15:31 PDT (History)
17 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
PDF file (122.19 KB, application/pdf)
2012-03-28 15:11 PDT, Marco Castelluccio [:marco]
no flags Details

Description Marco Castelluccio [:marco] 2012-03-28 15:07:41 PDT
pdf.js uses a lot of memory when opening big PDF files compared to other readers.

Sometimes I see high heap-unclassified numbers:
' 845.98 MB (100.0%) -- explicit
' ├──316.54 MB (37.42%) ── heap-unclassified
Comment 1 Marco Castelluccio [:marco] 2012-03-28 15:11:48 PDT
Created attachment 610320 [details]
PDF file

For example for this small PDF file, Firefox takes 50 MB.
Comment 2 Nicholas Nethercote [:njn] 2012-04-03 16:12:16 PDT
I'll look at the heap-unclassified and then decide what to do with this.
Comment 3 Nicholas Nethercote [:njn] 2012-04-15 23:58:45 PDT
I haven't run DMD yet but I was just browsing back and forth through http://njn.valgrind.org/pubs/phd2004.pdf and I got this:

1,230,914,043 B (100.0%) -- explicit
├────946,764,228 B (76.92%) ── heap-unclassified

Yikes.
Comment 4 Brendan Dahl [:bdahl] 2012-04-16 08:52:42 PDT
We're aware of one bug where some data was being sent from the worker to the main thread causing a big duplication.  We're also refactoring our API so all the data only lives in the worker thread.
Comment 5 Nicholas Nethercote [:njn] 2012-06-14 22:17:25 PDT
> I haven't run DMD yet but I was just browsing back and forth through
> http://njn.valgrind.org/pubs/phd2004.pdf and I got this:
> 
> 1,230,914,043 B (100.0%) -- explicit
> ├────946,764,228 B (76.92%) ── heap-unclassified

I just tried again and I couldn't get "explicit" above ~600MB and "heap-unclassified" was in the 20--25% range.

Marco, can you still reproduce this?
Comment 6 Marco Castelluccio [:marco] 2012-06-15 02:31:47 PDT
(In reply to Nicholas Nethercote [:njn] from comment #5)
> I just tried again and I couldn't get "explicit" above ~600MB and
> "heap-unclassified" was in the 20--25% range.
> 
> Marco, can you still reproduce this?

I get ~60 MB for the pdf file I've attached and ~150 MB for your file.
Comment 7 Nicholas Nethercote [:njn] 2012-06-15 02:51:16 PDT
> I get ~60 MB for the pdf file I've attached and ~150 MB for your file.

How does that compare with what you originally got?  i.e. can we close this bug?
Comment 8 Marco Castelluccio [:marco] 2012-06-15 03:09:38 PDT
(In reply to Nicholas Nethercote [:njn] from comment #7)
> How does that compare with what you originally got?  i.e. can we close this
> bug?

I originally got ~50 MB for the pdf file I've attached, so now it's using more memory than before (and it's just 2 pages long).

I still get really high memory usage with other pdf files compared to what I get with other readers, but not as much as before (i.e. not ~800 MBs, but ~400 MBs, when other readers take ~10 MBs). And heap-unclassified is now low (most of the memory is used by the worker and by the compartment for the PDF, so probably there's still that duplication that Brendan was talking about).

However I'm now testing with Linux, I don't know how Windows numbers look like.
Comment 9 Nicholas Nethercote [:njn] 2013-02-06 15:41:01 PST
(In reply to Nicholas Nethercote [:njn] from comment #5)
> > I haven't run DMD yet but I was just browsing back and forth through
> > http://njn.valgrind.org/pubs/phd2004.pdf and I got this:
> > 
> > 1,230,914,043 B (100.0%) -- explicit
> > ├────946,764,228 B (76.92%) ── heap-unclassified
> 
> I just tried again and I couldn't get "explicit" above ~600MB and
> "heap-unclassified" was in the 20--25% range.

Now I get ~400MB for "explicit" and 11% "heap-unclassified".  Good enough, I think.
Comment 10 Marco Castelluccio [:marco] 2013-02-06 15:49:05 PST
(In reply to Nicholas Nethercote [:njn] from comment #9)
> (In reply to Nicholas Nethercote [:njn] from comment #5)
> > > I haven't run DMD yet but I was just browsing back and forth through
> > > http://njn.valgrind.org/pubs/phd2004.pdf and I got this:
> > > 
> > > 1,230,914,043 B (100.0%) -- explicit
> > > ├────946,764,228 B (76.92%) ── heap-unclassified
> > 
> > I just tried again and I couldn't get "explicit" above ~600MB and
> > "heap-unclassified" was in the 20--25% range.
> 
> Now I get ~400MB for "explicit" and 11% "heap-unclassified".  Good enough, I
> think.

Do you get 400MB using another PDF reader?
I think 400MB is a lot of memory, above all if we want pdf.js on smartphones.
Comment 11 Nicholas Nethercote [:njn] 2013-02-09 12:22:30 PST
The bug has no clear end-point.  Please change the title so it has one.
Comment 12 Andrew McCreight [:mccr8] 2013-02-10 05:08:37 PST
meta bugs don't really need MemShrink tracking.  Add the tag to any blocking bugs that seem relevant to MemShrink.
Comment 13 Marco Castelluccio [:marco] 2013-02-10 06:47:57 PST
(In reply to Andrew McCreight [:mccr8] from comment #12)
> meta bugs don't really need MemShrink tracking.  Add the tag to any blocking
> bugs that seem relevant to MemShrink.

This bug is about investigating pdf.js high memory usage, in my opinion it should be MemShrink tagged until that investigation is carried through.
I mean, this is completely useless as a tracking bug if no investigation is done.

I don't know how to do this investigation, this is why I filed the bug. If you think pdf.js memory usage can't be improved, please close it.
Comment 14 Justin Lebar (not reading bugmail) 2013-02-11 06:08:59 PST
FWIW I'm OK having the [MemShrink] tag on this even though it's a metabug, if Marco thinks that would help him get the attention he needs to identify the issues with PDF.js that need to be fixed.
Comment 15 Marco Castelluccio [:marco] 2013-02-20 14:52:09 PST
(In reply to Justin Lebar [:jlebar] from comment #14)
> FWIW I'm OK having the [MemShrink] tag on this even though it's a metabug,
> if Marco thinks that would help him get the attention he needs to identify
> the issues with PDF.js that need to be fixed.

It's entirely up to you.
Comment 16 Marco Castelluccio [:marco] 2013-06-14 08:49:24 PDT
Looks like there's bug 881974 now.

*** This bug has been marked as a duplicate of bug 881974 ***
Comment 17 Nicholas Nethercote [:njn] 2013-06-14 15:31:36 PDT
> Looks like there's bug 881974 now.

Sorry I forgot about this one!

Note You need to log in before you can comment on or make changes to this bug.