812070 - Greatly speed up fix-linux-stack.pl

Status: RESOLVED FIXED

(Core :: General, defect)

Description

[Nicholas Nethercote [inactive]]

fix-linux-stack.pl is amazingly slow.  This makes using NewDMD (bug 717853) harder, because it leans heavily on that script.  I have a big improvement coming shortly.

(BTW, I have no idea what the correct product/component is for this bug.)

Comment 1

[Nicholas Nethercote [inactive]]

Attachment: (part 1) - Merge %pipes and %address_adjustment in fix-linux-stack.pl.

This patch does some preliminary rearrangement.

We currently have %pipes and %address_adjustment, both of which hold per-file
info.  This patch just combines them into a single hash called %file_infos.

Comment 2

[Nicholas Nethercote [inactive]]

Attachment: (part 2) - Speed up fix-linux-stack.pl by adding a cache for previously queried addresses.

This patch adds a table for each file, which saves the results of previous
queries.  This avoids many calls to addr2line, which are slow.

On a 94,240 line DMD output file, this reduced the processing time on my
machine from 18.5 minutes to 2 minutes.  On a truncated 4,000 line file, it
reduced the processing time from 65 seconds to 27 seconds.

Comment 3

[Andrew McCreight [:mccr8]]

I wonder if a similar improvement could be made for the OSX script.

Comment 4

[Justin Lebar (not reading bugmail)]

> This avoids many calls to addr2line, which are slow.

We have just one addr2line process, which we pipe all requests to, right?

I ask because that's much, much faster than launching a new process for each request.  But I don't want to read perl to figure out what we do.  :)

Comment 5

[Nicholas Nethercote [inactive]]

> We have just one addr2line process, which we pipe all requests to, right?

We have one per library/executable file (libxul.so, firefox, libstdc++.so, etc).  On my examples there are typically about 20 distinct files.

Upon further investigation it's clear that the piped requests are accounting for almost all the execution time.  Even in piped mode, addr2line is slow.  The very first lookup in libxul.so takes about 8 seconds.  For my 4,000 line example the cache takes care of 2,131 cases, and I end up doing 1,112 piped requests, each of which take 24ms on average.  Even if I ignore the 8 second start-up its about 17ms per request.  We're talking well under 100 requests per second -- slooow.

Then again, libxul.so is over 650 MiB on this machine when built with debugging info.  If I strip the debugging info (but keep the symbols) the processing time for a 4,000 line file drops from 27 seconds to 5 seconds, but then I don't get filenames and line numbers, which is much less useful.

Turning off demangling doesn't speed things up at all, which surprised me.

Comment 6

[Justin Lebar (not reading bugmail)]

> We have one per library/executable file (libxul.so, firefox, libstdc++.so, etc).  On my examples 
> there are typically about 20 distinct files.

Okay.  Since addr2line is a pure function, you could run multiple processes in parallel on the same file.

Comment 7

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 681852 [details] [diff] [review]
(part 1) - Merge %pipes and %address_adjustment in fix-linux-stack.pl.

r=dbaron

Comment 8

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 681853 [details] [diff] [review]
(part 2) - Speed up fix-linux-stack.pl by adding a cache for previously queried addresses.

Maybe change address2 to adj_address, or even skip the variable entirely and pass the whole thing to the printf call?

r=dbaronx

Comment 9

[Nicholas Nethercote [inactive]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a702c73d82a6
https://hg.mozilla.org/integration/mozilla-inbound/rev/e72aeaefb691

Comment 10

[Nicholas Nethercote [inactive]]

> I wonder if a similar improvement could be made for the OSX script.

fix_macosx_stack.py is much faster.  It processed a 100,000 line file in just over 30 seconds, which is about 4x faster than the improved fix-linux-stack.pl.  It uses otool, atos and c++filt, and I guess they're just much faster than addr2line.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/a702c73d82a6
https://hg.mozilla.org/mozilla-central/rev/e72aeaefb691