Closed Bug 72128 Opened 25 years ago Closed 10 years ago

hook up PR_Read stack trace searcher

Categories

(Core Graveyard :: Tracking, defect, P2)

Product:
Core Graveyard
Component:
Tracking
Platform:
All
Windows NT
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX
Milestone:
mozilla1.5alpha

People

(Reporter: cathleennscp, Unassigned)

References

Details

Attachments

(4 files)

Size of read chunks
14.16 KB, text/plain
Details
File reads on startup for 6 July opt mozilla build
938.37 KB, text/plain
Details
Summary of file reads (no stack traces)
35.68 KB, text/plain
Details
registry reads
2.33 KB, text/plain
Details
brendan/sfraser to hook up PR_Read stack trace searcher and hand off to dveditz to do some analysis...
Simon, which stack tracing juju were you gonna use? I was thinking of the xpcom/base/nsTraceMalloc.c stuff, but that doesn't work on Mac. Shall we port it, in order for you to use it for read calls? /be
Summary: hook up PR_Read stack trace searcher → hook up PR_Read stack trace searcher
I was thinking that we should catch all read calls on Linux too, in case there are non-XP ones that do not come from PR_Read. Or do all interesting reads funnel through PR_Read? /be
Ahem. The stack tracing in nsTraceRefCnt works fine on Mac (with some tweaking).
Blocks: 71874
/me should be clearing his throat, over the lack of unity he helped perpetrate in stack tracing. The nsTraceMalloc machinery has the advantage that it combines traces using a callsite tree, so it can compress a trace into a single callsite serial number. But quick and dirty wins here, especially if you have time to go strong on Mac (I'm still mired in drivers and js1.5 work). So why am I still here? Cathleen, I'm bowing out. /be
fine...! thanks brendan. :-) waterson was also going to look into using nsTraceMalloc to see if JS is using large amount of memory. If he confirms, we'll nee your help on instrumenting JS!
I've got this working in my tree; it does not seem like something that we should check in to NSPR, so I'm not sure what I should do with it, other than refactor the stack walking code into something that can easily be called from anywhere (including C code). Data from this are at <http://bugzilla.mozilla.org/showattachment.cgi?attach_id=28856> (before zip goodness) and <http://bugzilla.mozilla.org/showattachment.cgi?attach_id=29740> (after zip goodness).
Status: NEW → ASSIGNED
List of files that we read at startup, and how many times we read from them: 223 modern.jar 97 comm.jar 82 en-US.jar 50 toolkit.jar 36 localstore.rdf 24 cookies.txt 14 all-skins.rdf 13 messenger.jar 13 all-packages.rdf 12 cview.jar 12 chatzilla.jar 10 all-locales.rdf 9 US.jar 9 platformHTMLBindings.xml 8 htmlBindings.xml 7 overlays.rdf 6 user-skins.rdf 6 user-locales.rdf 5 en-mac.jar 3 userContent.css 3 ua.css 3 search.rdf 3 quirk.css 3 html.css 3 forms.css 3 D4B53221d00 3 charsetalias.properties 2 xpti.dat 2 maccharset.properties 2 langGroups.properties 2 charsetData.properties 1 xpinstall.js 1 xpconnect.xpt 1 xpcom.xpt 1 user.js 1 uriLoader.xpt 1 uconv.xpt 1 shistory.xpt 1 security-prefs.js 1 search.xpt 1 rdf.xpt 1 prefs.js 1 oji.xpt 1 nsLocale.xpt 1 nsIStringBundle.xpt 1 necko.xpt 1 mozBrowser.xpt 1 mailnews.js 1 macprefs.js 1 libpref.xpt 1 layout.xpt 1 initpref.js 1 editor.js 1 docshell.xpt 1 content.xpt 1 config.js 1 cache.xpt 1 bookmarks.xpt 1 bookmarks.html 1 autocomplete.xpt 1 appshell.xpt 1 all.js
Attached data shows the size of each PR_Read, sorted by filename.
Is the 1KB buffer too small in rdf_BlockingParse()? I'm pretty sure we're only reading the files once. (I should also fix bug 35816 so these files don't get so darn big.)
I collected some data on libreg's fread calls too, which are in bug 15115
Redux of libreg read data: File Total bytes read File size Times read --------------------------------------------------------- Application Registry 8140 4070 2.0 Mozilla Registry 543 543 1.0 Component Registry 3737086 366226 10.2 So we effectively read components.reg 10 times over. Nice.
putting this in moz0.9.1 to get it off the untargetted radar
Priority: -- → P2
Target Milestone: --- → mozilla0.9.1
Target Milestone: mozilla0.9.1 → mozilla1.0
What more is to be done here? I can't check in stuff into NSPR for this.
Notable issues from the data: * RDF files are being read in 1K chunks, which is below the usual 4k disk block size, so 4K might be better. * We're still reading a bunch of .xpt files in, even though we read in xpti.dat at the start. Why? * There are lots of reads from .jar files which are around 4060 in size, but variable. This seems a little weird. * The majority of the reads are reading files from .jar files. * XML files are read in 1K chunks. 4K might be better here too. * cookies.txt is read in 128 byte chunks (45 reads for me), which is suboptimal.
Simon, I don't see the registry reads in the summary. Are they captured by it?
No. Registy reads use fread, so don't go via PR_Read. Data coming.
> There are lots of reads from .jar files which are around 4060 in size, but > variable. This seems a little weird. When we open each .jar file we have to read the directory. We process it in 4k chunks, but if one of the structures is split at the end of the buffer (odds are it is) we move the part we've got to the beginning of the buffer and then read enough to fill the rest of the buffer. Makes the code a little simpler. The amount we have to read is directly proportional to the number of files in the archive so we can save time by - making sure we've weeded out obsolete chrome files - combining small chrome files - splitting out functional chunks that aren't always needed An example of the last point are skins, where we pay for parsing directory entries for Aim and Mail chrome even if all you're doing is using the browser. On the other hand it makes the Theme park work better. After the ~4k chrome reads you'll see a series of paired reads, first a 30 byte read of the file header in the archive and then the bytes of the file itself. We only have to do the 30 byte read the first time each chrome file is read, but since it's always followed by reading the file itself there's probably a better way to do this. Re-ordering the files in the archive into the order they're used would help us not fight the OS's disk caching and reduce wild seeks back and forth across the archives.
Dan, your last comment looks like a specific problem to fix. Has it been broken out into its own bug?
Bugs targeted at mozilla1.0 without the mozilla1.0 keyword moved to mozilla1.0.1 (you can query for this string to delete spam or retrieve the list of bugs I've moved)
Target Milestone: mozilla1.0 → mozilla1.0.1
don't move bugs that are in the 1.0 dependency tree. sorry.
Target Milestone: mozilla1.0.1 → mozilla1.0
Moving Netscape owned 0.9.9 and 1.0 bugs that don't have an nsbeta1, nsbeta1+, topembed, topembed+, Mozilla0.9.9+ or Mozilla1.0+ keyword. Please send any questions or feedback about this to adt@netscape.com. You can search for "Moving bugs not scheduled for a project" to quickly delete this bugmail.
Target Milestone: mozilla1.0 → mozilla1.2
Target Milestone: mozilla1.2alpha → mozilla1.5alpha
Assignee: sfraser_bugs → nobody
This is a mass change. Every comment has "assigned-to-new" in it. I didn't look through the bugs, so I'm sorry if I change a bug which shouldn't be changed. But I guess these bugs are just bugs that were once assigned and people forgot to change the Status back when unassigning.
Status: ASSIGNED → NEW
This looks obsolete.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WONTFIX
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: