95888 - [mac-only] 134.5Mb XUL FastLoad File and/or hang

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect, P1)

Description

[nnooiissee]

i have been eagerly awaiting bug 68045, so i grabbed Mozilla/5.0 (Macintosh; U;
PPC; en-US; rv:0.9.3+) Gecko/20010817 to try out fastlad as soon as something
landed. got the build and rand it about a half dozen times to the point it
completes displaying www.mozilla.org while timing (14 to 22 seconds), then added
user_pref("nglayout.debug.disable_xul_fastload", false); to my prefs.js and ran
once (28 seconds), twice (22 seconds), checked size of XUL FastLoad File and it
was about 7Mb, and then a third time which took well over 30 seconds, and
checked the fastload file again, and it was 134.5Mb. first two times i ran it
only until www.mozilla.org had finished displaying. third time i am still
running it. only window i have opened except for two browsers was the Find in
this Page window which took about 10 seconds the first time.

something simmilar happend earlier in the day when i downloaded this build at
work... but maybe i will try to reproduce that later.

Comment 1

[Greg K.]

As that bug has not yet been closed, these comments should be made on that bug. 
Marking duplicate.

*** This bug has been marked as a duplicate of 68045 ***

Comment 2

[Brendan Eich [:brendan]]

Ouch.  Mac I/O differences in NSPR or Necko, compared to Linux and Windows?

/be

Comment 3

[Phil Schwartau]

Verified Duplicate -

Comment 4

[nnooiissee]

okay, as of 2001082408 on my home mac i just got a 120 meg fastload file the
first time i ran mozilla with fastload on.

earlier today on a mac at work fastload ran fine, and i managed to get the
fastload file to 2 megs after hunting down every dialog i could find, then i
moved the fastload file and started again which took about 45 seconds (by my
count) and left me with a xul file a little over 300 megs. on a whim i dropped
the file onto stuffit and was rewarded with a 96k archive (i did not have a hex
editor available, and i forgot to bring home the archive). replacing the old
fastload file worked like a charm. also of note, the file i got at home
compressed by only 17% and was obviously messy, with even readable strings, all
the way through.

also, i shoud ask, should i have posted all this to bug 68045? if that was not
still open i would have two bugs here.

Comment 5

[Phil Schwartau]

> Also, I should ask, should I have posted all this to bug 68045?
> If that was not still open I would have two bugs here.

Yes, please post all comments on this issue to bug 68045 -

Comment 6

[John Morrison]

Yoiks! There is indeed something incredibly toxic happening on the Mac. 
I just tried this and I can't even start with fastload enabled with an 
8/23 trunk build -- it completely freezes my system (9.0.4 450MHz G4)
and I have to reboot the system.

Since this Mac-specific problem with fastload looks like it may be a 
non-trivial detour from the general fastload bug, I'm reopening this bug.

Comment 7

[John Morrison]

I had a little more luck after 'nglayout.debug.checksum_xul_fastload_file' to 
false, although that may be a red herring (e.g., pure luck).

However, I was still able to on occasion generate a monster file (446KB), and 
on other occasions to completely lock up Mac OS. I also had situations where
the file was either somewhat too big (~1MB just from the browser; extra bits 
were disk garbage) or somewhat too small (~216KB just from the browser; final 
bits in file were not the typical footer). 

It probably doesn't help that we are not cleanly deleting the fasl file in case 
of corruption (in the current trunk build; patch pending), although this bug
would still exist even if we were cleanly removing the file.

Comment 8

[John Morrison]

> monster file (446KB)

er, 446 MB

Comment 9

[Simon Fraser [no longer active]]

I tried this in a debug build with files, not .jars, in 'chrome'. It didn't work. 
We had to patch nsResChannel.

Comment 10

[Simon Fraser [no longer active]]

Attachment: mozilla/necko/protocols/res changes

Comment 11

[Simon Fraser [no longer active]]

With these changes, fastload works if you comment out the loading of hidden 
window. Putting that back in, there are still problems (pldhash foo) that brendan 
is working on.

Comment 12

[Brendan Eich [:brendan]]

Patch coming up to xpcom/io/nsFastLoadFile.cpp should fix all Mac woes.

/be

Comment 13

[Brendan Eich [:brendan]]

Need some r= and sr= love, for sfraser's patch and for my (forthcoming) one.

/be

Comment 14

[Brendan Eich [:brendan]]

Attachment: proposed xpcom/io/nsFastLoadFile.cpp fix

Comment 15

[Simon Fraser [no longer active]]

With that last patch, things run almost-ok, but I get a couple of assertions on a 
startup without an existing FastLoad file:

###!!! ASSERTION: redundant multiplexed document?: 'docMapEntry->mString == 
nsnull', file nsFastLoadFile.cpp, line 1284
(mString here is "chrome://wallet/content/walletOverlay.js")
###!!! ASSERTION: unknown aURI passed to EndMuxedDocument!: 'uriMapEntry && 
uriMapEntry->mDocMapEntry', file nsFastLoadFile.cpp, line 1439

Comment 16

[John Morrison]

With brendan's patch above applied (and Simon's too, but I was running with 
jar files) on Mac in debug, I get the same two assertions as Simon did when
starting with no existing fasl file.

I also at one point got this assertion, although I don't know what exactly I 
did to trigger it, and multiple attempts to get it again haven't worked.
User break at 2E32E0E0 dprintf(const char*, ...)+0008C
 ###!!! ASSERTION: too many strong refs in serialization: 'entry->
mInfo.mStrongRefCnt <= rc -
 2', file nsFastLoadFile.cpp, line 1606
 Calling chain using A6/R1 links
  Back chain  ISA  Caller
[snip]....
  18886600    PPC  2E00AAD4  nsXULDocument::ResumeWalk()+018B4
  18886330    PPC  2E0041B4  nsXULDocument::EndFastLoad()+00288
  18886280    PPC  2E40BD04  nsFastLoadFileWriter::Close()+00294
  188840F0    PPC  2E40B610  nsFastLoadFileWriter::WriteFooter()+00154
  18884080    PPC  2E3F397C  PL_DHashTableEnumerate+00068
  18884020    PPC  2E40B20C 
nsFastLoadFileWriter::ObjectMapEnumerate(PLDHashTable*, PLDHashEntryHdr*, 
unsigned int, void*)+000E8
  18883FD0    PPC  2E32E1E0  nsDebug::Assertion(const char*, const char*, const 
char*, int)+0005C

Comment 17

[Jaime Rodriguez, Jr.]

Nominating for nsbranch+.

Comment 18

[Phil Schwartau]

As per discussion with jrgm, changing QA contact to him -

Comment 19

[Asa Dotzler [:asa]]

0.9.4 is out the door

Comment 20

[Suresh Duddi (gone)]

nsbranch+ ing this. This needs to get resolved if we wanna take this for the branch.

Simon can you help drive this to closure.

Comment 21

[chris hofmann]

I don't think there are any plans to take this on the branch, are there?
this should go on the trunk so we can get fast load turned on

Comment 22

[chris hofmann]

branch-

Comment 23

[Brendan Eich [:brendan]]

Attachment: patch with dbaron's eagle-eyed fix to set uriMapEntry->mGeneration when revalidating mDocMapEntry

Comment 24

[Chris Waterson]

Comment on attachment 51828 [details] [diff] [review]
patch with dbaron's eagle-eyed fix to set uriMapEntry->mGeneration when revalidating mDocMapEntry

sr=waterson. brilliant!

Comment 25

[Brendan Eich [:brendan]]

dbaron r='ed, and can take that "brilliant!" compliment (unless you meant to
sound like Basil Fawlty, in which case I'll take it!).

I don't think we need sfraser's necko patch now that darin has overhauled
resource URLs and channels.  I'll verify tomorrow and try to close this bug, so
any new FastLoad probs can be addressed by new bug reports.

/be

Comment 26

[Brendan Eich [:brendan]]

Ugh, still haven't verified this.  But want to for 0.9.5.  Simon, are you able
to FastLoad without jars, using the latest build, and not using your old,
patched, res channel code?

/be

Comment 27

[Brendan Eich [:brendan]]

All fixed, sfraser verified.  I think the code in nsChromeProtocolHandler.cpp
could be simplified a bit, but I'm leery of throwing away the "ultimate
fallback" logic at
http://lxr.mozilla.org/mozilla/source/rdf/chrome/src/nsChromeProtocolHandler.cpp#705
that, upon failing to QI the URI inside a jar: URL into an nsIFileURL (which the
nsStdURL darin now resolves resource: URLs into), will make a new channel for
the non-file URI and then ask whether the channel is an nsIFileChannel.  Given
the degrees of freedom in Necko's interfaces, I can see this code being needed
again, some day.

Marking fixed.

/be