Closed Bug 46000 Opened 24 years ago Closed 23 years ago

Component Registry not pre-installed on MacOS, VERY slow first startup and runs out of memory quickly

Categories

(Core :: XPCOM, defect, P3)

Product:
Core
Component:
XPCOM
Platform:
PowerPC
Mac System 8.5
Type:
defect

Tracking

()

Status:
VERIFIED FIXED
Milestone:
mozilla0.9.1

People

(Reporter: mikepinkerton, Assigned: jj.enser)

References

Details

(Keywords: crash, perf)

Attachments

(1 file)

crash on startup stack trace with build 20001053011 (2)
48.08 KB, text/plain
Details 
- delete your component registry to force a rebuild
- launch mozilla (it will re-register everything)
- go to a couple of webpages, like my.netscape.com and foxnews.com

You'll hit a sysError 25 (out of memory) after only a couple of webpages. This 
only happens after the components are re-registered, probably because we load all 
the libs but never unload them. Subsequent runs are fine, memory wise.

This is pretty bad, as we crash very quickly on first run of the app. A great way 
to make a first impression!
Keywords: nsbeta2
It seems that we never unload all the DLLs we load when autoregistering. Most of 
the heap is thus being used by data (TOC etc) associated with loaded libraries on 
Mac, which never go away. The rest of the heap is sucked up with the double-
buffering GWorld back buffer.

Perhaps this is related to slowness issues of autoregistration on Mac.

A stop-gap solution might be to require autoregistration to always occur 
separately on the Mac.  We certainly need to solve the issue of unloadability, 
but it probably will not be a quick one.
Status: NEW → ASSIGNED
Depends on: 20743
Putting on [nsbeta2-] radar.  Not critical to beta2.  Adding "relnote" keyword 
for PR2 release. Adding nsbeta3 keyword.
Keywords: nsbeta3, relnote
Whiteboard: [nsbeta2-]
Target Milestone: --- → M20
Depends on: 45613
relnote2 keyword
Keywords: relnote → relnote2
I disagree that the ability to unload DLLs is a nsbeta3 function.
whoa, so you're telling me that it's ok that we're running out of memory on 
registration? ray?

There are lots of things that XPCOM does that are not OK.  But solving the whole
problem of being able to unload DLLs in nsbeta3 does not seem likely to happen.

There are ways you can work around the problem.  I could suggest a few if you 
like.  None are perfect, but it is possible to restart one or multiple times if 
registration has loaded too many things into memory.
Putting on [nsbeta3+] radar.
Whiteboard: [nsbeta2-] → [nsbeta2-][nsbeta3+]
A very simple work-around is to pre-build the component.reg file.

Unloading DLLs will not happen in beta3.

I am marking it "Future" and anyone who would like to come discuss this further 
may do so.  I am leaving it on the nsbeta3 for a few days so that those who 
marked it beta3 can decide what they want to do instead of unloading DLLs.  
There are many possible approaches.  After that, with no more input, I will 
remove nsbeta3.
Target Milestone: M20 → Future
What would some of those approaches be, pray tell, since the obvious one is being 
summarily dismissed?

cc'ing other smart people

Sorry to jump in here. But this bug has already been given a beta3+ by PDT.

Engineers are not allowed to future beta3 approved bugs. Ray if you really don't
think this should be fixed in beta3 then you should make your case to PDT (they
are the ones that said we need to fix this problem for the Mac for beta3). But
we can't future beta3 approved bugs at our level. Although just by scanning the
bug this sounds like a pretty severe problem IMHO...

anyway, just thought I'd point out that this bug shouldn't be marked future
until PDT decides they gave the beta3+ nomination in error (if they decide that)




Hold the phone.

We are *not* going to implement DLL unloading by nsbeta3. There are far, far too
many issues with DLL loading for us to even consider it. None of the modules
we've written (and I mean *none*) actually reference count the objects they've
created. So they don't even know if they can be safely unloaded or not. If we
start unloading modules, we will literally introduce *hundreds* of "random"
crashes.

But let's take a step back here. Look at "step one" that is required to
reproduce the bug:

  - delete your component registry to force a rebuild

We will ship the bits with a precompiled component registry. Why on earth would
a user delete their component registry? If they *do* delete it, then next time
they run the browser, it's going to gobble up memory. And maybe crash. Too bad.
It'll be okay after that. (Compare this to deleting any other "system" file
that's associated with an application: go ahead, I dare you to delete random
files out of /etc, /usr/bin,  your Windows system directory, or your Mac system
folder. You are asking for trouble.)

If the problem is that we're not shipping/installing a precompiled component
registry, then let's change the bug to mean that.

I'm tempted to mark this bug INVALID as reported, but I won't. Instead I'll
clear the nsbeta3+ that was assigned to it, and ask for this to be reconsidered.
Whiteboard: [nsbeta2-][nsbeta3+] → [nsbeta2-]
according to dp the last time I asked him about this, we are _not_ going to ship 
a preinstalled component registry. While this was bantered around at porkjockeys, 
and everyone there thought it was a done deal, no one outside those meetings ever 
took this idea seriously.

Since we're not doing that, this bug is perfectly valid. If we are doing that,  
where is the bug? Who is tasked with the work? When is it scheduled to land?

cc'ing dveditz. he might know.

Here's the real problem, there is supposed to be a pre-installed component 
registry (according to waterson), but for some reason it's either ignored or the 
date checking is buggy on macOS.

This is bad bad bad, however you slice it. What's up here? Who would know? Who is 
supposed to own the pre-install of the component registry?
Summary: After registering all components, run out of memory very quickly → Component Registry not pre-installed on MacOS
updating summary again
Keywords: perf
Summary: Component Registry not pre-installed on MacOS → Component Registry not pre-installed on MacOS, VERY slow first startup and runs out of memory quickly
Given this new information it sounds like we want to recommend an nsbeta3+ to
PDT for making sure the MAC comes with a pre-installed components.reg. This
works on windows and linux but as pinkerton says is aparently not working on the
Mac.

I think the summary of this bug now reflects that. Thanks pink.
I'm not convinced this is working on Linux or Windows. I just untar'd the
commercial Linux release bits from this AM, and there was no component.reg file
in there. I would love to be proven wrong.
cc'ing warren & dp.
Marking nsbeta3+
Whiteboard: [nsbeta2-] → [nsbeta3+]
adding [nsbeta2-]...this is work for nsbeta3
Whiteboard: [nsbeta3+] → [nsbeta3+][nsbeta2-]
I never suggested I would change the whiteboard status, but only the keywords 
and target milestone, which I understood should reflect my own analysis as the 
current owner of the bug.  I feel that it is my responsibility to do so.  The 
whole point is that the conflicting values should raise a flag with those who 
maintain the whiteboard status.

I will change the target milestone from "Future" when the bug becomes clearly 
about something that should be deliverable in a more-concrete time frame (or 
perhaps a new owner of the bug will).
This is why people will delete their component registry on Mac:
1. Rumors will start that deleting this file, and getting it rebuilt, will solve
   instability problems. This is inevitable.
2. Users see this stupid file with a generic icon in the Netscape folder, and 
   wonder why it is there. They will delete it, "but it keeps coming back. Is it
   a virus?"

My point is that this file is in the users face, it has no icon, and generally 
looks 'lost'. Naive users will be happy to delete it regularly.

The file should probably live in the cache directory (or a writable portion of 
"Essential Files"), along with all of hyatt's manifest.rdf files. They all 
should get the right icons, creator codes, etc.

Bug 42184 talks about the need to move these files for unix write permission 
issues.

Reassigning to myself.
Assignee: rayw → warren
Status: ASSIGNED → NEW
Depends on: 24312
Keywords: nsmac1
un-future this, since warren took it.
Target Milestone: Future → M18
we're less than a week away from b3, what's the status on this?
It's all up to the build team to make sure it goes out with the bits, right?
then maybe we should assign this bug to someone on the build team, perhaps?
--> leaf
Assignee: warren → leaf
Once upon a time, we ran regxpcom to generate this registry on mac; when it
started crashing the build machine and preventing deliveries of any kind
whatsoever from happening, we turned it off.

We still run this as part of the unix and windows automation, because it hasn't
crashed on these platforms.

Simon Fraser spent a lot of time trying to figure out why this was happening,
and it seemed to stop for a while. Unfortunately, it started happening again.

Any mac heads want to help debug this?
Status: NEW → ASSIGNED
We don't need to debug regxpcom on the mac to fix this bug for beta3, nor do we 
have time. We simply need to launch the browser, and it will automatically build 
the components.reg file to ship. We just have to make sure we do that before the 
bits go out the door.
fine, whatever. marking resolved remind, leaving in this milestone.
Status: ASSIGNED → RESOLVED
Closed: 24 years ago
Resolution: --- → REMIND
Reopening. Having Mac do autoreg on first run after an installation still sucks, 
and has the side-effect of causing serious low memory problems.
Status: RESOLVED → REOPENED
Resolution: REMIND → ---
Giving to JJ for some mac release machine loving.
Assignee: leaf → jj
Status: REOPENED → NEW
adding rtm keyword. this is very important that we fix!
Keywords: rtm
Blocks: 57835
Reasons for fixing this are described in bug 57835.

build release would be happy to do this as soon as you fix regxpcom.

rtm- til regxpcom is fixed.
Whiteboard: [nsbeta3+][nsbeta2-] → [nsbeta3+][nsbeta2-][rtm-] need regxpcom fixed.
Until RegXPCOM gets fixed (not by me), Release could use the following workaround 
for RTM.

since the automation is what prevents us from delivering a prebuilt component.reg 
on mac, I could to run Netscape by hand and add the newly generated component 
registry to the installer... I am willing to do that for RTM candidates -only-.
(and maybe set the file type & creator so that the registry has a Netscape icon 
and doesn't seem 'lost')

For those of you who have missed the first few episodes of this long story, take 
a look at related bugs 13129, 31858, and mainly 24312.
Status: NEW → ASSIGNED
jj: I recall discussing this with Leaf, and agreeing to run mozilla by hand for 
RTM candidate builds to generate the component.reg file. I think it would be 
great if you could do this!
Keywords: relnote2
QA Contact: leger → granrose
working on a "half-automated" solution where the component registry gets 
generated after the build is complete, then added to the browser.xpi installer 
module.

FWIW, I've testing RegXPCOM, and it still crashes in the same way.

Since 11/03, the packaging process launches mozilla/netscape after the build and 
delivers the component registry generated file.
This has proved to be a viable solution... as long as we don't have DOA builds ;-
)
This way, Release would discover and report crashes on startup before the bits 
even get to the somketesters!
Status: ASSIGNED → RESOLVED
Closed: 24 years ago → 24 years ago
Resolution: --- → FIXED
Status: RESOLVED → VERIFIED
verified fixed.
Reopening. Crashes have re-occured when launching Netscape at the end of the 
build, so this part of the automation has been turned off again since around 
March 20 :-(
I have 2 stack traces captured with Netscape 6, build 2001-05-30-11, coming up.

changing target milestone from M18 to ... 0.9.1?
Status: VERIFIED → REOPENED
Keywords: nsbeta2, nsbeta3, nsmac1
Resolution: FIXED → ---
Whiteboard: [nsbeta3+][nsbeta2-][rtm-] need regxpcom fixed.
Target Milestone: M18 → mozilla0.9.1
That log shows 2 crashes; which should we be looking at?

I attached both crashes (hence the (2) in the attachment name) because they both 
occured during test autoreg tests I did yesterday, but I now realize that the 
first one might be unrelated:

crash 1 occured after I did the following:
1. started up NS6
2. observed all components registration
3. waited until idle on www.netscape.com
4. pressed Cmd-Q
5. Boom!
(hence the domenu/menuSelected calls in the first stack trace)

This might be a totally different bug that might not affect the release 
automation since NS gets launches & quit using AppleEvents, not a Cmd key :-)

crash 2 occured a few minutes later as I was trying again.
1. Deleted previous Components Registry file
2. Launched NS 6
3. Observed Components registration in the splash screen
4. Boom!
Keywords: crash
Whiteboard: need help
jj: please test and see if you can reproduce these crashes.

I ran a basic script to see if this worked in an "automated environment":
	set myNetscape to "disk:mozilla tree:ns:dist:viewer:Netscape 6"
	tell application myNetscape to activate
	delay 5
	tell application myNetscape to quit

This ran just fine on both 06-01-11-trunk and 06-01-14-0.9.1 builds (tested on 
the branch right after the normal packaging automation ended), so I will flip the 
switch back on and pray that this is the last one I need to change this.

patch to BuildCentral Default Settings (applied on the branch verif. build mac 
only)

< #mz_make_comp_reg	false
< #ns_make_comp_reg	false
---
> #mz_make_comp_reg	true
> #ns_make_comp_reg	true
Status: REOPENED → RESOLVED
Closed: 24 years ago → 23 years ago
Resolution: --- → FIXED
Whiteboard: need help
Yeah! branch builds from the weekend and today's didn't crash. Looking at the 
build machines directories, the component registry was successfully created for 
all these builds.
Grace, please verify it's also installed by the installer.

Applying the changes to the trunk automation now.
QA Contact: granrose → gbush
Depends on: 64978
verified with RTM  build 2001072621 and trunk build 2001081704
Status: RESOLVED → VERIFIED
Component: XPCOM Registry → XPCOM
QA Contact: agracebush → xpcom
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: