239929 - (semi-single-profile) Semi-Single-Profile for firefox and the new-toolkit apps

Status: RESOLVED FIXED

(Firefox :: Migration, defect, P1)

Description

[Benjamin Smedberg]

This is, for the moment, a tracker-bug for base work for the semi-single-profile
stuff for firefox. See attachment 144252 [details] for an overview of the proposal.

Comment 1

[Benjamin Smedberg]

Attachment: main toolkit patch

This includes changes to toolkit/ only. The apps won't build without additional
patches that I'll put up soon.

Comment 2

[Benjamin Smedberg]

Comment on attachment 146763 [details] [diff] [review]
main toolkit patch

Darin, this doesn't need official sr=, but I would appreciate you looking at
this in some detail.

Comment 3

[Darin Fisher]

i will try to look over this patch within the next few days (maybe today even).

Comment 4

[Benjamin Smedberg]

Attachment: Design notes

Design notes for this patch. This is rather rudimentary, because I'm pressed
for time, but it's a start at least.

Comment 5

[Benjamin Smedberg]

Just a note, dbaron's patch in bug 241591 makes this patch perfectly stable (we
were leaking the RDF service and the prefservice because of that bug, which was
causing xpcom-restart problems).  I need to go back through and bulletproof the
prefservice/rdfservice so that even if we do leak them, they get reincarnated
with an XPCOM-restart.

Comment 6

[Darin Fisher]

Have you discussed the profile salting issue with the security group?  You
didn't mention who agreed that it isn't needed anymore.  Removing it sounds like
an additional risk.  This patch doesn't need anymore risks... why make more
trouble for yourself? ;-)

Comment 7

[Marco Pesenti Gritti]

Am I missing something or the usage of nsEmbedString will break the static 
build ? AFAIK embed and private strings will be linked in the same executable.

Comment 8

[Benjamin Smedberg]

Attachment: Core toolkit/profile and toolkit/xre changes

OK, per brendan's request, I'm distilling this patch into a couple pieces: this
first piece is the "core" changes in toolkit/profile and toolkit/xre that need
real review. app-startup is forked from xpfe/component/startup to
toolkit/components/startup and the native app support is forked from
xpfe/bootstrap to toolkit/xre, but those changes are purely to remove
quicklaunch cruft and other multi-profile cruft which is not needed; they
shouldn't need a detailed review.

Comment 9

[Benjamin Smedberg]

Attachment: Firefox profile-migration changes

These are the changes to firefox profile-migration.

Comment 10

[Benjamin Smedberg]

For those watching, we have come up with a plan to minimize the changes needed
for the 1.7 branch by using #ifdefs where necessary.

Comment 11

[Benjamin Smedberg]

Attachment: a few changes needed for mailnews

These changes are needed so the mailnews code uses the directory service
instead of the profile manager.

Comment 12

[Benjamin Smedberg]

Attachment: Minimal patch for 1.7 branch

This patch should work for the 1.7 branch. I need to update
mail/app/nsMailApp.cpp with the xulappapi. Maybe we won't need a special branch
for firefox.

This is tested on linux only. I suspect that nsAppRunner.cpp is going to need
to #include some OS-specific headers like windows.h in order to run the code in
LaunchChild. I won't have a chance to build win32 until tomorrow evening.

I noticed, when I was running this, that "ps" doesn't show the child process as
an actual child. Should it? Does NSPR have a way to explictly launch a child
process?

The changes in extensions/pref are a crash that I ran into during
autoregistration where NSPR logging wasn't initialized properly. I don't
understand why I uncovered that bug, but whatever...

Comment 13

[Benjamin Smedberg]

Comment on attachment 148581 [details] [diff] [review]
Minimal patch for 1.7 branch

darin/brendan/Ben: the parts of this patch that need special review attention
are

1) all of nsXREDirProvider.cpp
2) most of nsAppRunner.cpp ... please carefully check GetBinaryPath and
LaunchChild.
3) Please understand how profile-import works, which is different from
profile-migration: we just take the existing path ~/.phoenix/<foo> and use it,
instead of copying settings over to ~/.firefox/<foo>.

I really need to do a basic profile-import module for tbird which copies the
code from firefox-import, without all the profile-migration code.

Comment 14

[Benjamin Smedberg]

OK, further testing, already discovered a bug for non-linux unixes.
in nsAppRunner.cpp

-    if (stat(gBinaryPath, &fileStat)) {
+    if (stat(gBinaryPath, &fileStat) == 0) {

Comment 15

[Darin Fisher]

> Does NSPR have a way to explictly launch a child process?

Yes, see:

  http://lxr.mozilla.org/seamonkey/source/nsprpub/pr/include/prproces.h#85

I think we want something like this:
 
  int main(...)
  {
    ...

    if (relaunch)
    {
      PRProcess *child = PR_CreateProcess(...);
      PR_WaitProcess(child, ...);
    }
    return 0;
  }

Comment 16

[Benjamin Smedberg]

(In reply to comment #15)
>       PRProcess *child = PR_CreateProcess(...);
>       PR_WaitProcess(child, ...);

Well, that's what I'm doing, except that "ps" doesn't show it as a child
process. Whatever, it still waits for it properly.

However, I have a more interesting problem... when I show the profile manager as
a modal dialog (and presumably the EM dialogs will have the same problem), the
window is not made invisible or destroyed when it returns. I'm pretty sure that
this has something to do with the way modal windows are cleaned-up... IIRC they
post a "destroy" event to the event queue so that the function that shows the
modal window can retrieve information from it before it is destroyed. Since we
are never running the event loop, that destroy-message never gets processed.

Comment 17

[Darin Fisher]

> Well, that's what I'm doing, except that "ps" doesn't show it as a child
> process. Whatever, it still waits for it properly.

oops.. i originally searched your patch for PR_NewProcess :-(

perhaps we should just call "execv" on UNIXy platforms?  then we'd just keep
using the current process, right? :-)


> Since we are never running the event loop, that destroy-message never gets 
> processed.

aren't we at least pushing an event queue?  i mean, this code is running through
OpenWindowJS, right?  if that is the case, then when we pop the event queue, we
will as a side-effect process all pending events on that event queue.  sorry, if
i'm way off... i haven't looked closely at your patch yet.

Comment 18

[Benjamin Smedberg]

Attachment: 1.7 branch patch, updated for windows

This fixes the windows DDE code, I've tested and it seems to work correctly.
Also note, that the modal-windows-doesn't-go-away problem only occurs for my
gtk2 builds... I don't have that problem on windows, and I don't remember it
causing problems on gtk either.

Comment 19

[Scott MacGregor]

Comment on attachment 148494 [details] [diff] [review]
a few changes needed for mailnews

Can you make the tbird specific changes for abCardViewOverlay.js and
msgHdrViewOverlay.js?

Everything here looked find except for the changes to offlineStartup.js. I
didn't quite follow what was going on there. Can you verify that the behavior
still works correctly? if so then sr=mscott

Comment 20

[Scott MacGregor]

FYI, I already told Ben Goodger this in person. But now the aviary branch no
longer finds my thunderbird profiles when I start up. It comes up with a blank
screen. :(

Also these changes still don't build on Linux. I sent Ben the compiler errors
folks are seeing when they try to build tbird on linux on the branch.

Just adding a notation of the issues here or other folks who run into the same
problems.

Comment 21

[Scott MacGregor]

Here's the stack of the tbird build bustage for linux and windows:

-I/usr/X11R6/include -DMOZILLA_CLIENT -include                   
../../mozilla-config.h -Wp,-MD,.deps/nsNativeAppSupportBase.pp nsNativeAppSuppo
                  rtBase.cpp
gmake[4]: *** No rule to make target « nsNativeAppSupportDefault.cpp », needed
for « nsNativeAppSupportDefault.o ». Stopping.
gmake[4]: Leaving directory `/home/fred/logs/thun/mozilla/toolkit/xre'
gmake[3]: *** [libs] Error 2 

Comment 22

[Benjamin Smedberg]

(In reply to comment #21)
> Here's the stack of the tbird build bustage for linux and windows:
> 
> gmake[4]: *** No rule to make target « nsNativeAppSupportDefault.cpp », needed
> for « nsNativeAppSupportDefault.o ». Stopping.

I checked in nsNativeAppSupportDefault.cpp this morning. This should not be
causing windows bustage, though.

Comment 23

[Scott MacGregor]

i meant to say linux bustage not linux and windows.

Here's the current bustage my Mac OS X helper is now seeing:

nsChromeRegistry.pp nsChromeRegistry.cpp
nsChromeRegistry.cpp:2693: error: prototype for `nsresult
   nsChromeRegistry::UninstallPackage(const PRUnichar*, int)' does not match
   any in class `nsChromeRegistry'
nsChromeRegistry.h:76: error: candidate is: virtual nsresult
   nsChromeRegistry::UninstallPackage(const nsACString&, int)
make[4]: *** [nsChromeRegistry.o] Error 1
make[4]: Leaving directory `/Users/jon/build_tbird_branch/mozilla/rdf/chrome/src' 

Comment 24

[Darin Fisher]

bsmedberg,

the profile selection tooltip text does not show up properly on linux.  the '\n'
character in the .properties file does not render properly on GTK2+XFT builds. 
it seems that it is unable to find an appropriate glyph, so it just renders a
little box character with four numbers in it.  i'll try to debug the problem in
my build, but perhaps this is just a generic problem with the GTK2+XFT builds. 
or maybe it has something to do with my selected font.

Comment 25

[Darin Fisher]

also, i'm able to repro the problem with the profile selection dialog not going
away, so i'll look into that too.  though i suspect that will be solved by
moving the relaunch sequence to after NS_ShutdownXPCOM has been called.

Comment 26

[Benjamin Smedberg]

> moving the relaunch sequence to after NS_ShutdownXPCOM has been called.

The relaunch sequence is already after xpcom shutdown, at least for profile-UI
and profile-import situations.

Comment 27

[Darin Fisher]

> The relaunch sequence is already after xpcom shutdown, at least for profile-UI
> and profile-import situations.

Oh, ok...  I'll have to take a closer look then.

Comment 28

[Iain MacDonnell]

Just as another data-point, my nightly aviary thunderbird build also came up
with a blank profile (~/thunderbird/primary). Initially surprised by this, I
attempted to run "thunderbird -ProfileManager", which resulted in a miniature,
empty (white) window with the title "Gecko". I "got over it" and just populated
the new profile... 

Comment 29

[Scott MacGregor]

Now the thunderbird build doesn't start at all. After registering xpcom it just
quits.....

Comment 30

[Scott MacGregor]

And it still won't find my profile. It still creates a profile called "primary":

[Profile0]
Name=primary
IsRelative=1
Path=Profiles/primary

I think the profile gets loaded. Then it quits.

This really isn't what I had in mind when I created a nice stable 1.0 branch
away from the trunk to get some good builds going :). I had visions of a nice
happy safe place. How close are we to getting there? Help me get warm fuzzies
again please!

Comment 31

[Scott MacGregor]

Here are some of the run time errors I see on the console:

WARNING: NS_ENSURE_TRUE(NS_SUCCEEDED(rv)) failed, file c:/build/trees/tb10/mozil
la/toolkit/profile/src/nsINIParser.cpp, line 51
No Persistent Registry Found.
WARNING: NS_ENSURE_TRUE(NS_SUCCEEDED(rv)) failed, file c:/build/trees/tb10/mozil
la/toolkit/profile/src/nsINIParser.cpp, line 51
WARNING: NS_ENSURE_TRUE(NS_SUCCEEDED(rv)) failed, file c:/build/trees/tb10/mozil
la/toolkit/profile/src/nsINIParser.cpp, line 51

then we do regxpcom stuff

then I see more nsINIParser errors also on line 51.

Then I see:

WARNING: nsExceptionService ignoring thread destruction after shutdown, file c:/
build/trees/tb10/mozilla/xpcom/base/nsExceptionService.cpp, line 191
WARNING: NS_ENSURE_TRUE(window) failed, file c:/build/trees/tb10/mozilla/xpfe/ap
pshell/src/nsContentTreeOwner.cpp, line 643
WARNING: NS_ENSURE_TRUE(docShellElement) failed, file c:/build/trees/tb10/mozill
a/xpfe/appshell/src/nsXULWindow.cpp, line 1315
WARNING: NS_ENSURE_TRUE(windowElement) failed, file c:/build/trees/tb10/mozilla/
xpfe/appshell/src/nsXULWindow.cpp, line 1369
WARNING: NS_ENSURE_TRUE(windowElement) failed, file c:/build/trees/tb10/mozilla/
xpfe/appshell/src/nsXULWindow.cpp, line 978
WARNING: NS_ENSURE_TRUE(uriLoader) failed, file c:/build/trees/tb10/mozilla/docs
hell/base/nsDocShell.cpp, line 387
WARNING: Failed to create timer, file c:/build/trees/tb10/mozilla/dom/src/base/n
sJSEnvironment.cpp, line 1860

then the app shuts down

Comment 32

[Scott MacGregor]

Ben Goodger got Thunderbird profiles working again. Thanks Ben!

Howevever my comment here turned out to be prophetic:

"Everything here looked find except for the changes to offlineStartup.js. I
didn't quite follow what was going on there. Can you verify that the behavior
still works correctly? if so then sr=mscott"

Thunderbird insists on starting up in offline mode every time you start up. I
suspect the changes here to offlineStartup.js either aren't working the way they
were intended or there is more extension manager foo-ness going on here.

Comment 33

[Benjamin Smedberg]

(In reply to comment #32)
> Thunderbird insists on starting up in offline mode every time you start up. I
> suspect the changes here to offlineStartup.js either aren't working the way they
> were intended or there is more extension manager foo-ness going on here.

mscott, can you open new bugs on these kinds of issues? I can't reproduce this
behavior in my branch-tbird build, which seems to work correctly in both "ask
every time" and "use my last setting" modes.

Comment 34

[Scott MacGregor]

Anyone else getting errors like this:

###!!! ASSERTION: Potential deadlock between Monitor@3dfa58 and Lock@1e6ff60: 'E
rror', file c:/build/trees/tb10/mozilla/xpcom/threads/nsAutoLock.cpp, line 299
Break: at file c:/build/trees/tb10/mozilla/xpcom/threads/nsAutoLock.cpp, line 29
9

after bringing up the profile manager and selecting a profile?

Comment 35

[Benjamin Smedberg]

(In reply to comment #34)
> ###!!! ASSERTION: Potential deadlock between Monitor@3dfa58 and Lock@1e6ff60: 'E

Bug 209804, which was approval1.7- The patch has baked on the trunk for a while
without ill effect, maybe we should land it on the aviary branch.

Comment 36

[Scott MacGregor]

FYI, I spun off:

Bug #244203 to polish up the new profile UI.

Comment 37

[Scott MacGregor]

When running optmized bits with these changes I'm seeing two processes left in
my system tray. One process for the profile service of course and then the 2nd
process is the actual thunderbird process with my profile.

I would have expected the profile process to go away after I choose the profile.
I don't see this in debug builds.

Thunde~1.exe   Mem Usage: 12,636K (this is the profile process)
Thunde~1.exe   Mem Usage: 24,428K (this is the actual thunderbird process)

Anyone else seeing this?

Comment 38

[Darin Fisher]

bsmedberg: what needs to be reviewed in order to get this on the trunk and close
out this bug?

Comment 39

[Benjamin Smedberg]

I need superreview on bugs 239875 and 239925.

Comment 40

[Scott MacGregor]

Before we land this on the trunk, we should really figure out why the change to
offlineStartup.js makes Thunderbird startup in offline mode the first time you
upgrade to a build that uses this new profile code 

This has turned out to be the top issue with the 0.7 Release Candidate builds
with many users getting stuck in offline mode when upgrading. Similar to what I
saw in comment #32. Unfortunately we don't install the offline UI in the default
Windows install so there is no way to go back online with out requiring
upgraders to modify prefs.js and remove the network offline pref that is getting
set. 

Comment 41

[Benjamin Smedberg]

> Before we land this on the trunk, we should really figure out why the change to
> offlineStartup.js makes Thunderbird startup in offline mode the first time you
> upgrade to a build that uses this new profile code 

Who is "we"? I haven't seen a bug report about this (certainly not one assigned
to me). I changed the offline-startup code, and I am happy to fix regressions
that are brought to my attention in the form of a bugzilla bug assigned to me.

Comment 42

[Scott MacGregor]

International users haven't been able to use Thunderbird 0.7 if their profile
path contains 8-bit characters. We don't seem to be handling non ascii
characters in the profile directory name anymore. I wonder if there are some
8-bit character bugs in this patch?

See: Bug #247377 for an example. 

Comment 43

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

I just changed the tinderbox scripts so they don't check the return value when
they call -CreateProfile, since this changed success from non-zero to zero
(which is good, but now different builds are inconsistent).  We could reinstante
the test at some point if all builds are again consistent.

Comment 44

[Scott MacGregor]

Benjamin, I noticed you started to port this stuff back to the trunk today. Did
you mean to leave out the changes to the mailnews directory when you landed or
are those not needed on the trunk? (I'm guessing they are needed but wanted to
confirm).

I'll try to get all the Thunderbird changes necessary to support the new profile
stuff onto the trunk tonight. 

Comment 45

[Benjamin Smedberg]

> Benjamin, I noticed you started to port this stuff back to the trunk today. Did
> you mean to leave out the changes to the mailnews directory when you landed or

Oops, I merged them in my tree but didn't commit them. I will commit them when I
have a free hour today.

Comment 46

[Benjamin Smedberg]

this is finished on the branch and trunk

Comment 47

[Ryan Polk (Quark)]

switching fixed-aviary1.0 to keyword for searching purposes.
sorry for bugspam.

Comment 48

[(dormant account)]

Note Prbool errors in /browser/components/migration/src/nsProfileMigrator.cpp:
Lines 298 & 302 in macro NS_ENSURE_TRUE return a non 1/0 value in a prbool method.

298                     NS_ENSURE_TRUE(profileSvc, NS_ERROR_FAILURE);
302                     NS_ENSURE_TRUE(dirService, NS_ERROR_FAILURE);