12911 - Backend support for XP profile relative pathnames

Status: VERIFIED FIXED

(Core Graveyard :: Profile: BackEnd, defect, P2)

Description

[Akkana Peck]

I'm having a problem where my prefs are being saved in
/builds/akkana/.mozilla/akkana/prefs.js instead of
$HOME/.mozilla/akkana/prefs.js.  I switched my home directory some time ago from
/builds to /u, and hadn't realized 'til now that my prefs were still being saved
in the old location.  Where is it getting this path?  Not from anything in the
build tree (I pull a new tree nearly every day), and not from $HOME/.mozilla
since the whole problem is that it isn't looking there.  What do I have to
change to get it to use $HOME the way other apps do?

Comment 1

[selmer (gone)]

The original description of this bug resolved down to a problem of using
absolute pathnames in the registry file.  I'm changing this bug to capture the
idea that the registry file should store relative pathnames instead of absolute
pathnames.  The pathnames should be stored relative to the directory that
contains the registry file and the pathname handling must be careful to rebuild
the absolute name correctly (i.e. the CWD must NOT be used :-)

If the user enters a pathname for the profile directory, we must detect whether
it is an absolute or relative pathname and store/rebuild it accordingly.

The problem we're solving here is that on Unix the user's home directory was
copied to a new place - including the registry that still pointed to the old
place.  At best this is confusing, at worst it would fail because the old place
got deleted.  By using relative names, the whole problem is avoided.

Comment 2

[leger]

Putting on [PP] list.

Comment 3

[leger]

Removing [PP] since this is only relative to Unix.

Comment 4

[leger]

Moving all Profile Manager bugs to new Profile Manager Backend component.
Profile Manager component to be deleted.

Comment 5

[racham]

Reassigning to myself. Moving to M20.

Comment 6

[racham]

I need to examine the registry and debug this further.

If the directory doesn't exists (say deleted), we recreate the directory
depending on the path we get from the registry and launch the browser...we don't
crash.

After finding the actual work involved, I will consider nominating this for nsbeta3.

Comment 7

[racham]

Doing a mass reassign to Conrad Carlen.

Conrad is going address all current and upcoming profile manager issues. Please
do not hesitate to involve me in any of the issues.

Comment 8

[Conrad Carlen (not reading bugmail)]

We could use an object that represents an XP relative file spec. If the file
spec is truly XP, the only way to do it is relative. We could make the file
relative not to the registry but to any key defined by nsIDirectoryService that
is defined on all platforms (most of them). Then, we store a flattened
representation of this object in the registry. It's more complex than that
though - all files which were stored in prefs, etc, would have to use this
technique. If we had an XP way to specify relative files, you could even use
(copy) the same profile from one platform to another (yea!) as well as just move
your home dir on Unix. Making this object should go to the XPCOM folks though.

Comment 9

[Conrad Carlen (not reading bugmail)]

*** Bug 63204 has been marked as a duplicate of this bug. ***

Comment 10

[Peter Lairo]

wow, this is an old bug

Comment 11

[Akkana Peck]

For several other reasons, we need a url scheme to reference the profile dir
(e.g. profile:///prefs.js or profile:///chrome/userContent.css).  Such a scheme
would also solve the relative url problem, since prefs and other code needing to
refer to profile files could refer to them via the scheme.  For any of the uses
I've seen, we wouldn't need to flatten out the whole profile directory (e.g. it
would be okay to specify chrome/userContent.css instead of just userContent.css
-- the important thing is not to have to specify
/u/user/.mozilla/blah/blah/chrome/userContent.css (and then have security refuse
to load it because it's a file url).

What do we need to do to make this happen?  Do we need approval from anyone? 
Sounds like it just needs little bit of netlib love (should it be added to the
cached handlers in nsIOService.h?)  Besides, wouldn't we then get relative
pathnames for free if we still wanted them, via nsIoService::ResolveRelativePath?

Comment 12

[Brendan Eich [:brendan]]

I thought warren already solved a more general problem, whereby
res://profile/... or perhaps res://$profile/... did the trick. Isn't there a
one-level (host part) macro-like facility in res: or resource: already?

I'd rather see something generic and plugable that didn't add a new URI scheme
for each new macro.

/be


/be

Comment 13

[Conrad Carlen (not reading bugmail)]

There is something like that in the resource: protocol. See
http://lxr.mozilla.org/seamonkey/source/netwerk/protocol/res/src/nsResProtocolHandler.cpp#94.
Profile relative files are not dealt with their but I don't see why they
couldn't be. Akkana - maybe this isn't as relevant to your problem as I first
thought. More on this elsewhere. I need this on the nsLocalFile level for
profile registry and prefs.

Comment 14

[Doug Turner (:dougt)]

if anything, I would like to have this as a res protocol extension as suggested:

resource:///res/profile/dougt/xxx

We also have to make sure that this is cool with mstoltz who owns some code
which protects local URLs.  ccing him.

Comment 15

[Mitchell Stoltz (not reading bugmail)]

We went through a big effort to keep web content from being able to find your
profile directory. The resource URL pointing at the user profile directory seems
like a good idea, but we have to make sure it can only be used locally, not by
content.

Comment 16

[Akkana Peck]

I tried every combination I could think of of resource:/*[$]profile/filename and
they all stayed within the resource directory, none of them went to my profile
directory.

Something like Brendan suggests would be great (I don't care if it's profile:,
I'd be perfectly happy using resource:) but it doesn't appear to be hooked up.

Should I add some code to nsResProtocolHandler to implement the $profile syntax?
Where should the security issue (not allowing external pages to access the url)
be handled, using what mechanism?

Comment 17

[Brendan Eich [:brendan]]

akkana: see ccarlen's comment with the lxr link.  Someone needs to extend the
substitution list to include ProfileDir, and make sure "relative" paths work.

/be

Comment 18

[Mitchell Stoltz (not reading bugmail)]

I will double check but I think it's already the case that Web content can't
call resource: URLs. So we should be OK on security. Is there still such a thing
as "res:" (as opposed to resource:) URLs, and is that relevant here?

Comment 19

[Akkana Peck]

There's a res: protocol, which is handled by the same objects (under slightly
different rules) as the resource: protocol.

Brendan and I spent some time going through the code to figure out why the
special dirs don't work, e.g. resource://TempDir/foo.html (that appears to be
the correct syntax) doesn't work.  There appear to be several problems,
including TempDir being treated as a hostname and case folded, so it no longer
matches the key in the hash table; and if that problem is fixed, there are other
problems as well.  Perhaps this worked once. :-)

Comment 20

[Akkana Peck]

If you add tempdir as well as TempDir in nsResProtocolHandler.cpp, what happens
is that nsResChannel::AsyncRead calls EnsureNextResolvedChannel, which correctly
maps the url to file:///tmp/..., then AsyncRead does
mResolvedChannel->AsyncRead, then while we're waiting for the read,
EnsureNextResolvedChannel gets called again many more times from
nsResChannel::GetLocalFile (and perhaps from other routines), and this time it
doesn't find the right match, probably because mCurrentIndex is now wrong inside
Next.  This I think causes mResolvedChannel, which was set correctly before, to
get reset to null, so the actual load of the url fails (but there's no error
message because netlib now thinks it never had a url to load at all).

Netlib async hell.  Is there any chance we could get a netlib person to help
with getting this to work?

Comment 21

[Brendan Eich [:brendan]]

Cc'ing warren in case he can enlighten us as to how this stuff should work.

/be

Comment 22

[Brendan Eich [:brendan]]

And adding gagan.  The ToLowerCase (which could use <ctype.h>'s tolower, no?)
routine was added a year or more ago, and its use in nsStdURLParser.cpp almost
that far back.  What changed to break these resource://TempDir/... paths by
lowercasing to tempdir?  Or did they never quite work?

I think we should fix up this code, obviously.  Conrad, you up for it?

/be

Comment 23

[Conrad Carlen (not reading bugmail)]

I'm up for fixing it. I'll take a look at what pointed out on this bug in the
past day.

Comment 24

[Brendan Eich [:brendan]]

valeski was in the cvs annotate output; Jud, do you remember any of this
resource://MagicVar/... jazz, and why it stopped working?  The ToLowerCase'ing
of host parts is implicated, but that's old.

/be

Comment 25

[Conrad Carlen (not reading bugmail)]

If resource://MagicVar/ used to work and now it doesn't, I'd say it's time for a
new bug. This bug started out being "Use relative pathnames for profile
directories" and then morphed into a necko issue.

Comment 26

[Akkana Peck]

Good point.  I've split off bug 68950, and made this bug dependant on that one.

Comment 27

[Conrad Carlen (not reading bugmail)]

Front-burnering to 0.9.5
Also, changing the summary. This bug was originally about using relative paths
in the profile registry file. It also needs to include using relative names for
all paths *within* a profile dir. Mainly, paths in prefs.js which are used by
mailnews.

Comment 28

[Boris Zbarsky [:bzbarsky]]

*** Bug 99030 has been marked as a duplicate of this bug. ***

Comment 29

[Boris Zbarsky [:bzbarsky]]

*** Bug 102540 has been marked as a duplicate of this bug. ***

Comment 30

[Gervase Markham [:gerv]]

That last-marked duplicate has a list of all the places where we are using
absolute paths in the prefs. This is also a problem in the mail cache file,
panacea.dat.

Gerv

Comment 31

[Conrad Carlen (not reading bugmail)]

Copying that useful info to this bug:

However, mail stores absolute paths in
the following places:

a) Throughout the file panacea.dat

b) In the following pref keys:
mail.directory
mail.imap.root_dir
mail.newsrc_root
mail.root.imap
mail.root.nntp
mail.root.none
news.directory
mail.server.serverX.directory
mail.server.serverX.newsrc.file
(for multiple values of X)

Comment 32

[Conrad Carlen (not reading bugmail)]

-> 0.9.6

Comment 33

[Ben Bucksch (:BenB)]

The user's search provider (incl. default, I think) is stored as absolute path
to the sherlock file (last time I checked). (Note that this lives in the
instalaltion dir, not the profile dir.) That most likely the reason why the
default pref for the search provider doesn't work for me.

Comment 34

[Boris Zbarsky [:bzbarsky]]

*** Bug 109873 has been marked as a duplicate of this bug. ***

Comment 35

[Conrad Carlen (not reading bugmail)]

Mass move to 0.9.7

Comment 36

[R.K.Aa.]

*** Bug 111591 has been marked as a duplicate of this bug. ***

Comment 37

[Conrad Carlen (not reading bugmail)]

-> 0.9.8

Comment 38

[Conrad Carlen (not reading bugmail)]

*** Bug 76245 has been marked as a duplicate of this bug. ***

Comment 39

[Akkana Peck]

This was with 0.9.7, BTW.

Comment 40

[Matthias Versen [:Matti]]

*** Bug 116601 has been marked as a duplicate of this bug. ***

Comment 41

[Matthias Versen [:Matti]]

*** Bug 116685 has been marked as a duplicate of this bug. ***

Comment 42

[Matthias Versen [:Matti]]

*** Bug 119445 has been marked as a duplicate of this bug. ***

Comment 43

[Thommie Rother]

Just want to stress the fact that this would be a severe "ship stopper" for
people who want to use Mozilla with shared profiles on a network. In my case,
some directories
P:\Daten\Netscape\Thommie\w2patixq.slt\ImapMail\192.168.0-1.1
P:\Daten\Netscape\Thommie\w2patixq.slt\ImapMail\post.gaia-1.de
P:\Daten\Netscape\Thommie\w2patixq.slt\ImapMail\post.gaia-2.de
P:\Daten\Netscape\Thommie\w2patixq.slt\ImapMail\post.gaia.de
are created in the networked (Unix) /home/thommie directory when I open 
Mozilla. This is not a Unix-specific issue but a thing which is caused when the
profile is stored under Unix but mapped (e.g. through Samba) to a networked Win
client.

I would nominate this for mozilla1.0.

Comment 44

[Matthias Versen [:Matti]]

*** Bug 119707 has been marked as a duplicate of this bug. ***

Comment 45

[Christopher Aillon (sabbatical, not receiving bugmail)]

Nominating for mozilla1.0 on behalf of Thommie Rother in comment 43, and
changing OS to All.

Comment 46

[Conrad Carlen (not reading bugmail)]

Mass move to 0.9.9

Comment 47

[Jesse Ruderman]

*** Bug 118921 has been marked as a duplicate of this bug. ***

Comment 48

[olaf Schnapauff]

Also, make sure Downloaded Pop/IMAP Mail and Newsgroups Data can be shared
between different Intallations / OSes (e.g. Windows and Linux on same Laptop)

Comment 49

[Moritz Balz]

What about the seperator characters used in stored pathnames (Unix "/" vs.
Windows "\\")? If people want to use the same profile data across different
OSes, it needs to be standardized (-> "/").

At this time Windows-like separators are not recognized on Unix and vice versa.

Comment 50

[Conrad Carlen (not reading bugmail)]

That's right - the path separators will be taken care of. I plan on using a
UTF-8 encoded descriptor. This descriptor will have the same format on all
platforms. Internally, the nsLocalFile impls of each platform will convert this
to native format for that platform.

Comment 51

[A. Craig West]

I really don't think a utf-8 descriptor is necessary. Up unit now, the file is
quite editable in 7-bit ascii, unless the user is using higher characters. At
least in the Unix and Windows cases, '/' is quite acceptable for a path
separator. All of the internal windows API's accept '/' as a path separator.
On the other hand, I don't know anything at all about the Mac API's, so I don't
know what is acceptable there.
There is a related problem with line separators, that is not directly covered by
this bug. One of the primary purposes of this bug is to allow profiles to be
portable between OS's, so the line separator issue may be related.

Comment 52

[timeless]

F:\build\mozilla>cd js/src
F:\build\mozilla\js>

please stop making the assertion that / is well recognized by windows.  under 
XP this would work, on 2k it does not.

ccarlen's suggestion is fine. your justification for you suggestion is 
absolutely not.

Comment 53

[Matthew Miller]

Timeless -- he didn't say "the windows command prompt". he said "all of the
internal windows API's". There's obviously a difference there.

Comment 54

[A. Craig West]

Timeless, Mathew Miller pretty much covered my point. The internal Windows API's
are quite agnostic about '/' versus '\' going right back to DOS 2.0. In DOS 1.0
there were no subdirectories, so the issue mever came up The comand line
interface, which is a very different thing from the API's, had support for '/'
up until DOS 5.0, I believe. Once they went to a GUI, I believe that support for
'/' in the edit boxes was spotty at best.
Does anybody know what Apple uses in their API's? As of OS-X, I presume that the
Unix base for the underlying OS implies that '/' will work.
One of the strengths of the prefs.js file is the ability to edit it in an
emergency. If it depends on non-7-bit-ASCII characters, It will be much harder
to deal with.
This shouldn't be the only factor in the decision, of course, but I see no
reason to mess with a fairly widespread standard unnecessarily.
Ideally, the path separator could be either '/' or '\' (and any others in use as
well) and Mozilla versions from all OS's would recognize all of them.
I am by no means claiming that '/' is common to all OS's. There was a recent
thread on alt.folklore.computers that went into this at great length. Some of
the older OS's had positively byzantine methods of specifying the path. As far
as I could tell, a lot of them used ' ' as a separator, but they usually were
quite limited as to how far you could nest the directories.

Comment 55

[Ben Bucksch (:BenB)]

Can't we separate cross-platform profiles from this bug? That's a superset of
profiles being movable on one machine or on machines with the same architecture
(which is what this bug was about) and a less common case.

Comment 56

[timeless]

Here's the deal as i see it, you're arguing against an arbitrary character 
based on the assumption that it's a useful character to the end user.  / is not 
useful to me as an end user.

I can't |cd "/build"| (i can |cd "js/src"|) and your average dos/windows user 
will *not* figure out that quoting paths will make a few more paths work.

Entering "f:/build" into a windows filepicker gives me a nice error message, 
and even your average prefs.js editor does *not* write or know dos api calls, 
and isn't going to know that they can magically quote escape in some cases.

The simplest XP approach is to use file:/// or resource:/// for paths.  Both of 
these would be recognizable as taking / as their path separator and have well 
known escaping algorithms.  resource: can be extended to handle home 
directories and other magical things.  See comments up to 20.

As long as we make paths look like urls instead of native paths, there isn't a 
problem.

On MacOS classic, only : is a path sep, and using full paths is about the most 
evil thing you can do (it gets you stoned and worse).

Comment 57

[Matthew Miller]

Maybe I'm confused; I thought the talk was about the internal representation of
directory separators. Obviously the UI should use whatever is native to the
platform.

Comment 58

[timeless]

you're crazy. the goal of this bug is to make prefs portable. the only way 
reasonable way to make it portable is to use URLs. any 'internal' 
representation that isn't a URL is a disaster. I really don't understand why we 
have any need for this discussion.

Comment 49 caused the problems, here's a minor quote: "At this time 
Windows-like separators are not recognized on Unix and vice versa."  If you 
read the recognizer as users instead of operating systems then you're where I 
sit.  Before comment 49 we were clearly leaning to URLs, the fact that the bug 
hadn't been fixed is just the bug and not an indication of a rational 
direction.

Comment 59

[Matthew Miller]

Don't call me names. That's totally inappropriate. I'm wasn't even advocating
anything: just trying to clarify the situation. 

Note that comment 49 starts by saying "What about the seperator characters used
in stored pathnames [...]" -- *stored* pathnames, not user representation.

But anyway, I agree with you that using file:/// urls is the right and sensible way.

Comment 60

[Markus Gerstel]

Unfortunately file:/// URLs are not relative but absolute. See the Summary of
this bug for more details...

Comment 61

[timeless]

that's why we'd use resource: or an equivalent.

Comment 62

[Alec Flett]

this bug is about storing relative pathnames in your prefs.js file - not for how
you're calling APIs, what you see in the UI, etc etc. Conrad has already
asserted that the format he has in mind is both relative and cross-platform.

The actual storage format is irrelevant to all the above discussion, please take
this rediculous debate to the newsgroups.

Comment 63

[A. Craig West]

I agree that the paths should be stored as some form of resource: urls. Anything
is better than the current absolute paths, and a resource: makes a lot of sense. 
My main argument, appearances notwithstanding, is that it would be preferable to
avoid the use of a unicode(non-ascii) character for the separator. The
discussion of API's, command line options, etc. was entirely to establish that
the '/' shouldn't break anything in particular, and preserves compatibility with
existing editors. While it is certainly not a requirement for the file to be
compatible with ASCII editors, I also don't see any good reason to break that
compatibility just because we can.
It is quite possible that I misinterpreted Conrad's meaning when he said he
would use a UTF-8 encoded descriptor, but he seemed pretty clear (Unlike me,
apparently)

Comment 64

[Boris Zbarsky [:bzbarsky]]

You missed the point.  The separator is not in question.  The question is how to
point to files which may have non-ASCII characters in the filenames.  Conrad
said that he will just encode the filenames as UTF8 which means that as long as
your filenames are 7-bit clean your prefs file will be too.

Comment 65

[A. Craig West]

Unfortunately, my reading of Conrad's comment# 50 doesn't agree with that
interpretation. I expect that Conrad will be able to clear up any
misunderstandings that have arisen.

Comment 66

[Angus Scott-Fleming]

I'd also like the ability to specify a remote path so that users could store their profiles on a network 
server and have access to them from anywhere on the LAN.  This should be specified through the 
use of environment variables or usernames -- on a Windows workstation on a Netware network or 
SaMBA/Win network in the system login script a virtual drive letter like U: would be created 
pointing at the user's , and each user's profiles are in U:\Netscape\Users\Default.  On a 
*BSD/Linux network I'm sure the same procedure could be worked out.  This would allow a use to 
log in to the network from any workstation and have personal settings.  THIS IS NOT A ROAMING 
PROFILE where the profile is copied to the server when Mozilla shuts down and then copied to the 
local workstation when Mozilla starts.  The profile info stays on the server all the time.

Comment 67

[Daniel Jensen]

Unless I'm misunderstanding, the last comment asks for something which is
already in place- you can tell Mozilla where to find your profile using the
profile wizard as long as the profile is on a filesystem visible to your
machine. I'd just like to add my voice to ask for an implementation to fix this
ASAP. What's in the way of fixing it via a resource: url? Perhaps it would be a
good idea to split off the unicode pref encoding idea into a different bug and
resolve this quickly- after all, not that many people use non-ascii characters
in their directory and file names. Does the resource: fix have a shot at being
implemented for .9.9, the current target? With 23 votes and a dup count
approaching mostfreq [and definitely mostfreq if you count the bugs which are
dups of the dependent bugs 7067 and 58647 which appear to me to be dups of this
(with the additional request for big vs. little endian in 7067 and a UI mention
or hack in 58647)] this bug seems to deserve a quick squashing.

Comment 68

[Conrad Carlen (not reading bugmail)]

> this bug seems to deserve a quick squashing.

I'm glad you think this problem is so easy - why don't you supply a patch then ;-)

Seriously, my patch is coming along. I'm using a resource: URL, because it's
here, it works, etc. A resource URL is not quite as XP as what I had in mind.
Though escaped, non-ASCII chars will be in the native charset for that platform.
I don't think that's such a problem because all of the files created within a
profile dir are made programatically, under our control. We don't create files
within the profile dir with non-ASCII characters anyway. The path *to* the
profile dir can contain any characters of whatever charset. The laborious part
of this is fixing up the many callsites in mailnews. 

Comment 69

[Malx]

> profile dir are made programatically, under our control.
Actually not all - You could create Mail/Folder with local names (just tested).
It will then create directory or FILE + FILE.msf, where "FILE" contains
different charset (for me it is Russian languange + Koi8-r charset for Linux OS)

Also some RFE to this talk. Onece on discuassion forum I was asked "if it
possible to have mail/news archive on CD (CD-RW or CD-R) and still have ability
to read it from newsreader. The answer was "no for MS OE", "yes for Netscape4".
There you could specify different directory location for mail/news instead of
subdirectory of Profile. I do not know how to solve this problem if separate
"profile" URL whould be used, but just keep in mind that there is need in
separate storage for mail/news archives.

Comment 70

[Conrad Carlen (not reading bugmail)]

> Actually not all - You could create Mail/Folder with local names (just tested).
It will then create directory or FILE + FILE.msf, where "FILE" contains
different charset (for me it is Russian languange + Koi8-r charset for Linux OS)

You're right - but the folder names are not stored in the prefs as part of
absolute paths. The prefs which need to be fixed for this are the ones which are
absolute paths to the top-level mail directories. For example, "mail.directory",
"mail.server.server1.directory", etc. Once those top-level dirs are resolved,
folder names can be appended in whatever charset.

> There you could specify different directory location for mail/news instead of
subdirectory of Profile.

Yeah, relative paths cannot always be used - as in this case. If the file pref
to be saved is not a subdirectory of the profile dir, it will be written out as
an absolute file path. BTW, even if a file pref can be made relative, the
absolute path pref will still be written out until some point when compatibility
with older builds can be dropped (maybe never). Given that, we'll have to
support both kinds of paths anyway.

Comment 71

[Aleksander Adamowski]

>Seriously, my patch is coming along. I'm using a resource: URL, because it's
>here, it works, etc. A resource URL is not quite as XP as what I had in mind.
>Though escaped, non-ASCII chars will be in the native charset for that platform.

Ccarlen: would fix for bug 84186 and bug 124042 make it possible to use UTF-8
encoding for resource: URLs? If so, it might be worth to store everything
encoded as UTF-8...
Ascii is just a subset of UTF-8, so it would only be necessary to ensure
graceful support for rare cases where there are some multibyte characters in
such an URL (just a suggestion, correct me if I got something wrong).

Comment 72

[Matthias Versen [:Matti]]

*** Bug 127771 has been marked as a duplicate of this bug. ***

Comment 73

[Matthias Versen [:Matti]]

*** Bug 127771 has been marked as a duplicate of this bug. ***

Comment 74

[Conrad Carlen (not reading bugmail)]

Attachment: Patch for necko a allow resource://profiledir (and utilities to use them)

Comment 75

[Conrad Carlen (not reading bugmail)]

Attachment: Summary of diffs to mailnews

This snippet of the actual patch shows how the code is used in mailnews. There
are many callsites that need to be changed which will make for a long, boring
patch. Also, I changed nsIFileSpec usage to nsILocalFile in any method I
touched which makes the final patch bigger.

Alec, before I do the tedium of converting the rest of mailnews, can you tell
me what you think of the approach?

Comment 76

[Alec Flett]

Comment on attachment 71696 [details] [diff] [review]
Patch for necko a allow resource://profiledir (and utilities to use them)

I have to admit that I'm not fond of this approach. I feel like this would
increase our security risk w.r.t. profiles - essentially hiding the "salt"
directory name behind a URL, rendering it useless (in effect, it makes access
to  resource://default/prefs.js  incredibly easy if there is every a URL-based
security breach.

Honestly, it seems like this bug ought to be something handled internal to the
pref service

Comment 77

[A. Craig West]

Regarding Alec's comment, this bug is at least partially INTENDED to hide the
'salt' directory name. We have a conflict here between the security granted by
the 'salt' dir, and the overall usability of prefs file.
I'm usually pretty paranoid about security issues, but in this case I'm pretty
sure that we only have a problem if resource:// urls are accessible to the user.
If that happens, I imagine the lack of 'salt' protection is perhaps not the
greatest problem we will have to deal with.
On the other hand, at least some of the benefits of the relative paths could be
had if we use resource://profiledir/foo.SLT/prefs.js.
This happens to eliminate the benefits of relative paths that I am personally
interested in, but isn't totally bad.

Comment 78

[Alec Flett]

Woah there.. we can hide the salt directory within prefs, because the mechanism
is more convoluted than adding this capability to necko..

i.e. if I say that a pref has the value:
user_pref("wallet.SchemaValueFileName", "${profiledir}/abcd.w");

and then prefs internally handles this wacky syntax, then this does not give us
any further indication about salt directory name. It does give us the name of
the wallet schema file, but that risk is already present in the current codebase.

If we add this capability in a more global space, such as necko, then it makes
it easier for code which is searching for well-known filenames (such as
prefs.js) to read these files.

I've intentionally made a lame strawman syntax above.. I'm not suggesting that
we use the ${ } syntax, I just stole it from perl. I welcome other suggestions.
The resource:// syntax is interesting, but I'm against it mostly because it
implies that it works elsewhere, such as in necko, where I don't think it should!

Comment 79

[Conrad Carlen (not reading bugmail)]

Comment on attachment 71696 [details] [diff] [review]
Patch for necko a allow resource://profiledir (and utilities to use them)

Bad idea. Alec's right. Working on a new scheme which will solve problem with
resource:// URLs.

Comment 80

[A. Craig West]

I see what Alec is saying, now. It makes a lot of sense. I was working on the
assumption that it had already been decided to use resource: urls. I suppose
nothing is really decided until it is checked in to the trunk. I definitely
agree that there are security concerns with putting this functionality into Necko.
I'll go lurk again...

Comment 81

[Conrad Carlen (not reading bugmail)]

Attachment: diffs to prefs

Here's the funtionality needed in prefs to support this. Next, I'm working on
nsLocalFile::Get/SetRelativeDescriptor. At that point, we'll have support for
relative file prefs and I'll check that in as the first phase. It won't have
any effect until mailnews uses it everywhere but, with everything at once, the
patch will be frightenly large.

Comment 82

[Conrad Carlen (not reading bugmail)]

-> 1.0, in progress though...

Comment 83

[scottputterman]

Moving Netscape owned 0.9.9 and 1.0 bugs that don't have an nsbeta1, nsbeta1+,
topembed, topembed+, Mozilla0.9.9+ or Mozilla1.0+ keyword.  Please send any
questions or feedback about this to [e-mail].  You can search for
"Moving bugs not scheduled for a project" to quickly delete this bugmail.

Comment 84

[Conrad Carlen (not reading bugmail)]

Putting back to 1.0 since it's topembed now.

Comment 85

[Doug Turner (:dougt)]

the javascript component loader requires this functionality.

Comment 86

[tao]

Another profile directory bug.... It'd be nice if we fix this in MachV.

Comment 87

[Jaime Rodriguez, Jr.]

If this would be "nice for MachV", please explain why, and nominate it. 

Comment 88

[Conrad Carlen (not reading bugmail)]

> If this would be "nice for MachV", please explain why, and nominate it.

It already is topembed+ and in-progress.

Comment 89

[[not reading bugmail]]

so are relative pathnames part of the reason map-networked profiles dont work as
in bug 79419?

Comment 90

[Conrad Carlen (not reading bugmail)]

Attachment: diffs to xpcom

Comment 91

[Matthias Versen [:Matti]]

*** Bug 131054 has been marked as a duplicate of this bug. ***

Comment 92

[Conrad Carlen (not reading bugmail)]

Attachment: diffs to xpcom

This patch gets around the skanky *this = *(nsLocalFile *)targetFile business
in the previous patch. On the Mac, it was a little involved - I needed a new
iface for private getters in order to init a file with an existing file. On
other platforms, a GetPath()/SetPath() is cheap and non-lossy.

Comment 93

[Conrad Carlen (not reading bugmail)]

Adding Brian to review the prefs portion. Can Doug and Alec r=/sr= attachment
71905 [details] [diff] [review] and attachment 75079 [details] [diff] [review]?

Comment 94

[Brian Nesse (gone)]

I'm sorry, I must be stupid... in GetComplexValue() I don't see where you are
ever storing your object into the return value...

+    nsCOMPtr<nsIRelativeFilePref> relativePref;
+    rv = NS_NewRelativeFilePref(theFile, key.get(), getter_AddRefs(relativePref));
+    if (NS_FAILED(rv))
+      return rv;
+
+    NS_ADDREF(NS_STATIC_CAST(nsIRelativeFilePref*, *_retval));
+    return NS_OK;

Other than that, the prefs patch seems ok...

Comment 95

[Conrad Carlen (not reading bugmail)]

Attachment: diffs to prefs which works

D'oh - nice catch. I'm stupid. Here's a patch which was actually tested.

Comment 96

[Brian Nesse (gone)]

Comment on attachment 75295 [details] [diff] [review]
diffs to prefs which works

r=bnesse on the prefs patch.

Comment 97

[Alec Flett]

Comment on attachment 75079 [details] [diff] [review]
diffs to xpcom


that would simplify some of the code in SetRelativeDescriptor, you'd need less
XP_MAC stuff because you'd just be able to pass in targetFile



>+    void initWithFile(in nsILocalFileMac aFile);

I wonder if we should make InitWithFile() exist on 
nsILocalFile()? It seems like this would be useful to more than just the mac...
especially since the implementation you've given doesn't actually rely on
nsILocalFileMac...



>+#if defined(XP_MAC)
>+static const nsCAutoString kSlashStr("/");
>+static const nsCAutoString kESCSlashStr("%2F");
>+#endif

this is worrysome because you use these for ReplaceSubstring, and nsString.h
says
> void
> nsCString::ReplaceSubstring(const nsCString& aTarget,const nsCString& aNewValue){
> 
>   //WARNING: This is not working yet!!!!!

however, it kinda looks like the const char* versions work (or at least don't
have the warning)
I remember something about an infinite loop with the broken ReplaceSubstring()

>+    rv = GetUnicodePath(&thisPath);
>+    if (NS_FAILED(rv))
>+        return rv;
>+    rv = fromFile->GetUnicodePath(&fromPath);
>+    if (NS_FAILED(rv)) {
>+        nsMemory::Free(thisPath);

Can you use nsXPIDLString here?

>+    for (nodeIndex = branchIndex; nodeIndex < thisNodeCnt; nodeIndex++) {
>+      NS_ConvertUCS2toUTF8 nodeStr(thisNodes[nodeIndex]);
>+#ifdef XP_MAC
>+      nodeStr.ReplaceSubstring(kSlashStr, kESCSlashStr);
>+#endif

this is the spot I was worried about.


>+    nsMemory::Free(thisPath);
>+    nsMemory::Free(fromPath);

again, more nsXPIDLCString?

>+    const nsCAutoString kParentDirStr("../");

use NS_NAMED_LITERAL_CSTRING(kParentDirStr, "../"); - you'll get length for
free, plus no 
extra copying and no extra stack buffer

>+    
>+    nsCOMPtr<nsIFile> parentDir;
>+    while (FindInReadable(kParentDirStr, nodeBegin, nodeEnd)) {
>+        rv = targetFile->GetParent(getter_AddRefs(parentDir));
>+        if (NS_FAILED(rv))
>+            return rv;
>+        targetFile = parentDir;
>+        
>+        nodeBegin = nodeEnd;
>+        pos = nodeEnd;
>+        nodeEnd = strEnd;
>+    }
>+    

Not sure I understand the point of this loop.. are you removing "../"? can you
comment here?

>+    nodeBegin = nodeEnd = pos;
>+    while (nodeEnd.size_forward() > 0) {
>+      FindCharInReadable('/', nodeEnd, strEnd);
>+      nsCAutoString nodeString(Substring(nodeBegin, nodeEnd));      
>+#ifdef XP_MAC
>+      nodeString.ReplaceSubstring(kESCSlashStr, kSlashStr);
>+#endif
>+      targetFile->AppendUnicode((NS_ConvertUTF8toUCS2(nodeString)).get());
>+      nodeBegin = ++nodeEnd;

This last line is a little scary because you might advance past the end of the
string - nodeEnd exists just past the end of the 
string, if FindCharInReadable matched at the end of the string..
though now that I look at this loop, I am again confused :) Can you explain
what's going on here? :)

>+    }
>+
>+#ifdef XP_MAC
>+    nsCOMPtr<nsILocalFileMac> macFile(do_QueryInterface(targetFile, &rv));
>+    if (NS_FAILED(rv))
>+        return rv;
>+    return InitWithFile(macFile);
>+#else
>+    nsXPIDLCString nativePath;
>+    rv = targetFile->GetPath(getter_Copies(nativePath));
>+    if (NS_FAILED(rv))
>+        return rv;
>+    return InitWithPath(nativePath.get());
>+#endif

This is the bit that would be simplified by putting InitWithFile into
nsILocalFile

Comment 98

[Alec Flett]

Comment on attachment 75295 [details] [diff] [review]
diffs to prefs which works

>+ * 
>+ */
>+
>+[scriptable, uuid(2f977d4e-5485-11d4-87e2-0010a4e75ef2)]
>+interface nsIRelativeFilePref : nsISupports
>+{
>+  /**
>+   * file
>+   *
>+   * The file whose location is stored or retrieved.
>+   */
>+  attribute nsILocalFile file;
>+
>+  /**
>+   * relativeToKey
>+   *
>+   * A directory service key for the directory
>+   * from which the file path is relative.
>+   */
>+  attribute string relativeToKey;

Actually, I'm thinking this should be an ACString, because then:

>+inline nsresult
>+NS_NewRelativeFilePref(nsILocalFile* aFile, const char* relativeToKey, nsIRelativeFilePref** result)
>+{

this could take a |const nsACString&|

and then:

>+    nsCAutoString key(Substring(keyBegin, keyEnd));

this could become 
const nsAString& key = Substring(keyBegin, keyEnd);

>+    rv = directoryService->Get(key.get(), NS_GET_IID(nsILocalFile), getter_AddRefs(fromFile));

here you'd use PromiseFlatCString()

>+    rv = NS_NewRelativeFilePref(theFile, key.get(), getter_AddRefs(relativePref));

but here this could just be "key"


>+  nsCOMPtr<nsILocalFile> mFile;
>+  nsCAutoString mRelativeToKey;

and, possibly, this could be nsSharableCString..(that's not as important)
but even if this is an nsCAutoString, I think we should comment as to why this
is good
(i.e. that keys are never longer than xxx bytes, so we know we'll never
overflow autostring's buffer,
so this just saves us an allocation)

overall the approach looks great though, just these minor tweaks.

Comment 99

[Darin Fisher]

what is the charset of relativeToKey?  ASCII?  UTF-8?  platform dependent?

Comment 100

[Conrad Carlen (not reading bugmail)]

> what is the charset of relativeToKey?  ASCII?  UTF-8?  platform dependent?
It's ASCII. The charset isn't really an issue there - the key is just a constant
from nsDirectoryServiceDefs.h or nsAppDirectoryServiceDefs.h

Working on alec's points...

Comment 101

[Conrad Carlen (not reading bugmail)]

> I wonder if we should make InitWithFile() exist on nsILocalFile()?

Hmm - probably so.

> this is worrysome because you use these for ReplaceSubstring, and nsString.h
says

Yikes - thanks for catching that. Just checked and nsString.h doesn't say that
though it does in .cpp. It worked in my testing of that case though. I'll ask
scc or jag and, if that really is broken, not use it.

> >+    rv = fromFile->GetUnicodePath(&fromPath);
>+    if (NS_FAILED(rv)) {
>+        nsMemory::Free(thisPath);

> Can you use nsXPIDLString here?

I would sure like to. Problem was, I chop the string into substrings by
inserting NULL chars. Is it recomended to use GetWritableFragment() with an
nsXPIDL[C]String and operate on that?

> use NS_NAMED_LITERAL_CSTRING(kParentDirStr, "../"); - you'll get length for
 free, plus no extra copying and no extra stack buffer

OK

> Not sure I understand the point of this loop.. are you removing "../"? can you
 comment here?

Yes. While we have leading "../" they get removed and I get the parent of the
file. That's what "resolves" or normalizes the "../" in an XP way.

> This last line is a little scary because you might advance past the end of the
 string - nodeEnd exists just past the end of the 
 string, if FindCharInReadable matched at the end of the string..
 though now that I look at this loop, I am again confused :) Can you explain
 what's going on here? :)

what's going on here is that I'm taking each node, which are separated by '/'s
and appending that to the file. You're right that the loop is wrong. After
putting up this patch, I changed it. It'll be in the next patch.

> Actually, I'm thinking this should be an ACString, because then:

OK - A[C]String classes everywhere!

Comment 102

[Alec Flett]

oh! I didn't see that :)
You can use Substring() to get access to fragments of strings, though now that I
see what you're doing maybe that isn't the best option. Eh, nevermind :) back to
raw PRUnichar I suppose, unless you can find a neat way of doing Substring() tricks.

btw, you can do stuff like this:
const nsAString& sub = Substring(str, start, len);

if "str" is a any kind of nsAString

Comment 103

[Alec Flett]

> OK - A[C]String classes everywhere!

All the k00l kids are doing it :)

Comment 104

[Conrad Carlen (not reading bugmail)]

> back to raw PRUnichar I suppose
For this, it's hard to beat for efficiency as well as easy code. If only there
was such a thing as an nsMemoryAutoPtr ;-)

Comment 105

[Conrad Carlen (not reading bugmail)]

Attachment: new diffs to prefs

I made the API change to nsIRelativeFilePref to use ACString for the key. It
made things easier in the impl and saved some allocations. One suggestion I
didn't take was this one (though I tried it):

>+    nsCAutoString key(Substring(keyBegin, keyEnd));

this could become 
const nsAString& key = Substring(keyBegin, keyEnd);

>+    rv = directoryService->Get(key.get(), NS_GET_IID(nsILocalFile),
getter_AddRefs(fromFile));

here you'd use PromiseFlatCString()

Using the nsCAutoString in this case is better because no allocation ever
happens. If I were to use PromiseFlatCString on the nsACString& made from
Substring, PromiseFlatCString would be forced to allocate so it could make its
promised string null-terminated. I stepped though this, and saw how the
allocation happens.

Brian, can you r= again? Alec, can you sr?

Comment 106

[Alec Flett]

true enough. I think in the future stack allocation will happen automatically
though. Until that happens, I guess nsAutoString is best..I'll try to review
this today

Comment 107

[Brian Nesse (gone)]

Comment on attachment 76452 [details] [diff] [review]
new diffs to prefs

r=bnesse.

Comment 108

[Alec Flett]

Comment on attachment 76452 [details] [diff] [review]
new diffs to prefs

looks great!

sr=alecf

Comment 109

[Conrad Carlen (not reading bugmail)]

Attachment: new diffs to xpcom

This patch differs from the previous this way:
(1) Puts InitWithFile on nsILocalFile instead of only nsILocalFileMac. The Mac
has its own implementation though.
(2) Fixed up the logic in a loop in SetRelativeDesc
(3) Using the const char* versions of ReplaceSubstring()
(4) Using NS_NAMED_LITERAL_CSTRING where I can.

This patch also has the test program I've been using on this stuff.

Comment 110

[Alec Flett]

Comment on attachment 76482 [details] [diff] [review]
new diffs to xpcom

ok, everything looks good...except for one thing that I just realized with this
patch.

When you do the ReplaceSubstring() on a UTF8 string, you run the danger of
replacing a single byte in a fragment of a UTF8 character sequence, and thus
changing its UCS2-encoded value.
What I mean is, say you have a UTF8-encoded string and the first "character" is
a 4-byte sequence that would translate to a single UCS2 character. If the 3rd
byte of that sequence happens to be '/', then you'll be inserting "%2F" which
will make the last 2 bytes of that sequence "%2" which would mean the
translated UCS2 byte would be a different byte, plus you'd be adding in an
extra 'F' before the next character.

All this basically means is that you have to do the ReplaceSubstring() step on
the UCS2 string rather than the UTF8 string.. 

fortunately, SplitPath() is working on PRUnichar strings, so it is safe.

I see this in two places. It will unfortunately complicate the code because
you'll have to make a temporary nsAutoString somewhere, but I think its the
only way to avoid this problem.

other than that, the patch looks great!

Comment 111

[Darin Fisher]

> What I mean is, say you have a UTF8-encoded string and the first "character" 
> is a 4-byte sequence that would translate to a single UCS2 character. If the 
> 3rd byte of that sequence happens to be '/'

alecf: no, its not possible for '/' to appear as part of a UTF-8 multibyte
character.  that's in fact why UTF-8 is so nice.  the 8-th bit is set on each
byte in a UTF-8 multibyte sequence :-)

UTF-8 bits mapped to UCS2 bits goes like this:

  110xxxxx 10xxxxxx           ->  00000xxx xxxxxxxx
  1110xxxx 10xxxxxx 10xxxxxx  ->  xxxxxxxx xxxxxxxx

notice that the UTF-8 bytes all have the high bit set.

Comment 112

[Alec Flett]

Comment on attachment 76482 [details] [diff] [review]
new diffs to xpcom

oh! neat. I thougt it was only the leading byte of a multi-byte sequence that
had the high-bit set.

That explains why it can take so many bytes to represent one UCS2 character :)
sr=alecf then!

Comment 113

[Conrad Carlen (not reading bugmail)]

Attachment: patch to mailnews

Comment 114

[Alec Flett]

adding seth for account manager stuff (I don't know who the current mail account
manager owner is, but I'd like seth's input on this)

I'm a bit concerned about just going with the new pref, or at least just tacking
-rel onto the end of the pref name. I like the idea in general but I'm wondering
if mail will have any preferences in this area..

another possible approach is to add stuff to the pref migration code to convert
all the mail prefs over automatically. I don't think we've generally worried
about backwards-compatibility (i.e. at least making old browsers work with
current prefs) so its possible that a one-time migration may be all we really need.

Comment 115

[Doug Turner (:dougt)]

Comment on attachment 76482 [details] [diff] [review]
new diffs to xpcom

r-dougt.  this needs to land before mozilla 1.0.  After mozilla 1.0, these API
changes will not be allowed.

Comment 116

[Conrad Carlen (not reading bugmail)]

> another possible approach is to add stuff to the pref migration code to convert
all the mail prefs over automatically. I don't think we've generally worried
about backwards-compatibility

I think we should be worried about backward compatibilty. It is an headache for
 users who download nightlies. You'd be surprised at the number of bugs that
come in against profile mgr where people say "I ran the new build and it
corrupted the profile for my old build." Compatibilty in both directions should
always be maintained, at least for a few milestones. Then, as the number of
users who would run into this problem drop, support for the old pref can be dropped.

Comment 117

[Scott MacGregor]

Wow, this is some pretty scary stuff. Particularly for mailnews. May I ask a
silly question. I don't see this listed on the mozilla.org branch landing plan.
Nor do I see the branch name that this is on? Is it not a branch? A change this
size definetly needs to be on a branch AND it needs to be on the landing page.
Branch bits need to be given to QA for testing purposes as well. 

Given where we are in the cycle, I really would rather not see the mailnews
changes go into the tree for this. If you can land the prefs and xpcom stuff
without taking the mailnews changes, I'd prefer that. That's my take....

Comment 118

[A. Craig West]

While I agree that in some ways this is a scary change, that is the point of
having to get it in for 1.0. If it doesn't make 1.0, there will be no good way
to work it in later without breaking the prefs file. 

Comment 119

[Jaime Rodriguez, Jr.]

When would you be thinking about landing this, and where (1.0 branch/trunk)?

Comment 120

[Asa Dotzler [:asa]]

Comment on attachment 76452 [details] [diff] [review]
new diffs to prefs

a=asa (on behalf of drivers) for checkin to the 1.0 trunk

Comment 121

[Asa Dotzler [:asa]]

Comment on attachment 76482 [details] [diff] [review]
new diffs to xpcom

a=asa (on behalf of drivers) for checkin to the 1.0 trunk

Comment 122

[Jaime Rodriguez, Jr.]

adding myself.

Comment 123

[Jaime Rodriguez, Jr.]

adt1.0.0+ (on behalf of ADT) for checkin to 1.0, for all reviewed changes except
the mailnews patch.

Comment 124

[Andy Lyttle]

A couple quick comments on this:

1) Anything that currently uses a file path that is within the user's profile
directory needs to be converted to use a relative path, but absolute paths
should also be allowed.  For example, "ImapMail/mail/rules.dat" vs
"/var/foo/mail/rules.dat".  Perhaps this could be implemented as
resource://profile/ImapMail/mail/rules.dat vs file:///var/foo/mail/rules.dat? 
Would that be a bad idea?

2) From a UI perspective, I think the user should be shown only absolute paths,
and they should be converted to relative paths if they are within the profile
dir.  The average user won't notice a difference, until they move their profile
dir and find that everything still works.

3) Mac OS uses ":" as a path seperator and "/" is a valid character in a
filename.  If we're storing paths as URLs, "Folder:file/name.txt" can just be
encoded as "Folder/file%2Fname.txt" which probably already works in file://
URLs.  Obviously a filename like that isn't portable to Windows, but that's not
Mozilla's problem.

4) I think the ability to copy a profile between platforms is REALLY cool. 
Anything referencing an absolute path will break, but that should only be things
the user has explicitly defined (and thus is aware of).  If a Windows user keeps
his IMAP data at Q:\Imap\, he'll be expecting Mac OS not to be able to find it
there if he moves his profile over, and he'll know he needs to change the path
manually.  However, if he keeps everything in the default locations, he should
be able to boot Linux and point Mozilla to "/mnt/windows/Documents\ and\
Settings/username/Application\ Data/Mozilla/Profiles/default/a1b2c3d4.slt" and
have everything work perfectly.  That would rule.

Comment 125

[Thomas Swan]

FWIWk, I really think the mailnews prefs need to be able to use both relative
and absolute paths.

Comment 126

[Conrad Carlen (not reading bugmail)]

>3) Mac OS uses ":" as a path seperator and "/" is a valid character in a
filename.  If we're storing paths as URLs

That's taken care of in the current patch and, no, URLs are not being used.


> 1) Anything that currently uses a file path that is within the user's profile
directory needs to be converted to use a relative path, but absolute paths
should also be allowed.

> FWIWk, I really think the mailnews prefs need to be able to use both relative
and absolute paths.

The current patch does that.

Can people refrain from this sort of comment unless they've read the patches and
know what they do?

Comment 127

[Jaime Rodriguez, Jr.]

really adt1.0.0+ this time!

Comment 128

[Thomas Swan]

Re: Comment #126

Conrad, perhaps I should have referenced Comment #117 in the original comment. 
I can read the patches, I know what they are doing.  It seemed like a fairly
large dissenting voice, to not do the diffs to the mailnews area, and I thought
I would chime up, in favor of the changes not that my voice matters much.

Comment 129

[Conrad Carlen (not reading bugmail)]

Prefs and xpcom patches are landed. As soon as I can make test builds on all
platforms with the mailnews patch which makes use of this, that tests well, adt
wants it, this could be fixed. 

Comment 130

[Conrad Carlen (not reading bugmail)]

The part of this which was adt1.0.0+ (the facility for having XP relative file
descs) is in so I'm calling it fixed.

The new bug which will have mailnews use this (and actually do some good) is bug
137006. 

Comment 131

[Pete Boyd]

*** Bug 139514 has been marked as a duplicate of this bug. ***

Comment 132

[Grace Bush]

verified code fixes

Comment 133

[Jaime Rodriguez, Jr.]

If this was verified on 1.0.0 or 1.0.1, pls add the keyword vefified1.0.0 or
vefified1.0.1. thanks!

Comment 134

[Peter Lairo]

For better understanding:
1. Will this bug fix old profiles? Will old profiles be "converted"?
2. What are the *filenames* where the pathname is now stored as "relative"?
3. What are the things that users can *do*, now that this bug is fixed?

Comment 135

[Moritz Balz]

-> 3. Users can

(a) move their profile directories *without* getting all of their mail folders
lost (=> without having to correct all related path settings manually)

(b) use their profile directories across different os. this was impossible as
long as idividual mail folder paths were given as absolute names
(e:\mozilla\mail\mymail... not working under linux, /home... not under windows etc.)

just read the comments to find more problems described that ware issued by this bug.

Comment 136

[Conrad Carlen (not reading bugmail)]

Sorry, what was checked in from this bug offers no gain whatsoever to the end
user. See comment 117. That was the consensus - to check in the portion which
provided the facility of XP relative paths in the codebase but not the mailnews
code which would make use of the facility. That portion was split off into bug
137006 so that the part of this bug which had adt approval (prefs and xpcom)
could be closed out. Probably, when this was still on adt radar, it should have
been re-summarized as "Add support for XP relative paths" and 137006 should have
been given this bug's summary. I think it's better just to move along to bug 137006.

Comment 137

[Jackie Patti]

This is not fixed.  I've been using Mozilla a few months, starting with a 0.9
release, moving to 1.0 and finally 1.1

Hear my saga...

I run a home business and we had set up all the Windows98 computers to save all
user data to an NT server.  This means everything, the MyDocuments folder,
Favorites for IE, Navigator profiles, Outlook email folders, etc.  The reason we
did this is because I don't want work stopping because a computer is fubar'd (we
can't ghost easily because each machine ahs different hardware - we're using a
lot of used equipment).  If I have a work deadline, and my machine gets toasted,
I need to be able to move to another machine, login and have everything, and go
to work ASAP.  All our machines were setup so anyone could login and work on any
of them as needed.

When we decided to move to Mozilla for browsing, mail, news and calendaring, we
set up all the profiles on the same server.  At the same time, we set up our own
IMAP on a Redhat server and gathered our email via fetchmail and pointed to our
IMAP server.  This worked great.  Even in the field with a laptop (different
profile, of course), I just hit my IMAP server and had access to all my email.

Then we moved to a very rural location where we could not get cable, DSL or
wireless.  Being back on dial-up meant I could not access our IMAP server away
from home.  So I reconfigured Mozilla to grab my email via POP, turned off
fetchmail, and manually moved all my email from the IMAP folders to the newly
created Mozilla mail folders via the mail-and-news application.  

I did this on a stationary PC at home.  I then copied the profile to my laptop.
 Since Mozilla was entirely fubar'd, and a reinstall didn't help, I rebuilt my
laptop from scratch.  Then I then wrote a little DOS batch file to copy the
entire profile from my home box (which is on a network server) to my laptop
manually and vice-versa (and copy a few other things I need).

Basically, it took me over a week, and way more research than was necessary, to
fix my laptop due to all the absolute paths in various files within the profile.
 In spite of folders for each POP address already present, Mozilla made new ones
and I had to manually change the directory in each case.

It's taken me scores of hours to get Mozilla working properly on both boxes, and
my batch files are about ten times longer than necessary since I have to specify
the exact path of every single file to be copied (now that I know which are safe
to copy and which are not).  

And once that was working properly, Mozilla crashed on every form submission (an
*entire* crash of all applications) due to some problem with copying the .w and
.s files.

At a minimum, the location of the main profile directory needs to exist in one
human-editable file... all other paths should be relative to it.  

And frankly, the profile manager should have a button to click to copy a profile
from wherever it actually is to *exactly* the directory where I want it to be -
and auto-convert anything that eneds to be converted.  

I should not have needed to learn this much about Mozilla and what all the
specific files do in order to copy a profile.  It's been a bit much, and I can't
see the average user going through all this.  And it makes Mozilla look bad, my
co-workers know I am the only Mozilla person around, and also know I have been
unable to do my email at work for a few weeks now.

I'm not writing to criticize, I don't know enough programming to actually help.
 Sorry.  I'm writing to clarify the types of issues that come up.  I need to
store my profile on a server *and* on a local machine *and* be able to
synchronize the two.  I've been using Mozilla just a couple months and this
really drove me crazy to the degree of thinking seriously about going back to IE
and Outlook (god forbid!)  I can't go back now, I have learned too much.  ;)

Comment 138

[Conrad Carlen (not reading bugmail)]

Jackie, read comment #136.

Comment 139

[Grace Bush]

v

Comment 140

[Arne Falk]

Have you read the comments for bug 12911 ? Could be a bug caused by solving bug
12911

Comment 141

[OstGote!]

Please change someone the summary as mentioned in comment 136. It's confusing.

Comment 142

[Adam Katz]

a quote from http://www.mozilla.org/unix/customizing.html#key_example
> For editor and browser windows, create a file called userHTMLBindings.xml and 
> link it into res/builtin wherever mozilla is installed on your machine. 
> (Eventually we want this to live in the user's chrome directory, but that doesn't 
> work yet; watch bug 12911 for progress on this. We recommend, for now, that you 
> keep the real user bindings file in your chrome directory and make a symbolic 
> link (=shortcut on Windows, alias on Mac) from there to res/builtin.)

this bug is supposedly fixed (though bug 201011 is not), 
and that should be reflected in the documentation on that page.
the current text is confusing, mentioning this bug is not needed.

Comment 143

[GavinS]

(In reply to comment #142)
> this bug is supposedly fixed (though bug 201011 is not), 
> and that should be reflected in the documentation on that page.
> the current text is confusing, mentioning this bug is not needed.

I've raised bug#266695 about updating the key binding customisation docs

Comment 144

[Akkana Peck]

Conrad: could you or someone else involved with the patch explain how these
profile-relative paths are used?  Is it a resource://MagicVar url, or something
else?  I didn't get a response asking in bug 201011, but the xbl key binding
code needs to be fixed to reference a profile-relative file in order to fix that
bug.

Comment 145

[timeless]

akk: they kinda look like this:

mail.server.server4.directory-rel
[ProfD]../../../../../../../Documents and Settings/user/Application
Data/Mozilla/Profiles/default/random.slt/ImapMail/127.1
mail.server.server4.directory
C:\Documents and Settings\user\Application
Data\Mozilla\Profiles\default\random.slt\ImapMail\127.1

Especially when the code messes up, as it clearly did here.

A better version would be:
mail.server.server4.directory-rel
[ProfD]ImapMail/127.1

except, that the code is clearly not working well (fixing it is on my todo list).

note that code has to specifically pick a pref and treat it as a Complex pref w/
a certain type (nsIRelativeFilePref)