Closed Bug 137006 Opened 22 years ago Closed 21 years ago

Use nsILocalFile relative descs for mail prefs

Categories

(MailNews Core :: Backend, defect)

Product:
MailNews Core
Component:
Backend
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
mozilla1.5alpha

People

(Reporter: ccarlen, Assigned: Bienvenu)

References

Details

Attachments

(3 files, 1 obsolete file)

proposed fix
34.15 KB, patch
mscott
: superreview+
Details | Diff | Splinter Review
fix xpcom relative desc code
669 bytes, patch
sspitzer
: review+
mscott
: superreview+
Details | Diff | Splinter Review
patch to backport this to the 0.5 thunderbird branch
33.42 KB, patch
Details | Diff | Splinter Review 
This was split from bug 12911. The portion of that bug which added the
functionality of XP relative file descriptors to xpcom and prefs is checked in.
This bug is the remaining problem of using that funtionality in mailnews.
Attached patch patch (obsolete) — — Splinter Review 
patch from bug 12911
Status: NEW → ASSIGNED
Target Milestone: --- → mozilla1.0.1
  I think Alec suggested using messenger migration to do this just once. I
wanted to always have relative and absolute paths so that old clients could work
on prefs which had been updated.
  I think there's a safe way to do that: When messenger migration encounters a
prefs file that has not been updated, just back up the prefs.js file first,
then, replace any absolute file pref with one which can be made relative, remove
the old absolute prefs.
  This would allow us to have the prefs only in the newer format we prefer, but
backing up the old prefs file would allow somebody to go back to their old moz
in a pinch.

Alec, Seth - what do you think?
Blocks: 144712
QA Contact: gayatri → stephend
Seems to me that since this won't happen until after 1.0, it would be nice if
downgrading to 1.0 were seamless.  So, my personal suggestion is this:  if
prefs.js contains only absolute paths, add relative paths too.  If prefs.js
contains both relative and absolute paths, ignore the absolute paths, figure out
new absolute paths based on the relative paths, and store both.  If prefs.js
contains only relative paths, keep it that way.

Users should also be able to specify absolute paths if they want, so they can
store their mail on a separate drive or something.  These should be stored as
absolute paths stored in the new format, and handled like new relative paths for
the purposes above - two identical absolute paths would be stored, rather than
one absolute and one relative.

The only problem I see here is that if someone downgrades, then changes a path,
then upgrades again, the path would revert to the original relative path and the
absolute path would be ignored.

Comments?
Ahem. The patch already here does exactly what you suggest in your 1st paragraph.
I'm refraining from making a snide comment.
*** Bug 151141 has been marked as a duplicate of this bug. ***
*** Bug 54503 has been marked as a duplicate of this bug. ***
What's the status on this? Has it landed on the trunk? Is it still slated for
1.0.1 on the branch?
Whats happening on this bug?
Seems to me that once mozilla is using relative paths for profiles, it becomes a
lot easier to implement things like cross-os and cross-pc profile sharing as
well as things like "profiles stored on a server"
It's a few things back in my queue. Setting milestone.
Target Milestone: mozilla1.0.1 → mozilla1.3alpha
Blocks: 58647
Can someone tell me what the patch thats attatched to this bug is missing before
its able to go into the current trunk (for 1.2alpha one would assume)?
Maybe this patch should be applied ASAP before it starts to bit rot?

I've again (in 3 months time) had the need for this functionality - this time a
Mozilla nightly has corrupted the profile and I needed to migrate my mail
accounts to a fresh one - a nightmare! Replacing all those hard-coded paths in
prefs.js manually...
Apparently there is more need for this feature, because a project was started on
Sourceforge to convert linux profiles to windows (see
http://mozexp.sourceforge.net/). I don't dare use it though, because it might
not stay synced with the nightlies and mess me up bad. So better check this one
in quickly (pretty please). I'll help testing if I can fetch a build somewhere
that has the patch included.
this bug isn't fixed, so of course there's more work to be done here!

I suppose it might be too late for 1.2, but maybe we can think about trying to
reland this in 1.3
I'm quite willing to do some of the work. However, I have never done any hacking
on mozilla (never even compiled it myself yet!) and looking at the attached
patch, this issue is spread all over the tree. Doesn't feel like a project for a
newbie to get his feet wet.

However, if I can help in any indirect way like, say, testing, please let me know!
I'd like to add my me-too.

I really hope it isn't too late to get it checked in for 1.2.
I have a convoluted configuration of mail accounts - POP3, IMAP, NNTP, and they
give me lots of headaches. From time to time Mozilla looses them in prefs.js or
corrupts the profile in some way, and then I have to restore them.

My config would be a great testbed for this patch, and having the patch finally
checked in would save me much hassle in the future when Mozilla messes up the
profile again - restoring things from a backup would be _SO_ much easier.
*** Bug 174228 has been marked as a duplicate of this bug. ***
Alright - need to get this in early in 1.3. Though the patch has bitrotted, I
still like its approach of reading and writing both absolute and relative paths.
It's compatible in both ways which is crucial. Also, even with reading/writing
both types of paths, I think the patch here offers a net reduction in lines of
code :-)
*** Bug 177532 has been marked as a duplicate of this bug. ***
*** Bug 183029 has been marked as a duplicate of this bug. ***
1.2 is out.

Can we use this period of time when the tree is relaxed to check this in, hammer
on any bugs that the patch would cause, and have this feature ready for 1.3 ?

Pretty please?

I'm being hit by this bug all the time, having different machines with different
OS'es and drive layouts...
Yes, I'd like to get this in soon. There's still some debate about whether to
store both absolute paths under their current pref names for backward
compatibility and store relative paths under a new pref name. (That's what the
attached patch does.) Another approach suggetsed was to use a one-time migration. 

I'd like to get some opionion from acct mgr and mailnews-in-general folks.
Bhuvan, Seth?
I'm using different versions of Mozilla (trunk nightlies, stable english
releases from the past, polish releases etc.) on different OS'es (usually Linux
and Windows 2000, but occasionally Win9x).

So often there's a situtation where I (accidentally or not) launch an old
release/nightly. The approach where the old prefs are preserved for
backompatibility and new ones are generated seems more appropriate to me - there
are smaller chances for a profile screwup. One time migration is risky, because
people _do_ go back to older releases with their profiles (occasionally).
*** Bug 183204 has been marked as a duplicate of this bug. ***
*** Bug 186774 has been marked as a duplicate of this bug. ***
Wold it be possible to get the attached patch refreshed and checked in before
1.3 final, then wait until, say, 1.5 final and another commercial Netscape
release  and then create another patch which causes Mozilla to remove obsolete
pref names when found in prefs.js?

That would leave a safety period with some releases that handle both new prefs
and old prefs and don't dataloss. People going back to older releases wouldn't
be  a problem then.
there's no way this risky thing will make it into 1.3final... but we really do
need to get this into 1.4alpha...conrad? :)
What about extracting those parts of code from the patch that add support for
new style prefs, and apply that code before 1.3 final, so that 1.3 would support
those prefs, but wouldn't do any automatic conversion (so no risky action would
be taken, only that 1.3 would be the first release that would handle new style
prefs)?

That way testers and users going for some reason back to 1.3 from 1.4 and later
wouldn't lose access to their mail folders.
olo: No, 1.3beta is out and the only bugs that should be going into 1.3final are
stop ship, data loss, etc type bugs. This is not preventing anyone from using
the product.

Yeah, I'd really, really like to get this in 1.4. I asked a while back, comment
22, for opinion from mailnews people. Can we get that question decided and I can
move forward with this?
Target Milestone: mozilla1.3alpha → mozilla1.4alpha
Alec wrote:
> This is not preventing anyone from using the product.

Well, in a way it is; For all of us that try to move from Windows to Linux,
using multiboot to have both options available, this is really what is missing
since having dual mail folders creates a mess. OpenOffice has solved this for
the office part, but the Mozilla development has been really disappointing. The
main reason to use Mozilla is to have this type of cross-platform support. But
OK, it is not really 'preventing' anyone from using Mozilla, but it 'prevents'
me from using Linux :-(. Could a special 1.3.1 with this problem fixed be a
possibility? (Anyone knowing about some other good cross-platform
browser-connected mail reader that I can switch to if 1.4 is lost too?)
It may prevent people from using the product because it will (if it is postponed
to 1.4a) make 1.4a profiles incompatible with 1.3final. The newest stable and
the newest release are the most commonly-used mozillas, and this would make life
really difficult for people switching between these two.
You're not using the product in any way that was claimed to be supported, so it
hardly justifies your argument for WHY this should be fixed in 1.3.

mozilla doesn't brush my teeth, and I could argue that that is a broken feature,
and that its stopping me from using the product (and by your argument, that is
prevents me from having proper oral hygene)... and therefor someone ought to
whip up some tooth-brushing code and land it before 1.3final.

Look, please understand the concept of releasing software: you reduce risk as
you approach a release. you don't introduce new risk because 25 people vote for it.

If it helps, I'm one of the voters, and I agree that it's a little late to put
this in 1.3.  Get it in 1.4 - at least the partial support suggested in comment
28. :-)
That was what I was trying to say. Please consider checking in the portion of
the patch concerned with /reading/ relative paths, so that when (if) the entire
patch makes it into 1.4a, 1.3 and 1.4a can still share profiles.

Running the current 'latest and greatest release' and the current 'latest beta'
on the same machine is not a wild and unusual thing to do :)
Aleksander, Lewis, and Klas:

Firstly, no modifications to current mozilla versions, e.g. 1.3, would need to
be made to enable them to read profile data from versions with the patch (1.4a,
hopefully) if the patch's current method is used, since both relative and
absolute pathnames would be stored- see comments 3, 4, and 22.

Secondly, the bugs which will get fixes while the tree is in lockdown mode are
things like "Mozilla kills printer," "Mailnews account settings and data lost,"
"_______ causes repeated crash/hang," "bug in ____ responsible for O-ring seal
degradation, parachute deployment malfunction, plague, pestilence, famine,
global thermonuclear disasters, and loss of balance to the Force." There's
absolutely no way this qualifies. Yes, I sure wish it could have been checked in
for 1.3 before things went to lockdown- the fix has been delayed for quite a
while. But an extra 1 month delay in fixing this bug is not the end of the world.
aha, after reading comment 22 I thought that the decision in comment 3 / comment
4 had been undecided. I take it it's been re-decided again?

I guess I should keep up better - I didn't realise the tree was locked - comment
29 suggested there was a chance of it getting in before the freeze.
Blocks: 101953
Flags: blocking1.4a?
ok, 1.4alpha has arrived - how soon can we get this in? do we need a new patch?
Alec, see comment 30. I'm sure we need a new patch. If we can get some agreement
on the approach WRT backwards compatibility, and get mailnews leads to agree,
I'll move ahead with it.
nominating.

sounds like we want this.

ccarlen and I talked about this, and we think the right thing to do is leave
messenger migrator alone, and fix the path related pref getters (and setters)
to do the magic.

1.4 alpha would be the right time to land this.
Keywords: nsbeta1
The general idea is that in addition to storing absolute paths in prefs.js (like:

user_pref("mail.server.server9.directory", "C:\\Documents and
Settings\\Administrator\\Application
Data\\Mozilla\\Profiles\\default\\xxxxxxxx.slt\\ImapMail\\imap.mail.netcenter.com");

we'd store something like:

user_pref("mail.server.server9.directory.relative",
"ImapMail\\imap.mail.netcenter.com");

this way, if you moved your profile directory, things would work better. 

Better (but not perfectly) as I think there are other files (besides prefs.js)
that we store absolute file paths (panacea.dat, filter files, downloads.rdf,
secmod.db) which would also need to be addressed before the whole issue was
resolved.

We'd keep the old pref around in case the user went back to an old build
(Mozilal 1.0x, Netscape 7.0x).
from ccarlen:

It's actually user_pref("mail.server.server9.directory.relative",
"%Profile%ImapMail\\imap.mail.netcenter.com");

where %Profile% is substituted at runtime. There are two scenarios where this
can be used:

(1) For backup and restore, the user can have the contents of the profile
folder, xxxxxxxx.slt, backed up somewhere. To restore, they can make a new
profile, which will have a different xxxxxxxx.slt part, drop the contents of the
old profile dir into the new profile dir, and it will work. Right now, that
won't work because, when the xxxxxxxx.slt portion changes, the absolute path is
invalid.

(2) Once all paths in the profile are relative, when a profile is missing, or is
moved somewhere else, or a machine is restored, we can just ask them to find the
profile dir, and they'll be back in business. Currently, with the profile mgr,
if  a profile is missing, the user has no opportunity to do anything about it -
it's just gone.
Ad. (2) in previous comment:
It's a great idea.
I've watched people migrating to/from Mozilla, and that was always a problem.

The user has to be able to copy/move profile between machines, and between
different locations on one machine (think moving profile to a bigger partition
as mail archive is growing too big), and tell Mozilla where to look for the profile.
> I think there are other files (besides prefs.js)
> that we store absolute file paths

Maybe also training.dat?  Junk training and filters are my two biggest headaches
when moving among different machines -- I've tried copying training.dat and
whatever filter-related files I find, but it never seems to work and I always
end up having to build them all from scratch.  But I'm not sure this is a
pathname issue: seems like I've had problems migrating even to machines to which
I've copied an unsalted profile directory.
Blocks: 196119
Flags: blocking1.4a? → blocking1.4a-
Next alpha :-/
Target Milestone: mozilla1.4alpha → mozilla1.5alpha
*** Bug 200852 has been marked as a duplicate of this bug. ***
I apologize for stirring the pot, but I ran into this migration problem recently, which is why I submitted bug report 200852.  I see a lot of discussion among people who are hacking mozilla, but I am looking at it from the user point of view.  I did a lot of the copy and hacking mentioned in these comments, but lost some of the junk mail training etc when I migrated to linux.  

What I am looking for is a simple menu option to copy ALL of the information needed from a windows installation to a new linux installation, so that I can have  bookmarks, mail, newsgroups, junk mail training, and mail filters intact along with all local folders and their contents.

It seems that this would encompass more than just preferences, or am I mistaken?


Another problem I found was with the importing of email.  It appears that you cannot just take the Outlook mail file and import it.  You must actually have Outlook installed and runnig to do this.  Is that correct?

Thank you for your time.

If I can help, then email me at paulwheeler@qwest.net   I would need a little hand holding at first, but am willing to learn.

adt: nsbeta1-
Keywords: nsbeta1 → nsbeta1-
*** Bug 208085 has been marked as a duplicate of this bug. ***
*** Bug 210291 has been marked as a duplicate of this bug. ***
So, this patch was to be checked in for 1.4, but missed it.

Any chance it would get checked in for 1.5 alpha as targeted currently? Please?
We are willing to bear some regressions, and test for those, but please, give us
transportable profiles...
Can people from mailnews, prefs etc speak up about this so that we can get all
the issues regarding relative pathnames in profiles sorted out and actually get
down to business with the coding of relative pathname support?
*** Bug 216865 has been marked as a duplicate of this bug. ***
*** Bug 220044 has been marked as a duplicate of this bug. ***
*** Bug 139926 has been marked as a duplicate of this bug. ***
Regarding comment #44: I had no problem copying training.dat and message filters
between machines with different path schemes (e.g. from Linux with profile in
/home/user to Windows 2000 with profile in C:\Documents and
Settings\user\Application Data\Mozilla)...

Regarding the patch: 1.6 alpha comes. It's a wonderful moment to check in this
patch and start testing before more stable branches come. Testers are waiting to
try this out.
yes! Is there anyone on the CC that feels like bringing this patch up to date
for 1.6alpha?

I'll help review - but the patch is over a year old, I'm sure it will need to be
reapplied by hand...
yeah, I can help with this. I have some other stuff to finish up first, but I'll
have a crack at applying the patch.
Assignee: ccarlen → bienvenu
Status: ASSIGNED → NEW
Not much time left, as Asa said in bug 137698:

------- Additional Comments From asa@mozilla.org  2003-10-26 14:02 -------
Seth, Scott, time is short for 1.6a.  If this is going to happen, it needs to
happen quickly. 
I don't think this should go into 1.6a at all - it needs testing on the trunk
first, I'd say.
So you think this should wait until 1.6 branches to go on the trunk? No way.
This is more than a year overdue, has already missed target milestones 1.3a,
1.4a & 1.5a. Put it in the trunk today so that it gets maximum exposure before
1.6b and 1.6 are released. 1.6a is/will be, after all, alpha.
From what I can see from the roadmap, 1.6b is going to be shipped off the trunk.
If this change is checked into the trunk after 1.6a, it will still make it into
1.6b, and will have the advantage of baking for a few weeks on the trunk.
Flags: blocking1.6b?
We're days (not weeks) from 1.6b so it's probably too late for this.
Flags: blocking1.6b? → blocking1.6b-
What abbout 1.6 final?
Blocks: 226848
I've applied this patch to the trunk by hand (it doesn't apply cleanly anymore)
and run with it. It works OK unless you specify a local mail directory that's
outside the profile directory. I'll debug that when I get a chance.
OK, I have a fix for the problem that occurs when specifying a local mail
directory that doesn't exist - I think that was a long-standing bug, and didn't
have anything to do with this patch. I'll attach a complete patch in a minute.
Attached patch proposed fix — — Splinter Review 
I've been running with this patch and haven't seen any problems. I don't know
how to test the actual switching of the profile directory, however.
Attachment #78810 - Attachment is obsolete: true
Blocks: majorbugs
Attachment #136667 - Flags: superreview?(mscott)
1.6 has branched, could this patch possibly be checked in on the trunk?

It missed 1.0, 1.1, 1.2, 1.3, 1.4 and 1.5. That's 6 milestones!
this patch is required for the local file to correctly figure out the relative
path.
Attachment #137708 - Flags: superreview?(mscott)
Attachment #137708 - Flags: review?(sspitzer)
Attachment #137708 - Flags: superreview?(mscott) → superreview+
Attachment #137708 - Flags: review?(sspitzer) → review+
Attachment #136667 - Flags: review?(sspitzer)
Attachment #136667 - Flags: superreview?(mscott) → superreview+
fix checked in. There's currently no UI (that I know of) for changing a profile
directory, but it does allow you to copy your prefs between profiles much more
easily, which should help with mozilla->thunderbird migration, as Scott points
out. And on linux, you can change your home directory...
Status: NEW → RESOLVED
Closed: 21 years ago
Resolution: --- → FIXED
Comment on attachment 136667 [details] [diff] [review]
proposed fix

>Index: local/src/nsNoneService.cpp
>===================================================================
>@@ -117,12 +116,18 @@
... stuff deleted
>     NS_IF_ADDREF(*aResult);
>-    return rv;
>+    return NS_OK;
>+
>+
>+    NS_ENSURE_ARG_POINTER(aResult);
>+    *aResult = nsnull;
> }

This last lines are not reached.. I guess they could be removed
I've tested on Linux that a trunk build correctly creates relative directory
specifications (e.g. mail.server.server1.directory-rel) in prefs.js, and
preserves old ones for backward compatibitlity (e.g.
mail.server.server1.directory) for news and mail folders.

Going back to an older build works fine, nothing breaks.

I didn't have time to test whether the relative specifications do work though :(
Shouldn't the following path settings have relative variants?

mail.newsrc_root
mail.root.nntp
mail.root.none
mail.root.pop3

I'm curious about browser.cache.disk.parent_directory as well. It's a different
component, but I'd like to have *all* paths relative in my prefs.js
Message filter seems to be affected.

With 2003122108-trunk/Win-Me, already defined message filter did not work -
folder not found.
Message filter UI says target folder for "move to folder" is account root
instead of the folder pointed by mail path written in msgFilterRules.dat.
Re-choice of target folder from UI was required.
(Note: contents of msgFilterRules.dat was same as before since I choosed same
folder)

This seems to be mail.root.pop3 case suggested in comment #73. 

In addtion, problem in falback exists.

After this message filter recovery and successuful folder move by filter,
20031217 trunk build or Mozilla 1.6b(Release build) with same profile
(mail.server.server1.directory-rel is written) failed to move mails by message
filter.
Message Filter says target folder is accout root again.

Which bug should I report these insidents to?
these are generally not used, since they're only for backward compatibility with
Netscape 4.x:

mail.newsrc_root
mail.root.nntp
mail.root.none
mail.root.pop3

there's code that will make it so we set the -rel pref variant, if we ever set
the non-rel pref, but since that code is never used, we never set the -rel pref.

WADA, could you e-mail me your prefs.js, and your msgFilterRules.dat so I can
see what's going on? Did you try changing your profile dir, or did your problem
simply happen as a result of this code change?
David, but what's about browser.cache.disk.parent_directory ?
Should I fill up a separate bugreport for that?



the mail.root.* prefs are not 4.x compatibility prefs - they are prefs to
determine the default root directory for a new mail or news account... so if we
haven't updated them to use the new relative descriptors, existing profiles will
work fine, until a new account is created...
yes, please file a bug for browser.cache.disk.parent_directory

Re the mail.root dir prefs, the code has been updated to use them, if they
exist. The problem is, as I said before, since no one ever sets those prefs, the
new profile-rel prefs are never set. And since those prefs are only set by 4.x
profile migration, I suspect that a new mozilla profile will not have those
prefs set at all. Creating a new account works fine, and sets the profile-rel
dir correctly...
I've filled up bug 229596 for having relative
browser.cache.disk.parent_directory dec.
David Bienvenu, my message filter problem in Comment #74 is different from
mail.* related problem, was Bug 229345.
> filters with action to move to folder with "@" in the name disabled
"!" and "@" were escaped in filter rule by old code, but new code does not
escape these special characters, and almost all my folders used by filters
contains "@" or "!" for ease of maintenance. 
I'll add a comment to Bug 229345. 
ok, thx. Yes, afaik, there is no mail.* related rel prefs problem.
To Comment #77 From Alec Flett :

> the mail.root.* prefs are not 4.x compatibility prefs - they are prefs to
> determine the default root directory for a new mail or news account...
> so if we haven't updated them to use the new relative descriptors,
> existing profiles will work fine, until a new account is created...

Alec Flett, what should users do when user's prefs.js contains mail.* already?

My prefs.js contained mail.*.
According to Comment #75 From David Bienvenu, this is because my prefs.js was
migrated from Netscape 4.x.
And when Mozilla with fix for this bug was used, my prefs.js began to contain
both mail.* and mail.server.serverN.directory-rel.

What should user do?
 (1) Should do nothing?
 (2) Should delete mails.* after new Mozilla use?
 (3) Should delete mails.* before new Mozilla use?
How about in fallback to Mozilla without fix for this bug?
How about in fallback to old Mozilla after adding new accounts on new Mozilla?
for the sake of clarity, here is the patch used to back port the mailnews
portion of this patch to the THUNDERBIRD_M4_BRANCH.
fixed / back ported to the M4 branch
Blocks: 230700
*** Bug 235854 has been marked as a duplicate of this bug. ***
*** Bug 243815 has been marked as a duplicate of this bug. ***
Product: MailNews → Core
No longer blocks: majorbugs
Attachment #136667 - Flags: review?(sspitzer)
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: