Closed
Bug 116369
Opened 23 years ago
Closed 16 years ago
Large filter list gets incorrectly migrated; non-functional in Mozilla; and gets corrupt
Categories
(MailNews Core :: Filters, defect, P1)
MailNews Core
Filters
Tracking
(Not tracked)
RESOLVED
WONTFIX
mozilla1.2alpha
People
(Reporter: julien.pierre, Unassigned)
Details
Attachments
(5 files, 1 obsolete file)
I have a very large profile in Netscape Communicator 4.61 for OS/2, with about a half gigabyte of email and 50 filters, using POP3 mail. When I migrate it using Mozilla build 2001122019, I can view all my mail folders the first time I bring up Mozilla, including Inbox. All folder message count is displayed. However, if I close mozilla and then restart it, I can't see the messages in Inbox anymore. I can still access all other folders. However, the header message list on the top right does not display when the Inbox folder is selected. It is thus not possible to read the messages in Inbox. Closing and restart mozilla after that occurs does not help. I have noticed that in that situation, if new mail is fetched in Mozilla, the filters are not applied. All the messages end up going to Inbox, but Inbox cannot be read. If mozilla is closed after the new mail is stored in Inbox, then restarted, the Inbox gets its messages counted when clicking on the Inbox folder. Subsequently, as long as mozilla is not closed, the Inbox folder is properly viewable, and further mail messages retrieved are filtered properly to their respective folders. But the problem reappears if Mozilla is closed and restarted - the Inbox can then no longer be viewed and the scenario repeats itself. This has forced me to once again back out of Mozilla and revert to using Communicator 4.61 for my home e-mail :( I haven't done extensive testing yet, but it would appear that new profiles as opposed to migrated profiles do not exhibit this same problem with Inbox display & message filtering.
Comment 1•23 years ago
|
||
If you look in the filesystem, did your 1/2 gig of email get transferred over correctly?
Status: NEW → ASSIGNED
Reporter | ||
Comment 2•23 years ago
|
||
The migration did take up some space and create new files, yes. I will have to try again on a more recent build of mozilla.
Reporter | ||
Comment 3•23 years ago
|
||
I still have that problem on Mozilla build 2002012216. At the moment I can't open my inbox in Mozilla, and the "File/Get new messages" has no effect whatsoever. I have access to all my other migrated folders, just not Inbox, and I lost the ability to receive any new mail. This is a showstopper for me.
Comment 4•23 years ago
|
||
Can you check your encoding and see if it is set to Arabic?
Reporter | ||
Comment 5•23 years ago
|
||
Mike, View / Character-coding is set to Western (ISO-8859-1), not Arabic. Right now I'm able to view the inbox. I don't know how I got to that state. I will have to pick up a new build and migrate one more time to reproduce again. Currently, when I retrieve mail, mozilla complains about missing folders that apparently weren't migrated properly. So I am getting more convinced that this problem is related to filter migration. I have 39 filters though, so I'm not certain where the problem lies. More experimenting with this is needed and I don't have much time right now to do it.
Reporter | ||
Comment 6•23 years ago
|
||
OK, I retested this. This is an exact step by step summary of what I did : 1) I deleted my old mozilla build 2) I grabbed mozilla build 2002012908 . 3) I migrated my only Netscape 4.61 OS/2 profile on startup 4) I brought up Mozilla. Mail came up 5) I got prompted for password and hit "cancel" 6) I went to the mail preferences to set the POP to "leave messages on server"; also set the left mail window to display folder "unread" and "total" message counts 7) I hit "get msgs" 8) I got prompted for password again, and entered it 9) mozilla downloaded a couple messages into my Inbox, which I could read 10) I closed mozilla 11) I started mozilla 12) mail came up, prompting for POP password 13) I entered the POP password 14) there were no new msgs, but the inbox was invisible (nothing displayed) 15) I clicked on other folders, and got a message count and message list 16) I closed mozilla 17) go back to step 11, everything repeats the same after. Inbox is no longer visible.
Reporter | ||
Comment 7•23 years ago
|
||
In case this matters (and it well might), my 4.61 was set to come up with mail first, rather than the browser. So was the migrated mozilla browser profile. It might affect this bug.
Reporter | ||
Comment 8•22 years ago
|
||
FYI, in 0.9.8, I can access the inbox after migration. However, none of the filters work. Every message ends up in the Inbox, rather than in the folder where it's supposed to be. I looked at some of the filters in the dialog, and they appear to be fine. I will attach a copy of my rules.dat
Reporter | ||
Comment 9•22 years ago
|
||
Reporter | ||
Comment 10•22 years ago
|
||
FYI, I copied the OS/2 4.6 profile to a Win2K partition and installed Communicator 4.79. The profile worked fine there. Then I installed 0.9.8 on NT. It didn't see the communicator profile and didn't perform any migration. I filed a bug at 127632. Because of this I'm unable to verify if this is OS/2-only. I get stuck even earlier under Windows.
Component: Mail Window Front End → Profile Migration
Summary: Unable to access INBOX after migration, odd filter and display problems in INBOX → Filters don't work after migration
Reporter | ||
Comment 11•22 years ago
|
||
This bug still exists for me in the 2002032616 build for OS/2 . In the migrated profile, the mail filters just aren't working. Everything goes to the Inbox . This is very frustrating and I really need this fixed or I will not be able to migrate off communicator 4.61 for OS/2 ... I don't have any reason to believe the bug is OS/2 specific. The rest of my profile migrates fine. Even the filters look fine, except the rules don't work. It may be a bug in the filter engine. Does anyone care to try the rules file I attached and see if it works ? I'm using POP, FYI. You could setup a POP account with "leave messages on server" and see what happens. One of the filters migrated puts all mail to/from the netscape.com domain into one folder. So it's rather easy to verify if it's a migration bug or a filters bug.
Reporter | ||
Comment 12•22 years ago
|
||
OK, so I booted to NT, copied the rules.dat, and did the little pop account setup with "leave messages on server". I created a netscape subfolder out of inbox, and then sent a message to my address at netscape.com . When I fetched, the message ended up in my inbox, not the netscape subfolder. So the set of filters I have attached simply aren't working in mozilla. Please look into that. Since it isn't OS/2-specific, I'm setting this to all platforms.
OS: OS/2 → All
Priority: -- → P1
Hardware: PC → All
Target Milestone: --- → mozilla1.0
Reporter | ||
Comment 13•22 years ago
|
||
FYI : one of the filters at the top is for amazon.com in "to or cc". I tried to send mail to bogus@amazon.com, bcc'ing myself. The mail did end up in my amazon folder. But at the point of the netscape one, it fails. So there is probably a filter somewhere half-way in my 40-filter list that the Netscape filter engine doesn't understand and chokes on. You should be able to determine which one.
Comment 14•22 years ago
|
||
Please update this bug with an [adt1] - [adt3] impact rating (or take it off the list if it doesn't even rate adt3.) Thanks!
Reporter | ||
Comment 15•22 years ago
|
||
Steve, I am the reporter and set the target, because I consider it to be a critical mozilla mail bug, especially given that there appears to be no filter logging and therefore no way to diagnose what's going on. I don't know about the adt[x] priority though, since I don't directly work on the client but on NSS, so please set it up.
Summary: Filters don't work after migration → Some filters don't work (discovered after migration)
Updated•22 years ago
|
Target Milestone: mozilla1.0 → mozilla1.2alpha
Comment 16•22 years ago
|
||
Julien, Based on current comments 11, 12 & 13. I created a profile in 4.79 on my win98 system with a filter to a folder called "netscape" under my POP account Inbox. I sent myself a few messages that fit the filter criteria. The messages filtered OK. I launched the latest build with -installer so it would see the 4.79 profile that needed migrating. It migrated it then I opened it. I sent myself a message that fit the criteria for the filter and it worked OK. I will look at your filters to see if I can reproduce the netscape filter and reproduce the problem.
Reporter | ||
Comment 17•22 years ago
|
||
Esther, Which filter did you specifically test ? What I see is that some filters at the top of the list work, but after a certain point they are ignored. For example the "netscape" filter didn't work. That filter checks for jpierre@netscape.com in Sender or To or cc. This used when I send myself e-mail to/from home. I always Bcc myself on all my email since I use POP rather than use a Sent folder. So all the messages I send from home to jpierre@netscape.com end up in my "netscape" folder in Communicator. But in Mozilla, they end up in Inbox. Actually, I just found the problem. When I look at the filters through "message filters", some don't have a folder set for moving the messages to. Only the very first two filters are valid. "troll" is a delete filter, so it doesn't have a folder. "amazon" is showing a valid folder, "amazon on madbrain on pop..." . All subsequent filters, from Sonicwall on, show the folder as: "madbrain on pop.tsoft.net" . That's the name of my account and my pop server, but not of a specific folder. This is why none of these filters match. If I fix the folder by hand, then it works. I just tried with the Netscape filter by choosing the netscape folder in the filter, then sent myself a message, and it ended up in the proper folder. So based on this, I would say the bug is in the migration code.
Reporter | ||
Updated•22 years ago
|
Attachment #71281 -
Attachment is obsolete: true
Reporter | ||
Comment 18•22 years ago
|
||
Reporter | ||
Comment 19•22 years ago
|
||
Reporter | ||
Comment 20•22 years ago
|
||
Reporter | ||
Comment 21•22 years ago
|
||
I have just added three attachments : - the first one is the original rules.dat which runs fine in Communicator 4.61 (latest communicator for OS/2 ...) - the second one is the content of the rules.dat file in e:\internet\mozilla\bin\mozilla\profiles\default\nnoim9iv.slt\mail\pop. tsoft.net immediately after migration when mozilla.exe is first started . As you can see, it is identical to the rules.dat from Communicator, so the migration program didn't touch the rules - the third one is the content of rules.dat in e:\internet\mozilla\bin\mozilla\profiles\default\nnoim9iv.slt\mail\pop. tsoft.net after first getting messages in the mail program. As you can see, the "amazon" filter has been modified by Mozilla to look like this : actionValue="mailbox://madbrain@pop.tsoft.net/Inbox/amazon" All the other filters however are still the same as in Communicator, eg: actionValue="INBOX.sbd/Sonicwal" And those filters don't work. I hope this prrovides enough information for you to track the problem down.
Comment 22•22 years ago
|
||
Thanks, the new info helps a lot.
Comment 23•22 years ago
|
||
Julien, I read your rules.dat and created a profile in Communicator 4.79 added a pop account and added 4 filters with criteria similiar to yours. 1& 2 were Sender/Contains, the third was Subject/Contains and the 4th Sender/Contains, they all moved to folders under the Inbox. I confirmed they worked in 4.79 then ran NS6 with -installer so it would list the newly created profile from 4.79 as needing to be migrated. I migrated it and opened the Profile then exited so I could open another profile to send messages meeting the 4 filter criteria. I then opened the migrated pop profile, got messages and all messages filtered correctly. I then exited and looked at my rules.dat file and all of them showed the correct action values (actionValue="mailbox://xyz@xyz.com/Inbox/subtwo"). In comments #10, #11 and #12 you mention you manually copied profiles from place to place. You mention 0.9.8 didn't see the profile so you couldn't migrate it. Later you mention in the migrated profile the filters just don't work. Moving profiles and rules.dat files could be part of the problem you are seeing. I need to think about this more to see what to try next.
Reporter | ||
Comment 24•22 years ago
|
||
Esther, In comments 10, 11 and 12, I was trying to reproduce the problem in a Windows build. I didn't know if the problem was OS/2-specific or not, so I booted to my seldom-used Win2K partition at home. But I was completely unsuccessful at doing prrfile migration in Windows. Mozilla refused to recognize the presence of Communicator 4.79 on my Win2K partition, and therefore did not migrate any profiles or filters. This is why I was forced to copy the rules files manually in a new Mozilla profile on windows. Since I was stuck on Windows, all my other test cases of filters were done on OS/2 - which is what I want to use anyway. On OS/2, Mozilla correctly detected the presence of Communicator 4.61, and migrated the profile including the filter rules.dat . I never copied the rules.dat manually on OS/2. But the filters didn't work after the migration. The three files I attached today were all from Communicator and Mozilla builds on OS/2. Since I'm at the office right now, I think I may have more chance of getting the migration to work on Windows. I'll create a new Communicator 4.79 profile and try one more time with my filters here ...
Reporter | ||
Comment 25•22 years ago
|
||
OK, I was able to do the migration on Windows here at work. After migration, my filters didn't work either in Mozilla. The content of the rules.dat in C:\Documents and Settings\jpierre\Application Data\Mozilla\Profiles\madbrain\oi39lcjj.slt\Mail\pop.tsoft.net> was corrupt. I have attached the file. There is some binary garbage at the end. This probably means that there is a static buffer somewhere that's too small for my list of filters.
Reporter | ||
Comment 26•22 years ago
|
||
Reporter | ||
Updated•22 years ago
|
Attachment #77709 -
Attachment description: rules.dat after migration in Mozilla 2002040409 → rules.dat after migration in Mozilla 2002040409 on OS/2
Reporter | ||
Updated•22 years ago
|
Attachment #77711 -
Attachment description: rules.dat after first getting e-mail in Mozilla → rules.dat after first getting e-mail in Mozilla 2002040409 on OS/2
Reporter | ||
Updated•22 years ago
|
Attachment #77788 -
Attachment description: rules.dat file after getting mail in Mozilla on Windows → rules.dat file after getting mail in Mozilla 2002040403 on Windows
Reporter | ||
Comment 27•22 years ago
|
||
What is the timeframe for the "mozilla1.2alpha" target that you set for this bug ? I depend on this bug to be able to use mozilla for mail. Please don't tell me I have to wait 2 more years for that to happen ... Note that this is not a migration issue but a filter issue - the engine is not able to handle the large number of filters that I have, even if they aren't migrated. So I'm changing the component to filters instead of profile migration.
Component: Profile Migration → Filters
Reporter | ||
Comment 28•22 years ago
|
||
Setting a more accurate description of the problem. Hopefully this can be resolved easily and quickly. It appears that a static memory buffer is used to store filters in memory, and that buffer isn't large enough when you have many filters as I do. This is a regression from Communicator 4.x .
Summary: Some filters don't work (discovered after migration) → Filters don't work and get corrupt if they are numerous (50)
Reporter | ||
Comment 29•22 years ago
|
||
I finally got to the bottom of this problem. As you can see in my previous attachments, the migration process leaves actionValue="Inbox.sbd/folder" Only a couple of filters end up getting correct actionValue="mailbox://madbrain@pop.tsoft.net/Inbox/folder" I did a search and replace in my rules.dat file of Inbox.sbd to mailbox://madbrain@pop.tsoft.net/Inbox right after migration, but before getting any e-mail. This made all my filters work, and I did not get any corruption in the rules.dat file anymore. Therefore, this is primarily a problem with filter migration. I'd like to point out that there is still corruption in the filter code if the search/replace is not done. I was able to reproduce corruption of some entries in the rules.dat file before I figured out the search/replace. So there are 2 defects associated with this.
Summary: Filters don't work and get corrupt if they are numerous (50) → Large filter list gets incorrectly migrated; non-functional in Mozilla; and gets corrupt
Reporter | ||
Comment 30•22 years ago
|
||
Reporter | ||
Comment 31•22 years ago
|
||
I'd like this bug to get reconsidered for mozilla 1.0 . Since it is a migration bug, it will get in the way of people upgrading from 4.x . The fix for the migration part of the problem is probably not so involved, and it will make transition much smoother.
Comment 32•22 years ago
|
||
Has anyone interested in this looked to see if bug 163733 might be related?
Updated•20 years ago
|
Product: MailNews → Core
Comment 33•18 years ago
|
||
do you still see this problem?
Comment 34•17 years ago
|
||
Felix, Andrew... dupe or gone?
Assignee: sspitzer → nobody
Status: ASSIGNED → NEW
QA Contact: esther → filters
Comment 35•17 years ago
|
||
At this point in time I doubt anyone cares if Netscape 4 migration works.
Comment 36•17 years ago
|
||
(In reply to comment #35) > At this point in time I doubt anyone cares if Netscape 4 migration works. doesn't matter to me. Just want to see the bug resolved in some manner - duped or whatever. OTOH I see there is test data above.
Comment 37•17 years ago
|
||
perhaps ditto bug 47137
Comment 38•16 years ago
|
||
as with bug 45777... ->WONTFIX since 4.x migation has been dropped on trunk. If one has an overwhelming need to see this fixed on branch, please attach a patch and reopen the bug.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → WONTFIX
Assignee | ||
Updated•16 years ago
|
Product: Core → MailNews Core
You need to log in
before you can comment on or make changes to this bug.
Description
•