401396 - Mac Eudora importer truncates messages after any high-order (> x7F) character is encountered

Status: RESOLVED WONTFIX

(MailNews Core :: Import, defect)

Description

[David Ruggiero]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/419.3 (KHTML, like Gecko) Safari/419.3
Build Identifier: version 2.0.0.6 (20070728)

First, this is *not* a duplicate of bug 368372, which says it is specific to the Penelope product. This bug is in the normal, shipping, Thunderbird/OS X product.

When the Eudora import function is parsing a mailbox, and encounters a high-order byte (> x7F) in the mail message being imported, it will silently truncate the message and continue with the next message in the mailbox. This means many - if not most - HTML-ized mail messages are corrupted by the import process, as many common characters (for example, "smart" or "curly" quotes) have their high-order bits set.

Reproducible: Always

Steps to Reproduce:
1. Create a single email message in Eudora that contains at least one imbedded high-order character (for example, some "curly" quote marks.
2. Attempt to import the mailbox containing this message into Tbird.
3. See that the message is truncated after the high-order character is encountered.
Actual Results:  
Mail message is truncated at the point the high-order character was encountered.

Expected Results:  
Import should not treat high-order charactes as special; they should be imported along with the rest of the mail message, and the message should not be truncated.

I have no way of testing this on a PC platform - for all I know the bug exists in the Windows Tbird suite and importer code as well. 

Also, I have no way to test if the Communigate importer suffers from the same problem. It's possible it's inherent to it as well.

I regard this as a "major" severity problem because (a) it silently corrupts data, and (b) it may prevent users who are considering moving to Thunderbird from switching to it because they can't get their existing Eudora mailboxes converted.

Comment 1

[Magnus Melin [:mkmelin]]

If it still exists in Penelope I bet it's the same bug. It's the same importer afaik, perhaps with some additional patches that haven't made it to tb yet. So I'd say likely dupe of bug 368372.

Comment 2

[David Ruggiero]

If that's true, then it's more that bug 368372 is a duplicate of _this_ one (even though it was entered earlier). That bug says the problem is only in the Penelope code; this one says it's likely in _all_ the import code, and suggests that the Communigate import might be broken as well. I just don't want this one marked "dupe" and then someone else fixes 368372 only for Penelope - leaving the main code base unfixed.

Comment 3

[Magnus Melin [:mkmelin]]

Let's mark it as dependency...

Comment 4

[David Ruggiero]

That sounds like a perfect solution; solving this one will likely solve the other as well, if they're really all caused by the same code base problem.

Comment 5

[Ricardo]

There seems to be different problems for Eudora importing on Eudora 8 and Thunderbird. Thunderbird works better (it recognized all email messages whilst Eudora's didn't even go beyond recognizing one or two messages in each mailbox). But the bug is still running - in my case, since Portuguese uses accented vowels, the message cuts short the character right before that one.

Comment 7

[David Ruggiero]

Again, I do not believe this is a dupe of 368372. I am concerned that:

1) Bug 368372 refers only to the Penelope product, so it looks like this bug is less severe (affects fewer users) than it actually is; and

2) Any fix to the Penelope importer won't make it back to the main TB tree, where the importer is seriously broken.

Please read all the above comments before marking this bug a duplicate.

Comment 8

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

(In reply to comment #7)
> 2) Any fix to the Penelope importer won't make it back to the main TB tree,
> where the importer is seriously broken.

I'm not sure if this statement is true anymore.

Comment 9

[Matt Dudziak]

(In reply to comment #7)
> Again, I do not believe this is a dupe of 368372. I am concerned that:
> 
> 1) Bug 368372 refers only to the Penelope product, so it looks like this bug is
> less severe (affects fewer users) than it actually is; and
> 
> 2) Any fix to the Penelope importer won't make it back to the main TB tree,
> where the importer is seriously broken.
> 
> Please read all the above comments before marking this bug a duplicate.
> 


Regarding point #2 above, why do you say that fixes to the Penelope project will not make it into TB? The TB team is planning to take the importer improvements from Penelope (as Penelope is built on the Trunk).

I'll assign this to Gwenger as he is the one doing all the work on importers. He is better able to determine if the TB folks will take importer improvements that the Penelope team is making.

Matt

Comment 10

[David Ruggiero]

MD -

I'm happy to be wrong in that statement. I assumed that (a) the Penelope import routines were separate from the "normal" TB ones, and (b) that the development branch for Penelope had more-or-less forked and changes/fixes weren't getting passed back to the main TB code base. If either or both of those assumptions are wrong, fantastic - one fix, two bugs knocked down.

But I still insist that this bug isn't a dupe, in the sense that 368372 is only a subset of this one, even if it was entered earlier. Or at worst, they should be marked as completely dependent on one another, so that it's clear that the fixes to Penelope *have* to be moved to TB. I plead an ignorance of the organizational structure of TB development, though, so at this point I'll step aside and just hope that regardless of which bug gets worked on, it does get fixed in both code bases.

thanks /tsg/

Thanks so much for following up.

Comment 11

[Joshua Cranmer [:jcranmer]]

Jeff, can you give an update on the status of this bug?

Comment 12

[:aceman]

Importing from Eudora was removed from Thunderbird in bug 1243498.
Please see http://kb.mozillazine.org/Importing_from_Eudora_-_Thunderbird on how you can still import from Eudora if you need that.