Closed Bug 116443 Opened 23 years ago Closed 19 years ago

Deleted inbox after receiving virus infected mail (McAfee, Norton, AntiVir)

Categories

(MailNews Core :: Backend, defect)

Product:
MailNews Core
Component:
Backend
Platform:
x86
Windows 2000
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rocky-beach, Assigned: Bienvenu)

References

(Depends on 1 open bug,
URL
)

Details

(Keywords: dataloss, Whiteboard: [asaP1])

Attachments

(2 files)

add (currently) hidden pref "mailnews.downloadToTempFile" which makes pop3 download each message to a file before appending to inbox
24.81 KB, patch
mscott
: superreview+
Details | Diff | Splinter Review
fix multiple message case
1.06 KB, patch
mscott
: superreview+
Details | Diff | Splinter Review
From Bugzilla Helper: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.7+) Gecko/20011220 BuildID: 2001122003 Today for the third time my inbox was emptied by receiving a mail which was infected by the W32/Badtrans@MM virus. Reproducible: Always Steps to Reproduce: 1. Some mails are in the inbox. 2. I receive an infected mail. My NAV software pops up and warns me. 3. Actual Results: The messages in the inbox are still displayed but the Inbox file has file size 0 (some information of the inbox mails are still stored in Inbox.msf) so I think they are displayed in the inbox, but it is impossible to delete these messages. I had this problem (first time) also with the build from 17th of Dezember. Expected Results: Mails in the inbox should not be deleted.
Navin, don't you have a dup of this?
Assignee: bienvenu → naving
Status: UNCONFIRMED → NEW
Component: Mail Database → Mail Back End
Ever confirmed: true
I'm pretty sure this is a dup also. There's another bug out there with the suggestion that we store our attachments outside of the mail file so that in the worst case scenario just that attachment would be deleted.
Status: NEW → ASSIGNED
Now I have more information. NAV software moves - after receiving infected mails - the now infected Inbox file to quarantine. I think mozilla should prevent this while running.
Just had this hit a user I had converted over to Mozilla for his E-mail. Was running InoculateIT for anti-virus protection. There were reports of BadTrans in the AV log files, but unfortunately no dates attached. The entire inbox got wiped out. No real indication of what all happened. From what the user told me, he had just click on Get Messages while in the process of dragging older messages to folders outside his inbox to be filed. Prior to even seeing what had been sent, the inbox went blank. I went and checked his profile folder, and sure enough the inbox.sbd file showed a file size of zero. Don't know what to tell him at this point. Personally, I would concur with putterman's opinion that attachments be stripped off an E-Mail prior to being stored in the inbox. Prior to my seeing this bug I would have thought of that as a usability issue. Now it seems that this suggestion should be thought of as a security or stability issue. I've moved the severity of this bug to blocker and added "dataloss" to the keywords. I did so as I can't imagine how much worse a bug can get than actually losing so much critical stored data so quickly. I would further recommend that a target milestone be established for this due to the extreme severity.
Severity: major → blocker
Keywords: dataloss
Same user, got hit again wiping out everything in his inbox. This may be taken as a flame, but I'm moving this user off of Mozilla mail to something more stable. This user has lost significant data twice over now. The second time was doing nothing more than changing which folder was in view. I wish I had more info to provide, but that's pretty much all I could dig out. By the time I got to the box the data was lost beyond recovery. Based on what this user described this was not a virus issue, but a database corruption issue. No reliable way to reproduce that I can see. It just decides to blow up every week or so with unread mail permanently destroyed. Now I need to figure out how to get the mail out of Mozilla and onto something like Eudora or Pegasus. I just can't allow this user to trust his E-Mail to Moz any longer.
I don't think that it's a good idea to store attachments separate from the rest of the mails. This would violate the mbox file format and would make restoring the original mail (for export, proof, external processing, whatever) harder or impossible. If it's really the anti-virus software deleting the folders, I'd said that it's a severe bug in *that* AV software, not Mozilla. How could Mozilla possibly prevent that?
Eudora has been storing attachment files seperate from the messages for at least the last 2 major versions now. The folder for these files is placed by default in a folder along with the message body files. For backing up, you'd back up the entirety of the /Mail folder, just like you would with Moz or NS 4.7x. All the attachments go with. If an attachment no longer exists in that folder, Eudora notes the fact the file is no longer there, and retains the message body. Without exception, everyone I have ever personally discussed this with would LOVE to be able to remove an attachment from an E-Mail while retaining the original body text. That of course would be a usability issue, where as this is an integrity bug. As to how Mozilla should have prevented this from happening, much as I have already stated. By removing the attachment before entering the mail into the InBox no virus protection software would feel compelled to muck about with mail folders. The AV software is left scanning only the attachments, thus no conflict is possible. The alternative is to have every AV vendor be fully aware of every mail client vendor's method of storing mail. Conversely, Mozilla would need to be aware of every AV vendor's methods as well. This would simply lead to finger pointing on both sides as to who was at fault. All the while, users are suffering data loss. All I would seriously ask of any developer reading this bug is to give a hard look at how Eudora handles attachments. It's quite a popular feature amongst it's users, and insures a safe seperation between the message body and what needs to be scanned by AV products.
This is an antivirus issue. The anti virus should not be allowed to touch the mozmail inbox. Since InnoculateIT has no email protection (only file protection), that folder should be exclude when scanning (I know that this means no email scanning, but that is what you get for using an anti virus scanner that does not scan emails). OTOH, mork should be smarter about data corruption, though I doubt this can happen between now and 1.0 (if ever).
> everyone I have ever personally discussed this with would LOVE to be able to > remove an attachment from an E-Mail while retaining the original body text. That's another bug (already filed) and different from separating *all* attachments from the bodies. > The alternative is to have every AV vendor be fully aware of every mail client > vendor's method of storing mail. mbox is a very common storage method. They *have* to know that, if they mess with emails at l. Mozilla's (and 4.x's) usage of mbox was very helpful in the past. Removing mbox just because some AV software is braindead and can't behave is not a good move IMO.
Replying to Micheal Collette's post <a32f9g$9j73@ripley.netscape.com> in n.p.m.mail-news: > Eudora strips an attachment from an E-Mail, [...] The anti-virus > software never gets involved with the actual mail folders as they contain > nothing but plain text. This is not necessarily true. In fact, to do anything at all with Mozilla's attachments, the AV software has to MIME decode them, which would be special email knowledge. If they can MIME-decode, they really should know mbox, too. But you said that the website of InnoculateIT's AV software (one of the problematic ones, as it seems) says that it doesn't scan email. IIRC, some AV software scans for some typical "signatures" of virii, and that signature might be in the subject or body of the mail or the filename of the attachment. The circumstances suggest that this might be what happened here. In this case, a separation between bodies and attachments wouldn't help at all - in the worst case, all your mails (without attachments) are gone while the attachments (incl. the infected one) are still there.
_basic, this bug has nothing to with mork. Did you mean mbox?
heh! yeah typo, (should learn to not try to look at too many discussions at the same time ;-)
From what I've seen of InnoculateIT in action, I don't believe it takes any actions until the file is trying to be saved. Reason I say that is it turned out this user had a bunch of mail with viruses attached. As I converted him over to Eudora and these files were stripped that's when the AV caught the bulk of these, and removed them. The inbox was not touched. The reason BadTrans caused problems with Norton is that the mail included HTML that attempted to automatically run the attachment. This is going to trigger any AV app into some kind of action. Only Mozilla Mail, so far as I know, attempts to launch this virus. Mind you, I have zero evidence to suggest that the second wipe out had anything to do with AV software or even an attachment. There was no warning from the AV software, no was there even a message being viewed at that moment. The user was in the process of switching folders. The main thing I'm trying to get across here is that due to Eudora's handling of attachments, at least 20 (most likely many more, lost count) infected E-Mails were cleaned up with zero data loss in the actual E-mail. Where as in Mozilla mail the inbox was wiped out twice over, each time taking one other folder (seemingly at random) and wiping it out as well. How could I possibly sit here and recommend that he continue to use Mozilla? Telling him that Moz is compliant with the mbox format isn't exactly a selling point when a user hadn't had any AV problems under NS 4.7x or now under Eudora. One last thing to consider here is that most users aren't going to be able to figure out what the heck happened. This bug has tracked two major AV vendors that very possibly caused the dataloss in question. The only thing an end user is going to honestly know is that Mozilla lost their E-mail. Even if the mail box is just renamed and moved, how many folks actually know where to look for it? Just pointing to Norton and/or InnoculateIT and saying they're wrong isn't going to do any good for Mozilla or it's users. If McAfee gets to doing the same thing should I just tell my users that they shouldn't run AV software at all? It seems most likely that most users will react by switching to a mail client that is capable of living with the AV product they have installed rather than the other way around.
> when a user hadn't had any AV problems under NS 4.7x or now under Eudora. From what I understood about his bug, 4.x would be affected just as Mozilla, because they work very similar (both use mbox). > The only thing an end user > is going to honestly know is that Mozilla lost their E-mail. That's what the user *thinks*, yes. We all know that. If a virus does an |rm -rf ~/.*|, did "Mozilla lose all your email"? Did anyone contact the AV software vendors (not via the usual support channels) to make them aware of the problem? They are creating themselves one of the problems they claim to protect they users from... Does anyone have any other suggestions?
> They are creating themselves one of the problems they claim to protect they > users from... At least, that's what it looks like. Would be a good chance to test the assertion.
In my case I haven't contacted the AV vendor as this particular product no longer exists. It was purchased from Computer Associates by e-Secure. Although I have run Norton in the past, I haven't got this installed on any machines to do testing with. I also don't have a copy of the suspected virus that may have triggered the AV software. Worse yet, I don't actually know that the AV software was at fault here. The only thing we do know is that something is causing critical data loss, and we suspect 2 major AV apps might have something to do with it. One thing that is different from Netscape's mail is how it handled HTML in the message body that attempted to save to the users hard drive some attachment. NS simply wouldn't do this. Mozilla does. The office I admin is very nearly all NS 4.7x E-Mail users running InnoculateIT. I have not seen any reports or complaints from users that would suggest that similar things are happening with this combination. Been about 6-7 months of using that as our AV software. The user I'm referring to in this report is a friend of mine that had been using this same combination for at least as long, without corruption of his mail box. > That's what the user *thinks*, yes. We all know that. > If a virus does an |rm -rf ~/.*|, did "Mozilla lose all your email"? Good point, except the virus never launched. Had the virus actually launched this wouldn't be a Mozilla bug report. I feel I should mention that I have been a long time advocate of providing the ability for a user to delete an attachment from an E-Mail. I get asked how to do this by too many people to not believe this is a highly wished for feature. Prior to a few days ago I would not have been an advocate of stripping attachments on the fly as Eudora does. Seeing this bug in action has definitely changed my mind about that. Just thought I should add where I'm coming from on this.
> One thing that is different from Netscape's mail is how it handled HTML in the > message body that attempted to save to the users hard drive some attachment. NS > simply wouldn't do this. Mozilla does. Can you please investigate this and file a bug report (if needed with security flag on)?
Bug 109249 is already handling this aspect of it. Not sure if one of these should be considered a dependancy on the other. They seem to be related. At present, 109249 is listed as a MIME bug.
Sorry to interrupt your discussion. NS 4.78 and receiving a BadTrans virus the inbox get wiped out too.
Definitely not an interruption Titus. In fact, very much to the point. Which AV app was involved?
It was Norton AV Corporate Edition 7.50.846
Is there no backup copy of the inbox file that could be maintained until Mozilla could be sure that the updated inbox file was written succesfully?
Adding relnote, decreasing severity from blocker to critical.
Severity: blocker → critical
Keywords: relnote
*** Bug 136623 has been marked as a duplicate of this bug. ***
What about enhancement request 103487? I can't help but think that Moz might be able to work WITH the AV software proactively to prevent data loss. If an item were forcefully scanned before even being entered in the inbox, any deletions or quarantines should be isolated to that particular item. Any thoughts about this?
Just as additional info, a coworker's inbox was wiped today by McAfee's VirusScan 6.02 in response to an email labeled "Re: Your Password" (W32.Frethem.D@mm). The antivirus prompted for user choice (delete, quarantine, and others). The user selected "delete", and the messages remained in the listing, but where not available. When new messages arrived, the message list goes nuts (wrong message, etc)
I ran into this with Norton AV. FYI, for anyone who has this problem, you can go into the Norton tools, look in the quarantine and select the Inbox file and restore it to recover the data. It appears to strip the 'virus' email in the process. It's not an ideal solution, but if you're running Norton it's a way to recover lost data.
mass re-assign.
Assignee: naving → sspitzer
Status: ASSIGNED → NEW
Commnet #26: Actually, Mozilla and the AV software both worked correctly. The user choosed "delete", the AV did delete the file in which the virus was hiding - the Mozilla Inbox. This is correct behaviour IMHO. The user should know what he is doing when commanding an AV software. But there may be some workarounds: 1. The AV probably said which file it is going to delete and the user should have spotted that. 2. Mozilla can lock its mail files. Currently it locks its files in the way, that they can be read, but not opened for write. And not even copied. Maybe a more prohibiting locking should be implemented. If it could deny reading, the AV wouldn't spot the virus and there would be no problem. The virus will be detected when it is opened and saved. No problem, as I see it.
Sorry, I was a bit wrong at point 2. Mozilla locks only the .msf files of folder that were opened. These can't be copied. The mbox files (like Inbox) are freely accessible. That's probably why the AV succeeded in deleting the file.
I would argue that this bug should probably be a WONTFIX. Here's the argument, The user, in most cases, is in control of the virus scanning software and can control whether a file is left alone, moved, or deleted during the "realtime" scanning of the file system. As such, it is the user's responsibility to decide what the policy should be. AV vendors will not be able to constantly write new code to handle each new mail storage format. Thus most have gone to a "proxy server" technique where a local server is inserted in the mail retrieval loop. This avoids the problem with mail storage formats because the mail is scanned prior to storage. For local file scanning, the solution is to simply not delete the mailbox. Here is an excerpt from Symantec's knowledge base on the subject: (sorry for the long URL) http://service1.symantec.com/SUPPORT/ent-security.nsf/9d94c8571a91ba4788256bf3007f62b5/712247a53df336e088256a22002724ad?OpenDocument&prod=Symantec%20AntiVirus%20Corporate%20Edition&ver=8.x&src=ent&pcode=sav_ce&dtype=corp&svy=&prev=&miniver=sav_8_ce ----------- Netscape or Outlook Express Inbox is quarantined when infected email is detected Situation: Symantec AntiVirus Corporate Edition (Symantec AV) or Norton AntiVirus Corporate Edition (NAVCE) has detected an infected email, and the options are set to have infected files quarantined. You now find that your Netscape or Outlook Express 5.0 Inbox has been quarantined. Solution: Netscape and Outlook Express store email in a single file. Symantec AV or NAVCE scans the file, and quarantines the file if a virus is detected. Because all email is stored in a single inbox file, the entire inbox is quarantined in some circumstances. To avoid quarantining the entire inbox file, we suggest excluding the inbox file from being scanned. How to avoid having the entire Inbox placed in Quarantine Excluding the inbox file from being scanned prevents the inbox file from being quarantined while still allowing a virus to be detected. When a virus is found in an opened email message, rather than during a scan or when downloading, the opened message can be safely quarantined--or deleted--without causing problems with the Inbox itself. ----------- As the virus scanning vendor suggests that the user configure the scanner to leave the file alone, it would seem that it rests upon the user to take the action, rather than Mozilla. If not marked WONTFIX, I suggest that "dataloss" keyword be removed and severity be reduced to normal (I could do this, but I'll leave it to someone with more clout) A addition to the release notes should be made indicating to the users that the mail folders should not be scanned by file system scanners. Scanning of mail by "proxy servers" is still a viable option for scanning mail. Kevin
*** Bug 186930 has been marked as a duplicate of this bug. ***
*** Bug 216948 has been marked as a duplicate of this bug. ***
I strongly recommend looking seriously at this bug. Here's what happened to me: Symantec AntiVirus Corporate detected with its heuristic virus auto-detection (bloodhound) a virus in one of the attached emails (showing inline as my preferences) of a bugtraq email I got. The attached email had some malicious html code in it and it triggered bloodhound. Wiped out my mail folder. Since we are running a big network of computers, we don't let the user chose what to do with the infection. It's desinfect or quarantine. So this is kind of odd since folder deletion is not preventable. Notice that I've never opened an attachment or did anything that could lead to virus infection. I only had enabled inline attachment viewing and am using bloodhound with SAV (enabled by default). IMO, the problem wouldn't be that bad if it was triggered only when clicking on an attached virus (because it will actually prevent infection! the opposite of if they remove this folder from realtime protection like suggested above!) but as my case showed it was triggered automatically in an entreprise scenario (no control over AV at all) where we don't trust OE but do trust Thunderbird. I imagine that this will be complicated to fix. Goodluck! Great product!
Please report this to Symantec, because it's clearly *their* program which deleted your mailbox. The more customers complain about dataloss there, the more likely they are to fix it.
The thing I don't understand is that if it's *their* program's fault how come outlook express handles everything perfectly? I will report it to Symantec but I feel they'll tell me it's Thunderbird's fault.
Outlook Express stores all of your mail in one giant database instead of individual mailbox files. Symantec probably has a special case for dealing with the OE file, but not for individual mailbox files like INBOX. Usually this is done with file extensions (INBOX has no file extension). I forget what the OE db is called but it has a special file extension.
bienvenu, do you think it would have sense to give mboxes a file extension, at least on Windows?
No, that would be a nightmare. Virus checkers can be configured to deal with files w/ no extensions.
If this is a problem that should preferably be solved by symantec, shouldn't we call in some people from Evangalism to add some pressure to our case?
doesn't symantec already have the option to deal with files with no extension? I don't have a copy myself...
(In reply to comment #6) > If it's really the anti-virus software deleting the folders, I'd said that it's > a severe bug in *that* AV software, not Mozilla. How could Mozilla possibly > prevent that? While I agree that it's the anti-virus software that absolutely sucks for moving files out from under running problems, Mozilla should be more robust (at least since the anti-virus behavior is now a known problem and the severity of the data loss is so great).
Also see bug 235588 (Inbox zeroed out when filter target folder file quarantined).
NAV has something called "exclude at risk files", according to the manual this option will be available if the could not be deleted. Maybe it could be locked or something so that NAV cannot delete it?
If someone here could email me (bart at mozilla dot org) the name of ANY contact person at Symantec so we could discuss an arrangement for getting this problem solved, I'd be happy to try & help connect the dots. I just tried making a blind call over there and ran into a nightmarish experience ("pls call technical support") that got me nowhere (I think)...
*** Bug 233966 has been marked as a duplicate of this bug. ***
Hi. I just installed Mozilla mail in order to prevent Netsky virus to cause havoc in my computer with Internet Explorer (which for unknown reasons I cannot update). I had the exact same problem : all my emails were "erased" after Panda cleaned the messages infected with various sorts of viruses. In fact, I could read my mails in plain text after opening my inbox file with the Mozilla composer. I tried to find a solution after looking for bugs solutions (and discover that this bug hasn't been resolved) and found that if you uncheck the "Display Attachments Inline" in the "View" menu of the Mozilla Mail windows, then, even if Panda detects viruses and cleans them, then you keep all your messages without problem. I hope this aids. If this can be useful, I run Windows ME OS, and not Windows 2000.
(In reply to comment #47) > Hi. > I just installed Mozilla mail in order to prevent Netsky virus to cause havoc in > my computer with Internet Explorer (which for unknown reasons I cannot update). > I had the exact same problem : all my emails were "erased" after Panda cleaned > the messages infected with various sorts of viruses. In fact, I could read my > mails in plain text after opening my inbox file with the Mozilla composer. > I tried to find a solution after looking for bugs solutions (and discover that > this bug hasn't been resolved) and found that if you uncheck the "Display > Attachments Inline" in the "View" menu of the Mozilla Mail windows, then, even > if Panda detects viruses and cleans them, then you keep all your messages > without problem. > I hope this aids. > If this can be useful, I run Windows ME OS, and not Windows 2000. Hello, I have today the same problem, with Netsky virus : when arrives, deletes Inbox. For similar raisons, I have recently move from Opera 7 to Mozilla 1.6 (Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113). Note that (with the same mail), this problem doesn't happend with MS Outlook Express 6. When this problem will be solved ? Thanks.
(In reply to comment #48) > (...) > Note that (with the same mail), this problem doesn't happend with MS Outlook > Express 6. > (...) Does anyone know what are the reasons for this difference? Are viruses detected without deleting mail in other mail clients, or is there any different behavior? Possibly it would be interesting to know the general mechanisms for this. Perhaps the problem is that some antivirus software is currently developed with MS mail clients in mind, mainly. (?) In this case, it seems that it would be important to talk to them for collaboration with Mozilla to fix this bug.
Mailbox files of Microsoft clients have a unique extention and those are added to the exclusion list of most mailclients. Mozillamail (and thunderbird) use mbox format mailboxes without an extention. adding the mbx or mbox extention would make it much easier to exclude the files for the AV companies.
This seems a case where the Maldir format would show its strengths over mbox. In fact, in this particular case it would even display the expected behaviour: the infected message would be deleted leaving the rest unchanged. Anyway, I think changing the mail database format is out of the question. An alternate solution would be saving the incoming message to a temp file (each message to its file) on arrival, and only after that, if it was still there, it would be added to the inbox. This way the AV software would be able to catch *only* the infected messages. This way it wouldn't be necessary to contact *all* the AV software houses. OTOH, the mbox is a pretty well known format, so it would be expected that AV software producers would know that it's this kind of file they're dealing with (without the need to check the file extension) and delete only the infected message, levaing the rest in place...
(In reply to comment #51) > This seems a case where the Maldir format would show its strengths over mbox. In > fact, in this particular case it would even display the expected behaviour: the > infected message would be deleted leaving the rest unchanged. Anyway, I think > changing the mail database format is out of the question. An alternate solution > would be saving the incoming message to a temp file (each message to its file) > on arrival, and only after that, if it was still there, it would be added to the > inbox. This only works for incoming messages with known virusses. The inbox would still be deleted if a new virus snuck into your inbox and is detected on a schedulled scan two days later. > This way the AV software would be able to catch *only* the infected > messages. This way it wouldn't be necessary to contact *all* the AV software houses. That's true. Just changing teh extension and announcing this in public with a huge font and a note to software houses in press releases all over the world should get their attention don't you think? (probably not). But changing the extension would probably be the best way to get around this. I vote for renaming inbox -> inbox.mbx or inbox.mbox. same for all subfolders.
*** Bug 243162 has been marked as a duplicate of this bug. ***
*** Bug 245414 has been marked as a duplicate of this bug. ***
I am having the same problem, and as far as I can tell it is a bug in Symantec AntiVirus Corporate Edition. My office just upgraded from version 8.1 to 9.0 and it performed a full system scan (all files). A bunch of my mailboxes were immediately quarantined. To recover my mail, I restored mailbox files from the quarantine and parsed the messages into smaller sub-mailboxes. Then I scanned each sub-mailbox file to find the infected section. Many iterations later I found and deleted the infected message and then put the "healthy" messages back where they belong. Some of the infected mailboxes were large so this was a very long process (not done yet). Since I manually scanned each mailbox/sub-mailbox file outside of Mozilla, I don't see how this can be a Mozilla problem. As far as I know, mailbox files are just palin text files as far as the anti-virus program is concerned. Having said that, it appears that the anti-virus program does recognize Outlook pst mailbox files as it extracts and quarantines the infected message from the pst file and leaves the rest of the mailbox alone. I consider this a Symantec bug, and I think they should be notified. Unfortunately, the tedious process described above doesn't always work. I am still in the process of trying to fix my Inbox (fortunately I don't leave anything important there). When I parsed the messages into sub-mailboxes and scan each one they were all fine. When I put all the messages back in a single mailbox, Symantec found a virus and quanrantined the whole thing again. I'm already losing my hair, and by the time I'm done with this, I may not have any left. Of course our sysadmin says I should just switch to Outlook. The ability to strip attachments from email messages would certainly be helpful here. On top of that, I often get mail with attachments I don't need to keep or that need to be saved elsewhere, e.g. in a file management system. These attachments end up just bloating my mailboxes. It is important to keep the email messages themselves as they provide a record of when the mail was received and who sent it. For sent mail, it is often important to know to whom I sent a file and when, but I don't need to leave the file attached. So to whoever out there is counting votes for an option to strip attachments off email messages, please add mine. Sorry for the length of this posting, but I thought the detail might be helpful.
(In reply to comment #55) > (...) > > The ability to strip attachments from email messages would certainly be helpful > here. (...) > > So to whoever out there is counting votes for an option to strip attachments off > email messages, please add mine. If you wish, you may add yours to the current 355 votes of Bug 2920 ["Delete attachment from mail message in folder (remove/strip attached files from email messages)"]. Some programmers are trying to fix it, a not easy task within the complex Mozilla code.
Many have posted that this is a user problem, and that simply excluding the Mozilla inbox folder(s) from the scan fixes this problem. Many of us are on corporate networks where we don't have the luxury of being able to pick and choose exclude folder for virus scanning. In particular Norton's corporate edition tends to lock users out of any such configuration options. As has been stated many times, this is a problem that could be easily resolved by storing attachements as separate files. While I'll admit I don't know how much work will be involved in doing such a thing, it seems like a better solution than simply blaming the problem on the user. One of the advantages of Open Source is that in general- developers don't turn a blind eye on user problems. I love Thunderbird/Mozilla, but I have to seriously look at going back to a M$ product now that I have lost all of my inboxes for the fourth time in the last month.
(In reply to comment #57) > (...) As has been > stated many times, this is a problem that could be easily resolved by storing > attachements as separate files. (...) In this case (something like the Eudora automatic functionality as a possible Mozilla option), it's bug 9309 -"option to have attachments auto saved to chosen location"- rather than bug 2920. Naturally it would be great to have these two enhancements soon.
> this is a problem that could be easily resolved by storing > attachements as separate files. This is *everything* but easy. Not only would it be a lot of work, it would also be very dangerous in case of bugs in the new code. > better solution than simply blaming the problem on the user. I'm blaming the anti-virus product, not the user. mbox is a very old, standard and widely-used format, they wipe it, they have no excuse. Please call them, the more people the better.
Look at Symantec knowledge base Document ID:2000051809560948 which describes their solution...
(In reply to comment #60) > Look at Symantec knowledge base Document ID:2000051809560948 > which describes their solution... That's: Symantec Knowledge Base Netscape or Outlook Express Inbox is quarantined when infected email is detected http://service1.symantec.com/SUPPORT/ent-security.nsf/6fffc7260966992188256bf300818635/712247a53df336e088256a22002724ad
Blocks: 247126
Adding dependency on 9309 which would effectively contrinbute to solve this. Confirming this bug. It also deletes/moves the Junk mail and Trash folders and whatever folders have emails with viruses in them. These folders should not be easily modified/moved/deleted by other applications. Platform: PC OS: Windows 2000 Pro Product: Thunderbird Version: 0.7.2 (20040707) Component should be changed to Mail Window Front End This has also been reported on the MozillaZine forums: http://forums.mozillazine.org/viewtopic.php?t=101013 http://forums.mozillazine.org/viewtopic.php?t=75470 http://forums.mozillazine.org/viewtopic.php?t=79453 Workaround: http://kb.mozillazine.org/index.phtml?title=Thunderbird_:_FAQs_:_Anti-virus_Software
Depends on: 9309
*** Bug 254453 has been marked as a duplicate of this bug. ***
No longer blocks: 247126
*** Bug 247126 has been marked as a duplicate of this bug. ***
As mentioned once, the whole issue should be resolved by moving to the maildir format. Although, there is a clear disadvantage of switching to a different default mail storage format, there are several other advantages to the format, not the least of which is that new user wouldn't have to know ahead of time of this risk in order to avoid having their inbox deleted. I believe there's another bug about supporting the maildir format.
(In reply to comment #63) > *** Bug 254453 has been marked as a duplicate of this bug. *** Sorry for filing this bug again, I checked for other bugs but didn't see bug 116443. I read a little of the previous discussion on this bug. I think it is fine and prudent that a virus checker removes or quarantines an infected file - email or others - that's the whole point of a virus checker, after all. In this context, excluding files from the virus checker (comment #60/61) is precisely what I don't want to do because incoming mail is the number one potential virus source on my machine. After thinking I had a great idea involving a temp file before saving into the inbox, I read comments # 25, 51 and 52 and discovered that others had the same idea years before me. Now, my question is: why is not every single e-mail given some counter number or time stamp as a name and saved individually in text or mbox format in an inbox FOLDER? I realize that this increases storage space, but since we're talking GB meanwhile, it would be well worth it for me.
*** Bug 238499 has been marked as a duplicate of this bug. ***
Blocks: 233833
Blocks: 235588
*** Bug 200287 has been marked as a duplicate of this bug. ***
Blocks: 256034
I wanted to note that this is something that Mozilla needs to handle. The latest versions of the two biggest (I believe) corporate virus scanning clients, McAfee and Norton, seem to have problems with this, and these problems do not occur when used in conjunction with Outlook or Eudora. Also, in the corporate environment, anti-virus program settings may be locked down, so the MailNews/Thunderbird mailbox files cannot be exempted and individuals in these situations flat out CANNOT use MailNews/Thunderbird as their e-mail client. As long as incoming files were written to an individual file first, then appended to the mailbox after the scanner has a chance to react, many (not all) of these problems would go away. Maildir would be the ultimate fix, but in the mean time, something MUST be done if corporate deployment is a goal.
Blocks: eudora
Summary: Deleted inbox after receiving virus infected mail → Deleted inbox after receiving virus infected mail (McAfee, Norton)
Blocks: 229235
No longer blocks: 256034
I'll mark a number of bugs as dups (currently deps), which are about different situations (filtering, compacting folders, Eudora import), but essentially the same cause.
No longer blocks: 229235, 233833, 235588
*** Bug 229235 has been marked as a duplicate of this bug. ***
*** Bug 235588 has been marked as a duplicate of this bug. ***
*** Bug 233833 has been marked as a duplicate of this bug. ***
(In reply to comment #70) > I'll mark a number of bugs as dups (currently deps), which are about different > situations (filtering, compacting folders, Eudora import), but essentially the > same cause. OK, but does that imply that whoever fixes this needs to test that the fix actually works in all of these situations? It may turn out that the cause is not quite as identical as you assume.
Flags: blocking-aviary1.0?
*** Bug 245538 has been marked as a duplicate of this bug. ***
*** Bug 260384 has been marked as a duplicate of this bug. ***
*** Bug 260855 has been marked as a duplicate of this bug. ***
Flags: blocking-aviary1.0? → blocking-aviary1.0-
Anyone interested in having this solved should also check 9309 and vote for it / contribute to it.
(In reply to comment #78) > Anyone interested in having this solved should also check 9309 and vote for it / > contribute to it. That should be restricted to those who think that the best way to solve this problem is a complete redesign of Mozilla's data formats, although the experts seem to say that that is not the best way. To put it another way, I don't agree that there is real a dependency on bug 9309. An alternative and probably much simpler way of solving this problem would be to save and reopen each incoming message (or each one with an attachment) as a separate temporary file before adding it to the inbox. This would trigger a virus scan of the temporary file. This is the same kind of strategy as the good one used with downloaded executable files, that they are always saved and reopened before being executed, and so any virus scanner has the chance to scan them. But I'm not sure if the performance hit would be acceptable. Is anyone seeing this problem with latest versions of anti-virus software? I haven't seen it since I have been using Norton AV 2003 and now 2004, with e-mail protection. Maybe that is because they catch the virus before it makes it to the inbox. (Or maybe it is because my ISP is catching the viruses before they get to me.) None of the recent duplicates mention exactly what anti-virus software is in use or whether e-mail protection is enabled.
Saving the file to a temp location and then adding won't help if you're behind with signiature files (happens to the best, just 10 minutes behind can be 10 minutes too many); the virus creeps into the inbox file; signiatures are downloaded; inbox get's deleted. I think you're no longer seeing this with NAV 200[345] because most virusses are stopped during teh POP3 transfer, but the above situation applies to this aswell.
How is this solved in ms outlook? It saves all it emails in one file, too, doesn't it? Or is this file just encrypted, so the anti-virus programms can't scan it?
(In reply to comment #80) > Saving the file to a temp location and then adding won't help if you're behind > with signiature files (happens to the best, just 10 minutes behind can be 10 > minutes too many); the virus creeps into the inbox file; signiatures are > downloaded; inbox get's deleted. NAV 2004 at least supports automatic update of signature files, if enabled as recommended. Of course they still come out after the viruses first appear. But in this scenario the anti-virus program should not delete an infected file, but should clean it or quarantine it (and quarantined files can be retrieved). The basic problem here as I observed it is that Mozilla overwrites the inbox before the AV program gets the chance to quarantine it. I think what is happening is that Mozilla can't read a file, because the AV program is locking it, and so instead Mozilla overwrites it, which is not blocked by the AV program - all before the user has had the chance to respond to the "what do you want to do about this virus" dialog. The fallacy here is Mozilla's assumption that because it cannot read a file that file may be safely overwritten; there are many reasons other than virus problems why this is a bad assumption e.g. multi-user locks, network glitches. This is certainly happening when an inbox is deleted while being compacted: Mozilla can't read the inbox because it is infected, so deletes it - see my report of bug 233833 of which this is considered a duplicate. By the way, ;-) would the problem be solved if mailbox files were given the extenstion .pst? That would automatically exclude them from at least some AV scans. Or is this extension a Microsoft copyright? ;-)
In response to Christian Fersch's question. My experience is with Outlook and McAfee, where McAfee hooks itself into Outlook and can scan the mail messages and delete them using Outlook's COM API.
*** Bug 264172 has been marked as a duplicate of this bug. ***
(In reply to comment #82) > > The basic problem here as I observed it is that Mozilla overwrites the inbox > before the AV program gets the chance to quarantine it. I think what is > happening is that Mozilla can't read a file, because the AV program is locking > it, and so instead Mozilla overwrites it, which is not blocked by the AV program I'm not sure if this is what really happens. When Norton AntiVirus 8.0 found the virus in my inbox, an alert box popped up that stated: Action taken: Clean failed : Quarantine succeeded : Access denied (see my dup 264172 for further infos) I read into that, that NAV tried to clean my mbox file, but didn't succeed because of a lock. However, it succeeded with moving the file to the quarantine. Furthermore, does anyone know, why I could use Thunderbird / NAV for over half a year now with hundreds of viruses trickling in my Inbox before this bug occurred twice this week (with the same kind of virus)? Does this affect only viruses found with bloodhound? Anyway. It's a nasty situation for anyone that gets into it. Even if it might be the AV-Vendors fault, it won't be easy to convince all of them to provide a fix, so a solution on the Mozilla side would really be appreciated.
This problem also exists with McAfee's VirusScan Enterprise 8.0i; I got my junk mail folder deleted even though I have only set it to automatically clean files and move into quarantaine them if the clean failed. Would be alot better if McAfee's Email Scanner would also do Thunderbird/Mozilla/Netscape 7 (which is all the same product).
Despite all the finger-pointing at anti-virus applications, this still appears to be an E-mail client problem. I use Norton Anti-Virus 2003. My E-mail client is Eudora Lite 3.0.6 (yes, several years old). I do NOT have this problem. Why? Because viruses are not found in the E-mail messages themselves, only in the attachments to those messages. Eudora Lite merges messages into long files, one per folder; but it saves the attachments separately, each attachment as an individual file. Thus, a virus results in the deletion, repair, or quarantine of the afflicted attachment file, leaving the merged message file untouched. I know the process used by my Eudora Lite is somewhat old-fashioned. But it works. The more modern process used by Mozilla and Thunderbird is broken with respect to anti-virus protection. Some questions: * How likely do you think it would be for all anti-virus application vendors to change their products for compatibility with Mozilla products? * Given the unending problem of computer viruses, which is more likely -- someone disabling their anti-virus application to use a Mozilla E-mail client or someone enabling their anti-virus application and refusing to use a Mozilla E-mail client? I must agree with comment #85 and suggest that this be fixed in the affected Mozilla products. There should be no room for an attitude that says: "We're right and we don't care how adversely it impacts our users or the future growth of our organization."
*** Bug 264916 has been marked as a duplicate of this bug. ***
I Have had the same problem with Norton AV 2004. It detected copies of netsky downloaded into the mailbox file and so quite sensibly blocked access to that file. To get mozilla mail working again I had to log in as admin, turn off the AV, manually edit the mailbox file to remove the content of said bad e-mails and then run mail and hope it all worked ok. As soon as I had it working 2 more Netsky mails came in and I had to do it all again :( Possible solutions: - Don't download attachments until the user tries to open them (as a pref) - Don't store the e-mail attachments in the mailbox as mentioned earlier. Prehaps store a ref to the file that contains the attachment? (Much better solution although both combined would be nice) This is getting to be quite an issue as I have had to go and fix quite a few machines with this issue. It also appears to be getting mozilla mail a bad reputation as "outlook never did this". I have to agree with previous comments about the fact that this needs fixing. Is there a good reason why the attachments are stored in the mailbox or just a legacy thing? (See Also: BUG 9309 and 2920)
(In reply to comment #89) ...> Possible solutions: > > - Don't download attachments until the user tries to open them (as a pref) Please, no! This would be disastrous for those who download e-mail by sometimes expensive dial-up and read it later off-line. Anyway, I don't suppose POP3 supports this. But then I would suggest using an ISP which filters viruses - not 100% of course. Or save and reopen the file as I suggested in comment 79 - again not 100% but not far short. ... > This is getting to be quite an issue as I have had to go and fix quite a few > machines with this issue. It also appears to be getting mozilla mail a bad > reputation as "outlook never did this". > > I have to agree with previous comments about the fact that this needs fixing. Agreed. This is potentially a major, almsot show-stopping problem with Mozilla/Thunderbird. It's not enough to say this is not Mozilla's problem, anything which means that Mozilla appears not to work is a PR disaster for Mozilla.
(In reply to comment #90) > (In reply to comment #89) > ...> Possible solutions: > > > > - Don't download attachments until the user tries to open them (as a pref) > > Please, no! This would be disastrous for those who download e-mail by sometimes > expensive dial-up and read it later off-line. Anyway, I don't suppose POP3 > supports this. Was a suggestion to provide a user pref so you could choose to do this, not enforce it on all users. On reflection I am fairly sure you are correct about POP3's lack of support for this :(
Is it possible to download new messages into separate files in a tmp directory, then merge them into the mailbox? This gives AV software an opportunity to detect problems and delete individual messages and attachments, but then Thunderbird can work with singular mail box files as designed. This also gives Thunderbird a chance to insert a replacement message with a note about what happened where the offending message would have appeared.
yes, pop3 doesn't support not downloading attachments (imap does). But, you can configure mozilla not to download pop3 messages over a certain size (not sure how big the nevsky virus usually is).
(In reply to comment #93) > yes, pop3 doesn't support not downloading attachments (imap does). But, you can > configure mozilla not to download pop3 messages over a certain size (not sure > how big the nevsky virus usually is). Depending on which of the mails I got the sizes were 41.2K & 42.1K So under the moz 50k limit
you can make the moz limit whatever you want - 50K is just the initial default
(In reply to comment #95) > you can make the moz limit whatever you want - 50K is just the initial default Fair point, however the people using mail that are showing the problems would like to be able to accept word documents, pdfs and similar, which are usually as large as the netsky option. Certainly the suggested solution of using maildir or the suggestion from comment 92 would provide some resolution of this problem. I don't think any normal user would see the whole mailbox going due to a vius being recied but not run as good. I agree that the AV vendours SHOULD understnad mbox, however the major ones apppear not to, and as such no only is this issue causing data loss and user problems, it's also generating some very bad PR for mozilla.
I'd just like to point out the I'm using McAfee 4.5.1, which is a fairly old version (kept up to date of course), but it has *NEVER* tried to delete my inbox, and I have had numerous virii sent to me. It only prompted after *I* saved the attachment. So McAfee used to do the right thing...
(In reply to comment #97) > I'd just like to point out the I'm using McAfee 4.5.1, which is a fairly old > version (kept up to date of course), but it has *NEVER* tried to delete my > inbox, and I have had numerous virii sent to me. It only prompted after *I* > saved the attachment. > > So McAfee used to do the right thing... Only if it's really the right thing to allow your inbox to remain infected. I would expect anti-virus software to find a virus even when it is in encoded form in my inbox - and to do something sensible when it finds it. Norton knows not to delete an entire zip archive, or even quarantine it, when it finds a virus in one, so it and McAfee could use a similar strategy for viruses in mailboxes.
> Only if it's really the right thing to allow your inbox > to remain infected. Note that your mailbox is _not_ infected. Is anything executing the virus code when the virus data is sitting in your mail file? No. (The virus can't get executed unless the virus is decoded (from e.g., MIME) into some executable form (e.g., saving as an executable file).) > I would expect anti-virus software to find a virus even when > it is in encoded form in my inbox - Okay, _detecting_ the virus data is reasonable. > and to do something sensible when it finds it. Like what? If you find a virus attachment in an e-mail file while some application (Mozilla) is using the file, what's a sensible thing to do? Unless you know something specific about the application, what can you do? Moving the file out from under the application (e.g., quarantining it by renaming it or otherwise making it appear to be deleted) is NOT a reasonable thing to do. > Norton knows not to delete an entire zip archive, or even > quarantine it, when it finds a virus in one, so it and > McAfee could use a similar strategy for viruses in mailboxes. That's what Norton doesn't do, but what does it do? (What is its strategy for viruses (virus data) in mailboxes? (What can it be other than to leave the mailbox files alone and catch the viruses at a subsequent stage?)) Of course, even though Mozilla can't be responsible for working perfectly if someone comes along and hides its files while it's running, Mozilla should be more robust given the severity of deleting the entire mailbox file's contents. Mozilla's file-compaction logic should probably be a little paranoid, checking things every step along the way (checking that all reads and writes worked, verifying that the compacted-to file has the expected structure and contents before actually deleting the original file, etc.)
> I agree that the AV vendours SHOULD understnad mbox, > however the major ones apppear not to, ... Are you sure? The fact that a virus is detected in Mozilla's mailbox file means that the AV software _does_ understand the format enough to detect the virus (e.g., find the message headers, find the MIME header and boundary, base-64-decode the virus attachment, etc).
RE: comment 100 > The fact that a virus is detected in Mozilla's mailbox > file means that the AV software _does_ understand the > format enough to detect the virus (e.g., find the > message headers, find the MIME header and > boundary, base-64-decode the virus attachment, etc). OK so they are decoding it, so obviously understand the encoding method. However they blatantly don't realise that the code is only a small part of the mbox file. Ideally they shoul.d do what I did by hand: - Find the infected message. - Delete the section of the infected message and replace it with a string such as "Some idiot sent you something nasty so we removed it for you" in it's place . A program called mailscanner does this when it scans incoming messages so it can't be that hard. The problem is that all current and 2005 releases that are out there will trash a mozilla mailbox. Even though it isn't directly mozilla mails fault, it looks bad on mozilla and a work around should be used. In relation to my previous comment about maildir, this could cause problems on file systems with an insufficient number of inodes, so it appears that saving an attachment as a temporary file, letting the AV do it's evil, and if it's gone replacing it with a simple message of: "The attachment '<FILE NAME>' was removed by a third party product (possibly Anti Virus)." Anyway fingers crossed for a solution soon or I may have to seriously consider outlook again :(
Don't suppose anybody can tell me in which files the code that handles the downloading of mail is. I found a few bits but couldn't find anything that appeared to do this. Guess I must be looking in the wrong place :(
(In reply to comment #99) > > Only if it's really the right thing to allow your inbox > > to remain infected. > > Note that your mailbox is _not_ infected. Is anything > executing the virus code when the virus data is sitting > in your mail file? No. Well, this is a matter of definition, but as I understand it, something which contains a viable virus etc (whether medical or software) is considered infected, whether or not the virus etc is active at the time. ... > > Like what? If you find a virus attachment in an e-mail file > while some application (Mozilla) is using the file, what's a > sensible thing to do? Unless you know something specific > about the application, what can you do? > > Moving the file out from under the application (e.g., > quarantining it by renaming it or otherwise making it appear > to be deleted) is NOT a reasonable thing to do. > Agreed. One sensible strategy is described in comment 101. > > > Norton knows not to delete an entire zip archive, or even > > quarantine it, when it finds a virus in one, so it and > > McAfee could use a similar strategy for viruses in mailboxes. > > That's what Norton doesn't do, but what does it do? (What > is its strategy for viruses (virus data) in mailboxes? > (What can it be other than to leave the mailbox files alone > and catch the viruses at a subsequent stage?)) > In fact what Norton does with infected zip archives is only warn the user. It could do the same with mailboxes, as an alternative strategy. This is a lot better than doing nothing and also a lot better than deleting or quarantining the whole mailbox. > > Of course, even though Mozilla can't be responsible for > working perfectly if someone comes along and hides its files > while it's running, Mozilla should be more robust given > the severity of deleting the entire mailbox file's contents. > > Mozilla's file-compaction logic should probably be a little > paranoid, checking things every step along the way (checking > that all reads and writes worked, verifying that the > compacted-to file has the expected structure and contents > before actually deleting the original file, etc.) > Indeed. The initial problem which I saw was not that anti-virus was deleting the mailbox, but that it was hidden from Mozilla and so Mozilla simply overwrote it.
> > > Only if it's really the right thing to allow your inbox > > > to remain infected. > > > > Note that your mailbox is _not_ infected. Is anything > > executing the virus code when the virus data is sitting > > in your mail file? No. > > Well, this is a matter of definition, but as I understand it, something which > contains a viable virus etc (whether medical or software) is considered > infected, whether or not the virus etc is active at the time. > ... I tend to agree that if the code is there, even in a dormant form, it's still a virus. It's likle dormant smallpox is still smallpox and hence still nasty. > Indeed. The initial problem which I saw was not that anti-virus was deleting > the mailbox, but that it was hidden from Mozilla and so Mozilla simply > overwrote it. Yes it does quarantene the file, however on some versions of norton at least it says could not fix, shall I delete the virus infected file? Again a request to anybody who knows in which file the code that handles download of e-mails & parsing of MIME are as I can not find them in the source tree :(
the code that writes to the mailbox file is in mozilla/mailnews/local/src/nsPop3Sink.cpp - in particular, nsPop3Sink::WriteLineToMailbox writes to m_outFileStream. see http://lxr.mozilla.org/mozilla/source/mailnews/local/src/nsPop3Sink.cpp#647 Optionally using a temp file is going to be a non-trivial amount of work, though it's doable...you'll have to deal with message filters moving the message, among other things...
I'd like to point out that this particular problem has exposed a somewhat larger issue: We need a How To or FAQ for installing Mozilla, Thunderbird and Firefox in corporate environments with some tips on dealing with other applications that may need to be tuned to peacefully coexist with something other than Outlook and IE. For example, there is a known method of excluding the Inbox from scanning by Symantec AV or Norton Corporate AV (same product - names recently changed). As was pointed out some months ago, other email clients use similar indox file structures, so it isn't an entirely new issue for the AV vendors (even if they choose not to fix it but offer a workaround). This would be very helpful for IT people attempting to build desktop packages for testing and mass deployment (including documentation of installation procedures), as well as those of us who build 3 or 4 systems at a time once every few months. And it should anticipate questions like: if I build a Ghost image with Mozilla as the default browser and mail client, will I need to change the id number of the default client on each cloned computer to avoid potential identity problems between 2 clients sharing the same id number?
(In reply to comment #104 and comment #105) Thanks, I'll have a look at the code, I'm probably getting in a bit deep as Java is my main language rather than C, however I'll have look over it and see whats what and if I can do anything with it. (In reply to comment #106) > For example, there is a known method of excluding the Inbox from scanning by > Symantec AV or Norton Corporate AV (same product - names recently changed). As > was pointed out some months ago, other email clients use similar indox file > structures, so it isn't an entirely new issue for the AV vendors (even if they > choose not to fix it but offer a workaround). > > This would be very helpful for IT people attempting to build desktop packages > for testing and mass deployment (including documentation of installation > procedures), as well as those of us who build 3 or 4 systems at a time once > every few months. The issue is probably more severe if your building 10+ machines at once, and rolling out few hundred in a matter of days. The FAQ can be useful, however the comment about not scannig certain fails for worms & viruses is not really a safe option. I am sure it wouldn't take much for somebody to write something that was downloaded and run (by the user clicking) and then looking over inbox, MIME Decoding the viral code and then running it, posibly even hidden from the AV. That would be a disaster and is probably a good enough reason to find a permanent solution.
A footnote on the behavior of Symantec Anti Virus 9.0 when you manually exclude the inbox (and enter the incoming port of your email feed if different from the default). This is different from that of it's predecessor - Norton Corporate AV 7.6. Upon opening the message in Mozilla, SAV displays the message: "Symantec Email Proxy deleted the following email message: From: xxx@xxx.com To: yyy@yyy.net Subject: Mail Delivery (failure yyy@yyy.net)" It isn't clear when it actually deletes the virus payload, but I suspect it is upon reciept, as it seems to be inserting itself as a proxy (since it requires the incoming port #) before Mozilla. If so, this may be of some help to those who fear leaving a virus payload intact if it can be verified.
Norton 2004 etc are supposed to install a proxy mail server for pop3 whereby it recieves the message and then scans it and passes it on to the mail program. Have to say I am not sure why it works with netscape, but appears to not be set up with the install of mozilla mail I have. The message you have does sound like it has removed a nasty attachment or mail mind :)
(In reply to comment #105) > the code that writes to the mailbox file is in > mozilla/mailnews/local/src/nsPop3Sink.cpp - in particular, > nsPop3Sink::WriteLineToMailbox writes to m_outFileStream. I've had a look over the code and don't currently trust my C skills to be sufficient to not create a buffer overflow or some odd pointer error while hacking at that. (Sorry, looks like I got too used to Java managing memory etc for me, so I will have to brush my C up). Regarding the comment about message filters moving the message, how about grabbing the message before any of the filters have a chance to do things with it and then popping it back where it came from?
To at least spare users from the worst data loss, would auto-creating a backup of the mail account on startup be an easy enough work around? I realize that would be a lot of data for some people, but still. That way, if users experience this bug the damage is somewhat limited.
This seems to be a fork in the road branding issue - there are workarounds, but they require our users to be sufficiently motivated to seek them. BUT, if Mozilla/Thunderbird is going to compete in the consumer market against IE, it must come out of the box in such a way as to be able to cope with this issue. Otherwise Moz will not retain the perception of quality that people who want something better than IE will expect. I realize this is a really tough issue that is going to take some really serious work by some highly skilled people (I'm certainly not that good) on a volunteer basis. This is one of the weaknesses of the open source paradigm - but it can be a real strength as well if our collective passion to build a better mousetrap is sincere!
(In reply to comment #112) ... > I realize this is a really tough issue that is going to take some really serious > work by some highly skilled people (I'm certainly not that good) on a volunteer > basis. This is one of the weaknesses of the open source paradigm - but it can be > a real strength as well if our collective passion to build a better mousetrap is > sincere! It is certainly a weakness of the open source paradigm, or at least of Mozilla's implementation of it, that a "critical" issue like this one is in fact not being dealt with at all because the person it is assigned to, Seth Spitzer, is "not reading bugmail". No doubt Seth has good reasons, but if his absence is not temporary this bug needs to be reassigned. And since there are those who have a commercial interest in the success of Mozilla, and since this is an issue which threatens that success and is officially listed as "critical" and causing data loss, perhaps it is a candidate for reassignment to one of the few who are not volunteers.
Symantec/Norton Anti Virus corporate (7+) and Intel LanDesk (6+) store thr exclusion directories/extentions in the registry at the following locations: HKEY_LOCAL_MACHINE\SOFTWARE\INTEL\LANDesk\VirusProtect6\CurrentVersion\Storages\Filesystem\RealTimeScan - keyname "ExcludedExtensions" collon separated list of extentions (all caps) eg. "MSG,TST,EXT" HKEY_LOCAL_MACHINE\SOFTWARE\INTEL\LANDesk\VirusProtect6\CurrentVersion\Storages\Filesystem\RealTimeScan\NoScanDir - key name is the full path eg "c:\dir\subdir" and the value is 0 (non-recursive) 1 (recursive) It would not be hard to detect the existence of this key and add the profiles directory upon installation. I know similar registry locations exist for McAfee (I'll look them up tomorrow at work) and other Anti-Virus programs. The least we can do is warn the user on startup that their mailbox needs to be excluded if the exclusion is not detected. Offering to do this automatically and supply a link to the knowledgebase of the AV supplier, or a FAQ page on the Thunderbird homepage. This would also be a great addition to the (upcoming) msi package.
(In reply to comment #109) > Norton 2004 etc are supposed to install a proxy mail server for pop3 whereby it > recieves the message and then scans it and passes it on to the mail program. > Have to say I am not sure why it works with netscape, but appears to not be set > up with the install of mozilla mail I have. I can confirm that NAV 2004 (and various earlier versions) use a proxy scanner that scans all inbound and outbound e-mail (even if it's not going through the Windows registered mail client). However, it doesn't always seem to catch stuff when it comes in. My mail provider scans and cleans before delivery so I don't normally get to see how NAV responds to e-mail worms, but in September a new batch of worm-spawn edged out the filters and made it to my inbox. Strangely, NAV didn't alert me until I had the message open for about 15 seconds (either to read or forward). For what it's worth, here's the excerpt from the NAV log: ========== BEGIN CUT N' PASTE ========== Source: Unknown00000000.data Description: The compressed file Unknown00000000.data within Unknown0000288A.data within E:\winutils\Thunderbird profile\Default User\Mail\pop.sbcglobal.net\After Thunderbird.sbd\Spam.sbd\Contain virus is infected with the Download.Trojan virus. Click for more information about this virus : Download.Trojan ========== END CUT N' PASTE ========== I don't know if it wasn't detected until that point because it was "compressed" or what, but I was frustrated to find that the scan didn't catch it as it entered my system. My $0.02US. -Neil R.
(In reply to comment #115) > I can confirm that NAV 2004 (and various earlier versions) use a proxy scanner > that scans all inbound and outbound e-mail (even if it's not going through the > Windows registered mail client). However, it doesn't always seem to catch stuff when it comes in. Thats quite interesting. I wonder if it only recognises certain versions that possibly co-incide with Netscapre releases. Prehaps my proxy got dumped during an upgrade. Certainly however the AV needs to understand mbox, however Mozilla/firefox should not leave itself open to this. Regarding the option of just excluding the mailbox, I feel that is unacceptable as we still have dormant viral code left on the computer. What are the work arounds that are refered to? Are they proper solutions or just quick and dirty hacks?
I'd like to point out that many users, such as a co-worker and myself, might use Thunderbird in Corporate environments to check their personal mail. I have no access to the corporate settings (well, technically I do, but I'm not about to explain to the CTO that I need to exclude a file on my machine) and therefore are in a situation where they probably need to use Outlook or Eudora to check their mail. I've lost my inbox twice in the last month alone. Just food for thought for those trying to determine if this is worth fixing or not ...
I sufered the same problem, and i am sure this is not AV problem, but Thunderbird's. Until recently I used Mozilla mail and I also use Outlook Express. I had no problem of this kinf with them. I use Kaspersky Antivirus set to report virus only. Kaspersky locks file and reports that it found virus. Nothing more. Thunderbird has problem with locked file and for some reason it truncates it. In other mail readers, when virus occures, I turn of antivirus, delete virus from mailbox, pack mail database and turn Antivirus On. With Thunderbirt it does not work. When file is locked for the first time, and I turn off antivirus, file remains locked for Thunderbird until i exit and start it again. So, now, when Antivirus reports virus, I have to disable Antivirus, exit Thunderbird, start it again, erase virus mail, pack database and then enable antivirus. If I fail doing that it is likely that mailbox will be truncated by Thunderbird. I am experienced user and I can bet this is Thunderbird's bug. It seems it tries to ignore the fact that file is locked which results in unwanted consequences. What it should do: - when it finds mailbox locked it should say so and do not remember that state - it should check if file is locked each time it tries to access it, not to remember the first occurence of lock - if file is locked, Thunderbird should awoid any operation that may truncate file. It should report the problem to user, instead, like saying "Hey, I cannot do that, mailbox is locked, possibly by your antivirus"
*** Bug 270839 has been marked as a duplicate of this bug. ***
Product: MailNews → Core
*** Bug 233223 has been marked as a duplicate of this bug. ***
*** Bug 271578 has been marked as a duplicate of this bug. ***
I second (or third, fourth... :-)) the suggestion of switching to maildir format. Not only does it provide security against the entire mailbox being deleted when a subsequent AV signature update causes a previously undiscovered virus to be detected, it is also significantly faster: the real-time scanners cause a significant CPU load and delay when saving mails to a large inbox or other folder, because they scan the entire file. This problem is magnified when the profile or mail directory is on a network drive. To those who suggest excluding the profile directory from AV scanning: 1. What if I choose to save my mail in a different directory? I can do this on Account Settings / Server Settings / Local directory. 2. What if I add another profile? Do I have to manually add the exclusion to the AV program? 3. (Perhaps a little contrived, but given that we're aiming for millions of users, possible:) What if a user manages to save an attachment with a not yet detected virus to their profile folder, and later (after an AV update would detect it) open the file? They then have a virus on their system which would otherwise have been caught. I am not at all familiar with the Thunderbird codebase, but surely it cannot be an immense problem to switch the mail storage system. There must be some storage subsystem which tells the front end which messages are in a folder, and delivers the message content, because IMAP works completely differently to POP3. Looking at an existing IMAP account, the .msf files exist, but no mbox files until the folder is marked for offline use. There's probably enough mail program code with a BSD (or Apache?) license that could be used as a starting point for maildir support.
my $0.02: this is a data loss bug, there should be no known data loss bugs in any shipping software. Whatever it takes to fix it, workaround it, whatever, needs to be addressed ASAP. I would also think maildir format would be better for this bug as well as I would assume there would be performance/memory advantages of being able to access individual emails not indexing into a large (many, many MB in my case) file.
First let me say that I applaud the work so far of the Thunderbird developers. You have the makings of a potentially great mail client here. With the release of 1.0 I was getting ready to recommend it to people I know as well as making it the standard mail client in the school district in which I work as head of IT. Then I found this bug, and I've had to stop dead in my tracks. It is almost unbelievable to me that a critical dataloss bug three years old has not been fixed. I've been looking at the posts here, and I have seen lots of excuses. "It's the AV companies' fault." You're not going to change them all, if any, especially with TB's small market share. "It's ignorant users' fault." The world is full of ignorant end users, or corporates who can't control their antiviruses. Do you want to write off all of these people? There have been some fixes suggested here. Changing the mailbox format? I don't know if it will work. How about changing the mailbox file extension? My NAV, for instance, doesn't scan .mbx (OE) mailboxes by default. How about changing to that? The problem doesn't have to be solved tomorrow, and it may not be easy. However, it will never be solved by talking it to death. C'mon, coders, let's please get an solution to this problem so that I and others can use Thunderbird and recommend it as what it should be - a world-class e-mail client.
Sorry, but in my last comment I meant that by default NAV does not scan .dbx files (not .mbx) and that using .dbx on the mailbox files might be considered.
Changeing the mailbox format by stripping out attachments is probably not a good idea, it potentially breaks other software. E.g. what would happen to imap folders if I read the mail from another imap client. Then the attachment would already be stripped out and residing on some other client unavailable to me. If this can be fixed without any such side effects fix it, if not, mark it as wontfix. By the way don't forget to complain to your AV software vender if you are affected by this.
(In reply to comment #126) > Changeing the mailbox format by stripping out attachments is probably not a good > idea, it potentially breaks other software. > ... > If this can be fixed without any such side effects fix it, if not, mark it as > wontfix. ... I agree that changing the mailbox format is not a good idea, at least certainly not as the best quick way to fix this urgent bug. But fortunately there are ways to fix it, or at least greatly reduce its severity, without changing the format. One of those is the one I mentioned in comment 79: to save each attachment to an incoming message as a temporary file, and then reopen it and save the reopened version in the mailbox. This should force the virus scanner to scan the attachment before it ever reaches the mailbox file. If this has a serious performance hit, it can be made optional. I know there can still be problems if a virus gets into the mailbox before the anti-virus data is updated to catch it. But this should catch the great majority of incoming viruses.
(In reply to comment #126) > E.g. what would happen to imap folders if I read the mail from another imap > client. Then the attachment would already be stripped out and residing on some > other client unavailable to me. What other mail client would be accessing Thunderbird's LOCAL cache of IMAP folders? Other IMAP client would acces the server's version which would still have the attachments on each message. We're talking about the local cache of the IMAP folders and they are in the TB profile folder so no other program should be touching them.
A lot has been said in this bug and people are beginngin to repeat what others have said. There are a couple of solutions that will help solve this problem, one is better than the other and at present there is no solution that has been decided on. To recap: Add a extension to the mbox files (be it .bdx, .mbx, .mbox, .tbm wtc) pro's - The extension can easily be added to a list of excluded filetypes in all antivirusapplications con's - The virus would remain in the file unnoticed until it is finally extracted by a user to a location on the harddrive/floppydrive. - Large coorporations are probably not willing to exclude extensions for different kinds of files because of the above. Switch to maildir format pro's - each file itself would be scanned upon access and cleaned or removed. This prevents the whole mailfile from being deleted. - Compacting of folders will be a thing of the past (unless an index of files is kept for other performance enhancements. con's - Lots of files on a harddrive cause defragmentation and thereby performance degradation of the whole system. On windows systems the swapfile is configured to reside on the same disk as the maildir would be lowering performance even more. - Complete change in format breaks compatibility with older versions and would require a lot of extraction/conversion upon upgrades. Extact binaries to a different folder altogether pro's - Infected attachments would reside in a different folder, virusscanners would not delete the mailfolder if a virus is detected. - Changes to the mailformat would be small and can probably be worked around by inserting file:// references to the attachments so that older clients still know where to find the attachments. con's - Many plain text pieces in messages are detected as virusses (scripting that manipulates outlook into doing bad things for example). These would still reside in the mailbox file. Detection of these would still cause a virusscanner to delete the whole maildir file. - Naming issues or the attached files (what to do with attachments that contain files with the same name), this could be solved by using a foldername that is unique for each message, potentionally causing a lot of folders to be created. - Small breakage of the current format - Still need to extract all attachments upon upgrade. Extracting an attachment (or save the whole message), see what the virusscanner does and re-include the attachment into the mbox file pro's - Known virusses cannot end up in the mbox file. - Known scripted messages could also be deleted by the virusscanner before infection of the mbox file occurs. - No changes to the file format causing no breakage with older versions. delete con's - Virusses that are not yet known (due to older definition files) will still be included into the mbox file causing the mbox file to be deleted upon next access. Nag the viruscompanies to fix their scanners till they've done so pro's - No changes to the file format con's - Needs coorporation of all anti-virus companies (which they've been very reluctant to give) - Does not solve the problem for people stuck on older versions of the virusscanner software Encrypt the mbox format pro's - encrypted format is probably undetected by the antivirus software con's - Virus is still dormant in the mbox file - Needs conversion upon upgrade - Breaks compatibility with the older versions Integrate with anti-virus software using MS-office integration pro's - Uses a way to scan the mbox file using a documented? format - directly integrates with current versions of anti-virusscanners con's - Currently unknown how to do this integration - Currently unknown if this will actually work. Split the mbox into partitioned files of x Mb each pro's - Format is relatively unchanged - Only part of mail history is lost - Defragmentation of prevented due to larger filesize con's - Requires conversion upon upgrade - breaks compatibility with older versions - Dataloss still occurs To conclude: Safest method is to save messages in seperate files, but this will cause performance loss due to defragmentation. Easiest to implement is add an extension to the mbox files and adding those to the exclusionlist (which is unlikely to happen in corporate environment). Also easy is to implement is to encrypt the mbox files with a simple encryption/decryption stream. Whichever method is chosen, none of them are perfect and none of them will completely solve all problems. It is however very important that a solution is chosen that at least minimizes dataloss.
(In reply to comment #129) ... > > Switch to maildir format ... > con's > - Lots of files on a harddrive cause defragmentation and thereby > performance degradation of the whole system. On windows systems > the swapfile is configured to reside on the same disk as the maildir > would be lowering performance even more. ... > > Safest method is to save messages in seperate files, but this will cause > performance loss due to defragmentation. Another con here, if you really mean saving each message (and not just each attachment) as a separate file, is that it can be very wasteful of disk space in some filesystems. A high proportion of the many thousands of messages in my folders are 1 KB or 2 KB. But the minimum file allocation is 16 KB I think. The result could be many thousands of wasted blocks of 14-15 KB. Not everyone has gigabytes to spare on their disks. ... > > Whichever method is chosen, none of them are perfect and none of them will > completely solve all problems. It is however very important that a solution is > chosen that at least minimizes dataloss. Agreed. But who is in a position to choose and implement a solution? It seems that Seth Spitzer has taken responsibility for this bug and gone away (or been sent). Is there any way the assignment can be reversed - if Seth is not coming back? Is the situation being officially monitored e.g. by our QA contact Esther?
Believe me, this is high on my priority list. My plan is to do the temp file thing first, since that's fairly easy. It will probably be optional - I'll have to see. I'm also thinking of doing the single file per message thing, though that will definitely be optional, and is a lot more work. The rest of the approaches (storing attachments separately, etc) are not likely at this point given our mailbox format. Though I'd also like to add support for removing attachments for other reasons, while still keeping our mailbox format.
Assignee: sspitzer → bienvenu
(In reply to comment #129) > There are a couple of solutions that will help solve this problem, one is > better than the other and at present there is no solution that has been > decided on. Bearing in mind comments #123 and #124 (i.e. the release change from 0.x to 1.0 and the effects this will have on expectations and production environments), wouldn't it be pragmatic step forward if there was a new warning screen in the install wizard, to tell the user to configure the virus scanner to quarantine files instead of deleting them? It might not help every layman but at the very least this would alert system administrators to the risks involved giving them the chance to postpone (#124) or configure the virus scanners properly. Bearing in mind the consequences of this issue (meltdown) and the length of time that developers have been aware of the potential disaster - anything short of such a warning would be cavalier. Alan
*** Bug 273969 has been marked as a duplicate of this bug. ***
(In reply to comment #132) ... > wouldn't it be pragmatic step forward if there was a new warning screen in the > install wizard, to tell the user to configure the virus scanner to quarantine > files instead of deleting them? Unfortunately this doesn't work round the problem properly. The problem is that a quarantined file is unreadable. And at least some parts of Mozilla, including the routine to compact a folder and maybe also the one to append a new incoming message to it, seem to assume that an unreadable file is empty and so overwrite it. This suggests another relatively quick fix, or partial fix: if a folder file is found to be unreadable (but the file does exist), do something sensible like flagging an error, but do not attempt to write anything to that file. See comment 82.
> configure the virus scanner to quarantine files instead of deleting... > Unfortunately this doesn't work round the problem properly. Not wanting to distract from your quick-fix I'd just like to point out that configuring the virus-checker to quarantine files is the suggestion from the issues FAQ and it's disappointing to hear that this doesn't solve the problem after all. How about a instal warning saying that the following anti-virus programs CAN be used with thunderbird ... and that others may lead to lost inboxes?
Summary: Deleted inbox after receiving virus infected mail (McAfee, Norton) → Deleted inbox after receiving virus infected mail (McAfee, Norton, AntiVir)
(In reply to comment #129) > Switch to maildir format > pro's > - each file itself would be scanned upon access and cleaned or removed. > This prevents the whole mailfile from being deleted. > - Compacting of folders will be a thing of the past (unless an index of > files is kept for other performance enhancements. > con's > - Lots of files on a harddrive cause defragmentation and thereby > performance degradation of the whole system. On windows systems > the swapfile is configured to reside on the same disk as the maildir > would be lowering performance even more. What if the maildir was zipped? Most virus scanners can scan inside a zip file -- can they clean/delete a file within a zip file? Just put compression on low for the zip (for better performance) and there could even be a user option for the compression level so they can trade-off performance<--->disk-space. > - Complete change in format breaks compatibility with older versions and > would require a lot of extraction/conversion upon upgrades. Why do we need to maintain backwards compatibility with older versions? It's not like a document editor where we have older clients opening files distributed by newer clients. We're (discussing) making a one-time conversion to a new format. I don't think it's too much to require an initial conversion on an upgrade either.
>> - Complete change in format breaks compatibility with older versions and >> would require a lot of extraction/conversion upon upgrades. > > Why do we need to maintain backwards compatibility with older versions? It's > not like a document editor where we have older clients opening files distributed > by newer clients. We're (discussing) making a one-time conversion to a new > format. I don't think it's too much to require an initial conversion on an > upgrade either. Thunderbird isn't the oinly client using the mbox format. A host of unix based emailclients use this. I know a lot of people who use Thunderbird under X-Windows and a text/commandline client when connected through SSH. So it is not only a backwards compatibility issue. You must also keep in mind that if a newer version isn't as much liked as an older version, or if the newer build contains a nasty bug, people will want to be able to downgrade until a workaround/fix/enhancement is made. Completely changing the format makes this impossible.
(In reply to comment #137) > >> - Complete change in format breaks compatibility with older versions and > >> would require a lot of extraction/conversion upon upgrades. > > > > Why do we need to maintain backwards compatibility with older versions? It's > > not like a document editor where we have older clients opening files distributed > > by newer clients. We're (discussing) making a one-time conversion to a new > > format. I don't think it's too much to require an initial conversion on an > > upgrade either. > > Thunderbird isn't the oinly client using the mbox format. A host of unix based > emailclients use this. I know a lot of people who use Thunderbird under > X-Windows and a text/commandline client when connected through SSH. So it is not good point, I know that many others use mbox but I hadn't considered that other clients might share mbpx folders with TB. It does warrent consideration, however, how many other mainstream email programs take that case (other email programs sharing thier data) into consideration? By mainstream I'm mainly referring to Eudora, Outlook, whatever AOL uses. Maybe Evolution too, not sure what it does. TB (well the community) has to decide whether they are attempting to have TB become widespread (like Firefox) or remain an email program mainly for experienced users. If we are targeting the general computer users, then we can't be concerned with storing our email in a non-standard format and can't worry about if some other program wants to share our data without going through our APIs. I'm not saying we shouldn't TY to play nice like that, however I think it's more important that the bugs get fixed (especially data loss bugs) then it is to worry about other programs directly accessing and using our data without going through TB. Note: obviously we document the storage format completly so anyone can read the data if they want to add code for that > only a backwards compatibility issue. You must also keep in mind that if a newer > version isn't as much liked as an older version, or if the newer build contains > a nasty bug, people will want to be able to downgrade until a > workaround/fix/enhancement is made. Completely changing the format makes this > impossible. Well, what if TB introduces a bug that corrupts my data or gets all my data wiped by, oh, I dunno, a virus scanner. I can't downgrade to fix that. I'm not trying to be annoying, I'm serious. Right now, there's a data loss bug in the software. I'll be the first person to step up and say 'assume you won't be perfect' and would normally be all over that agreeing with you. However, in this case, the tradeoff is fix a bug where you could permanently lose all your data but people may have to deal with not being able to downgrade, or take the chance of complete data loss. We have a large pool of alpha/beta testers and recent experience has shown that the testing process works, and builds released for public consumption (0.8, 0.9, etc.) have not had the major bugs in them that force people to downgrade -- that's why we test before releasing a milestone -- and fixes to major bugs have landed quickly. So, I would say, yes, I agree that that tradoff (not supporting downgrades) is there, but I think the positives of fixing that bug outweight the negatives of disrupting downgrades.
*** Bug 250401 has been marked as a duplicate of this bug. ***
(In reply to comment #130) > (In reply to comment #129) While using Mdir could be a good idea, a person would have to configure file systems such as ext2/3 to cope with a very large number of nodes. If they don't do this they waste a very large amount of disk space and if they do do this then the filesystem will be very slow. As suggested mbox with save to temp files, then merge scanned file into mbox seems like a safe idea for now. In reply to people saying that this would allow undiscovered code/scripts in and then loose data afterwards consider that: - Not scanning will let said code trash your system. - Any form of scanning could trash your system. - Realistically, if the code got past your AV you probably have more things to worry about.
*** Bug 275027 has been marked as a duplicate of this bug. ***
Ok.. Had it another 2 times... Really frustrating and a good waste of my time reconstructing mail boxes. I am affraid that mozilla/thunderbird is just not worth it at present. Will be changing peoples mail clients, and will revisit the options when this is fixed.
*** Bug 275372 has been marked as a duplicate of this bug. ***
(In reply to comment #142) I was up to the point to leave TB as mail client due to this bug, when I installed the new Symantec Norton Antivirus version 9 yesterday. It installs a mailproxy that intercepts the mailstream and deletes viruses in attachments on the fly. The result is a message like: ****** quote cleaned mail ******* Symantec Email Proxy deleted the following email message: From: re-mail_system@hotmail.com To: Mail-User@planet.nl Subject: illegal signs in your mail . ****** end quote ****** This suggests that you can still use TB or Mozilla Mail as your preferred mail client, as long as you keep your antivirus program up to date :-) Harold
*** Bug 275649 has been marked as a duplicate of this bug. ***
*** Bug 276186 has been marked as a duplicate of this bug. ***
This patch adds support for a (currently) hidden pref "mailnews.downloadToTempFile" which makes pop3 download each message to a file before appending to inbox - if a virus checker deletes the temp file before we can append it to the inbox, we ask the user if they want to skip this message. If they skip it, we won't try to download it again (and if leave on server isn't set, we will delete the message from the server). We may want to add a UI for this in advanced settings...but I don't think we want to default it to true.
Attachment #170512 - Flags: superreview?(mscott)
Blocks: 241894
David - importing mail also triggers this - see bug 241894.
*** Bug 247223 has been marked as a duplicate of this bug. ***
*** Bug 235232 has been marked as a duplicate of this bug. ***
Same thing: t-bird 0.9 incoming mail infected with Netsky_32 Norton report, complete inbox deleted without a trace....
*** Bug 278580 has been marked as a duplicate of this bug. ***
I have also installed Symantec AntiVirus 9 and without even realising it at first have since noticed it intercepts all my mail as it comes in. I didn't have to configure anything. However, even though this is the case, with the "mailnews.downloadToTempFile" idea, if an email passes through the scanner at first but is then detected a week later due to newer definition files, we are back in the same spot where we'll lose our Inbox. Likewise with Symantec AntiVirus 9... If mail passes through at first and is then detected as infected later, we too will lose our Inbox. Regardless of the mail scanner we use, we are in a bad situation if we keep all our mail in the one file. To address this situation once and for all we need to change the way mail is handled to be either individual files, or encrypted so as to avoid initial detection. Then when the mail is re-opened, you could copy the email to a temp file, let the automatic virus scanner have it's way and amend the database accordingly if it gets removed.
Encryption makes the mailboxes much less portable. The ultimate solution to this problem, I think, is maildir. However, maildir has it's own problems one of them being ease of implementation. I think the most reasonable course of action until maildir can be implemented would be the file extension idea. Granted, there will be messages in the box that have viruses, however they will be detected if the attachments are saved or launched. It probably wouldn't be too difficult to have an option that allows the user the specify the extension, with the default being the same as Outlook's mailbox extension. (In reply to comment #153)
Encryption makes the mailboxes much less portable. The ultimate solution to this problem, I think, is maildir. However, maildir has it's own problems one of them being ease of implementation. I think the most reasonable course of action until maildir can be implemented would be the file extension idea. Granted, there will be messages in the box that have viruses, however they will be detected if the attachments are saved or launched. It probably wouldn't be too difficult to have an option that allows the user the specify the extension, with the default being the same as Outlook's mailbox extension. (In reply to comment #153)
I think the most reasonable course of action is to separate the attachments from the messages so that if a virus is sent, only the attachment is deleted and not the message. This has other advantages too, like, I can delete large attachments but save the message it came with. Eudora does this and has been very successful. Just put a marker with a reference to the attachment in the message that is stored in the mbox file. This should be a portable, backwards compatible solution if the reference is just a small text line/file inserted where the attachment would be in the mbox file. Other mail readers would just see a text attachment and Moz would interpret that attachment as a refrerence to the actual file somewhere else on disk. Moz can support both systems (actual file and reference) simultaniously.
(In reply to comment #156) > I think the most reasonable course of action is to separate the attachments from > the messages so that if a virus is sent, only the attachment is deleted and not > the message. ... Although this would reduce the frequency of the bug, it doesn't actually fix it. For a virus may be included in the main text portion of the message. I'm not sure if it could be active in such a case (it would be if encoded with something like UUENCODE), but it would still be detected by anti-virus scanners and as a result the inbox or folder would be quarantined - although not usually deleted completely.
Actually I hate that feature of Eudora. In my experience, the attachments always end up disassociated with the messages and either the user deletes the message and the attachment stays...or the user clicks on the attachment and cannot pull it up...or the program forgets which messages have attachments because the TOC gets corrupted. This feature also makes it very difficult to move away from Eudora if you decide you do not like it. Also, as someone else indicated, what if the message itself contains something the virus checker thinks is a problem? An example of this is a paypal phishing scam. (In reply to comment #156)
Also see 260539 - a warning when this happens, so the user can retrieve the inbox from quarantine.
(In reply to comment #159) > Also see 260539 - a warning when this happens, so the user can retrieve the > inbox from quarantine. I managed to sucessfully retrieve tha mailbox from quarenteen many times, but if the user has large mailboxes it can become a pain in the a**e. I'm going to apply the patch to the latest mozilla and see where I get from there. With regard to maildir, as I have pointed out before, this causes problems when the file system hasn't been created for this, eg Maildir will use lots of inodes to store a medium size mailbox. This could result in the FS running out of inodes. If enought inodes are created then the FS will become slow for normal operations. I think the current save to temp file first is a good option.
*** Bug 280052 has been marked as a duplicate of this bug. ***
Comment on attachment 170512 [details] [diff] [review] add (currently) hidden pref "mailnews.downloadToTempFile" which makes pop3 download each message to a file before appending to inbox How about simplifiying the wording for the text. I don't think end users need to be informed about the temporary file. %brandname% encountered an error trying to download the following message: \n From: \n%S Subject: %S\n This message may contain a virus or there is not enough disk space. Skip this message? (Note the spaces before the From so we can indent the message info a little bit). When getting the prompt service in HandleTempDownloadFailed, shouldn't we just get the nsIPromptService, passing in the dom window from nsIMsgWindow. Or are we supposed to go through the docshell? I'm not sure.
Attachment #170512 - Flags: superreview?(mscott) → superreview+
I'll change the message...re the doc shell window, your way does seem the "right" way, passing in the true parent, etc, though I think it's more code :-) I'll switch over to it anyway...the code I copied was probably older code.
patch checked in - the hidden pref is "mailnews.downloadToTempFile". If you set it to true in prefs.js or user.js, pop3 mail will get downloaded to a temp file first. If anyone wants to try it out with a virus checker and virus e-mails, and let me know how it goes, that would be great. If it works OK, adding a UI for this would be the next step (probably under advanced?)
Flags: blocking-aviary1.1?
I tried the new pref and it causes some problems when trying to download more than one mail at once. For each mail following the first one I receive the error message as decribed in bug 166111.
Sven, what platform are you running on?
I'll look into the problems with this pref set tomorrow.
(In reply to comment #167) > I'll look into the problems with this pref set tomorrow. Well, the problem is, the temp file is deleted in IncorporateComplete() after handling the first message. But we only go through BeginMailDelivery() (where the file is created) for each retrieve session, not for each message. So when trying to write the first line of the second message (WriteLineToMailbox() from IncorporateBegin()), we fail because the temp file doesn't exist anymore.
(In reply to comment #168) ... > Well, the problem is, the temp file is deleted in IncorporateComplete() after > handling the first message. But we only go through BeginMailDelivery() (where > the file is created) for each retrieve session, not for each message. ... We really need separate temporary files for each message, as we don't want a whole batch to be deleted or quarantined because one has a virus. So the problem is not with the deletion, but that a new file is not created for each message.
thx, Christian. Yes, of course the idea is that we'd have a separate temp file for every message. I just need to make sure it gets created for every message.
truncate temp file instead of deleting it after incorporate complete. We'll still delete the temp file when done with mail delivery.
Attachment #173650 - Flags: superreview?(mscott)
Attachment #173650 - Flags: superreview?(mscott) → superreview+
There is another case this patch will not fix. Take a user whose virus definitions auto update and does an automatic system scan weekly. If the user gets an infected email before the new virus definition, his inbox still gets nuked at the end of the week. The same happens to users who only scan weekly instead of every incoming email, either because of misconfigured software or expired subscription. This is the users fault but still makes Mozilla look like the bad guy.
(In reply to comment #172) I think this issue can not be fixed 'cause of the internal design - all messages are stored in UNIX mailbox format, which is a single file per folder.
(In reply to comment #172) > There is another case this patch will not fix. Take a user whose virus > definitions auto update and does an automatic system scan weekly. If the user > gets an infected email before the new virus definition, his inbox still gets > nuked at the end of the week. > > The same happens to users who only scan weekly instead of every incoming email, > either because of misconfigured software or expired subscription. This is the > users fault but still makes Mozilla look like the bad guy. This scenario seems less likely to look like Mozilla/Tbird is the guilty party, since the deletion happens as a direct result of the virus scan. The data loss I hit due to this bug was worst when a wave of several back-to-back virus arrivals caused a month of inbox data to get shuffled to Norton's quarantine area in chunks: the inbox was quarantined, new inbox was auto-created, mail arrived, inbox was quarantined, new inbox was created, mail arrived, etc. If the scan is a once-per-X (week, day, month) incident, it is unlikely to have a 2nd incoming virus lead to inbox files getting repeatedly quarantined or even overwritten (depending on how recklessness the antivirus software is). Hopefully, a user would notice the disappearing inbox before the next full-system scan. If, on the other hand, they are scanning all incoming email, but are just getting updates on a once-per-week basis, they'll get the scanner update, lose their virus-laden inbox, then they'll start losing each infected temp file. Last, anyone relying on weekly virus updates is getting close to unsupportable for being too far from following best practices. While fixing that issue might be possible, it's a much more pressing issue to create support/KB documents that talk users through reconfiguring A/V software or using a different A/V package. But hey, that's just my opinion-- I'm probably wrong. Oh, and 1e6 *KUDOS* for the design/development of the tempfile workaround!!!
*** Bug 282898 has been marked as a duplicate of this bug. ***
Quick question. Does the final patch save a per e-mail temporary file or a per retrieve temporary file. If the first is true then that is all good, if the second is the case then surely the whole retrieval will be lost if any virus is found in any of the downloading mails. Also I agree with the comment that the person who scans once a week is outside of supportability. If you scan once a week, then you know that you stand a chance (a very high one these days) of getting infected in that week and having to do some cleaning up. If a person is doing this they are probably aware of the risks and also probably know what they are doing with the AV as they will regularly encounter viral files. Thanks for the fix mind, looks like I will be revisiting mozilla/thunderbird as a mail client again :)
Sorry brain dead moment. Regarding the pref, could it be made to default to true, or provide a UI dialog at some realistic point in the future. I just feel this feature would be useful to the homoe users. Thanks
we create a temp file per message. Or to put it another way, there's never more than one message in the temp file. Yes, a UI for this would be useful. I was hoping to get some testing with the hidden pref first...
(In reply to comment #178) > we create a temp file per message. Or to put it another way, there's never more > than one message in the temp file. Yes, a UI for this would be useful. I was > hoping to get some testing with the hidden pref first... No problem I'll test it on some of the systems I have access to and give feed back after a week of running or so.
Well it's now running on a system I have with AV and there were no problems getting mail. Havn't had anything for the AV to shout and scream about so far, but seems good enough. Mark
Geez! Some debate going here! I'm afraid it's too much tech talk for me, but perhaps you're able to help us? I still haven't deleted my email bow, however Kaspersky AVP has detected a nest of worms in : C:\WINDOWS\Profiles\max\Application Data\Thunderbird\Profiles\default.pac\Mail\mail.esmedia.cz\Inbox/[From "Rita I've deleted the independent emails from my email, however they still show in the virus scan, and it seem like they are spreading. I'm not happy about deleting the intire inbox of a few 1000 mails, just because of a few infected mails...besides; they keep coming every week, so it's an ongoing problem for all of us?! I've also mailed Kaspersky, as I'm not sure whether it's the anti-virus programmers who should improve, or whether it's a specific [silly] Mozilla issue... Any ideas? The worms are: Email-Worm.Win32.NetSky.x, Email-Worm.Win32.NetSky.af, Email-Worm.Win32.Bagle.at, Email-Worm.Win32.Mydoom.m.... Thank you for your interest! Max
Max, you need to compact the folder in question to actually remove the deleted e-mails, right click on the folder and pick "compact this folder"
(In reply to comment #178) > we create a temp file per message. Or to put it another way, there's never more > than one message in the temp file. Yes, a UI for this would be useful. I was > hoping to get some testing with the hidden pref first... Ok had a week of testing and nothing seems to have fallen over. Mails have been fetched ok and I purposely sent a copy of on of the various netsky worms to one of the machines without the inbox being deleted. So far looks good from where I am. Unfortuanately the Test PC is going to be rebuilt soon, so that's about all the testing I can do for now
I believe this is normal and expected function. When NAV finds a virus (etc), it can only work at the file level. Emails are stored in a single, large file, not individually. So, NAV can't simply remove the one affected email. It has to remove the entire file which in this case is the entire, full, mail folder (one example, the INBOX file).
I thought this one was working fine, but now today my inbox is gone! PLEASE FIX ASAP!!!!!!!
I thought this one was working fine, but now today my inbox is gone! PLEASE FIX ASAP!!!!!!! Thunderbird 20050309
> I thought this one was working fine, but now today my inbox is > gone! Did your anti-virus software simply delete or quarantine your Inbox file, or did it confuse Mozilla and cause Mozilla to wipe out some data (including preventing restoring a quarantined file)? (Note that if it's that your anti-virus software is configured to simply delete files it thinks are infected, there's not much Mozilla can do about it (besides going to an unrecognized and less-standard file format). Besides, that's a pretty bad way to configure anti-virus software (deleting instead of quarantining or whatever)--complain to your system administrator, software vendor, or yourself. If the case is that your anti-virus software is configured to quarantine files it thinks are infected, then clean the file if possible and unquarantine it. If the anti-virus software can't clean it, then try to configure your anti-virus software to exclude the Inbox file from checking. (You'll also need to exclude the Trash folder's file and probably the Junk folder's file too.) If you can't change the anti-virus software's configuration so that you can use the original application (Mozilla) to remove the virus (delete the mail message carrying the virus), that's an irresponsible restriction too--complain to your system administrator (or software vendor). However, if the anti-virus software quarantines the file, causes Mozilla to get confused, and Mozilla does something that prevents the anti-virus software from restoring the file (e.g., Mozilla creates a corrupted partial copy of the Inbox file), then that is clearly something that Mozilla could handle better. (I wouldn't argue that it's a full Mozilla bug (Mozilla's fault), since no software (e.g., the anti-virus software) should ever be f***ing around with and deleting private files out from under a running application in the first place. (Few applications accept the burden of running without problems if their files are asynchronously deleted.) However, since Mozilla does seem to get confused and possibly prevent restoring of a quarantined file, I would call it a robustness bug.) Daniel
(In reply to comment #187) > ... > > However, if the anti-virus software quarantines the file, > causes Mozilla to get confused, and Mozilla does something that > prevents the anti-virus software from restoring the file..., > then that is clearly something that Mozilla could handle better. > > (... I would call it a robustness bug.) > I wonder if Sasquatch's problem is related to what I wrote in comment 82 to this bug. This problem is that in at least one case, compaction, when Mozilla cannot read an inbox because it has been locked or quarantined by anti-virus software, Mozilla deletes that inbox. I don't think this part of this bug has been fixed, or addressed at all. And, while arguably it is not Mozilla's problem, it is indeed a robustness issue, and also one of public relations: if Sasquatch's problem gets wide publicity, Mozilla's reputation could be seriously damaged. The fix should be a fairly simple one, but maybe at some deep level in the code: if a file which Mozilla expects to find exists but is unreadable, it should never be deleted or overwritten, but probably the activity should be aborted with an error message.
(In reply to comment #186) > I thought this one was working fine, but now today my inbox is gone! > > PLEASE FIX ASAP!!!!!!! > > Thunderbird 20050309 Is this patch included in any builds of Thunderbird, as the bug appears to be marked as NEW and not confirmed or resolved.
I think part of the problem is that the NAV and Thunderbird are both working at the same time. The mail comes in, one with a virus gets picked up, then the inbox gets deleted, then another bunch come in (making a new inbox!), then another virus gets the inbox deleted again, etc. Not good.
(In reply to comment #190) > I think part of the problem is that the NAV and Thunderbird are both working at > the same time. The mail comes in, one with a virus gets picked up, then the > inbox gets deleted, then another bunch come in (making a new inbox!), then > another virus gets the inbox deleted again, etc. > > Not good. Ther eis a patch attached which saves the mail to a temporary file which then allows the AV to do it's evil and not delete the entire inbox. From the symptoms your getting I would say that whatever build of T-Bird your using doesn't have this patch applied. I don't know if it has actually made it into nightly builds yet, so you may need to check out of cvs or patch the source & build it yourself. I know this isn't much help, but it's all I can think of at the moment.
The patch doesn't fix what was already described in comment #52 (virus already in your inbox and (days) later the virus scanner finds it). The only way to fix this would be to change the mailbox storage format to maildir instead of mbox (quite a major change but not impossible). This is best discussed by the drivers or those with authority on the future of thunderbird/mozilla. - IMHO I'd rather go maildir to fix this and other issues. I think this horse was whipped some time ago but I'm not sure where I read it (as to mbox vs maildir). Would someone on the thunderbird/mozilla mail team like to comment and put us all in our place? ;-)
I believe the better option is to provide alternative storage methods, than replacing mbox with something else. Pegasus supports two alternative storage methods - its default system or mbox. Note further that there are four different incompatible mbox formats http://en.wikipedia.org/wiki/Mbox , so mbox is not only one standard supported by all linux formats... My recommendation would be to make mailbox formats plugin based, giving room for all types of formats. Thunderbird to have at least two alternative box formats. Note that there are not only one common mbox format, but four different incompatible mbox formats! I think the better option of this reason would be to make storage formats plugin based adding two plugins by default - one for the current mbox format and one that would store each message in individual files - for example as a file-based xml database. This way we add flexibility for adding new formats with unique capabilites - some maybe storing the mail on a server and only maintaining a local index for quick searches, some encrypted mailboxes, some compressed formats, and whatever other mailbox format people are willing to make a plugin for.
*** Bug 256043 has been marked as a duplicate of this bug. ***
Whiteboard: [asaP1]
sounds like the work for this fix is completed. blocking1.1- renominate if this is incorrect.
The patch does work for me, and I have no reason to believe that moz/TB can be expected to do anythign about a virus that has come in and got through the AV and is later deleted by the AV. This fix prevents Moz from looking bad, by having it trash the inbox when a virus comes in. I addition it also focuses the problem back onto the AV vendors who should not be carrying out whole file deletes/quarantine for a commonly used file format. Anyway fingers crossed this work around should protect anyone with an up to date AV where a mail happens to get past the now common mail proxy systems.
This is still not marked as fixed. Is it or isn't it fixed? Thanks.
u can tell this has been resolved by going to privacy>anti-virus. now, av can delete individual messages instead of the whole mailbox. this should be marked fixed.
OK, marking fixed. We've already got a bug open for supporting maildir.
Status: NEW → RESOLVED
Closed: 20 years ago
Resolution: --- → FIXED
(In reply to comment #198) > u can tell this has been resolved by going to privacy>anti-virus. now, av can > delete individual messages instead of the whole mailbox. this should be marked > fixed. I turned on on the option to quarantine individual messages, and now when I am prompted to skip a message that might have a virus in it, I sometimes get this error: Unable to write the email to the mailbox. Make sure the file system allows you write privileges, and you have enough disk space to copy the mailbox. It doesn't happen all the time, but when it happens, it stops downloading all other messages as well. I have tons of disk space available, and I do have write privs to that folder.
Using Thunderbird 20050413. Just got bit by this again, and this time I had "Allow anti-virus clients to quarantine individual incoming messages" turned on. Virus came in, my entire Inbox got quarantined. I don't think this is really fixed.
(In reply to comment #201) > Using Thunderbird 20050413. Just got bit by this again, and this time I had > "Allow anti-virus clients to quarantine individual incoming messages" turned on. > > Virus came in, my entire Inbox got quarantined. > > I don't think this is really fixed. same thing for me with the last build and "Allow anti-virus clients to quarantine individual incoming messages" turned on too. huge problem for professional use.
Flags: blocking-aviary1.1?
*** Bug 296043 has been marked as a duplicate of this bug. ***
Is it worth reopening this bug to avoid a flood of duplicates. It does appear that there is still an issue with the original fix.
I'm sorry, but is this one fixed in 1.0.6 (last official "release" version, right?)? I just had someone who had this error in this version today. I've been using trunk for a while, but my PC is in for repairs right now. I can't seem to find the setting anywhere in 1.0.6. Is this trunk only, or am I missing something? Thanks.
It happened to me for the 3d time now, PLEASE fix it properly (its over 1 year I reported it) Stopped using Thunderbird for the moment.
I still don't understand why there can't be an option to add a file extension to the mailbox. This way to virus scanner could be configured to ignore the mailboxes all together. The solution of having a temp file simply doesn't make sense. This is always going to result in having the mailbox quarantined at some point. Unless you subscribe to the conspiracy theory that anti virus companies write the the viruses, it should seem pretty obvious that the temp file solution is not going to work in the long term (or in some cases the short term). After all, scanners can only check for viruses after they are known. Therefore there is a window between when you can receive the virus and when the virus scanner will detect the virus. If the virus comes in during this window, your entire mailbox will get quarantined. This seems rather obvious to me and lot less sensible then simply excluding the mailboxes all together.
To: junk@path.berkeley.edu Most virus scanners use huristic scanning which will try to detect new viruses based on previous viruses. Sure, this won't catch all, but this is the way life goes. We won't be able to change this and so there's no point dwelling on it. Some virus scanners allow you to exclude entire directories. Why not try excluding your mail directory. This way it's safe. Then, when you try gaining access to an attachment, it'll be saved to a temporary location and your virus scanner will then pick up on it. Cheers, Jeremy
Marking this one to block any future possible branch builds. This is a dataloss and possibly could even be considered a security issue.
Flags: blocking-aviary2.0-
Flags: blocking-aviary1.0.7?
> Some virus scanners allow you to exclude entire directories. Why not try > excluding your mail directory. This way it's safe. Then, when you try gaining > access to an attachment, it'll be saved to a temporary location and your virus > scanner will then pick up on it. The reason why is because I manage a large number of workstations. The virus scanning is centrally managed. Yes I can exempt specific directories, however, since the mail directories change on a per machine/user basis I would have to handle exclusions for all the machines individually. With a file extension based scheme, I can globally tell the virus checker to ignore the files across all machines. In addition, I would not consider excluding an entire directory the best practice. It gives viruses a very easy place to hide. I'm of the opinion that most viruses should be stopped before the user's machine if possible anyway. Our site uses virus scanning at the mail server, and in addition uses attachment file renaming and mime retyping for things like .exe, .scr, .pif, etc. Most viruses don't make it to the machines, and of the ones that do, almost all of them are neutralized. In order for a user to execute a virus, they have to save it and rename it thus forcing them to think about it instead of just clicking on them. Granted it's not great to have viruses sitting in the mailbox, however, it's not too bad if they can't be clicked on and executed. Having the mailbox quarantined, however, is unacceptable. This issue is preventing me from deploying Thunderbird on a wide scale within my organization (something I'd really like to do because of it's cross platform compatability, junk filtering capabilities...and it's not outlook). > Most virus scanners use huristic scanning which will try to detect new viruses > based on previous viruses. Sure, this won't catch all, but this is the way life > goes. We won't be able to change this and so there's no point dwelling on it. Exactly. This is why the current solution is no solution.
(In reply to comment #210) ... > The reason why is because I manage a large number of workstations. The virus > scanning is centrally managed. Yes I can exempt specific directories, however, > since the mail directories change on a per machine/user basis I would have to > handle exclusions for all the machines individually. This is even more pronounced because of these bugs: Bugzilla Bug 247973 Bugzilla Bug 233746 (if it also applies to Thunderbird=?)
> all machines. In addition, I would not consider excluding an entire directory > the best practice. It gives viruses a very easy place to hide. So does a filename extension...
Sry for this agression, but wtf is going on here? There is an extremely heavy bug in Thunderbird, that makes it unusable and stops it from getting the chance to ever run in a corporate environment. Just marking this bug as fixed doesn't make it true. It's a fact, that if your virus definitions get updatet after you recieved a virus-infected mail, your mailbox gets deleted. That's not acceptable, so this bug is definitely not fixed. It's a fact that very few virus-scanners implement a feature to exclude a directory, but nearly every one of them has an "exclude extension" feature. Everyone here knows, that the profile-paths include the username, so it is impossible to set up a virus-scanner to exlude all the profile-directorys for a whole company. It's not our problem if the solution is secure or not, because it's the problem of the anti-virus supplier. They seem to believe in the sercurity of their "exclude extension" feature, otherwise they wouldn't include it in their scanners. If they want a more secure solution, they will have to build in support for every mailbox format out there. But as said before, that's not our problem. So why can't just someone give the mailbox files an extension, so thunderbird can finally compete with other email-clients in corporate environenments?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
re comment #211: Those bugs have got nothing to do with this re comment #213: So exclude the whole of %appdata%\Thunderbird\Profiles (or wherever the profiles are stored for your machine/company), or simply disable the scanning of mime-encoded files. Or get better AV software. re the issues of adding an extension or switching to maildir: If they aren't already, file enhancement requests. They are beyond the scope of this bug. Anyone who still wants to moan about the way this has been fixed: read https://bugzilla.mozilla.org/etiquette.html
Status: REOPENED → RESOLVED
Closed: 20 years ago → 19 years ago
Flags: blocking-aviary2.0-
Flags: blocking-aviary1.0.7?
Resolution: --- → FIXED
Excellent. So pointing out that a bug marked as FIXED isn't really fixed is now considered "moaning?" The point is that the "save each message as a temporary file so the AV software can scan it and potentially quarantine it" approach still results in the entire In box getting quarantined on occassion. Given this: > It's a fact that very few virus-scanners implement a feature to exclude a > directory, but nearly every one of them has an "exclude extension" feature. Your solution is this: > exclude the whole of %appdata%\Thunderbird\Profiles (or > wherever the profiles are stored for your machine/company), or simply disable > the scanning of mime-encoded files. Or get better AV software. ? If you want Thunderbird to be taken seriously in the corporate world, you can't seriously expect everyone to upgrade their AV software simply so they can use your software. Actually, if you want Thunderbrid to be taken seriously in the consumer world, you can't expect that either. This bug is extremely serious because it appears to be (and in some ways is) data loss. There's no indication that your In box get eaten by your AV software. It looks like Thunderbird lost it. Refusing to fix this bug isn't going to help anyone. I mean, look at Comment #206. You've already lost one user. And that person was helping to test.
Done. I did a search and didn't find it so I added it. https://bugzilla.mozilla.org/show_bug.cgi?id=305354 I honestly am surpised at this attitude. I cannot understand how anyone could consider the temp file fix to be a real fix to this problem, and thus making the matter resolved. It's obvious it isn't. > re the issues of adding an extension or switching to maildir: If they aren't > already, file enhancement requests. They are beyond the scope of this bug. > > Anyone who still wants to moan about the way this has been fixed: read > https://bugzilla.mozilla.org/etiquette.html
re comments 215, 2nd part of 216 Please remember that this entire problem is caused by bug(s) in certain AV software. The fix checked in worksaround the problem for the case of a virus being detected as it arrive, which is the most common complaint in this bug by far. The real fix can only come from the makers of the affected AV software. For the case of the AV software detecting a virus after it's been sitting on your system for a while, I'd say you've got more serious problems. A more comprehensive Mozilla 'fix' for this *AV* problem, would require significantly more work, work that obviously isn't felt worth it at this moment of time by the *2* people who currently do work on Thunderbird. As always in open source - patches are welcome.
> For the case of the AV software detecting a virus after it's been sitting on > your system for a while, I'd say you've got more serious problems. Um...virus comes in...virus definition comes in. It doesn't have to be a while. In fact the time differential may be an hour or less. There's not much that can be done about that. We're not talking about viruses that have been there a month...or week...or even a day.
Got stung by this bug 2 more times this past week!!!!! Mozilla Corp looks like fools. )-:
We can't fix this "properly" (this means a way to have only the infected mail quarantized). That's correct - that's the job of the AV-supplier. BUT we CAN give thunderbird the abillity to compete with other email clients, wich all suffer from the same problem. You understand? We can't fix it - all we need to do is to get on the same level with the other email clients. This can be done by simply adding a file extension to the mailboxes. And please don't tell me that this would be too much work.
(In reply to comment #220) > We can't fix this "properly" (this means a way to have only the infected mail > quarantized). That's correct - that's the job of the AV-supplier. BUT we CAN > give thunderbird the abillity to compete with other email clients, wich all > suffer from the same problem. > You understand? We can't fix it - all we need to do is to get on the same level > with the other email clients. This can be done by simply adding a file extension > to the mailboxes. And please don't tell me that this would be too much work. See comment #39, by the developer assigned to this bug. If you think you know better than please submit a patch. Please - no more comments in this bug!
*** Bug 264936 has been marked as a duplicate of this bug. ***
It seems someone had a similar problem with a nightly branch and "NOD". http://forums.mozillazine.org/viewtopic.php?t=330908 Can't there be a new beta release to help these poor people? Anyone think Thunderbird should release BEFORE Firefox?
What if the "message filter" feature be extended in such a way that an external program can scan individual emails and give yes or no answers according to whatever criterion it carries. Then we can do virus-scaning like this: if Message-Body get-yes-when-scanned-by "Virus-Scanning-Prog-A" move Message to "mail with virus" folder Even better, it could be this (requires further extending, though): if Message-Body get-yes-when-scanned-by "Virus-Scanning-prog-A" process Message with "Virus-killing-prog-B" I'm proposing this because message filtering can be run both on message retrieval and later on user demand ("Run filters on folder"). Users are given a chance to identify infected mails ALREADY IN THE MBOX FILE. I really need this feature. I've used Thunderbird for more than a year without doing any virus scanning on mails until recently. (Yes I use linux and I'm very lucky.) I installed clamav only a few days ago to find 2 virus in my inbox. Without this feature I can't imagine a way to pick out the 2 infected mails from a 200M+ mbox.
This one seems to have come back recently with a 1.5 version of Thunderbird and Norton/Symantec anti-virus. Anyone else? I restore from the av software, and the inbox comes back, but then gets taken out again after a new scan. Help!
(In reply to comment #225) > This one seems to have come back recently with a 1.5 version of Thunderbird and > Norton/Symantec anti-virus. You may see, for Thunderbird and the Mozilla Suite: Antivirus software - MozillaZine Knowledge Base http://kb.mozillazine.org/Antivirus_software (with the sections "Keeping your antivirus software from deleting your Inbox", "Antivirus programs with compatibility problems", etc.).
The point is that I thought that quarantining individual message IS the fix, and that setting has been and still is checked off on the Thunderbird in question. The trouble is that it isn't working.
No. The "fix" stores a copy of the message outside the inbox before adding it to the inbox to give a virus scanner the chance to scan the message before being put in the inbox. As is mentioned many times earlier in this thread, this really isn't a fix at all because it depends on virus scanners being able to detect the message when it arrives. If it gets detected later, it eats your inbox. Thunderbird really needs a method of adding a file extension to mailboxes that can be exluded from a centralized virus scanner on a global level. Exclusion is currently very difficult because profiles have random names and this makes deployment across many machines very difficult. In your case you can probably tell the scanner to avoid your mail directory (kind of cripples the effectiveness of scanning though).
So then what is the point of quarantining individual messages? For that matter, what is the point of antivirus software if you can't run it on your email which is the most likely point of entry lately?
Good questions. Ideally your virus scanner would understand the mbox format and quarantine only the infected messages. Your (most) virus scanners are dumb and don't do this. Scanning the message when it comes in, gives you a chance to prevent quarantining the inbox. If used with something that would exclude inboxes from being scanned it would be helpful. In my opinion, you have to rely on user education once the message makes it past the scanner. After all, if the scanner misses it (Even if it had the ability to quarantine individual messages instead of the whole box) the message will sit in the user's box until the scanner was updated. Therefore the user has to know not to click it and to delete it, anyway.
I know this has been raised before (e.g., comment #2). However, viruses do seem to be spread not in the body of messages but in their attachments. This whole mess could be resolved by separating out the attachments the way some other E-mail clients do (e.g., Eudora). Until this is at least an option, this problem will not go away. Note that Norton Anti-Virus notifies the user every two weeks to update the virus definitions if the user has not enabled automatic live update. Except when there is a serious attack, daily updates are really not necessary. Thus, the recently added capability to place new incoming messages in a separate temporary file will mean either a proliferation of such files until the next virus definition update or else a loss of the entire inbox upon the next update even if those temporary files were retained for a week. There is a recommendation to keep the inbox almost empty to avoid serious loss from this problem. Be honest. How many of you have fewer than 100 messages in your inbox? (No, don't answer here. Just think about it.) I keep filing messages in other folders and deleting messages, but my inbox has over 400 messages. In any case, it's possible that the loss of only one very important message could be catastrophic. And anyway, this recommendation could merely shift the loss to other mail folders. I won't use Thunderbird for E-mail until a real fix is implemented.
I personally hate that feature of Eudora. It makes it difficult to migrate to another mail client, Eudora often loses its association between messages and attachments, and often when you delete the message the attachments stay. Maildir is a much better option where each message is saved as a separate file. I think implementing maildir in thunderbird would be a significant undertaking though.
Has it been seriously considered to develop an "Antivirus API" for Thunderbird? The minimal requirements for such an API would include being able to: * Scan individual e-mail messages every time the user accesses them. * Scan the raw e-mail as well as individual parts/attachments as they are parsed by Thunderbird. * Add, read and modify certain headers in the messages. * Remove individual parts/attachments. * Modify individual parts/attachments, including changing the name and MIME type. * Instruct Thunderbird to remove/delete the entire message (even based on scanning just one part/attachment of the message) and compact the folder. * Instruct Thunderbird to block access to individual parts of the message or the entire message and inform the user of the reason (infected message). I'm aware of the setting [Tools->Options->Privacy->Anti-virus], but neither a POP3/IMAP scanner nor individual spool files are a good solution. (Why is this setting under Privacy and not Advanced? What's the privacy implication?) If you look at [http://kb.mozillazine.org/Thunderbird_:_FAQs_:_Anti-virus_Software] then one of the recommendations are "[C]onsider waiting a while before opening the attachment. This gives your AV program's manufacturer a chance to provide a perhaps necessary new update." To do this for the entire e-mail (since there is HTML/Javascript-based malware) the message would need to be rescanned every time it is accessed after a virus signature file update. For that an API or a method to plug into / hook into Tunderbird is necessary. Scanning the entire mbox is a waste of CPU and has performance issues if only a few e-mails in a large mailbox are being accessed. We at FRISK Software are interested in developing an F-PROT Antivirus plugin for Thunderbird that would be a part of our product, similar to our current Microsoft Outlook plugin. P.S. Also posted to http://forums.mozillazine.org/viewtopic.php?p=2740502 and discussions on similar topics can be found in bugs 247223 and 369847. Sorry for the spamming.
(In reply to comment #236) > Has it been seriously considered to develop an "Antivirus API" for Thunderbird?... > I'm aware of the setting [Tools->Options->Privacy->Anti-virus], but neither a > POP3/IMAP scanner nor individual spool files are a good solution. (Why is this > setting under Privacy and not Advanced? What's the privacy implication?) ... Or a junk/virus/spam API.
The quickest, simplest, safest solution is to give all mbox files a ".pst" or ".dbx" file extension so that they are, by default, excluded by most old and new antivirus software. With such a solution in place, can we later worry about evangelizing Thunderbird to the AV developers to fix their mbox scanner.
Product: Core → MailNews Core
Keywords: relnote
Blocks: 1144999
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: