1694942 - maildir converter should ignore more files, which are not mail files (such as filterlog.html)

Status: ASSIGNED

(MailNews Core :: Backend, defect)

Description

[Magnus Melin [:mkmelin]]

The converter is not strict enough about files in the profile that are non-mail. E.g. filterlog.html in the mbox will create a maildir folder named "filterlog.html" and subfolders cur+tmp.

https://searchfox.org/comm-central/rev/30dd9287808adc0d4c6e008f48f9cb547ff4e391/mailnews/base/src/converterWorker.js#319

It should probably mirror nsMsgLocalStoreUtils::nsShouldIgnoreFile
https://searchfox.org/comm-central/rev/30dd9287808adc0d4c6e008f48f9cb547ff4e391/mailnews/local/src/nsMsgLocalStoreUtils.cpp#24

Comment 1

[Ben Campbell]

Yup. Pretty sure this is responsible for Bug 1529929 at least, and likely some other ones.

Notes to self:

• It looks like NNTP accounts don't use filterlog.html, but use another naming scheme instead, possibly a per-folder one (see nsMsgFilterList::GetLogFile()). And I don't think even nsMsgLocalStoreUtils::nsShouldIgnoreFile() takes account of this.
• I think the mbox->maildir conversion should probably also have some sort of heuristic to avoid files that are obvious non-mboxes. Maybe peeking at the first few lines to see if it can spot some headers, say...
• The root cause of all this is insufficent segregation between actual mail storage files/dirs and all the other files. I think it'd be worth trying to prise them apart one day (eg, have a fixed subdir to hold only mail). But waaay out of scope here.
• Another side effect: all the corner cases where some names are invalid folder names. You just know we'll get a bug report one day from someone who desperately needs to be able to name one of their folders "filterlog.html" or "sort.dat" or whatever...

Comment 2

[Ben Campbell]

Attachment: Bug 1694942 - Update whitelist of "special" files for mailstore conversion. r=mkmelin

Comment 3

[Ben Campbell]

I think that patch should fix things (and include the problem in a simple test case), but I think there are a few improvements to follow up on.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/49aa76a575ce
Update whitelist of "special" files for mailstore conversion. r=mkmelin