1736197 - Removal of movemail support should still give users access to data

Status: RESOLVED WONTFIX

(Thunderbird :: Account Manager, defect)

Description

[Gareth McCaughan]

Initial note

This is not a protest at the fact that movemail is no longer supported. (I wish it were, but clearly that argument has already been lost.) The bug I am reporting is that the way in which movemail has stopped being supported is needlessly destructive.

Reproduction recipe

• Begin with an older version of Thunderbird.
• Set up a "movemail" account retrieving mail from a local mail spool.
• Use it for a while, setting up many folders and moving emails into them.
• Upgrade to Thunderbird 91.

Expected behaviour

• Since (alas) movemail is no longer supported, of course the account will become unable to retrieve new emails.
• However, it should still be possible to access its existing saved email.
• It should be possible to change the account's type from the no-longer-supported "movemail" to, say, IMAP and continue using it.

Actual behaviour

• The account has completely disappeared along with all its mail folders and emails.
• There is no obvious way to regain access to the old email.
• The files and folders still exist on disc, so in some sense the information hasn't been permanently lost, but ...
• ... if there is any way to associate them with a new account, I don't know what it is.
• Furthermore, all this happens silently. There isn't a message saying "Sorry, movemail is no longer supported so we disappeared all your data" or even "E4019 ABEND PARSE ERROR IN PREFS.JS". Just a Thunderbird window from which your account and all its emails have vanished.

Further comments

Loss of function is inevitable, given the decision to stop supporting movemail. Loss of data is inexcusable. In this case it's access to data rather than the data itself that's been lost, but that's nearly as bad.

Ideally, it should be possible just to change the account type and keep using it. Failing that, there should be a documented procedure for transferring the account's saved emails to a newly created account. Failing that, there should be sufficient documentation to know, e.g., whether creating a new account risks deleting all the saved emails of which Thunderbird has apparently lost all awareness.

Comment 1

[Gareth McCaughan]

In case anyone cares, the setup I had which provoked this was:

• Retrieve email from my ISP using getmail
• Local machine runs a mail server
• Thunderbird retrieves email from /var/spool/mail/my_username with movemail, sends email with SMTP
• Mail folders in Thunderbird are under the "localhost" movemail account, rather than using "Local Folders"

and (after trying a couple of other hacky things that resulted in silent failure or outright data loss, but fortunately I have backups) the recovery procedure that seems to be working for me (which I mention for the benefit of anyone reading this who is in the same situation as mine) is

• Edit ~/.thunderbird/my_profile/prefs.js to change mail.account2.type from "movemail" to "none"
  • (this enables Thunderbird to admit the existence of the account again and show me all my saved emails)
• Create a new account to retrieve my email (still locally; I do have an IMAP server running even though I wasn't using it in Thunderbird)

At this point I have all my mail and can send and receive things but the organization is kinda stupid: all my saved email is under an account that I'm not going to use any more. So now I moved them all to be under "Local Folders".

• Except that Thunderbird doesn't allow you to move folders, so instead I copied them.
• Except that when I dragged a bunch of folders from one account to another, what Thunderbird did was to copy some of their contents and then stop, so then I identified things that hadn't been copied, deleted them, and dragged them over again.
• Except that that didn't help, because apparently Thunderbird assumes that if you haven't opened a folder recently you don't care about the contents, or something, so I had to go through my hundreds of old email folders explicitly opening them one by one before Thunderbird considered their contents worth copying.

But after those simple steps everything seems like it more or less works.

Comment 2

[Magnus Melin [:mkmelin]]

Probably easiest to on the file system level, copy over the mbox folders of the former movemail account to under the Local Folders, and restart.

I agree it's a bit unfortunate, but it's not super clear that's what the user would want. Especially now when some users already may have taken action, I think we'll just have to live with what we have.

Comment 3

[Gareth McCaughan]

I think just copying over the mbox folders to under "Local Folders" was the first thing I tried, and what happened was that Thunderbird sat unresponsively using 100% of a CPU core for what seemed like it was going to be for ever, and when I interrupted it after about 2 hours and restarted a lot of the newly-copied folders simply weren't there any more. That was one of the "other hacky things that resulted in silent failure or outright data loss". I didn't keep careful records, though, so I'm not 100% sure.

An alternative to "just living with what we have" (which loses users' data) would be to make a future Firefox release do something more graceful, such as just treating "movemail" the same as "none" so that messages remain accessible and it just won't fetch new messages any more. That won't do anything to help people who have already upgraded and run into trouble, but it will help users who upgrade in the future, if they get the newer version.