1656898 - Thunderbird v78+ causes data & settings loss on downgrade to v68 and earlier

Status: RESOLVED INVALID

(Thunderbird :: General, defect)

Description

[Eyal Rozenberg]

Despite Thunderbird's weird choice of creating a different profile for every version, many, and probably most people, use the same profile with all versions.

Also, a significant fraction of users - and most developers - have to downgrade their version for one of a variety of reasons. A typical scenario is noticing that an extension is not compatible with the new version.

Now, beginning with Thunderbird 78 (and perhaps earlier), there's an error message when you try to access your profile, and you have to use a command-line option to bypass it. This is the first problem.

The more serious problem is that some of the profile data is either reset, deleted, or hidden after downgrade. So far, downgrading from TB 78.1 to TB 68.8, I've noticed:

• Folder collapsed state is reset
• Address book is emptied! (luckily, abook.mab.bak is still there)

This is unacceptable IMHO - both in terms of reduced usability and manageability, and more importantly - data loss! Now, maybe the data exists in a way you can access it with TB 78, so it's not complete data loss - but it's effective loss.

Therefore - those TB versions which make changes to profile data must do it so as not to hamper the access to this data by previous versions. Some of the damage is already done, but I ask that a new version be issued ASAP addressing this issue.

Comment 1

[Thomas D. (:thomas8)]

(OT: I'd think in standard English mustn't mustn't be spelled as musn't, that's slang... anyway, let's drop it.)

Comment 2

[Avner Falk]

I entirely agree with Eyal Rozenberg. Not being as skilled as he is at using the Command Prompt for installing Thunderbird, and not knowing how to downgrade without losing my entire profile, I twice had to restore my entire computer system from a backup just to go back to my earlier version, due to numerous bugs in TB 78, of which this is only one. The program also kept freezing on me. I know you all have the best intentions, but PLEASE test new versions thoroughly before releasing them. Thanks.

Comment 3

[Eyal Rozenberg]

Avner, if you've found other bugs, please file them (separately from this one). Also, was the data loss only due to having to downgrade the version? If not, then pretty-please-with-sugar-on-top file another bug about that.

Comment 4

[Magnus Melin [:mkmelin]]

The address book was migrated from mork to sqlite, so the old data can't be used anymore. Downgrades are NOT supported, exactly because such conversions and database alterations cannot reasonably support that. Once you start using for the shortest time, data can have changed, so you can't go back even if there were migrators - some data would be lost and there's no knowing which is dispensable.

Comment 5

[Eyal Rozenberg]

Magnus, you don't just brush away bugs - certainly not data loss bugs.

First, you have not addressed the clobbering of settings (and possible data) other than the address book. Now, on the address book issue

The address book was migrated from mork to sqlite

That's fine, but it is no reason to delete the abook.mab file. If we are so concerned with disk space, deletion of critical user data (even if it is kept in another format) should at most be an option, or require specific confirmation.

"Downgrades are not supported"

That's a meaningless phrase. Downgrades happen by users uninstalling a newer version and reinstalling the older one; they don't need specific support. What is needed - and has essentially always been the case with Thunderbird and Seamonkey so far - is that newer write their new data to new files, or different pref tree locations etc. - so that older versions may still run.

And this is essentially the case with the address book migration too: We create a new sqlite file. This is good. But why delete the abook.mab file?

Also, in a political context: How can one of the leaders of the Thunderbird project tell users who experienced data loss due to a necessary downgrade that we are ok with their effective-loss of data and that it's their fault for having to downgrade?

Comment 6

[Magnus Melin [:mkmelin]]

The folder collapsed state is not important enough to migrate.

The mab file is deleted so that we know it was migrated. We keep a backup of that file, just with an additional extension (abook.mab.bak). That's the best we can do or the user would have dataloss if they added something in the new addressbook and then migrated.

It's pointless discussing if downgrades are supported: they are not. The UI never allows you to do so anymore, and gives warnings. It's really up to advanced users to copy paste file around if they need to do something more. We can't copy over a potentially very huge profile either and start using that.

So, there's no brushing, you just need to accept realities.

Comment 7

[Eyal Rozenberg]

The mab file is deleted so that we know it was migrated.

Use a preference instead. Or check whether a non-empty sqlite addressbook exists. Or any combination of the above. Deleting the file effectively means sabotaging downgrades.

It's pointless discussing if downgrades are supported

It's not pointless, it's meaningless. Downgrades happen, period. Certainly in applications which have an auto-upgrade feature. And because extensions mostly don't support TB 78 yet, downgrades happen with not-insignificant frequency. They don't stop happening because you declare "Ah, but that's not supported". We need to account for them, enough for people not to lose things like their address book.

Additionally, downgrade safety is important for extension developers.

We keep a backup of that file,

Even that one is not the latest abook.mab, IIANM.

Comment 8

[Wayne Mery (:wsmwk)]

For Thunderbird to have real functional improvements for users, backward compatibility cannot always be guaranteed.

Yes, we have had some releases that don't break compatibility. That might be the norm, but that doesn't mean every release gets a guarantee of backward compatibility. We have past releases where backward compatibility did not exist, for example where the format of filters, search database or message folders has changed.

Your request simply isn't going to happen in this particular case.

Comment 9

[Eyal Rozenberg]

@Wayne:

Backwards compatibility "cannot always be guaranteed", so you decided to delete abook.mab ? Why? It almost seems like it's for spite, or to prove a point.

for example where the format of filters, search database or message folders has changed.

A search database can be rebuilt. And my message folders are, AFAICT, in the same MBOX format they've always been.

But in any of those cases - has an unused file ever been intentionally deleted?

Also, none of these happened when extensions were mostly not ready and incompatible with the new release, requiring a downgrade to continue working normally.

Just stop deleting the file. Auto-upgrades have not happened yet, right? So there's still time until the larger slew of downgrades.

Comment 10

[Magnus Melin [:mkmelin]]

Like I wrote, the data is in the .bak file. Advanced user actions are needed to rescue it yes (== finding, knowing, renaming). That would be required to go back anyway, so the UI explicitly prevents you from going back.

Comment 11

[marcolopespt]

I UPGRADED today to TB 78!

It MESSED UP my address books!

I LOST few address books (NOT converted!)

The ones converted have a .bak original MAB!

I always make backups, but was stupid enough not to make today!

I tried UPGRADING again using the MAB files / BAK files, at no avail... it's an ugly BUG :\

CAN I DOWNGRADE to 68 having ALL my profile info backup on 68 (using the mail data from 78?)

Comment 12

[Magnus Melin [:mkmelin]]

Most, but not all data is compatible. To use the 68 profile you need to start with --allow-downgrade.
If you're willing, send me a copy of the .bak file so I can try to reproduce the bug.

Comment 13

[marcolopespt]

I mean, i have a FULL backup of v68... except the MAIL folders!
Has anything changed on the MAIL format?
I'm doing a test, recovering the 68 backup, replacing the MAB files backups and MAIL folders from the 78

DONE... the pop folders are ok, the GMAIL IMAP folders do not appear :\

Comment 14

[marcolopespt]

I gave up!

I EXPORTED all the address books on v68 (CSV)
UPGRADED to v78
DELETED all the address books from the UI
RESTARTED
DELETED few new empty address books from the UI (there was something really wrong after de update!)
IMPORTED all the address books (CSV)
All address books were created in order (SQL version)

NOTE: had to review ALL the mail RULES, because they were messed up (when the books were deleted)

A guess that's the best approach. I will keep the latest version.

Comment 15

[Upgeya Pew]

I think I may have posted this in the wrong place, so I'm posting here:

I'm having a similar problem to marcolopespt's on the upgrade to TB 78. Since there have been multiple updates in the last few days, I'm not sure at what sublevel the problem occurred, or exactly when - but probably in the last week or two. I'm now at 78.4.3.

On the upgrade, most of my 20 or so address books have been emptied. The only ones that remain populated are those with a "list" in them, and the Personal Address Book, and an address book called "Collected Addresses", which is specified on the Options->Composition property sheet under "Addressing" as the place to automatically add outgoing e-mail addresses.

I'm not trying to down grade.
How do I restore the addresses in my empty address books?