Closed Bug 1505038 Opened 2 years ago Closed 1 year ago

Passwords wiped out updating from 52.9.1 to version 60.3.0, possible workaround: comment #25, delete "pkcs11.txt" from profile

Categories

(Thunderbird :: Security, defect)

Desktop
All
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 1510212

People

(Reporter: ndswaebe, Unassigned)

References

Details

(Keywords: dataloss)

Attachments

(1 file)

Attached image Version 60.3.0.JPG
Notified of available update to version 60.3.0.
Allowed update to install, after installation my passwords and security information  was gone.

As I don't usually have a problem with Thunderbird updates I was going to run it with out first doing an email backup with Mozbackup, GLAD I DIDN'T!

Re-installed version 52.9.1 and restored Thunderbird from my backup.
Priority: -- → P1
Summary: update from 52.9.1 to version 60.3.0 → Passwords wiped out updating from 52.9.1 to version 60.3.0
Gone as in you couldn't find the password files?
Group: mail-core-security
Priority: P1 → --
Duplicate of this bug: 1505056
> Wayne, any more reports like this one?

It's not an area I tend to follow in support fora.
Flags: needinfo?(unicorn.consulting)
Sadly this seems to be a real problem. As a developer I have long stopped using TB 52.x, but if I create a new profile with that version, enter a password and then visit the profile with TB 60, I get prompted again. If I click cancel and visit the saved passwords, they are all gone. Returning back to TB 52, they are there again and I'm not prompted again.

I think I've noticed password prompts when switching between 52 and 60+ on test profiles for testing purposes, but I didn't pay much attention to it. I should have :-(

I think there were changes to the password storage, I just have to find the M-C person to ask ... which it turning out to be difficult. Let's try Marco (toolkit) and Dana (security).

Marco and Dana, what has changed in the way passwords were saved between 52 and 60? As you know, TB uses the toolkit password manager to store e-mail passwords. Was there a migration? It doesn't look like it works for us.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(mak77)
Flags: needinfo?(dkeeler)
Bug 1475775 was uplifted to ESR60 (60.2 I guess?) so potentially bug 1496736 could be what's going on if this is on a redhat-based system (and assuming thunderbird is based on esr?).
Flags: needinfo?(dkeeler)
Hmm, I'm on plain Windows 10 and I don't have a master password. Yes, TB 60.3 is is based on mozilla-esr60 at 60.3.
Actually, I upgraded my wife's laptop from TB 52.7 to TB 60.3 without *any* problem and I didn't have to re-enter a single password. So there must be some special circumstances under which this happens.
(In reply to Jorg K (GMT+2) from comment #7)
> Actually, I upgraded my wife's laptop from TB 52.7 to TB 60.3 without *any*
> problem and I didn't have to re-enter a single password. So there must be
> some special circumstances under which this happens.

Possible Windows ver? Just asking?
Can someone else reproduce this?  (copying some of our giant testers)

My key question is whether only 52>60.3.0 is affected?  Or is 60.2.1 also affected?
And as Jorg asks, under what circumstances?

Calling this dataloss until we know better. 
I think THIS issue is unlikely to be OS specific. (The original report is Windows.)  
However, there may be other issues. And cannot rule out OS specific. 
Casting a wide net for potential support postings - some of these sound similar to this bug report, others not so much:

https://support.mozilla.org/en-US/questions/1237460  10/17 linux, downgrade from 60.2.1 to 52.x and passwords work
https://support.mozilla.org/en-US/questions/1237698  10/20 linux also
https://support.mozilla.org/en-US/questions/1237790  10/21 Mac (unknown if user updated)
https://support.mozilla.org/en-US/questions/1238103  10/24 Mac (user had NOT updated to 60.x)
https://support.mozilla.org/en-US/questions/1237531  10/18 Win (constant issue for user, not just updates)
https://support.mozilla.org/en-US/questions/1237482  10/17 Win (unknown if user updated)
https://support.mozilla.org/en-US/questions/1237811  10/21 linux, user updated to 60
Flags: needinfo?(o.e.ekker)
Flags: needinfo?(albert)
Keywords: dataloss
No recent bug reports that match this issue, at least not filed in Thunderbird product.  Here's a selection from https://mzl.la/2JKnohE (some are filed against TB52)
Bug 1503829 - impossible to send messages: password refused
Bug 1448769 - Thunderbird suddenly forgot mail server password, now not remembering passwords
Bug 1503829 - impossible to send messages: password refused

Also what I didn't state in the prior comment, is password issues are so prevalent that 
a) often the support posts don't all get looked out - so we can easily miss things
b) it's near impossible to distinguish non-bugs from bugs, and new bugs from old - so we can easily miss things

In short, we should get our act together to make the entire password area more robust, perhaps like was done with Firefox sessionstore.
OK, here's some testing:
1) Created profile with TB 52, setup account, saved password.
   Checked saved passwords in PW manager. IMAP and SMTP there. All good.
2) Visited profile with TB 60.3. I did NOT get prompted to enter the password again,
   I could visit my IMAP folders, the PW manager showed the saved passwords. All good.

So a regular user just moving from TB 52 to TB 60 will NOT have any problem. That is my experience from upgrading two people (friend/family).

3) Went back to the profile with TB 52. All still good. Passwords visible in PW manager.
   IMAP working.
4) Went back with TB 60.3. All still working.

5) I went in with TB 64 beta. All still working. Then 52 and 60 again. No problem.

So on a new profile, I don't see any problem whatsoever.

I wasn't drunk when I wrote comment #4, but I didn't tell the full truth. The failing profile wasn't new, it had been created with TB 65 Daily.

If I create a profile with TB 64 beta (or TB 65 Daily), then the passwords are not accessible in TB 52. If I enter them again, then they become available in 52, 60 and 64.
I'm not the right person to tell you about passwords, maybe MattN?
Flags: needinfo?(mak77)
I'm not sure when, but NSS changed from legacy NSS database files cert8.db and key3.db to newer SQLite database files cert9.db and key4.db files. Maybe that is causing problems in some cases? I recently answered a support question which might be related, although the user specified he was fiddling around on two computers: https://support.mozilla.org/en-US/questions/1238886?utm_campaign=questions-reply&utm_medium=email&utm_source=notification#answer-1168728
Flags: needinfo?(o.e.ekker)
Happened for me as well - tested on macOS / German

1) downloaded latest 52.x
2) added test account
3) saved password
4) checked that password is saved under Preferences > ...

meanwhile TB downloaded latest version 60.x

5) restarted
6) TB asked for PW
7) checked Preferences and PW was gone
Flags: needinfo?(albert)
OS: Windows 10 → All
I am not a programmer so this may not help, since I reported this bug 1505038 and re-installed 52.9.1 I looked at the "troubleshooting report" and found I had a crash of version 60.3.0 521 seconds of "uptime" from install.
I am including the link here if it helps let me know or should I just stay back and watch!

LINK https://crash-stats.mozilla.com/report/index/b2d1bdf2-5912-4e43-9ec8-3fa910181106#tab-bugzilla

Norm
In https://support.mozilla.org/en-US/questions/1239638 the user shares the profile between OS
I tried 52.9.1 -> 60.3.0 upgrade on linux. Passwords were kept.
Duplicate of this bug: 1505630
Wayne, bug 1505630 seems to be the other way round (about downgrading from 60.3). Were the passwords created in 60.3 that got lost? As it appears to me the reports are that if you upgrade from 52 to 60 and then downgrade back, you get the passwords back (as the original file is used again).
(In reply to :aceman from comment #19)
> Wayne, bug 1505630 seems to be the other way round (about downgrading from
> 60.3). Were the passwords created in 60.3 that got lost? As it appears to me
> the reports are that if you upgrade from 52 to 60 and then downgrade back,
> you get the passwords back (as the original file is used again).

OP of bug 1505630 here. All my login data was created/saved in 52.9.1 or earlier. Upgraded to 60.3, which asked me to define a new master password (I didn't comply), then downgraded, then noticed the loss of my login data.
In the past the passwords were stored in the file signons.sqlite in your profile. But I think that was long before TB 52. Now we use the file logins.json.
There should have been an automatic migration from one file to the other, keeping the passwords.

Can you see if you have those files and what their date of modification is?
Hi Matt, according to Marco's comment #12 you might be the person to ask about stored passwords.

We have contradicting reports of problems when switching from TB 52 to TB 60 (and back). I had no problems with the former (comment #11), but others report problems (comment #14).

The bug was originally filed by someone who lost his passwords after the 52->60 upgrade, but there are other reports where this caused no problem (comment #7, comment #11, comment #17).

So the whole thing is a bit of a mystery :-(
Flags: needinfo?(MattN+bmo)
(In reply to Wayne Mery (:wsmwk) from comment #16)
> In https://support.mozilla.org/en-US/questions/1239638 the user shares the
> profile between OS

That sounds like bug 1453372.

(In reply to Jorg K (GMT+1) from comment #22)
> We have contradicting reports of problems when switching from TB 52 to TB 60
> (and back). I had no problems with the former (comment #11), but others
> report problems (comment #14).

It's expected that logins saved in a new profile created after bug 783994 (defaulting to key4.db) will not be able to be read in an older build expecting key3.db. i.e. if you have no key3.db, only key4.db, then logins can only be decrypted in TB60+. It didn't make sense to support downgrades on a profile that never previously used the old version.

> The bug was originally filed by someone who lost his passwords after the
> 52->60 upgrade, but there are other reports where this caused no problem
> (comment #7, comment #11, comment #17).

There have been no changes to password manager storage that I can think of. The only changes are related to the encryption that Dana mentioned in comment 5. You should read the two bugs mentioned there and look at the timing of bug 783994 to help solve this.

NFS and the environment variable NSS_DEFAULT_DB_TYPE could also be relevant to your investigation. Dana knows much more about the NSS/PSM side.
Flags: needinfo?(MattN+bmo)
On a personal note, in the TB52 >60 migration I did almost daily for months,  I had issues with oauth passwords and caldav passwords being requested at odd times.  But standard POP/MAP/Chat accounts never needed to be re-entered.  I had assumed that the constantly changing user agent was causing providers to challenge the passwords, but perhaps it was a side effect of this bug.
Flags: needinfo?(unicorn.consulting)
https://support.mozilla.org/en-US/questions/1239779

This topic suggests passwords were missing and fixed by deleting "pkcs11.txt" from the profile or starting in safe mode.
(In reply to Matt from comment #25)
> https://support.mozilla.org/en-US/questions/1239779
> 
> This topic suggests passwords were missing and fixed by deleting
> "pkcs11.txt" from the profile or starting in safe mode.

Trying to understand this completely. I have had a lot of headaches around the update from 52.9.1 to 60.3.0
Are you saying after the upgrade I should "delete pkcs.txt" from the profile and this will re-instate my passwords?

I have reinstalled 52.9.1 numerous times and finally deleted all reference to ThunderBird went to a clean install of 52.9.1 reinstalled my email settings from a Mozbackup file being careful to be disconnected from the internet as the default is to immediately load the 60.3.0 update!

Now running 52.9.1 email back up and running and "AUTO UPDATE" turned OFF!
Flags: needinfo?(unicorn.consulting)
I guess, this Problem is much older; may I suggest having a look at my bug report:
https://bugzilla.mozilla.org/show_bug.cgi?id=1209803

Also, more recently, same Problem with 52.x -> 60.x
@Matt: Thanks for pointing me into the right direction with "pkcs11.txt" - the problem seems to be resolved!

@mozdev: Changes between old and new file:

--- <unbenannt>
+++ <unbenannt>
@@ -1,8 +1,5 @@
-library=libnsssysinit.so
+library=
 name=NSS Internal PKCS #11 Module
-parameters=configdir='sql:/home/USER/.pki/nssdb' certPrefix='' keyPrefix='' secmod='secmod.db' flags= updatedir='' updateCertPrefix='' updateKeyPrefix='' updateid='' updateTokenDescription='' 
-NSS=Flags=moduleDBOnly,internal,critical trustOrder=75 cipherOrder=100 slotParams=(1={slotFlags=[RSA,DSA,DH,RC2,RC4,DES,RANDOM,SHA1,MD5,MD2,SSL,TLS,AES,Camellia,SEED,SHA256,SHA512] askpw=any timeout=30})
+parameters=configdir='sql:/USER/buddy/.thunderbird/0000000.default' certPrefix='' keyPrefix='' secmod='secmod.db' flags=optimizeSpace updatedir='' updateCertPrefix='' updateKeyPrefix='' updateid='' updateTokenDescription=''  manufacturerID='Mozilla.org' libraryDescription='PSM-interne Krypto-Dienste' cryptoTokenDescription='Allgemeine Krypto-Dienste' dbTokenDescription='das Software-Sicherheitsmodul' cryptoSlotDescription='PSM-interne Kryptographie-Dienste' dbSlotDescription='PSM private Schlüssel' FIPSSlotDescription='FIPS 140 Krypto-, Schlüssel- und Zertifikat-Dienste' FIPSTokenDescription='das Softw.-Sicherh.modul (FIPS)' minPS=0
+NSS=trustOrder=75 cipherOrder=100 slotParams={0x00000001=[slotFlags=RSA,RC4,RC2,DES,DH,SHA1,MD5,MD2,SSL,TLS,AES,SHA256,SHA512,Camellia,SEED,RANDOM askpw=any timeout=30 ] }  Flags=internal,critical
 
-library=/usr/lib/x86_64-linux-gnu/nss/libnssckbi.so
-name=Mozilla Root Certs
-NSS=trustOrder=100
Duplicate of this bug: 1506407
Flags: needinfo?(unicorn.consulting)
Duplicate of this bug: 1509618
Summary: Passwords wiped out updating from 52.9.1 to version 60.3.0 → Passwords wiped out updating from 52.9.1 to version 60.3.0, possible workaround: comment #25, delete "pkcs11.txt" from profile
See Also: → 1510212
Progress is being made on this issue in bug 1510212.  If you have concerns for yourself or for other users you should read the recent comments, and then comment there about any potential fixes or actions that might be taken.
Duplicate of this bug: 1517889
Severity: major → normal
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1510212

A patch was created in BUG 1564284.
I believe this BUG has some relationship. Just updating... : )

Flags: needinfo?(ndswaebe)
Flags: needinfo?(ndswaebe)

Marcus, have you seen that this bug is already marked as resolved, because it is a duplicate of bug 1510212 ?
Do you have any reason to believe it isn't a duplicate of bug 1510212 ?

Hi Kai.

This bug was in my list for some time. I realized it was resolved right after my comment. : )
Sorry and thanks!

You need to log in before you can comment on or make changes to this bug.