1377100 - Passwords don't sync because sync never recovers from master password reset / failure to decrypt logins / corrupt/missing logins.json

Status: RESOLVED FIXED

(Firefox :: Sync, defect, P2)

Description

[[e-mail]]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0
Build ID: 20170608105825

Steps to reproduce:

Using Sync to synchronise between a Firefox for Android device and a Firefox instance on a PC.


Actual results:

Since the update to FF 54 and consequent re-authentication on the 23rd of June, 324 out of 336 saved passwords are no longer available either via the Password Manager or the Edit Passwords add-on. They are still present on the mobile device.

About Sync shows all the 336 total passwords as server records and the 324 it needs to download under ClientMissing. However, although they are evidently fetched, they are not written to local storage.

about:sync-log shows lots of the following error, ever since the re-authentication:

https://api.accounts.firefox.com/v1/certificate/sign failed: 2152398878 - NS_ERROR_UNKNOWN_HOST
1498648337388	Hawk	WARN	hawk request error: [Exception... "NS_ERROR_UNKNOWN_HOST"  nsresult: "0x804b001e (NS_ERROR_UNKNOWN_HOST)

This happens irrespective of whether I am on an enterprise network, VPN'd into it, or at home - therefore the problem seems to be with this web service of yours. Requesting https://api.accounts.firefox.com/v1/certificate/sign explicitly fails with a 404 error and errorno 999 ("Unspecified error"), as does anything under /v1/. 
And yes, I've clicked on the link to confirm the sign-in (I just pulled up the e-mail and did so again, and was informed I already had done). 

Timing strongly suggests that either FF 54 or release v1.89.0 of fxa-auth-server is guilty.


Expected results:

My passwords should get written to the local store.

Comment 1

[[e-mail]]

WORKAROUND: disconnect affected device from sync, re-add it, all is love again.

Comment 2

[Mark Hammond [:markh] [:mhammond]]

I'm glad it seems to be working for you again, but in the interests of finding the underlying issue, could you please upload all logs from about:sync-log?

Comment 3

[[e-mail]]

OK, here's a link: https://www.dropbox.com/sh/kf3wzl07lkz4yoy/AAC7nY-R9ppK6F6Y-F1jr1u2a?dl=1

Comment 4

[Mark Hammond [:markh] [:mhammond]]

Thanks - those logs are helpful. I'll get back to this.

Comment 5

[Mark Hammond [:markh] [:mhammond]]

I thought we already had a bug for this, but I can't find it.

(In reply to [e-mail] from comment #0)
> Since the update to FF 54 and consequent re-authentication on the 23rd of
> June,

I'm not sure what happened as you updated, but whatever it was, I expect all your logins were suddenly unable to be decrypted. This is roughly the same thing that will happen if you set a master password then reset it. In short, Sync doesn't know the data has been lost, so doesn't attempt to refetch the logins and re-save them. Sadly, even if it did, currently it would still fail due to bug 621846, but hopefully that will be fixed soon.

What *might* be possible is for Sync to count how many logins are unable to be decrypted and store that in preferences. If it ever finds that number has changed since the last sync, it could do a local "reset" and force all new logins to be downloaded and applied.

(This still wouldn't solve the issue if the problem is actually that logins.json was deleted or became corrupt - we could possibly solve that by storing a GUID in logins.json and in sync preferences, and if sync finds them different it also does a reset.)

Comment 13

[Mark Hammond [:markh] [:mhammond]]

I found a little time to work on this.

Comment 14

[Mark Hammond [:markh] [:mhammond]]

Attachment: Bug 1377100 - Store sync metadata in the LoginManager. r?lina,MattN

This is done so that if the login manager loses all data sync is "reset",
meaning will do a full reconcile with the server and pull all sync records
down and apply them locally - ie, sync will be ble to recover passwords for
many users.

The sync GUID is stored encrypted, so that even if the login manager data
isn't lost, but the encryption key is, it will still reset and recover.

This will also make restoring an older logins.json a little more reliable for
sync - if an old version is restored then sync should "rewind" itself to the
state it was in the backup.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0d918902af47
Store sync metadata in the LoginManager. r=lina,MattN

Comment 16

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/0d918902af47