1421082 - Reports of Sync Not Working After Successful Login

Status: RESOLVED FIXED

(Firefox for iOS :: Sync, defect, P1)

Description

[Alex Davis [:adavis]]

On Twitter last week, there were just a couple of users that signed into their sync, confirmed their emails but then sync did not seem to work but then today it also happened to Nick Nguyen.

When he goes to Send Tab, he does not see his device list. There is no message saying to confirm his email. When the device syncs, it displays a last sync date in September.

Exact steps Nick did to run into bug:
- Migrated to a new iPhone from backup
- Logged into Sync (although we didn't say to) since the session didn't carry over.

Expect result:
- Device syncs

Actual result:
- Device looks like it is syncing but it isn't and there is no error. Last sync date is September.

Comment 1

[Lina Butler [:lina]]

We can try to reproduce this in the emulator and see if we can diagnose the issue.

Comment 2

[Alex Davis [:adavis]]

We just tried the following on Nick's phone:

- Disconnecting from Sync
- Logging back in again

Sync still did not work after that. We had to delete the app from his phone and then login again for it to finally work.

Comment 3

[Justin D'Arcangelo [:justindarc]]

:eoger, is this something you can look into?

Comment 4

[Richard Newman [:rnewman]]

My wild speculation is that this is related to the keychain. I wonder if login storage works, either?

You might not be able to repro in the Simulator: backup and keychain edge cases are in the set of things that aren't really solid concepts in the Simulator, IIRC.

The last sync date will carry over as part of prefs, but it won't advance if we're totally failing to sync.

We have no real mechanism for reporting sync failures to the user, so that would explain what you're seeing.

Grabbing the profile from an affected device (debug menu or iTunes backup) should show something useful in the logs.

Comment 5

[Alex Davis [:adavis]]

Just a heads up that Spencer Hui ran into the same problem today but unlike many of the others I've helped lately, he hadn't restored from a backup lately. His last sync date was yesterday. I'm not sure if it is related but it also took reinstalling the app to work.

Comment 6

[Edouard Oger [:eoger]]

I've tried to reproduce this at the office last week but didn't succeed. Maybe because I was using the same physical device?

Comment 7

[Justin D'Arcangelo [:justindarc]]

Closing this as we have no real steps to reproduce for right now. Feel free to re-open if we can repro.

Comment 8

[Alex Davis [:adavis]]

Apparently matej ran into this problem recently. 

Can you share with us how you encountered it? Were you restoring from a backup?

eoger: when you tested this, did you do an iOS backup restore?

Comment 9

[Richard Newman [:rnewman]]

I'm also interested to know if you have iCloud Keychain enabled.

Comment 10

[Justin D'Arcangelo [:justindarc]]

Catalin, can you try and repro this device migration scenario?

Comment 11

[Catalin Suciu [:csuciu]]

I was able to reproduce this issue by backing up the data from an iPhone 6 and restoring it on an iPhone 8.

After the restore, I had to re-sign in, but the sync was no longer working (no new data is transferred to the device of from it).  

STRs:

On iPhone 6 running iOS 10.3:

1. Install Firefox 10.6 - App Store version
2. Sign into sync and synchronize some data between desktop and mobile to make sure that sync is working. 
3. Using iTunes, perform a backup of the device on your computer.

On iPhone 8 running iOS 11.2.6:

4. Reset (Erase all Content and Settings) the device.
5. Restore the device from the iTunes backup.
6. Open Firefox and re-sign into sync.
7. Create some new data on mobile and desktop and sync.

Result: The sync is no longer working.

Comment 12

[Alex Davis [:adavis]]

Thank you so much for confirming the bug!

Comment 13

[Justin D'Arcangelo [:justindarc]]

Edouard, do you have any bandwidth to take this on?

Comment 14

[Edouard Oger [:eoger]]

Sorry I'm pretty busy these weeks I can't take this.

Comment 15

[Justin D'Arcangelo [:justindarc]]

Do we know if uninstalling/re-installing the app fixes the issue?

Comment 16

[Catalin Suciu [:csuciu]]

(In reply to Justin D'Arcangelo [:justindarc] from comment #15)
> Do we know if uninstalling/re-installing the app fixes the issue?

It does.

Comment 17

[Alex Davis [:adavis]]

Yes, uninstalling/re-installing resolves the issue.

This also requires to log back in and go through an email verification loop.

That's a lot of friction to get Sync up and running again!

Comment 18

[Justin D'Arcangelo [:justindarc]]

Catalin,

I'm curious if when you tested this if you had "iCloud Keychain" turned on or off. I believe that this backup/restore may behave differently depending on whether or not "iCloud Keychain" is turned on and I would like to confirm this. Is there anyway you could either verify whether or not you were using it when you tested this or if you could test again in both cases (iCloud Keychain on and off).

Comment 19

[Justin D'Arcangelo [:justindarc]]

(In reply to Justin D'Arcangelo [:justindarc] from comment #18)
> Catalin,
> 
> I'm curious if when you tested this if you had "iCloud Keychain" turned on
> or off. I believe that this backup/restore may behave differently depending
> on whether or not "iCloud Keychain" is turned on and I would like to confirm
> this. Is there anyway you could either verify whether or not you were using
> it when you tested this or if you could test again in both cases (iCloud
> Keychain on and off).

Actually, we need to test this a bit differently to give us a better understanding of whether or not this is a *Keychain* issue or if it is a *Firefox Sync* issue.

1. On a clean profile, store a saved login
2. Backup device to
   a.) Unencrypted iTunes Backup
   b.) Encrypted iTunes Backup
   c.) iCloud Backup (with iCloud Keychain OFF)
   d.) iCloud Backup (with iCloud Keychain ON)
3. Restore device from backup in step 2
4. Check to see if saved login is still saved

Comment 20

[Justin D'Arcangelo [:justindarc]]

Initial results from my tests:

a.) Restoring an unencrypted iTunes Backup to same device
    - No saved logins show in Firefox Settings (spinner never resolves)
b.) Restoring an encrypted iTunes Backup to same device
    - No saved logins show in Firefox Settings (spinner never resolves)
c.) Resetting and restoring an iCloud Backup to same device (with iCloud Keychain OFF)
    - *DID NOT TEST*
d.) Resetting and restoring an iCloud Backup to same device (with iCloud Keychain ON)
    - Saved logins show up successfully in Firefox Settings

NOTE: The iTunes Backups were simply restored to the device without "resetting" by clicking the "Restore Backup..." button in iTunes.

ALSO NOTE: When I restored the encrypted iTunes Backup, I got an error afterwards that my device ran out of storage. This *may* have adversely affected my results.

Comment 21

[Justin D'Arcangelo [:justindarc]]

UPDATE:

I tried RESETTING and then restoring my encrypted iTunes Backup and my saved logins persisted in Firefox Settings. So, either there's an issue with restoring without resetting first, or the fact that my device ran out of storage space when trying this the first time contributed to this affecting result `b.)` in Comment 20.

Comment 22

[Justin D'Arcangelo [:justindarc]]

I just want to re-iterate my test results here because I've now re-confirmed that test result `b.)` in Comment 20 is wrong. Clearly, when my device ran out of storage space restoring the backup, it adversely affected my test results.

My revised findings are:

a.) Restoring an unencrypted iTunes Backup to same device
    - No saved logins show in Firefox Settings (spinner never resolves)
b.) Restoring an encrypted iTunes Backup to same device
    - Saved logins show up successfully in Firefox Settings
c.) Resetting and restoring an iCloud Backup to same device (with iCloud Keychain OFF)
    - *DID NOT TEST*
d.) Resetting and restoring an iCloud Backup to same device (with iCloud Keychain ON)
    - Saved logins show up successfully in Firefox Settings

Tomorrow, I'll follow this same test plan with logging into FxA and enabling Firefox Sync.

Comment 23

[Justin D'Arcangelo [:justindarc]]

I just completed testing backup/restore between two different devices. Here are my results:

Prep:
- Signed into FxA and enabled Firefox Sync on "iPad A" (contained saved logins)
- Not signed into FxA on "iPad B" (contained saved logins)
- iTunes Backups were performed using iTunes 12.7.4.76 on macOS High Sierra 10.13.4
- iPads were wiped to factory state using "Restore iPad..." button in iTunes
  - This process downloaded the iOS 11.4 image and flashed it to wipe the device to its original "on-boarding" state
- Both iPads were running iOS 11.4

Backups:
a.) Backed up "iPad A" to un-encrypted iTunes Backup
  - Clicked "Back Up Now" button in iTunes with "Encrypt local backup" un-checked
b.) Backed up "iPad A" to encrypted iTunes Backup
  - Clicked "Back Up Now" button in iTunes with "Encrypt local backup" checked
c.) Backed up "iPad A" to iCloud (with iCloud Keychain disabled)
  - Tapped "Back Up Now" button in Settings app under Accounts & Passwords > iCloud > iCloud Backup
d.) Backed up "iPad A" to iCloud (with iCloud Keychain enabled)
  - Tapped "Back Up Now" button in Settings app under Accounts & Passwords > iCloud > iCloud Backup
e.) Backed up "iPad B" to iCloud (with iCloud Keychain enabled)
  - Tapped "Back Up Now" button in Settings app under Accounts & Passwords > iCloud > iCloud Backup

Restores:
a.) Restored "iPad B" with un-encrypted iTunes Backup of "iPad A"
  - Clicked "Restore iPad..." button in iTunes to factory reset the device, then selected the un-encrypted iTunes Backup to restore after reset
b.) Restored "iPad B" with encrypted iTunes Backup of "iPad A"
  - Clicked "Restore iPad..." button in iTunes to factory reset the device, then selected the encrypted iTunes Backup to restore after reset
c.) Restored "iPad B" with iCloud Backup (with iCloud Keychain disabled) of "iPad A"
  - Clicked "Restore iPad..." button in iTunes to factory reset the device, then unplugged and followed the on-boarding on the device to restore the iCloud Backup
d.) Restored "iPad B" with iCloud Backup (with iCloud Keychain enabled) of "iPad A"
  - Clicked "Restore iPad..." button in iTunes to factory reset the device, then unplugged and followed the on-boarding on the device to restore the iCloud Backup
  - After restoring the device, re-enabled iCloud Keychain in Settings (it was off after restoring)
e.) Restored "iPad B" with iCloud Backup (with iCloud Keychain enabled) of "iPad B"
  - Clicked "Restore iPad..." button in iTunes to factory reset the device, then unplugged and followed the on-boarding on the device to restore the iCloud Backup
  - iCloud Keychain was enabled automatically after restoring (presumably because the backup was of the same original device)

Results:
a.) BROKEN - Restoring un-encrypted iTunes Backup to new device
  - History tab showed "Synced Devices - 4 device(s) connected" even though FxA appeared to not be logged in
  - Viewing saved logins in "Settings" resulted in an empty list and an infinite spinner
  - Was able to log into FxA successfully, but Firefox Sync was broken (as described by this bug)
    - The sync appears to complete quickly without error
    - Nothing new is synced and nothing appears under "Synced Devices"
b.) WORKED - Restoring encrypted iTunes Backup to new device
  - FxA was logged in
  - Firefox Sync was working
  - Saved logins were present in "Settings"
c.) BROKEN - Restoring iCloud Backup (with iCloud Keychain disabled) to new device
  - History tab showed "Synced Devices - 4 device(s) connected" even though FxA appeared to not be logged in
  - Viewing saved logins in "Settings" resulted in an empty list and an infinite spinner
  - Was able to log into FxA successfully, but Firefox Sync was broken (as described by this bug)
    - The sync appears to complete quickly without error
    - Nothing new is synced and nothing appears under "Synced Devices"
d.) BROKEN - Restoring iCloud Backup (with iCloud Keychain enabled) to new device
  - History tab showed "Synced Devices - 4 device(s) connected" even though FxA appeared to not be logged in
  - Viewing saved logins in "Settings" resulted in an empty list and an infinite spinner
  - Was able to log into FxA successfully, but Firefox Sync was broken (as described by this bug)
    - The sync appears to complete quickly without error
    - Nothing new is synced and nothing appears under "Synced Devices"
e.) WORKED - Restoring iCloud Backup (with iCloud Keychain enabled) to same device
  - FxA was logged in
  - Firefox Sync was working
  - Saved logins were present in "Settings"

Comment 24

[Justin D'Arcangelo [:justindarc]]

Attachment: GitHub Pull Request

Comment 25

[Justin D'Arcangelo [:justindarc]]

I believe I've solved the issue here. So, as thoroughly documented in Comment 23, after restoring from a backup, we're left in a state where we have our SQLite databases, but no keys in the iOS Keychain. This automatically prevents us from logging into FxA/Sync, however, we still have some state leftover from before that leaves us in a weird state. Namely:

1.) Previous count of synced devices/remote tabs
2.) Previous encrypted logins.db

In the case of #1, this is what caused the "Remote Tabs" item in the History panel to show a state that appeared as though we were connected when we really weren't. But, in the case of #2, the inability to even open logins.db is what caused the entire Sync process to short-circuit and bail out. We never recovered. My patch now fixes both of these issues. Specifically, for #2, it moves the old encrypted logins.db to a backup location, generates a brand new encryption key in the iOS Keychain and creates a new empty logins.db. This lets the user log in as usual and actually successfully Sync again.

Comment 26

[Farhan Patel [:farhan]]

merged to master.