Closed Bug 768102 Opened 12 years ago Closed 12 years ago

Sync account lost on device restart or upgrade if Firefox is on the SD card

Categories

(Firefox for Android Graveyard :: Android Sync, defect, P1)

Product:
Firefox for Android Graveyard
Component:
Android Sync
Platform:
All
Android
Type:
defect

Tracking

(firefox14 verified, blocking-fennec1.0 .N+, fennec14+)

Status:
RESOLVED FIXED
Milestone:
mozilla14
Tracking Flags:
Tracking Status
firefox14 --- verified
blocking-fennec1.0 --- .N+
fennec 14+ ---

People

(Reporter: nalexander, Assigned: nalexander)

References

Details

(Keywords: dataloss, user-doc-needed) 

From http://www.reddit.com/r/IAmA/comments/vkwjz/iama_significant_portion_of_the_firefox_for/c55dquc:

"I've been using Firefox Nightly for Android for a while. My biggest annoyance with it is that when I set up sync a day or two later I have to set it up again, and this continues to happen no matter how many times I set up sync. This happens on both GB and ICS. Why does this happen? I've read that having multiple versions (Nightly, Aurora, Beta) causes issues with sync, so now I only have nightly installed, but the issue persists."
I've seen this when Beta gets upgraded via Play.

Ideas: background update only? Update that fails? Update during browsing or syncing?

I'd love to get a log during an upgrade that causes this.
Severity: normal → major
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: qawanted
Priority: -- → P1
In fact, several:

http://www.droidforums.net/forum/bugless/119998-facebook-twitter-accounts-disappearing.html

http://www.droidforums.net/forum/tech-issues-bug-reports-suggestions/18509-exchange-settings-account-disappears.html

Hypothesis: whenever Android decides to reload the list of accounts — e.g., when an update is installed — your account is lost if the app is stored on the SD card.

The workaround for this might simply be "don't allow Firefox to be moved to the SD card", but that's a pretty crappy workaround.
App data itself doesn't seem to go missing (and perhaps neither do prefs?), so we might be able to catch an onUpgrade and persist our account so we can restore it afterwards…

QA: could you try installing, moving to SD, setting up Sync, then updating, see if you can get this to happen?
On my Nexus One Android v2.2, I can not reproduce this.  Installed Aurora build from 20120623, moved it to the SD card, setup Sync (confirmed it was working properly), checked for updates, then installed the found update.  The Sync account remain and is functional.
A few people have reported this, or a similar issue, at

https://support.mozilla.org/en-US/questions/930599

And one person confirms that things work after moving the app off the SD card.
So, it seems to me it has nothing to do with upgrade.  I restarted my device with Fx on the SD card, then the account disappeared. :(
Keywords: qawanted → dataloss
Would it be wrong for me to shake my fist at Android right now?

OK, this leaps straight to the top of the list.
Severity: major → critical
(In reply to Richard Newman [:rnewman] from comment #4)
> App data itself doesn't seem to go missing (and perhaps neither do prefs?),
> so we might be able to catch an onUpgrade and persist our account so we can
> restore it afterwards…

There are security implications all over this.  Also, why would we receive onUpgrade at any point during a normal reboot cycle?  (Because Android.)  We might be able to hook account deletion:

http://developer.android.com/reference/android/accounts/AccountManager.html#addOnAccountsUpdatedListener%28android.accounts.OnAccountsUpdateListener,%20android.os.Handler,%20boolean%29
(In reply to Nick Alexander :nalexander from comment #9)

> There are security implications all over this.

Sure, but we could write into the same protected storage as the passwords DB, and delete when done. Let's try to come up with a solution before we whittle down.

> Also, why would we receive
> onUpgrade at any point during a normal reboot cycle?

Suggested this before Tracy observed disappearance during reboot.

> We might be able to hook account deletion:

If we had a way to distinguish that from an intentional deletion :/
(In reply to Richard Newman [:rnewman] from comment #10)
> (In reply to Nick Alexander :nalexander from comment #9)
> 
> > There are security implications all over this.
> 
> Sure, but we could write into the same protected storage as the passwords
> DB, and delete when done. Let's try to come up with a solution before we
> whittle down.

Fair enough.  I have been thinking about how secure storage is on an Android device and didn't realize we had access to *any* non-Account based secure storage.
(In reply to Nick Alexander :nalexander from comment #11)

> Fair enough.  I have been thinking about how secure storage is on an Android
> device and didn't realize we had access to *any* non-Account based secure
> storage.

Perhaps I should say "no less secure than where we write your password database". It's only accessible to users or other apps on rooted phones.
Putting this on some radars.
tracking-fennec: --- → ?
status-firefox14: --- → affected
Summary: Android Sync nightly upgrade not maintaining Sync account → Sync account lost on device restart if Firefox is on the SD card
I've lost my account and never restart my device.
Summary: Sync account lost on device restart if Firefox is on the SD card → Sync account lost on device restart or upgrade if Firefox is on the SD card
> Hypothesis: whenever Android decides to reload the list of accounts — e.g.,
> when an update is installed — your account is lost if the app is stored on
> the SD card.
> 
> The workaround for this might simply be "don't allow Firefox to be moved to
> the SD card", but that's a pretty crappy workaround.

After doing quite a bit of reading, and some investigation, this is looking better by the minute.  I'm going to try registering a broadcast receiver for all sorts of events and see if *any* of them get through to an App re-enabled after an SD card is re-mounted.  So far, the obvious events do not get through.

The collective wisdom of the internet doesn't mention any workarounds.
Firefox Sync users must not put Fx on the SD card. If they did and lost their Sync setup, their workaround is to move Fx back to the phone, then reconnect to their sync account.
More technical details on investigation:

Android just doesn't support installing Apps that define long-lived Services and/or own Account types onto the SD card.  The documentation says not to do it.  There are hordes of developers who want to do it, and have tried to register for almost every "package installation changed" broadcast intent that Android supports.  They all explicitly state that the package that has changed does *not* receive the broadcast intent, thereby preventing an App from re-establishing its state.

I think we could re-install a Sync account if the user runs Fennec or similar, but I think we are better off accepting this limitation of the Android ecosystem and considering installation to the SD card if and when we move the Sync account architecture out of the Fennec APK.
(In reply to Nick Alexander :nalexander from comment #17)
 
> Android just doesn't support installing Apps that define long-lived Services
> and/or own Account types onto the SD card.  The documentation says not to do
> it.

Reference?
(In reply to Richard Newman [:rnewman] from comment #18)
> (In reply to Nick Alexander :nalexander from comment #17)
>  
> > Android just doesn't support installing Apps that define long-lived Services
> > and/or own Account types onto the SD card.  The documentation says not to do
> > it.
> 
> Reference?

Long reference:

http://developer.android.com/guide/topics/data/install-location.html

<quote>
Services
    Your running Service will be killed and will not be restarted when external storage is remounted. You can, however, register for the ACTION_EXTERNAL_APPLICATIONS_AVAILABLE broadcast Intent, which will notify your application when applications installed on external storage have become available to the system again. At which time, you can restart your Service.
</quote>

Only problem is that intent doesn't work:

http://code.google.com/p/android/issues/detail?id=8485

<quote>
Sync Adapters
    Your AbstractThreadedSyncAdapter and all its sync functionality will not work until external storage is remounted.
</quote>

http://android-developers.blogspot.com/2010/07/apps-on-sd-card-details.html

<quote>
When not to install on SD card?

The advantage of installing on SD card is easy to understand: contention for storage space is reduced. There are costs, the most obvious being that your app is disabled when the SD card is either removed or in USB Mass Storage mode; this includes running Services, not just interactive Activities. Aside from this, device removal disables an application’s Widgets, Input methods, Account Managers, Device administrators, Live wallpapers, and Live folders, and may require explicit user action to re-enable them.
</quote>
Rebooting will cause bug 755638.  Mounting via USB Storage will cause this bug.
So we have two possible solutions for this.

The first -- preferable because it has less impact on low-end devices -- is to pickle your Sync account on a regular basis, catch account removal intents, and unpickle the account either when the SD card is remounted or when Firefox is next launched.

This is kinda janky, it might not work, and it's not ideal, but it might allow us to preserve the ability to install to SD card.

Note that this is *not* a long term solution as we start to add more long-term services; when we've got AITC and Notifications, we will probably encounter more hurdles that will require us to be memory resident.

Part two of this solution is to introduce messaging when creating an account with Firefox installed on SD, and perhaps even watch for move events and show a warning. These would link to SUMO.


The second solution is to disable SD card installs, and prioritize profile/cache shrinkage work: catching low space intents to clear cache, expiring history, improving favicon storage, etc.

If we flip that bit, we'll probably need UI in Sync setup to prevent existing users continuing with setup if they're currently installed on SD, because I doubt Android will move us back.


We're going to start investigating the former, aiming for a point release, and fall back on the latter if we need to.
(In reply to Naoki Hirata :nhirata from comment #20)
> Rebooting will cause bug 755638.  Mounting via USB Storage will cause this
> bug.

Well well. Android preserves account UIDs across reboots, but changes app UIDs when remounting?

*throws brick through window*
Scratch rebooting will cause 755638.  I had multiversions.

Rebooting will cause the sync account to still look like it's there, but there's nothing in the bookmarks and history.  Maybe bug 769165 might be a dup?
No longer blocks: 755638
blocking-fennec1.0: --- → ?
(In reply to Naoki Hirata :nhirata from comment #23)
> Scratch rebooting will cause 755638.  I had multiversions.

Yay.

> Rebooting will cause the sync account to still look like it's there, but
> there's nothing in the bookmarks and history.  Maybe bug 769165 might be a
> dup?

To clarify those last two sentences, you:

* Installed Firefox, so it's the only thing on the device
* Moved it to SD card
* Set up Sync, synced
* Verified that bookmarks and history existed
* Rebooted
* Verified that Sync account still existed
* Bookmarks and history are gone

?
Crud.  Turns out that my sync account was locked today.  Sync seems to work fine on the nightly after reboot on the SDcard...

* Installed Firefox, so it's the only thing on the device
* Moved it to SD card
* Set up Sync, synced
* Verified that bookmarks and history existed
* Rebooted
* Verified that Sync account still existed
* Bookmarks and history are still there.

STR for this bug:
* Installed Firefox, so it's the only thing on the device
* Moved it to SD card
* Set up Sync, synced
* Verified that bookmarks and history existed
* Plug device to computer
* set device connect to USB Storage
* Verified that Sync account still existed

Actual: Sync account is missing.
(In reply to Naoki Hirata :nhirata from comment #26)
> Crud.  Turns out that my sync account was locked today.  Sync seems to work
> fine on the nightly after reboot on the SDcard...

OK, good.

> Actual: Sync account is missing.

Yup, that's this bug.
tracking-fennec: ? → 14+
blocking-fennec1.0: ? → .N+
Depends on: 769745
Depends on: 769749
Actions from Comment 21 are now the two blocking bugs. Will file additional messaging bugs later.
Nick, Richard: Any update on the "pickle" approach?
(In reply to Mark Finkle (:mfinkle) from comment #30)
> Nick, Richard: Any update on the "pickle" approach?

Yep, moving on it now.  Due to history, Android Sync stores account prefs in multiple places, so we're centralizing that while implementing the pickle.  I'll update the ticket as we move forward.
(In reply to Nick Alexander :nalexander from comment #31)
> (In reply to Mark Finkle (:mfinkle) from comment #30)
> > Nick, Richard: Any update on the "pickle" approach?
> 
> Yep, moving on it now.  Due to history, Android Sync stores account prefs in
> multiple places, so we're centralizing that while implementing the pickle. 
> I'll update the ticket as we move forward.

This doesn't sound like its going to make a 14.0.1 build for Monday though, is it?
(In reply to JP Rosevear [:jpr] from comment #32)
> (In reply to Nick Alexander :nalexander from comment #31)
> > (In reply to Mark Finkle (:mfinkle) from comment #30)
> > > Nick, Richard: Any update on the "pickle" approach?
> > 
> > Yep, moving on it now.  Due to history, Android Sync stores account prefs in
> > multiple places, so we're centralizing that while implementing the pickle. 
> > I'll update the ticket as we move forward.
> 
> This doesn't sound like its going to make a 14.0.1 build for Monday though,
> is it?

The centralization of account prefs: no (Bug 761682).

A work-around for this ticket is in place and landed on m-i Friday afternoon: Bug 769745 and Bug 735842.  (That last addresses Bug 769749.)

So this could definitely make it for a 14.0.1 build.  I wrote these patches and think they are fairly low risk.  There are a few risk points:

1. we could re-create an Android Sync account the user intentionally deleted.  This is annoying, but definitely not data-loss.

2. we could affect disk performance by checking for pickle file every Fennec run.  I don't think this will be significant.

Can't think of other issues right at the moment.
(In reply to Nick Alexander :nalexander from comment #33)

> A work-around for this ticket is in place and landed on m-i Friday
> afternoon: Bug 769745 and Bug 735842.  (That last addresses Bug 769749.)

These hit m-c yesterday at noon, so they're in this morning's Nightly. I'm pretty confident in this having no repercussions that would be worse than the original problem, and the fixes are self-contained, so I'd be happy to uplift the workaround after it's had a QA pass.
(In reply to Richard Newman [:rnewman] from comment #34)
> (In reply to Nick Alexander :nalexander from comment #33)
> 
> > A work-around for this ticket is in place and landed on m-i Friday
> > afternoon: Bug 769745 and Bug 735842.  (That last addresses Bug 769749.)
> 
> These hit m-c yesterday at noon, so they're in this morning's Nightly. I'm
> pretty confident in this having no repercussions that would be worse than
> the original problem, and the fixes are self-contained, so I'd be happy to
> uplift the workaround after it's had a QA pass.

Said it better than I did.
See Bug 769745 Comment 6 for non-cheery news. Nick, I hope you're up early!
QA, please test apk in #37
Keywords: qawanted
(In reply to JP Rosevear [:jpr] from comment #38)
> QA, please test apk in #37

Already done: https://bugzilla.mozilla.org/show_bug.cgi?id=769745#c16.
Assignee: nobody → nalexander
Status: NEW → ASSIGNED
yes, verified per depends bugs. Nick, this can be marked FIXED, yes?
status-firefox14: fixed → verified
Keywords: qawanted
This made it to fx 14.0.1 beta, so I tagged it target-milestone:firefox14.  Please correct if that's wrong.
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla14
Product: Mozilla Services → Android Background Services
Product: Android Background Services → Firefox for Android
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.