1095157 - [email/activesync] Device ID mitigation for manual QA testing

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::E-Mail, defect)

Description

[Andrew Sutherland [:asuth] (he/him)]

Bug 825538 changed to create a new device ID when adding an ActiveSync account.  The problem is that ActiveSync servers seem to not have a smart LRU-eviction policy but instead require manual intervention to clear out old devices (usually a 10 or 100 device limit for most configs).  We are also unable to detect and report the situation when too many devices were registered (filed as bug 1095144), which makes things bad.

There are a lot of trade-offs here between privacy concerns, Web API limitations, hassle for QA testers, bug triage effort for reported bugs from this, developer effort required, etc.

There was some existing discussion about this from when this totally broke the Python based automated tests (bug 1028192 with bug 1033923 being the bug to allow an automated testing workaround):

=== asuth said in https://bugzilla.mozilla.org/show_bug.cgi?id=1028192#c21 ===

Hm, I was somewhat assuming from the spec that ActiveSync servers would just do a least-recently-used (LRU) eviction policy for Devices from its synchronization table.

So yeah, the only practical solution to the problem for automated tests is to have each device use the same device ID every time.  And I mean *each* device; multiple devices cannot/must not use the same device ID or we're back where we started.

There are three scenarios we need to be concerned about here:
A) These automated tests using a pool of devices and potentially performing aggresive wiping in-between.
B) Manual QA testing.  In this case, the device may not be quite so thoroughly wiped between runs, but the accounts are likely to be repeatedly added.
C) Real users who ideally are only rarely deleting and re-adding the account when there is a problem, and even more rarely completely wiping their devices.

Situation 'A' is very special.  We have the ability to poke/magically hack things about the state of the email app to allow workarounds like forcing the use of a specific device ID.  It's also the only one where this is basically required.

Situation 'B' is probably the most troubling because we cannot really expect manual QA testers to reliably jump through special debugging hoops.  Mitigations for this that jump out at me are:

- Keep the device id's we have created in the database so that if the user deletes [e-mail] and then re-creates it, we can reuse that same device ID again.

- If there's a way to tell an ActiveSync server that we are unbinding the device, we should totally 100% do that when the user deletes the account.  We'd do that opportunistically but we could have an enhancement bug to make sure we eventually do it if we were offline at the time.

If it really takes 100 devices to make an ActiveSync server angry, we're probably okay in the 'C' case most of the time, but the mitigation of persisting the device ID even when an account is removed from step 'B' is likely the most useful thing we can do.

Jim, any thoughts?

=== Jim said in https://bugzilla.mozilla.org/show_bug.cgi?id=1028192#c22 ===

We could add a pref that you can use to hard code the device ID, and then (A) and (B) users can just be expected to set that pref. Otherwise, I think we're doing the right thing. If we can maintain the ability to change the device ID by deleting and recreating the account, I think that's ideal, since it helps reduce fingerprinting.

=== asuth said in https://bugzilla.mozilla.org/show_bug.cgi?id=1028192#c24 ===

We could add a pref that you can use to hard code the device ID, and then (A) and (B) users can just be expected to set that pref. Otherwise, I think we're doing the right thing. If we can maintain the ability to change the device ID by deleting and recreating the account, I think that's ideal, since it helps reduce fingerprinting.

=== Jim said in https://bugzilla.mozilla.org/show_bug.cgi?id=1028192#c25 ===

Can we just do some kind of build-time config?

I think it's useful to be able to reset the device ID in order to, say, work around buggy servers. I can imagine that if something goes awry, a particular device ID might be "poisoned", and give back erroneous/unexpected data.

Also, you might want to reset your device ID if you're afraid your account got hijacked (and if you have access to see the IDs of all devices connecting to your account), since then you'd be able to see if there were any devices using something other than your new device ID.

Granted, these are edge cases and I'm kinda just making excuses here, but I'd rather be a little too paranoid than not enough.

=== end quotes ===

Comment 1

[BMO Automation]

Firefox OS is not being worked on