1332028 - Live account tests no longer work because of unverified test accounts

Status: NEW

(Firefox for iOS :: Firefox Accounts, defect)

Description

[Stephan Leroux [:sleroux]]

We've recently had the following live account tests fail consistently on BuddyBuild:

SyncTests:LiveStorageClientTests:testLive()
SyncTests:LiveStorageClientTests:testStateMachine()
AccountTests:SyncAuthStateTests:testLive()

The issue seems that the test account credentials now return status code 400, errno 104: attempt to operate on an unverified account whenever we try to fetch our keys. 

According to BuddyBuild, the tests last worked on Dec. 20 2016 and started to fail Dec 22. The FxA team did a push to the production environment on the 21st with the following changes: https://github.com/mozilla/fxa-auth-server/blob/master/CHANGELOG.md#1761-2016-12-21.

:vbudhram was able to create a new test account and had the tests run successfully on his machine using the new credentials but when I tried to run it on my machine the same tests fails. The running hypothesis is that the FxA server doesn't require verification if the account is used from the same IP the account was created from and this change was part of the push made on the 21st.

For the time being, the tests have been disabled until we can find a way workaround for the unverified issue.

Comment 1

[Ryan Kelly [:rfkelly]]

IIUC, we should be able to re-enable these tests by adding the email addresses to the new SIGNIN_CONFIRMATION_SKIP_FOR_EMAIL_ADDRESS config variable in FxA:

https://github.com/mozilla/fxa-auth-server/blob/master/config/index.js#L651

We did this for a test account used by lockbox:

https://github.com/mozilla-services/cloudops-deployment/pull/2750/files#diff-363c9077b281aadc0efcbf84a089315eR139

:justindarc what email address(es) do these tests use?

Comment 2

[Ryan Kelly [:rfkelly]]

(If they're restmail accounts, the other thing you could do is fish the sign-in confirmation email out of restmail and manually confirm each signin, but what a waste of your time if we have the allowlist available server-side).

Comment 3

[Justin D'Arcangelo [:justindarc]]

Ryan,

It appears as though the only account being used in a test environment is currently hard-coded as: [e-mail]

https://github.com/mozilla-mobile/firefox-ios/blob/master/AccountTests/LiveAccountTest.swift#L134

Comment 4

[Ryan Kelly [:rfkelly]]

How much of a pain would it be to create a new account for this purpose with a more descriptive name, like "[e-mail]" or similar? I'm just thinking about future maintainability of the config on the FxA side, someone's going to look at "[e-mail]" in a year's time and wonder what it's for :-)

Comment 5

[Justin D'Arcangelo [:justindarc]]

Not a pain at all. I would gladly welcome a more meaningful account name :-)

[e-mail] sounds good to me!

Comment 6

[Ryan Kelly [:rfkelly]]

Proposed config change in https://github.com/mozilla-services/cloudops-deployment/pull/2984

Comment 7

[Ryan Kelly [:rfkelly]]

ni? Shane for comment on the above change; I also tagged you for review on the github issue, but I'm not sure if it'll notify you or not given that it's the private ops config repo.

Comment 8

[Shane Tomlinson [:stomlinson]]

(In reply to Ryan Kelly [:rfkelly] from comment #7)

ni? Shane for comment on the above change; I also tagged you for review on the github issue, but I'm not sure if it'll notify you or not given that it's the private ops config repo.

This is a perfectly valid reason to disable signin confirmation for that email address. r+