1093583 - [DSDS] Add test to make a phone call with default SIM

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Johan Lorenzo [:jlorenzo]]

Base user story : Bug 946866 - [DSDS][Dialer] On-the-fly SIM change when calling

Let's just make a phone call with 2 SIMs and SIM 2 is the default to place phone calls.

Comment 1

[Johan Lorenzo [:jlorenzo]]

Attachment: Gaia PR

WIP. I'll ask for review once bug 1094185 has landed.

Comment 2

[Johan Lorenzo [:jlorenzo]]

Hi guys,

Now that bug 1094185 has landed, can one of you put a second SIM in one of our Flame in the lab? Can you also update the credentials repo for this phone?

Thank you!

Comment 3

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

I'm a little hesitant to go make hardware changes without understanding exactly what needs to be done, and verifying that the infrastructure has all other dependencies taken care of. I'm more than happy to be the hands, but need someone else to point.

Needinfoing Dave to coordinate.

Comment 4

[Parul Mathur [:pragmatic] - out of office - please ni? [:bajaj] instead]

On a temporary basis, an AT&T SIM Card has been added to device b2g-14.1 as SIM2. The device already has a T-Mobile SIM Card as SIM1.

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

How are you planning on targeting this node for tests? b2g-14.1 is currently allocated to the performance testing pool, so I don't think this will be any use for you. Even if it was part of the UI testing pool, any tests that require multiple SIMs will fail unless they happen to run on this device.

Comment 6

[Parul Mathur [:pragmatic] - out of office - please ni? [:bajaj] instead]

The temporary AT&T SIM Card has been withdrawn from device b2g-14.1. A dedicated SIM Card for testing this scenario may be requested from ServiceNow, if needed.

Comment 7

[Johan Lorenzo [:jlorenzo]]

After talking to :davehunt on IRC, I split the dependencies of that bug. I update the estimation accordingly.

Comment 8

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8517458 [details] [review]
Gaia PR

Started this adhoc job: http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.dsds.adhoc/3/console

I created a new manifest file in the dialer folder because some dialer tests can't pass on a DSDS setup.

At first, I wanted to check which phone number SIM was used thanks to plivo. But plivo intermittently add/remove some digits in my French phone number. Ex: "33123456789" sometimes becomes "133123456789" and sometimes "3312356789". In the end, I only checked the "Via SIM" label on the callscreen. Do you think it's enough, Doug?

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #8)
> I created a new manifest file in the dialer folder because some dialer tests
> can't pass on a DSDS setup.

Why is the new manifest needed? Shouldn't the job be only running tests that are marked as dsds in the manifest file, so we would avoid running any tests that would otherwise fail? Alternatively, what are the failures, and can we improve these tests so they can run on devices with muliple SIMs?

Comment 10

[Johan Lorenzo [:jlorenzo]]

(In reply to Dave Hunt (:davehunt) from comment #9)
> Why is the new manifest needed?
My bad, I had a leftover of --type in the command line with which I tested. I updated the PR.

The tests that are failing are:
> TEST-UNEXPECTED-FAIL | test_dialer_add_contact.py TestDialerAddContact.test_dialer_add_contact
> | AssertionError: u'55527102717\nSIM1' != '55527102717'
> TEST-UNEXPECTED-FAIL | test_dialer_miss_call_from_known_contact_notification.py 
> TestReceiveCallFromKnownContactNotification.test_dialer_miss_call_from_known_contact_notification
> | AssertionError: u'(SIM 1) Missed call' != 'Missed call'
Seem easy to fix

> TEST-UNEXPECTED-ERROR | test_dialer_find_contact.py TestDialerFindContact.test_dialer_find_contact
> | NoSuchElementException: NoSuchElementException: Unable to locate element: .handled-call.outgoing
> TEST-UNEXPECTED-ERROR | test_dialer_set_up_conference_call.py TestSetUpConferenceCall.test_set_up_conference_call
> | NoSuchElementException: NoSuchElementException: Unable to locate element: .handled-call.outgoing
Might be related to the SIM Picker. I have to look into it.

Comment 11

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Thanks. There certainly no need to fix those tests in this bug.

Comment 12

[Doug Sherk (:drs) (inactive)]

Comment on attachment 8517458 [details] [review]
Gaia PR

I'm arbitrarily setting r- as I think that this requires more discussion, though it may be ready to go as-is afterwards.

(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #8)
> At first, I wanted to check which phone number SIM was used thanks to plivo.
> But plivo intermittently add/remove some digits in my French phone number.
> Ex: "33123456789" sometimes becomes "133123456789" and sometimes
> "3312356789". In the end, I only checked the "Via SIM" label on the
> callscreen. Do you think it's enough, Doug?

In the Gaia code, we have a library called SimplePhoneMatcher that strips out international codes, + symbols, etc., and allows us to compare just local numbers to each other. Perhaps we could use something similar within the UI tests. I don't know if such a thing exists, but if not, it would be a helpful addition for the future.

Aside from that, I left some comments on the PR.

Comment 13

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8517458 [details] [review]
Gaia PR

I updated the test case to use a parameterized test. Re-asking Bebe to check this change.

(In reply to Doug Sherk (:drs) (use needinfo?) from comment #12)
> In the Gaia code, we have a library called SimplePhoneMatcher that strips
> out international codes, + symbols, etc., and allows us to compare just
> local numbers to each other.

The main issue is that Plivo also removes a digit in the middle of the phone number like "33123456789" and "3312356789" (the 4 is missing, it took me some time to notice it). I tried about 20 times and I don't have the same result depending on the time where I run the tests. That's why I prefer avoiding to rely on it.

Comment 14

[Doug Sherk (:drs) (inactive)]

Comment on attachment 8517458 [details] [review]
Gaia PR

(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #13)
> The main issue is that Plivo also removes a digit in the middle of the phone
> number like "33123456789" and "3312356789" (the 4 is missing, it took me
> some time to notice it). I tried about 20 times and I don't have the same
> result depending on the time where I run the tests. That's why I prefer
> avoiding to rely on it.

Ok, I see. I missed that in your initial explanation.

Comment 15

[Florin Strugariu [:Bebe]]

https://github.com/mozilla-b2g/gaia/commit/87c40a61ceae544d825a1ba746faf2377b1bb867