Closed Bug 1093583 Opened 6 years ago Closed 6 years ago

[DSDS] Add test to make a phone call with default SIM

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jlorenzo, Assigned: jlorenzo)

References

Details

Attachments

(1 file)

Base user story : Bug 946866 - [DSDS][Dialer] On-the-fly SIM change when calling

Let's just make a phone call with 2 SIMs and SIM 2 is the default to place phone calls.
QA Whiteboard: [fxosqa-auto-s3][fxosqa-auto-points=6]
Attached file Gaia PR
WIP. I'll ask for review once bug 1094185 has landed.
Hi guys,

Now that bug 1094185 has landed, can one of you put a second SIM in one of our Flame in the lab? Can you also update the credentials repo for this phone?

Thank you!
Flags: needinfo?(parul)
Flags: needinfo?(gmealer)
QA Whiteboard: [fxosqa-auto-s3][fxosqa-auto-points=6] → [fxosqa-auto-from-s3][fxosqa-auto-s4][fxosqa-auto-points=6]
I'm a little hesitant to go make hardware changes without understanding exactly what needs to be done, and verifying that the infrastructure has all other dependencies taken care of. I'm more than happy to be the hands, but need someone else to point.

Needinfoing Dave to coordinate.
Flags: needinfo?(gmealer) → needinfo?(dave.hunt)
On a temporary basis, an AT&T SIM Card has been added to device b2g-14.1 as SIM2. The device already has a T-Mobile SIM Card as SIM1.
Flags: needinfo?(parul)
How are you planning on targeting this node for tests? b2g-14.1 is currently allocated to the performance testing pool, so I don't think this will be any use for you. Even if it was part of the UI testing pool, any tests that require multiple SIMs will fail unless they happen to run on this device.
Flags: needinfo?(dave.hunt)
The temporary AT&T SIM Card has been withdrawn from device b2g-14.1. A dedicated SIM Card for testing this scenario may be requested from ServiceNow, if needed.
After talking to :davehunt on IRC, I split the dependencies of that bug. I update the estimation accordingly.
QA Whiteboard: [fxosqa-auto-from-s3][fxosqa-auto-s4][fxosqa-auto-points=6] → [fxosqa-auto-from-s3][fxosqa-auto-s4][fxosqa-auto-points=3]
Comment on attachment 8517458 [details] [review]
Gaia PR

Started this adhoc job: http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.dsds.adhoc/3/console

I created a new manifest file in the dialer folder because some dialer tests can't pass on a DSDS setup.

At first, I wanted to check which phone number SIM was used thanks to plivo. But plivo intermittently add/remove some digits in my French phone number. Ex: "33123456789" sometimes becomes "133123456789" and sometimes "3312356789". In the end, I only checked the "Via SIM" label on the callscreen. Do you think it's enough, Doug?
Attachment #8517458 - Flags: review?(zcampbell)
Attachment #8517458 - Flags: review?(florin.strugariu)
Attachment #8517458 - Flags: review?(drs.bugzilla)
(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #8)
> I created a new manifest file in the dialer folder because some dialer tests
> can't pass on a DSDS setup.

Why is the new manifest needed? Shouldn't the job be only running tests that are marked as dsds in the manifest file, so we would avoid running any tests that would otherwise fail? Alternatively, what are the failures, and can we improve these tests so they can run on devices with muliple SIMs?
(In reply to Dave Hunt (:davehunt) from comment #9)
> Why is the new manifest needed?
My bad, I had a leftover of --type in the command line with which I tested. I updated the PR.

The tests that are failing are:
> TEST-UNEXPECTED-FAIL | test_dialer_add_contact.py TestDialerAddContact.test_dialer_add_contact
> | AssertionError: u'55527102717\nSIM1' != '55527102717'
> TEST-UNEXPECTED-FAIL | test_dialer_miss_call_from_known_contact_notification.py 
> TestReceiveCallFromKnownContactNotification.test_dialer_miss_call_from_known_contact_notification
> | AssertionError: u'(SIM 1) Missed call' != 'Missed call'
Seem easy to fix

> TEST-UNEXPECTED-ERROR | test_dialer_find_contact.py TestDialerFindContact.test_dialer_find_contact
> | NoSuchElementException: NoSuchElementException: Unable to locate element: .handled-call.outgoing
> TEST-UNEXPECTED-ERROR | test_dialer_set_up_conference_call.py TestSetUpConferenceCall.test_set_up_conference_call
> | NoSuchElementException: NoSuchElementException: Unable to locate element: .handled-call.outgoing
Might be related to the SIM Picker. I have to look into it.
Thanks. There certainly no need to fix those tests in this bug.
Attachment #8517458 - Flags: review?(florin.strugariu) → review+
QA Whiteboard: [fxosqa-auto-from-s3][fxosqa-auto-s4][fxosqa-auto-points=3] → [fxosqa-auto-from-s3][fxosqa-auto-from-s4][fxosqa-auto-s5][fxosqa-auto-points=3]
Attachment #8517458 - Flags: review?(zcampbell)
Comment on attachment 8517458 [details] [review]
Gaia PR

I'm arbitrarily setting r- as I think that this requires more discussion, though it may be ready to go as-is afterwards.

(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #8)
> At first, I wanted to check which phone number SIM was used thanks to plivo.
> But plivo intermittently add/remove some digits in my French phone number.
> Ex: "33123456789" sometimes becomes "133123456789" and sometimes
> "3312356789". In the end, I only checked the "Via SIM" label on the
> callscreen. Do you think it's enough, Doug?

In the Gaia code, we have a library called SimplePhoneMatcher that strips out international codes, + symbols, etc., and allows us to compare just local numbers to each other. Perhaps we could use something similar within the UI tests. I don't know if such a thing exists, but if not, it would be a helpful addition for the future.

Aside from that, I left some comments on the PR.
Attachment #8517458 - Flags: review?(drs.bugzilla) → review-
Comment on attachment 8517458 [details] [review]
Gaia PR

I updated the test case to use a parameterized test. Re-asking Bebe to check this change.

(In reply to Doug Sherk (:drs) (use needinfo?) from comment #12)
> In the Gaia code, we have a library called SimplePhoneMatcher that strips
> out international codes, + symbols, etc., and allows us to compare just
> local numbers to each other.

The main issue is that Plivo also removes a digit in the middle of the phone number like "33123456789" and "3312356789" (the 4 is missing, it took me some time to notice it). I tried about 20 times and I don't have the same result depending on the time where I run the tests. That's why I prefer avoiding to rely on it.
Attachment #8517458 - Flags: review?(florin.strugariu)
Attachment #8517458 - Flags: review?(drs.bugzilla)
Attachment #8517458 - Flags: review-
Attachment #8517458 - Flags: review+
Comment on attachment 8517458 [details] [review]
Gaia PR

(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #13)
> The main issue is that Plivo also removes a digit in the middle of the phone
> number like "33123456789" and "3312356789" (the 4 is missing, it took me
> some time to notice it). I tried about 20 times and I don't have the same
> result depending on the time where I run the tests. That's why I prefer
> avoiding to rely on it.

Ok, I see. I missed that in your initial explanation.
Attachment #8517458 - Flags: review?(drs.bugzilla) → review+
See Also: → 1104667
Attachment #8517458 - Flags: review?(florin.strugariu) → review+
You need to log in before you can comment on or make changes to this bug.