1112191 - Add a test to check that touchtones are responsive during phonecall

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Johan Lorenzo [:jlorenzo]]

We saw that regression in bug 1039844. This is a critical feature of a phone. 

Plivo allows us to record a call[1]. Once it's created, you can download to all the records made[2] in MP3 (or wav). To check that a single DTMF tone are working, we can download one of the record and verify the amplitude, the frequency and the length of the silence. Then we would repeat the same test for the 12 tones present on the keypad.

In other words:
STR
1. Call plivo
2. Record it from there
3. Play a single tone
4. End the recording
5. End the call
6. Download the record
7. Check amplitude
8. Check frequency
9. Check silence lenght

[1] https://www.plivo.com/docs/api/call/record/
[2] https://www.plivo.com/docs/api/recording/

Comment 1

[Johan Lorenzo [:jlorenzo]]

I started a draft and this might not be doable with the current Plivo. I was able to get a record but the record didn't contain any DTMF tone. That's unexpected because they do support these tones from their service to a regular phone[1]. I reproduced the issue by tapping the keypad with my fingers and I only heard the impacts of the fingers on the screen[2].

I contacted Plivo to see if DTMF tones are actually implemented from a phone to their service.

[1] https://www.plivo.com/docs/api/call/dtmf/
[2] http://s3.amazonaws.com/recordings_2013/3726de02-87a4-11e4-a4bc-bc764e088315.mp3

Comment 2

[Johan Lorenzo [:jlorenzo]]

Retested today, Plivo didn't record the tones. No news from them. It's more prudent to drop that task until we have a response from them.

Comment 3

[Johan Lorenzo [:jlorenzo]]

I've got an answer by email today (ticket #13303):
> Hello Johan,
> 
> Unfortunately there is no option to record DTMF tones on call.
> 
> Regards,
> Priya 
> Plivo Support Team

Please reopen this bug if this feature appears in the future.

Comment 4

[Johan Lorenzo [:jlorenzo]]

I checked if Twilio[1] was able to record DTMF tones. They behave exactly the same as Plivo. One thing I haven't noticed with Plivo is that they allow to recognize which digit is typed[2]. This might help to write this test. I reopen this bug to see if we can go this way.

[1] https://www.twilio.com/
[2] https://www.plivo.com/docs/xml/getdigits/

Comment 5

[Johan Lorenzo [:jlorenzo]]

Attachment: Gaia PR

I couldn't get you in touch over IRC. Feel free to redirect the reviews.

Comment 6

[Johan Lorenzo [:jlorenzo]]

Attachment: Plivo Handler - Server

This is the server created to handle the requirement asked by Plivo if you want to get the digits input[1]: you need to have an HTTP server accessible from Plivo's server. As the Jenkins is only accessible through the VPN, I hosted this server there[2]. 

// How does it work?

[2] behaves like a standard REST API. 
When Plivo has finished receiving touch tones, it makes a POST to[2] with a body containing: the CallUUID, the Digits recognized, and some other details that are not relevant for the moment.
This server receives these 2 pieces of data and store them in its database. So from now on, the call information is accessible at [3]
So, in the Gaia test, we wait until [3] is accessible. Once it is, it gets the digits stored and compares them to what has been input on the keypad.

After the test, we clean up the resource at [3] with a DELETE request.


// Implementation details

As we're moving towards MarionetteJS, I chose NodeJS + ExpressJS. This choice was also motivated by the fact that a generator[4] is also present for this king of app. I used it and cleaned up what was not necessary for us.
Finally, I used MongoDB as a database here. The main reason is we only store 2 fields for now, but this might change in the future. So maintaining a database schema sounded like a big task with not much value. On the other hand, MongoDB was already present in the generator, which allowed me to not spend too much time with setting up a DB connection and its transactions.


// Additional notes

The link given is not a regular pull request. It points to a set of commits on my personal repo (I removed the commits coming from the generator or the clean up). If we're good to go with the current implementation, I propose to transfer it to the Mozilla-B2G account. If not, I'll add a bunch of new commits. 


What do you guys think?


[1] https://www.plivo.com/docs/xml/getdigits/
[2] http://plivo-handler.paas.allizom.org/api/v1/calls
[3] http://plivo-handler.paas.allizom.org/api/v1/calls/:callUuid
[4] https://github.com/DaftMonk/generator-angular-fullstack

Comment 7

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #6)

> What do you guys think?

Wow! This is a heck of a thing.

Before we stand up a server like this, I think we need to ask and answer a few questions though:

1) Who will officially "own" the server?
2) What's the maintenance going to be?
3) Where do we host it--does it stay on Allizom?
4) Are there security concerns? Do we need a review?
5) How many tests will this benefit or enable?

Ultimately, I was hoping that the grand majority of standing infrastructure would be owned by the FxOS Automation Team, with us as consumers. That would keep us focused on testing rather than system administration and maintenance. 

I'm NI'ing Dave for comment as to what might make sense, though I know such a conversation would almost certainly have to go further.

Comment 8

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

Should say, just so that I'm not sounding dismissive, that I do see a number of possibilities for a general call-control server, which is what this could be extended to using Plivo XML. 

Essentially it could be used as a very powerful mock object for receiving calls, if rather than specializing to getDigits we allowed it to take a chain of request/response a la expect (the scripting language). 

Add a test "session ID" which is put on the callback URL, and it could control a number of telephony tests simultaneously without needing a ton of state, chain aside--receive a callback with session ID, reference the current chain link for that ID, validate response and send next chained XML request, repeat.

So yeah, lots we could do with this, more a question of whether we expect to do so and what the costs are.

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Geo Mealer [:geo] from comment #7)
> (In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #6)
> 
> > What do you guys think?
> 
> Wow! This is a heck of a thing.
> 
> Before we stand up a server like this, I think we need to ask and answer a
> few questions though:
> 
> 1) Who will officially "own" the server?
> 2) What's the maintenance going to be?
> 3) Where do we host it--does it stay on Allizom?
> 4) Are there security concerns? Do we need a review?
> 5) How many tests will this benefit or enable?
> 
> Ultimately, I was hoping that the grand majority of standing infrastructure
> would be owned by the FxOS Automation Team, with us as consumers. That would
> keep us focused on testing rather than system administration and
> maintenance. 
> 
> I'm NI'ing Dave for comment as to what might make sense, though I know such
> a conversation would almost certainly have to go further.

I have the same questions. There are obvious concerns with tests relying on another service, however if the scope is understood I don't see this being an issue so long as the service is hosted and maintained sensibly. Also, how much does this tie us into Plivo, and would it make changing provider difficult? Adding James, who may have some further thoughts on this.

It may also be worth asking our device lab partner if they have any experience with such tests.

Comment 10

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8549746 [details] [review]
Gaia PR

Clearing the reviews while we discuss the questions raised.

Comment 11

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8549748 [details]
Plivo Handler - Server

See comment 10.

Comment 12

[Johan Lorenzo [:jlorenzo]]

Thank you for bringing these points up.

> 1) Who will officially "own" the server?
The QA team might need to add a couple of new features in order to enable new tests. But I agree on the infrastructure point. 


> 2) What's the maintenance going to be?
From a development standpoint, it might be driven by the QA team's need. Adding new end points to the REST API is facilitated by the generator which gives you these files[1] with this command[2]. After that, we need to specify the model, add some unit tests, and if needed, tweak the controller (like to handle a specific request from Plivo). 
About the build: it's currently automated by grunt.
Concerning the deployment, it depends on where we host it. Currently, with Stackato, I haven't found a way to generate some keys to publish the server. I have a to manually run "stackato push". 

[1] https://github.com/JohanLorenzo/plivo-handler/tree/master/server/api/call
[2] https://github.com/DaftMonk/generator-angular-fullstack#endpoint

> 3) Where do we host it--does it stay on Allizom?
Mozilla already has some of prod apps running there[4]. They don't provide a "production grade" SLA though[4]. We can roughly estimate the number of request by D*t*R/(T*Δ), with:

D, the number of devices running at a given time
t, the number of tests using this server
T, the grand total of tests
R, the number of request made during a test
Δ, the duration of a test, in minutes

Taking this worst case scenario of:
D=900, the max number of devices in the next lab
t=10 (see question 5)
T=100 (I underestimated the current number of functional tests)
R=10 (with the current implementation of the server, it takes 3 requests to get the digits)
Δ=1 minute

We would end up with 900*10*10/(100*1) = 900 requests/minute. It may be worth it to ask IT about what kind of load they're able to support on PaaS or not.

[4] https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=30081453#MozillaPaaS%28Stackato%29-Kindsofapplications?


> 4) Are there security concerns? Do we need a review?
For now, the data posted by Plivo contains:
* Like said above, the CallUUID and the digits entered
* The remote phone number
* Our plivo phone number
* The direction of the call
* The caller's name we set here[5]
* The bill rate per minute
* The call status (in-progress/done)
* The UUID of the request made by Plivo.
* The event name which triggered this request (in the digit case: the event is call "Redirect", like XML attribute defined in the GetDigits Documentation[6]).

I think the remote phone number is the most sensible information here, especially if we want to avoid spam on our phone. So, this sounds like a good idea to set up an HTTPS connection and hence, asking for a security review (at least for this point).


[5] https://github.com/mozilla-b2g/gaia/blob/74bc79d1ff6e0a643d08910b6e99feeba73ec2a5/tests/python/gaia-ui-tests/gaiatest/utils/plivo/plivo_util.py#L45
[6] https://www.plivo.com/docs/xml/getdigits/

> 5) How many tests will this benefit or enable?
I haven't dug into Moztrap to get a number, yet. On top of my head, I currently see these new tests:
 * Check that the mute button doesn't silence the DTMF tones.
 * Same thing while using the speaker.
 * The DTMF tones shouldn't be heard on the other side if the call is paused
 * Same thing when the Flame switches to another call
 * Or check that the tones are heard during a Conf Call

> How much does this tie us into Plivo, and would it make changing provider difficult? 
A serious alternative to Plivo would be Twilio. It works relies on the same concepts as Plivo: You need to provide an XML to describe how it answers the call[7] and you need to have a server to get some responses like the digits entered[8]. The attributes are different and sometimes the grammar differs but changing to them wouldn't cost a big architectural modification.

I haven't found any other concurrent which enables us to place phone calls to regular phone numbers[9].

[7] https://www.twilio.com/docs/api/twiml
[8] https://www.twilio.com/docs/api/twiml/gather
[9] http://alternativeto.net/software/twilio/ http://alternativeto.net/software/plivo/

> It may also be worth asking our device lab partner if they have any
> experience with such tests.
Agreed

Comment 13

[Johan Lorenzo [:jlorenzo]]

> So, this sounds like a good idea to set up an HTTPS connection
> and hence, asking for a security review (at least for this point).

I just noticed that HTTPS is already delivered: https://plivo-handler.paas.allizom.org/api/v1/calls
We would need to force the redirection from HTTP to HTTPS then.

Comment 14

[Johan Lorenzo [:jlorenzo]]

NI James to get his point of view about the ownership.

Comment 15

[James Lal [:lightsofapollo]]

Spoke /w Johan over IRC... IMO QA (or some group of owners) would "own" this but we can deploy it in such a way to minimize uptime issues/maintenance..

We have a new corp heroku account setup and papertrail accounts which can provide good uptime/alerts which should make this fairly easy to keep running... 

I don't have the entire context here but bundling this as a docker thing and shipping it alongside your tests is also possible in the near future which would completely avoid maintenance/uptime burden but you still may have issues if we are relying on a remote service.

Comment 16

[Johan Lorenzo [:jlorenzo]]

Gaia UI tests are not run or maintained anymore.