997248 - Semiauto Telephony tests fail on 1.3

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Certification Suite, defect)

Description

[Robert Wood [:rwood]]

The semi-auto webapi telephony tests (test_telephony_incoming, and test_telephony_outgoing) fail on Tarako 1.3T. These same tests pass on Inari with 1.3. Perhaps the 1.3T branch uses the newly changed telephony API code (in master) that uses promises?

Comment 1

[Hsin-Yi Tsai (she/her) | away [:hsinyi]]

(In reply to Robert Wood [:rwood] from comment #0)
> The semi-auto webapi telephony tests (test_telephony_incoming, and
> test_telephony_outgoing) fail on Tarako 1.3T. These same tests pass on Inari
> with 1.3. Perhaps the 1.3T branch uses the newly changed telephony API code
> (in master) that uses promises?

No, TelephonyAPI in 1.3T doesn't use 'Promise' yet.

Comment 2

[Robert Wood [:rwood]]

Thanks Hsin-Yi. I'll dig into it further and see what is happening. As :jgriffin suggested, perhaps it is an OOM issue with our CertTest App and the Dialer App.

Comment 3

[Robert Wood [:rwood]]

In the latest 1.3T build, the telephony tests get further than before (the tests successfully initiate/receive live calls) however the tests fail and test runner hangs when hanging up an active call. 

When TelephonyCall.hangUp() is called, Tarako runs out of memory and the device is killing the CertTest App itself, causing the test to fail (and the test runner hangs up indefinitely). Sometimes the active call is terminated successfully first (the 'ondisconnected' call event is received) and then the device runs OOM; other times the device runs OOM right after the TelephonyCall.hangUp() and kills the CertTest App and actually leaves the live phone call ongoing.

From logcat:

E/OomLogger(   83): [Kill]: select 374 (CertTest App), adj 10, size 5177, to kill
E/OomLogger(   83): [Kill]: send sigkill to 374 (CertTest App), adj 10, size 5177

Next I will try running the telephony tests in a different context (i.e. dialer) instead of the CertTest App and see if they work in that case.

Comment 4

[Robert Wood [:rwood]]

Update: I can get the semi-auto telephony tests to run successfully and pass on tarako if I modify /certapp/certapp.py to run the webapi test inside the context of the existing phone/communications app, instead of launching and using the certtest app.

The memory on tarako is just too minimal to be able to run the telephony tests (and possibly others) inside the certest app. Do we want to go the route where we run certain tests inside the existing gaia apps context, like in this case? Do we modify the semi-auto harness to use existing gaia apps based upon the particular webapi being tested? This could get messy, especially when it comes to permissions. Any other ideas how to solve this?

Comment 5

[Malini Das [:mdas] - Away, not checking bugmail]

(In reply to Robert Wood [:rwood] from comment #4)
> Update: I can get the semi-auto telephony tests to run successfully and pass
> on tarako if I modify /certapp/certapp.py to run the webapi test inside the
> context of the existing phone/communications app, instead of launching and
> using the certtest app.
> 
> The memory on tarako is just too minimal to be able to run the telephony
> tests (and possibly others) inside the certest app. Do we want to go the
> route where we run certain tests inside the existing gaia apps context, like
> in this case? Do we modify the semi-auto harness to use existing gaia apps
> based upon the particular webapi being tested? This could get messy,
> especially when it comes to permissions. Any other ideas how to solve this?

We originally wanted to have everything run in the certtest app because we don't have the guarantee that Gaia will be running on the phones that will be tested. Even if it is there, we won't know which apps will be there and what their permissions would be, or how to launch them.

I'm actually most interested in why it doesn't run in the certtest app itself. When you get a call, does the default communications application also get instantiated? If so, perhaps there's a way to suppress that?

Comment 6

[Robert Wood [:rwood]]

Yes, when the certtest app is running and the device receives an incoming call, the default communications app does start up and appears over top of the certest app. The certtest app / telehony test answers the call via the webapi, then terminates the call via the webapi, and at that point the gaia communcations app closes and returns to the certest app.

Same thing happens on outgoing calls, even though the certtest app uses the webapi to initiate the outgoing call, the default communications app starts over top the certest app and displays "dialing" etc. and stays up until the call is terminated.

That's a good question mdas, Hsin-Yi, is there a way to disable the default gaia phone/communications app so that it will not grab incoming/outgoing calls?

Comment 7

[Hsin-Yi Tsai (she/her) | away [:hsinyi]]

(In reply to Robert Wood [:rwood] from comment #6)
> Yes, when the certtest app is running and the device receives an incoming
> call, the default communications app does start up and appears over top of
> the certest app. The certtest app / telehony test answers the call via the
> webapi, then terminates the call via the webapi, and at that point the gaia
> communcations app closes and returns to the certest app.
> 
> Same thing happens on outgoing calls, even though the certtest app uses the
> webapi to initiate the outgoing call, the default communications app starts
> over top the certest app and displays "dialing" etc. and stays up until the
> call is terminated.
> 
> That's a good question mdas, Hsin-Yi, is there a way to disable the default
> gaia phone/communications app so that it will not grab incoming/outgoing
> calls?

Per my understanding communications app will be launched by System app which is monitoring telephony events, when there's a new call coming/outgoing. So, maybe we could try to disable there? Let's ask Etienne for further suggestions.

Comment 8

[Etienne Segonzac (:etienne)]

Everything go through the DialerAgent [1] now.
So doing a |window.dialerAgent.stop()| should do the trick (window being the system app window).

[1] https://github.com/mozilla-b2g/gaia/blob/master/apps/system/js/dialer_agent.js

Comment 9

[Robert Wood [:rwood]]

Thanks!

I can't seem to access the dialerAgent, from the test or from certapp.py. Do I need to add DialerAgent to the certtest permissions or something?

Comment 10

[Malini Das [:mdas] - Away, not checking bugmail]

(In reply to Robert Wood [:rwood] from comment #9)
> Thanks!
> 
> I can't seem to access the dialerAgent, from the test or from certapp.py. Do
> I need to add DialerAgent to the certtest permissions or something?

How are you trying to get the DialerAgent? on today's v1.3t, I can do:

m.switch_to_frame() # go to system app
m.execute_script("return window.wrappedJSObject.dialerAgent.stop();")

which works. I think you want to call this in the system app to prevent it from taking control

Comment 11

[Robert Wood [:rwood]]

Ahhh I was missing the "wrappedJSObject" from the ...dialerAgent.stop() call. Added that, and it works.

Closing this issue as I can get the telephony tests to run successfully now from within the certtest app, by disabling the system dialer agent as noted above. Thanks all!!

Comment 12

[Robert Wood [:rwood]]

Hi Etienne,

I cannot seem to disable the default gaia dialer agent in 1.4 on flame or hamachi. It works fine in 1.3 (tarako). Has there been a dialer agent change in 1.4? If so, is it still possible to disable it?

Thanks!

Comment 13

[Etienne Segonzac (:etienne)]

(In reply to Robert Wood [:rwood] from comment #12)
> Hi Etienne,
> 
> I cannot seem to disable the default gaia dialer agent in 1.4 on flame or
> hamachi. It works fine in 1.3 (tarako). Has there been a dialer agent change
> in 1.4? If so, is it still possible to disable it?
> 

The callscreen app is not currently part of 1.4 so stopping the DialerAgent won't work.
Can you apply a patch as part of this automation process?

If yes, the easiest way if probably to remove this line [1] from dialer.js.
The patch shouldn't bitrot since this part of the file shouldn't change.

If no, then we'll have to be more creative I guess :)

[1] https://github.com/mozilla-b2g/gaia/blob/v1.4/apps/communications/dialer/js/dialer.js#L375

Comment 14

[Robert Wood [:rwood]]

Hi Etienne, thanks for the info. No we can't apply a patch as the certsuite will run on partner devices. Looks like the same problem exists in the latest 1.3 now also. Any other ideas how to get around this?

Comment 15

[Robert Wood [:rwood]]

Note: Disabling the dialer agent still works on tarako 1.3T, but not on hamachi 1.3 or 1.4/master, updating the summary

Comment 16

[Etienne Segonzac (:etienne)]

(In reply to Robert Wood (PTO until Jul 10) [:rwood] from comment #14)
> Hi Etienne, thanks for the info. No we can't apply a patch as the certsuite
> will run on partner devices. Looks like the same problem exists in the
> latest 1.3 now also. Any other ideas how to get around this?

1.4 is finally getting the separate callscreen app!

Which leaves 1.3. Since we probably can't land new code on 1.3 nor add custom patches on top we're a bit empty handed. How strong is the demand for those tests to run on 1.3?

Comment 17

[Jonathan Griffin (:jgriffin)]

If we have a solution for 1.4 (and 1.3T still works) I think we can get away without running these on 1.3.

Comment 18

[Robert Wood [:rwood]]

Hi Etienne,

Where can I find the code in 1.4 for the new callscreen app (I've been digging around but can't seem to find it) and also is there an easy way to kill it, similar to how I was killing the old dialer agent? Thanks!

Comment 19

[Robert Wood [:rwood]]

Good news, disabling the new callscreen app now works in 1.4 by using the same call that was used to disable the old dialer agent (dialerAgent.stop) in 1.4.

In the patch for bug 1023345 I will enable the telephony tests once again for 1.4. Need to do it in that patch as the test code needs to be updated to use the new 1.4 dialing promise before being enabled for 1.4.

Leaving this bug open until the semiauto certsuite is fixed on tarako and I can check on 1.3T where we are at with regards to disabling the dialer.

Comment 20

[Robert Wood [:rwood]]

Good news, the telephony tests are running fine on tarako 1.3t once again (disabling the dialer is working again with the latest 1.3t build).

Comment 21

[Robert Wood [:rwood]]

The tests pass on tarako 1.3t, but fail to disable the dialer in the main 1.3 branch.

Comment 22

[Jonathan Griffin (:jgriffin)]

This is something we won't be able to fix, so we're intelligently disabling the tests on devices where we can't disable the dialer agent.