1367874 - nexus-4-9 is not quitting but is instead waiting to be killed during s1s2tests

Status: RESOLVED FIXED

(Testing Graveyard :: Autophone, defect)

Description

[Bob Clary [:bc] (inactive)]

The reason that nexus-4-9 / remote-nytimes is falling so far behind is that it is not quitting pages as it should.

https://treeherder.mozilla.org/logviewer.html#?job_id=101337571&repo=mozilla-central&lineNumber=269

2017-05-25 08:47:53,439       nexus-4-9 INFO     S1S2TestJob mozilla-central 20170523195543 opt api-17 android-api-15 remote-nytimes killing org.mozilla.fennec

This means we wait for up to a minute for each page load before killing fennec and attempting the next load. This takes the time to run the test from 6-7 minutes to 20 minutes or more.

nexus-4-7 / remote-twitter, nexus-4-8 / remote-blank do not have this problem.

Comment 1

[Bob Clary [:bc] (inactive)]

regression:

bad  20170512114218 https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=20e72e5cc536b4ad9e041f3ae298101abb0a3c68
good 20170505120444 https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=d69be43cf123d41a98629440c4a7b5fdd4b4fb11

affects samsung gs3 sdk15 and nexus 4 sdk17.

working on bisecting now. snorp?

Comment 2

[Bob Clary [:bc] (inactive)]

Actually, I'm not sure what is up. mozregression is kind of broken due to

E/PackageManager(  538): Package org.mozilla.fennec_aurora has no signatures that match those in shared user org.mozilla.
fennec.sharedID; ignoring!
W/PackageManager(  538): Package couldn't be installed in /data/app/org.mozilla.fennec_aurora-1.apk

and manually I get inconsistent results...

Comment 3

[Bob Clary [:bc] (inactive)]

Attachment: bug-1367874.patch

The basic issue is that Android 4.0/4.2 fire the onload event properly but
the call to AutophoneQuitter.quit does not succeed for the nytimes page which contains a reference to a missing script file and throws exceptions due to errors.

We work around the problem by invoking the quit method from a setTimeout 5 seconds after the script is loaded. I've also switched the load handler to being attached to the document rather than the window object. This has resolved the issues locally for all remote s1s2 tests.

r=self.

Comment 4

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/7e1beca7af430c4454c9f882e94b25d659f88965
deployed 2017-05-27 00:00:00

Comment 5

[Bob Clary [:bc] (inactive)]

Although this worked reliably locally, this does not work for nexus-4-9. We will have to work around it another way. 

We can't use am set-inactive followed by am force-stop since am set-inactive is not available until android 6.0 sdk 23.

By the time we call wait_for_fennec after running the test, we really want the app to terminate immediately. We can change the wait time for wait_for_fennec and kill the app quicker if necessary. I'll test this out locally.

Comment 6

[Bob Clary [:bc] (inactive)]

Attachment: comment out the eideticker.js reference

Commenting out the inclusion of the missing eideticker.js file appears to fix the issue here locally without needing to change quitter.js.

I hate to change the test file, but if this works with no other change it is the best choice.

Comment 7

[Bob Clary [:bc] (inactive)]

Actually that is wrong. I was testing master which had the quitter.js change still in effect. More testing is required.

Comment 8

[Bob Clary [:bc] (inactive)]

Attachment: bug-1367874-revert-quitter.patch

The previous patch did not help as much as I had hoped. Also, I changed the local version of nytimes to not have js errors and it had no effect either.

reverting, r=self.

Comment 9

[Bob Clary [:bc] (inactive)]

Attachment: bug-1367874-am-stop.patch

Use am to bring the home screen to the front putting fennec/geckoview into the background, then use am to force stop, kill and if necessary the shell kill command to kill the process.

Comment 10

[Bob Clary [:bc] (inactive)]

PS. Also shortens the period we wait during wait_for_fennec.

https://github.com/mozilla/autophone/commit/c10579c681cc1e1f46f104c0839d3d054b5814ce
https://github.com/mozilla/autophone/commit/089394c9fa276b9008bb1002e5e49c0eb4441e97

We'll still be killing the process on nexus 4 but it will be cleaner and won't take as long.

deployed 2017-05-28 23:55