1133580 - Autophone - support job retriggers and cancel notifications from Treeherder

Status: RESOLVED FIXED

(Testing Graveyard :: Autophone, defect)

Description

[Bob Clary [:bc] (inactive)]

Once bug 1113281 lands, Autophone should listen to the pulse exchange and allow Treeherder users to retrigger and cancel jobs and tests.

Comment 1

[Bob Clary [:bc] (inactive)]

Mark, I'm going to attach the individual patches here. I'm going to try to get my personal repo for autophone in sync and attempt to provide a github pr for the review request but wanted to record the patches before I mess up my repo. ;-)

Comment 2

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-1.patch

add utils.py

Comment 3

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-2.patch

use utils.py

Comment 4

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-3.patch

fix missing error in cache response in builds.py

Comment 5

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-4.patch

Remove autophonepulsebuildmonitor.py

Comment 6

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-5.patch

Add autophonepulsemonitor.py

Comment 7

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-6.patch

hookup new pulsemonitor, consolidate compatible build detection, use new jobs format in autophone.py

Comment 8

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-7.patch

implement new jobs format including new tests table in jobs.py

Comment 9

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-8.patch

remove enable_unittests as a BuildCache property in builds.py

Comment 10

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-9.patch

update autophonetreeherder.py
    
- begin divorcing PhoneWorkerSubProcess from AutophoneTreeherder
- make AutophoneTreeherder.submit_complete require tests.
- submit a privatebuild artifact to Treeherder to record the build used in a test
- catch potential exceptions while uploading logs

Comment 11

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-10.patch

update worker.py
    
- clean up device error/disconnection/disable states
- add new FETCHING state.
- move compatible job detection into autophone.py
- delete tests from the jobs database as they are completed.
- use test status RETRY when a test must be retried while
  creating a new job for the test(s) to be retried.
- process retrigger, cancel job actions

Comment 12

[Bob Clary [:bc] (inactive)]

Attachment: bug-1133580-v1-11.patch

incorporate job action processing in tests

Comment 13

[Bob Clary [:bc] (inactive)]

Attachment: PR 14

Mark, here goes.

Comment 14

[Mark Côté [:mcote]]

Comment on attachment 8581582 [details] [review]
PR 14

Made a bunch of comments; nothing serious, although there are a few areas that could be refactored to make them a bit less verbose and hence more readable.

Comment 15

[Bob Clary [:bc] (inactive)]

Attachment: PR 15

Comment 16

[Mark Côté [:mcote]]

Comment on attachment 8582473 [details] [review]
PR 15

Still think you should reduce the redundancy of all those test-result calls, but not a huge deal.

Comment 17

[Bob Clary [:bc] (inactive)]

Comment on attachment 8582473 [details] [review]
PR 15

Ok. this should do it.

Comment 18

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/37045563b2930e0e9a70bbbeb9f6fdd85d70499f
https://github.com/mozilla/autophone/commit/8080d9ae1b22720aa1f3c8850e9928ab5c0d06da
https://github.com/mozilla/autophone/commit/65f0067d9dcf2f3e7405574945a637b364d4dbff
https://github.com/mozilla/autophone/commit/2264d074a6bd72741778fe30a2e1c96322298f21
https://github.com/mozilla/autophone/commit/d78bc8dc2c56e4c305edd49799d68411b31ae570
https://github.com/mozilla/autophone/commit/0bd108b6f310979c214d2ba2a2588108fc9cabea
https://github.com/mozilla/autophone/commit/2e7c3ec57fcbfc0b1619a02f753c56c73e203dbf
https://github.com/mozilla/autophone/commit/4576a1ffbfea80c5eabc7b4f4fd616ab2f032296
https://github.com/mozilla/autophone/commit/19a86a16055c8d10d9f480e89aef68f34fd7dedf
https://github.com/mozilla/autophone/commit/b0a2e595a2a9c1b5ee780e3c1ae623da50da1d45
https://github.com/mozilla/autophone/commit/161d7c4244c3c0592f9806b0cbf9bdfdf6d104b7

Comment 19

[Bob Clary [:bc] (inactive)]

I screwed up one of the changes due to the review by removing the parens in a conditional. I run tests with it locally but did not see the error until I started the production instance of autophone. Self reviewed. Sorry.

https://github.com/mozilla/autophone/commit/5bbc94886ff85989858722f1ede1b4e6845a9904

Comment 20

[Bob Clary [:bc] (inactive)]

I'm not sure what is up, but there are new errors appearing in production which should have been caught during testing. I think I didn't handle the new pr process properly and have messed up the repository. Autophone production is currently down while I resolve this.

Comment 21

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/f122c4133e17f051389cd6af8e1c84360948c063

simple format string error in test/device/repo selection. This wasn't caught during testing since I wasn't using test/device/repo selection in the configuration. The error wasn't related to my use of pull requests or multiple repositiories at all. Autophone is currently up and running but is not responding to retrigger requests.

Comment 22

[Bob Clary [:bc] (inactive)]

Ok, these stuck and Autophone on production Treeherder now supports retriggers and cancels.