bug 1133580 caused several regressions. * The test status reported to Treeherder is not set properly upon failure. Some tests which fail are being reported as Green even though job details shows an error. * The PhoneTest.job_details properly is not reinitialized when a test begins running. Previously the job_details was initialized in AutophoneTreeherder.submit_pending, but now that submit_pending is called from autophone.py when the build is first detected, and submit_running/submit_complete are called on a different when the test runs/completes, the job_details is not initialized. Other issues which are not really regressions: * #moc has implemented rate limiting on ftp.m.o and the utils.get_remote_text can fail due to 503 errors especially during trigger_runs.py from outside the moco network. They suggest retrying after random waits.
Created attachment 8583271 [details] [review] pr 16 - v1 Due to the severe nature of the bugs, I have been running this in production since about 11 AM PT. You can see the example results at: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=autophone&exclusion_profile=false https://treeherder.mozilla.org/#/jobs?repo=fx-team&filter-searchStr=autophone&exclusion_profile=false Note the mochitest exceptions do not appear to be related to the patch but instead appear to be related to sutagent issues. I'm reaching out to stephend for help investigating the devices in the lab.
Attachment #8583271 - Flags: review?(mcote)
For an example of the mochitests running without sutagent issues see a result from my local testing on staging: https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&revision=6465d9a4d0dc&exclusion_profile=false&filter-searchStr=autophone
https://github.com/mozilla/autophone/commit/57123d38251d60ad42a5ac0787b37492150920c7 https://github.com/mozilla/autophone/commit/9e37b83d5bbb0f6f0e9e445044d3d8b7c89f2c36 https://github.com/mozilla/autophone/commit/57123d38251d60ad42a5ac0787b37492150920c7
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.