1206954 - Autophone - better handle failures to submit results to PhoneDash

Status: RESOLVED FIXED

(Testing Graveyard :: Autophone, defect)

Description

[Bob Clary [:bc] (inactive)]

https://treeherder.mozilla.org/#/jobs?filter-searchStr=autophone&exclusion_profile=false&repo=mozilla-beta&revision=62eace42cee0 caused the nexus 5 device to submit requests to phonedash for webappstartup which failed due to 400 Bad Request and it appears the s1s2 tests may have been failing due to unable to run fennec exceptions but it is not clear if they would have given up or just continued retrying.

Not sure what is up, but filing this for later investigation.

Comment 1

[Bob Clary [:bc] (inactive)]

Attachment: Phonedash - bug-1206954-webapi.debug.patch

print has no effect in webpy handlers. We need to use print >> web.webapi.debug.

Before:

192.168.1.50:33165 - - [22/Sep/2015 16:08:40] "HTTP/1.1 POST /api/s1s2/add/" - 400 Bad Request

After:

Request {u'productversion': u'', u'throbberstop': 5365.0, u'machineid': u'SCH-I535', u'testname': u'webappstartup', u'rejected': False, u'productname': u'org.mozilla.firefox_beta', u'cached': True, u'osver': u'4.0.4', u'throbberstart': 4064.0, u'starttime': 440.0, u'blddate': 1442860778, u'phoneid': u'samsung-gs3-3', u'bldtype': u'opt', u'revision': u'http://hg.mozilla.org/releases/mozilla-beta/rev/651a0b2fc204'}: productversion  is not clean
192.168.1.50:33165 - - [22/Sep/2015 16:08:40] "HTTP/1.1 POST /api/s1s2/add/" - 400 Bad Request

Comment 2

[Bob Clary [:bc] (inactive)]

Attachment: Phonedash - bug-1206954-empty-is-clean.patch

Yesterday the versioning of beta was fouled up which resulted in no version being reported. This totally broke Autophone. The problem was that the is clean check in the Phonedash handler considered an empty field to be unclean. While it isn't great that productversion was reported as empty, it should not have been rejected.

Comment 3

[Bob Clary [:bc] (inactive)]

Attachment: Phonedash - bug-1206954-empty-is-clean.patch

Comment 4

[Bob Clary [:bc] (inactive)]

Attachment: bug-1206954-autophone.patch

Don't raise an exception if we can't submit results to Phonedash.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8664551 [details] [diff] [review]
bug-1206954-autophone.patch

Review of attachment 8664551 [details] [diff] [review]:
-----------------------------------------------------------------

how can we monitor these errors?  I assume the status would be emailed out or placed on a dashboard in the future?

Comment 6

[Bob Clary [:bc] (inactive)]

They are currently mailed to the admins:

https://github.com/mozilla/autophone/blob/master/tests/perftest.py#L190

 I'll place you on the distribution list on the next deployment.

Comment 7

[Bob Clary [:bc] (inactive)]

Comment on attachment 8664542 [details] [diff] [review]
Phonedash - bug-1206954-webapi.debug.patch

mcote is on pto.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8664547 [details] [diff] [review]
Phonedash - bug-1206954-empty-is-clean.patch

Review of attachment 8664547 [details] [diff] [review]:
-----------------------------------------------------------------

cool!

Comment 9

[Bob Clary [:bc] (inactive)]

https://github.com/markrcote/phonedash/commit/8da556364bd10dbb22139ddc611871e5dea644fa
https://github.com/markrcote/phonedash/commit/f931fe8558b86e425163b80cbc4445df53bf1a74

deployed to phonedash.mozilla.org, phonedash-dev.allizom.org

Comment 10

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/8932b011e95e850c0cbf98f4e712399adfd50261