704325 - Submit update tests after each test, rather than all at the end

Status: RESOLVED INVALID

(Mozilla QA Graveyard :: Mozmill Automation, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

The motivation is that we can monitor the results during release testing rather than discovering any issues at the end.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2] (away July 24th - August 16th)]

What do you mean? Results are getting pushed after each update test. Means after a partial/full update path and after a possible (if selected) fallback update path.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

ashughes had told me the results were not available until the end.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2] (away July 24th - August 16th)]

(In reply to Nick Thomas [:nthomas] from comment #2)
> ashughes had told me the results were not available until the end.

No, results are visible / queryable immediately from our dashboard. You don't have to wait until the end of the whole test-run.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Awesome!

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2] (away July 24th - August 16th)]

A query to retrieve the results as JSON blob will be:

http://mozmill-release.brasstacks.mozilla.com/_view/update_reports?startkey=[%22All%22,%22All%22,%222011-11-22T23:59:59%22]&endkey=[%22All%22,%22All%22,%222011-11-21T00:00:00%22]&descending=true

More details about the query parameter can be found in the repository on Github:
https://github.com/whimboo/mozmill-dashboard/blob/master/_attachments/js/dashboard.js#L751