672954 - Add support for handling the new soft assertions (expect)

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Result Dashboard, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

With the new assertions backported to the current shared modules on bug 668810, we have to update the dashboard code to handle those soft assertions (expect). Failures are getting listed as fail and not exception. We have to support both types.

Comment 1

[u415141]

A Pivotal Tracker story has been created for this Bug: https://www.pivotaltracker.com/story/show/16049031

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I will attach two patches on this bug. The first one will be for the Sammy application, while the second one will take care of the CouchDB views.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Patch v1 (sammy) [checked-in]

Patches the report views of all test-runs to show messages of the new assertion types. Also it now supports multiple failure messages per test function.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 547447 [details] [diff] [review]
Patch v1 (sammy) [checked-in]

Landed as:
https://github.com/whimboo/mozmill-dashboard/commit/748f8d1c23214eb51877d1a3141491a1343430fc

Also changes are now live on all dashboard instances.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Patch v1 (couchdb views)

Changes to the couchdb views.

We don't need the platform branch anymore, so I have removed it from everywhere. Also for soft assertions only the functional failures view have had to be updated.

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 547543 [details] [diff] [review]
Patch v1 (couchdb views)

As mentioned on IRC the top failures don't match up. Where there should be two different failures occurring twice, the same failure is reported four times.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Patch v2 (couchdb views)

Not sure why but when I create the r object outside of the loop and set the message via r.message it will always be the one from the last iteration. Could be a bug in the spidermonkey version of CouchDB.

For a live view see:
http://mozmill-archive.brasstacks.mozilla.com/#/functional/failure?branch=All&platform=All&from=2011-07-14&to=2011-07-21&test=%2FtestSessionStore%2FtestUndoTabFromContextMenu.js&func=testUndoTabFromContextMenu

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Landed as:
https://github.com/whimboo/mozmill-dashboard/commit/84cdab77189740a7d9d5c649d87941a9048e1915