Closed Bug 1433390 Opened 2 years ago Closed 2 years ago

Enable assertion introspection for support.asserts module in wdspec tests

Categories

(Testing :: geckodriver, enhancement, P2)

Version 3
enhancement

Tracking

(firefox59 fixed, firefox60 fixed)

RESOLVED FIXED
mozilla60
Tracking Status
firefox59 --- fixed
firefox60 --- fixed

People

(Reporter: whimboo, Assigned: whimboo)

References

Details

Attachments

(2 files)

Log lines for failures like the following are pretty useless at the moment:

https://treeherder.mozilla.org/logviewer.html#?job_id=158638024&repo=try&lineNumber=1425

[task 2018-01-26T07:39:52.934Z] 07:39:52     INFO -             if e["type"] != "mousemove":
[task 2018-01-26T07:39:52.935Z] 07:39:52     INFO -                 assert e["pageX"] == point["x"]
[task 2018-01-26T07:39:52.935Z] 07:39:52     INFO - >               assert e["pageY"] == point["y"]
[task 2018-01-26T07:39:52.935Z] 07:39:52     INFO - E               AssertionError

There is basically no way to identify by the log what the problem is. I really want to see array introspection working here. While it is working fine if assertions are in the tests, it fails for helper modules like the above.

After searching a bit I found the following stackoverflow article:

https://stackoverflow.com/questions/41522767/pytest-assert-introspection-in-helper-function

We should enable that introspection for all of our helpers.
Summary: Enable assertion introspection for helpers of wdspec tests → Enable assertion introspection for support.asserts module in wdspec tests
Attachment #8945737 - Flags: review?(james)
Attachment #8945738 - Flags: review?(james)
Attachment #8945737 - Flags: review?(mjzffr)
Attachment #8945738 - Flags: review?(mjzffr)
Comment on attachment 8945738 [details]
Bug 1433390 - Use register_assert_rewrite to enable introspection in assertion helper.

https://reviewboard.mozilla.org/r/215844/#review221720
Attachment #8945738 - Flags: review?(james) → review+
Comment on attachment 8945737 [details]
Bug 1433390 - Move all assert helps into support.asserts.

https://reviewboard.mozilla.org/r/215842/#review221722

Assuming you tested the fll changeset actually works.
Attachment #8945737 - Flags: review?(james) → review+
Attachment #8945737 - Flags: review?(mjzffr)
Attachment #8945738 - Flags: review?(mjzffr)
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3f90bf2ca184
Move all assert helps into support.asserts. r=jgraham
https://hg.mozilla.org/integration/autoland/rev/955a54d5f82b
Use register_assert_rewrite to enable introspection in assertion helper. r=jgraham
https://hg.mozilla.org/mozilla-central/rev/3f90bf2ca184
https://hg.mozilla.org/mozilla-central/rev/955a54d5f82b
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
Small change which drastically helps in test failure discovery in log files. It would be great to see uplifted to beta.
Whiteboard: [checkin-needed-beta]
You need to log in before you can comment on or make changes to this bug.