1512531 - Investigate adding a "rapid screenshot" or "screen recording" mode when running tests

Status: NEW

(Testing :: General, enhancement, P3)

Description

[Andrew Halberstadt [:ahal]]

I was watching an Iris demo, and it has a neat little feature that records screenshots throughout the duration of the test, so you can debug what happened if something goes wrong. The images show up in a little carousel gui that you can click next on to rapidly scroll through them.

This got me thinking about when we try to debug intermittent timeouts and the like in CI. Some harnesses like mochitest take a single screenshot on timeout, but this might miss something when it goes wrong as you only get to see the end state (which is normally just a blank page).

What if we had a flag which tells the harnesses to take continuous screenshots (or better yet record the screen) for a given test? This would give us a better sense of what actually happened. Did a Windows update dialog appear and subsequently get hidden behind the browser? We could have a view of the entire manifest run from browser startup all the way to the issue (or shutdown).

This couldn't be enabled by default, but we could add a --capture flag or something similar that developers (or ourselves) could pass in from |mach try|.

This bug is just a brain dump so the idea isn't lost to the ether.

Comment 1

[Geoff Brown [:gbrown]]

Android test tasks support "timed screenshots", which could easily be generalized to desktop:

https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/testing/mozharness/configs/android/android_common.py#55-58

That's fairly new. So far they haven't been very useful. I also tried screen recording (video) for Android, but the artifacts were huge after just a couple of minutes.