973690 - Use eideticker to measure "checkerboarding" on FirefoxOS

Status: RESOLVED FIXED

(Testing Graveyard :: Eideticker, defect)

Description

[William Lachance (:wlach)]

Filing this bug to track adding some kind of way to measuring "checkerboarding" (periods where we are waiting for the screen to redraw completely after a scroll) in FirefoxOS.

Comment 1

[William Lachance (:wlach)]

Attachment: Measure overall entropy during capture

Here's an initial attempt where we just get the sum of all the frame entropies in the whole capture. Since frames where there is checkerboarding have lower entropies, this should in theory tell us when we have regressed or improved on this benchmark.

Comment 2

[Mason Chang [Inactive] [:mchang]]

Comment on attachment 8383409 [details] [diff] [review]
Measure overall entropy during capture

Review of attachment 8383409 [details] [diff] [review]:
-----------------------------------------------------------------

::: src/eideticker/eideticker/metrics.py
@@ +45,4 @@
>          capture, threshold=analysis_props['animation_threshold'])
>      metrics['checkerboard'] = videocapture.get_checkerboarding_area_duration(
>          capture)
> +    metrics['overallentropy'] = videocapture.get_overall_entropy(capture)

Nit: add the space back between metrics and the if here.

Comment 3

[William Lachance (:wlach)]

Pushed with nit addressed: https://github.com/mozilla/eideticker/commit/b6ecbb54cdb3c1ddcc72ac4f0f9d13a16c599ed8

Going to leave this open for a few followups to make it easier to visualize entropy-related data and possibly change the reporting default to make these results more obvious for (e.g.) the scrolling tests.

Comment 4

[William Lachance (:wlach)]

This is as fixed as it's going to be for the forseeable future. Eideticker for FxOS is not under active development.