Need a reference for verifying performance measurement

RESOLVED WONTFIX

Status

Testing Graveyard
Eideticker
P3
normal
RESOLVED WONTFIX
4 years ago
4 months ago

People

(Reporter: geo, Unassigned)

Tracking

({perf})

Details

(Whiteboard: [c=automation p= s= u=])

Filing under Eideticker, though this might be more generally useful.

We should have a reference tool for validating Eideticker external measurement (or any other technique we use to do the same). That way we can better understand any potential inaccuracy, as well as verify the correctness of Eideticker's measurements after code changes.

The tool should be capable of reproducing the various scenarios Eideticker measures in a highly predictable fashion. The analogy would be having a tuning fork available to test the accuracy of an electronic tuner.

Currently the scenarios to reproduce would be:

fps -- display an animation (preferably a canned scrolling and/or panning animation, since that's what's most typically measured) at various speeds, like 5fps, 10fps, 15fps, 30fps, 60fps.

timetostableframe -- run through a number of variable frames for a set amount of time, then land on an unchanging frame. Amount of time can be varied.

I'll go out on a limb for a couple of suggestions for the tool name: Eidetuner or, my favorite, TuningFox.
Summary: Need a reference for verify performance measurement → Need a reference for verifying performance measurement
Adding the flashing corners demo as a dependency. Think we can probably figure out more tests too, so not closing out as a dupe yet.
Depends on: 962728

Updated

4 years ago
Keywords: perf
Whiteboard: [c=automation p= s= u=]
Priority: -- → P3
See also - 874781 - Should add flashing corners at every frame. If you can test that with a High Speed Camera, then we should be able to verify FPS.
See Also: → bug 874781
(In reply to Mason Chang [:mchang] from comment #2)
> See also - 874781 - Should add flashing corners at every frame. If you can
> test that with a High Speed Camera, then we should be able to verify FPS.

Is that guaranteed to flash at 60 fps no matter what?

The flashing corners demo, for example, turns out not to run at an actual FPS at all. 

The rough conclusion I've come to so far is our reference will need to be built outside FxOS in native code with guaranteed framerate--since FxOS is the system under test, assuming it's stable enough to be a reference is problematic unless we identify an older version that's rock solid.
"at an actual 60 FPS" re: flashing corners. Problem there is that requestAnimationFrame doesn't run at 60fps reliably on Gecko on any platform, afaict.

Comment 5

4 months ago
Eideticker has been discontinued, see bug 1361056
Status: NEW → RESOLVED
Last Resolved: 4 months ago
Resolution: --- → WONTFIX

Updated

4 months ago
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.