1158032 - Implement Frame Timing API

Status: RESOLVED WONTFIX

(Core :: DOM: Core & HTML, enhancement, P5)

Description

[Hiroyuki Ikezoe (:hiro)]

This bug will be a tracker bug to implement Frame Timing API.

The spec is: https://w3c.github.io/frame-timing/

Comment 1

[Hiroyuki Ikezoe (:hiro)]

Attachment: [WIP] buffer type API

* does not work on E10S yet.
* sourceFrameNumber is not set yet. (I have no idea how to deal with the number between main and compositor threads yet)
* PerformanceCompositeTiming does not work (I do not know why, need to investigate)
* not considered :visible change issue.


This patch breaks unified build, needs a patch. I will post the patch in a new bug for the unified build issue.

Comment 2

[Ilya Grigorik]

Awesome to see quick progress -- great stuff! :)

As a heads up, both Chrome and IE are planning to block and ship FT on Performance Observer. To be specific, the plan is to update the FT spec and remove the buffer based API entirely and only provide access to FT events via Performance Observer. Related GH bug: https://github.com/w3c/frame-timing/issues/20 - we should probably make this more clear in the spec itself as well.

[1] https://github.com/w3c/frame-timing/issues/20

Comment 3

[Hiroyuki Ikezoe (:hiro)]

Ilya, thanks for the info.
I've opened a new bug for Performance Observer and set the dependency. Bug 1165796.

My plan here is to implement buffer base Frame Timing API and confirm its basic behavior first, then rewrite it with observer.

Comment 4

[Hiroyuki Ikezoe (:hiro)]

Attachment: [WIP] buffer type API v2

Changes from previous one:

* Add basic mochitests.
* Works on e10s partially. (will not work in iframe)

Comment 5

[Ilya Grigorik]

Yep, makes sense.. We're taking the same approach in Chrome as well. Let me know if you run into any questions or issues while you're at it :)

Comment 6

[Hiroyuki Ikezoe (:hiro)]

Attachment: [WIP] buffer type API v3

Changes from v2
* Add mochitests in iframe
* Works in iframe

Comment 7

[Hiroyuki Ikezoe (:hiro)]

Attachment: [WIP] buffer type API v4

This patch relies on the patch for bug 1167487.

The buffer type API works fine in most cases.

Remaining issues

* layout/base/tests/chrome/test_bug533845.xul crashes on Windows 8 and XP.
* The startTime in PerformanceRenderTiming does not match rAF time on Android opt builds
* :visited link handling

Comment 8

[Hiroyuki Ikezoe (:hiro)]

For reference a try server result with attachment 8610286 [details] [diff] [review]:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=203fc1d4ced2

Comment 9

[Hiroyuki Ikezoe (:hiro)]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #7)
> * The startTime in PerformanceRenderTiming does not match rAF time on
> Android opt builds

It turned out that the failure is intermittent and happens both opt and debug debuilds. And the failure reason is that the main thread on Android emulator is too busy to get RenderTiming in mochitest.

Comment 10

[Hiroyuki Ikezoe (:hiro)]

I am going to handle :visited link issue in a separate bug. Bug 1169531.

Comment 11

[Hiroyuki Ikezoe (:hiro)]

Attachment: [WIP] buffer type API v5

This patch relies on attachment 8612703 [details] [diff] [review] for bug 1167487.

Changes from v4:

* Adapted to the latest nsPerformance changes by bug 1155761
* Fix mismatch of rAF time partially. Still fails on Android 2.3 emulator.
* Rewrite test with testharness.js

The crash on Windows XP/8 will be handled in another bug. I will post a patch against the crashed test in the bug.

Comment 12

[Hiroyuki Ikezoe (:hiro)]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #11)

> The crash on Windows XP/8 will be handled in another bug. I will post a
> patch against the crashed test in the bug.

Filed 1170448.

Comment 13

[Hiroyuki Ikezoe (:hiro)]

Attachment: [WIP] buffer type API v6

Changes from v5:
* Fix time lag between rAF time and render startTime.
* Add some test cases
* Split test into smaller test cases
  some of these tests could be merged into web platform tests
* Fix a problem that render event is recorded even if there is no paint event.

Comment 14

[Hiroyuki Ikezoe (:hiro)]

Attachment: [WIP] buffer type API v7

This patch no longer relies on the patch for bug 1167487
This patch makes web platform tests for Frame Timing API (with a few modifications for Firefox) [1] success.

[1] https://github.com/w3c/web-platform-tests/pull/1894

In the previous patch, each frame number is advanced in each vsync frame, but current Chromium implementation of Frame Timing API does not so. The behavior of the frame number on Chromium is similar to transaction id in gecko.  So I've changed out implementation to adapt the Chromium's frame number behavior.

An example of Frame Timing entries on a CSS animation

Firefox with this patch
=======
render
[ startTime,  frameNumber ]
[ 1049.048, 19 ]

composite
[ startTime,  frameNumber ]
[ 1066.255, 19 ]
[ 1082.349, 19 ]
[ 1099.668, 19 ]
[ 1115.791, 19 ]
[ 1132.987, 19 ]
[ 1149.133, 19 ]
[ 1166.272, 19 ]
[ 1182.384, 19 ]
[ 1199.567, 19 ]
[ 1215.962, 19 ]
[ 1232.884, 19 ]
[ 1249.018, 19 ]
[ 1266.123, 19 ]
[ 1282.230, 19 ]
[ 1299.262, 19 ]
[ 1316.366, 19 ]
[ 1332.454, 19 ]
[ 1349.563, 19 ]
[ 1365.671, 19 ]
[ 1382.788, 19 ]
[ 1398.863, 19 ]
[ 1415.947, 19 ]
[ 1432.043, 19 ]
[ 1449.169, 19 ]
[ 1466.384, 19 ]
[ 1482.530, 19 ]
[ 1499.671, 19 ]
[ 1515.883, 19 ]
[ 1533.045, 19 ]

Chromium 45.0.2438.0
========

render
[ startTime,  frameNumber ]
[ 1144.354, 2 ]
[ 1161.020, 2 ]
composite"
[ startTime,  frameNumber ]
[ 1144.354, 2 ]
[ 1161.020, 2 ]
[ 1177.686, 2 ]
[ 1194.369, 2 ]
[ 1211.034, 2 ]
[ 1227.717, 2 ]
[ 1244.393, 2 ]
[ 1261.053, 2 ]
[ 1277.704, 2 ] 
[ 1294.382, 2 ]
[ 1311.052, 2 ]
[ 1327.724, 2 ]
[ 1344.380, 2 ]
[ 1361.048, 2 ]
[ 1377.724, 2 ]
[ 1394.395, 2 ]
[ 1411.032, 2 ]
[ 1427.726, 2 ]
[ 1444.381, 2 ]
[ 1461.048, 2 ]
[ 1477.725, 2 ] 
[ 1494.367, 2 ]
[ 1511.055, 2 ]
[ 1527.703, 2 ]
[ 1544.386, 2 ]
[ 1561.058, 2 ]

As you can see there are some differences:

1. Firefox starts compositor events from a subsequent frame but Chromium starts it in the same frame.
2. Chromium renders something in the second frame.

Comment 15

[Ilya Grigorik]

Note that the spec has been updated to use the "slow-only" model:
- background: https://github.com/w3c/frame-timing/pull/48
- new draft: https://w3c.github.io/frame-timing/

Comment 16

[Tyler Gross]

Did this die?

Comment 17

[Tyler Gross]

I suppose I can provide more clarity as well:

I’m on the Photoshop for Web team at Adobe. We’re looking for a solid way to programmatically track frames per second on a number of devices. Right now I’m just doing this with requestAnimationFrame and timers, but it doesn’t feel terribly accurate.

Comment 18

[Brian Birtles (:birtles)]

I believe this spec has been abandoned. According to https://wicg.github.io/frame-timing/

This work is NO LONGER BEING PURSUED. It's left here for historical purposes.

Comment 19

[Brian Birtles (:birtles)]

Regarding requestAnimationFrame, you can use the timestamp passed-in the to callback to get the frame time (or query document.timeline.currentTime for the same value) although I agree it is less than ideal.

Comment 20

[Tyler Gross]

Thank you for these tips! These definitely look more accurate than using performance.now. I had never even heard of document.timeline.currentTime.

Comment 21

[Marco Castelluccio [:marco]]

If the spec has been abandoned, should we close this and linked bugs as WONTFIX?

Comment 22

[Brian Birtles (:birtles)]

Yes, sounds good.

Comment 23

[Brian Birtles (:birtles)]

Spec has been abandoned.