Last Comment Bug 1158032 - Implement Frame Timing API
: Implement Frame Timing API
Status: NEW
: dev-doc-needed
Product: Core
Classification: Components
Component: DOM (show other bugs)
: unspecified
: Unspecified Unspecified
-- normal with 1 vote (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
: Andrew Overholt [:overholt]
Mentors:
https://w3c.github.io/frame-timing/
Depends on: 1169531 1198548 1158731 1164366 1165796 1168726 1191178 1197003
Blocks: 1163901
  Show dependency treegraph
 
Reported: 2015-04-24 00:25 PDT by Hiroyuki Ikezoe (:hiro)
Modified: 2016-07-11 23:26 PDT (History)
15 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
[WIP] buffer type API (36.51 KB, patch)
2015-05-13 01:28 PDT, Hiroyuki Ikezoe (:hiro)
no flags Details | Diff | Splinter Review
[WIP] buffer type API v2 (46.46 KB, patch)
2015-05-17 23:14 PDT, Hiroyuki Ikezoe (:hiro)
no flags Details | Diff | Splinter Review
[WIP] buffer type API v3 (49.60 KB, patch)
2015-05-19 20:41 PDT, Hiroyuki Ikezoe (:hiro)
no flags Details | Diff | Splinter Review
[WIP] buffer type API v4 (57.75 KB, patch)
2015-05-25 17:38 PDT, Hiroyuki Ikezoe (:hiro)
no flags Details | Diff | Splinter Review
[WIP] buffer type API v5 (60.67 KB, patch)
2015-06-02 00:23 PDT, Hiroyuki Ikezoe (:hiro)
no flags Details | Diff | Splinter Review
[WIP] buffer type API v6 (67.17 KB, patch)
2015-06-17 23:40 PDT, Hiroyuki Ikezoe (:hiro)
no flags Details | Diff | Splinter Review
[WIP] buffer type API v7 (72.16 KB, patch)
2015-06-21 18:53 PDT, Hiroyuki Ikezoe (:hiro)
no flags Details | Diff | Splinter Review

Description User image Hiroyuki Ikezoe (:hiro) 2015-04-24 00:25:28 PDT
This bug will be a tracker bug to implement Frame Timing API.

The spec is: https://w3c.github.io/frame-timing/
Comment 1 User image Hiroyuki Ikezoe (:hiro) 2015-05-13 01:28:05 PDT
Created attachment 8605118 [details] [diff] [review]
[WIP] buffer type API

* does not work on E10S yet.
* sourceFrameNumber is not set yet. (I have no idea how to deal with the number between main and compositor threads yet)
* PerformanceCompositeTiming does not work (I do not know why, need to investigate)
* not considered :visible change issue.


This patch breaks unified build, needs a patch. I will post the patch in a new bug for the unified build issue.
Comment 2 User image Ilya Grigorik 2015-05-13 08:59:18 PDT
Awesome to see quick progress -- great stuff! :)

As a heads up, both Chrome and IE are planning to block and ship FT on Performance Observer. To be specific, the plan is to update the FT spec and remove the buffer based API entirely and only provide access to FT events via Performance Observer. Related GH bug: https://github.com/w3c/frame-timing/issues/20 - we should probably make this more clear in the spec itself as well.

[1] https://github.com/w3c/frame-timing/issues/20
Comment 3 User image Hiroyuki Ikezoe (:hiro) 2015-05-17 23:09:27 PDT
Ilya, thanks for the info.
I've opened a new bug for Performance Observer and set the dependency. Bug 1165796.

My plan here is to implement buffer base Frame Timing API and confirm its basic behavior first, then rewrite it with observer.
Comment 4 User image Hiroyuki Ikezoe (:hiro) 2015-05-17 23:14:42 PDT
Created attachment 8606860 [details] [diff] [review]
[WIP] buffer type API v2

Changes from previous one:

* Add basic mochitests.
* Works on e10s partially. (will not work in iframe)
Comment 5 User image Ilya Grigorik 2015-05-18 07:55:52 PDT
Yep, makes sense.. We're taking the same approach in Chrome as well. Let me know if you run into any questions or issues while you're at it :)
Comment 6 User image Hiroyuki Ikezoe (:hiro) 2015-05-19 20:41:52 PDT
Created attachment 8607918 [details] [diff] [review]
[WIP] buffer type API v3

Changes from v2
* Add mochitests in iframe
* Works in iframe
Comment 7 User image Hiroyuki Ikezoe (:hiro) 2015-05-25 17:38:18 PDT
Created attachment 8610286 [details] [diff] [review]
[WIP] buffer type API v4

This patch relies on the patch for bug 1167487.

The buffer type API works fine in most cases.

Remaining issues

* layout/base/tests/chrome/test_bug533845.xul crashes on Windows 8 and XP.
* The startTime in PerformanceRenderTiming does not match rAF time on Android opt builds
* :visited link handling
Comment 8 User image Hiroyuki Ikezoe (:hiro) 2015-05-25 17:40:58 PDT
For reference a try server result with attachment 8610286 [details] [diff] [review]:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=203fc1d4ced2
Comment 9 User image Hiroyuki Ikezoe (:hiro) 2015-05-28 00:38:13 PDT
(In reply to Hiroyuki Ikezoe (:hiro) from comment #7)
> * The startTime in PerformanceRenderTiming does not match rAF time on
> Android opt builds

It turned out that the failure is intermittent and happens both opt and debug debuilds. And the failure reason is that the main thread on Android emulator is too busy to get RenderTiming in mochitest.
Comment 10 User image Hiroyuki Ikezoe (:hiro) 2015-05-28 22:57:00 PDT
I am going to handle :visited link issue in a separate bug. Bug 1169531.
Comment 11 User image Hiroyuki Ikezoe (:hiro) 2015-06-02 00:23:51 PDT
Created attachment 8613901 [details] [diff] [review]
[WIP] buffer type API v5

This patch relies on attachment 8612703 [details] [diff] [review] for bug 1167487.

Changes from v4:

* Adapted to the latest nsPerformance changes by bug 1155761
* Fix mismatch of rAF time partially. Still fails on Android 2.3 emulator.
* Rewrite test with testharness.js

The crash on Windows XP/8 will be handled in another bug. I will post a patch against the crashed test in the bug.
Comment 12 User image Hiroyuki Ikezoe (:hiro) 2015-06-02 00:42:50 PDT
(In reply to Hiroyuki Ikezoe (:hiro) from comment #11)

> The crash on Windows XP/8 will be handled in another bug. I will post a
> patch against the crashed test in the bug.

Filed 1170448.
Comment 13 User image Hiroyuki Ikezoe (:hiro) 2015-06-17 23:40:41 PDT
Created attachment 8624050 [details] [diff] [review]
[WIP] buffer type API v6

Changes from v5:
* Fix time lag between rAF time and render startTime.
* Add some test cases
* Split test into smaller test cases
  some of these tests could be merged into web platform tests
* Fix a problem that render event is recorded even if there is no paint event.
Comment 14 User image Hiroyuki Ikezoe (:hiro) 2015-06-21 18:53:31 PDT
Created attachment 8625217 [details] [diff] [review]
[WIP] buffer type API v7

This patch no longer relies on the patch for bug 1167487
This patch makes web platform tests for Frame Timing API (with a few modifications for Firefox) [1] success.

[1] https://github.com/w3c/web-platform-tests/pull/1894

In the previous patch, each frame number is advanced in each vsync frame, but current Chromium implementation of Frame Timing API does not so. The behavior of the frame number on Chromium is similar to transaction id in gecko.  So I've changed out implementation to adapt the Chromium's frame number behavior.

An example of Frame Timing entries on a CSS animation

Firefox with this patch
=======
render
[ startTime,  frameNumber ]
[ 1049.048, 19 ]

composite
[ startTime,  frameNumber ]
[ 1066.255, 19 ]
[ 1082.349, 19 ]
[ 1099.668, 19 ]
[ 1115.791, 19 ]
[ 1132.987, 19 ]
[ 1149.133, 19 ]
[ 1166.272, 19 ]
[ 1182.384, 19 ]
[ 1199.567, 19 ]
[ 1215.962, 19 ]
[ 1232.884, 19 ]
[ 1249.018, 19 ]
[ 1266.123, 19 ]
[ 1282.230, 19 ]
[ 1299.262, 19 ]
[ 1316.366, 19 ]
[ 1332.454, 19 ]
[ 1349.563, 19 ]
[ 1365.671, 19 ]
[ 1382.788, 19 ]
[ 1398.863, 19 ]
[ 1415.947, 19 ]
[ 1432.043, 19 ]
[ 1449.169, 19 ]
[ 1466.384, 19 ]
[ 1482.530, 19 ]
[ 1499.671, 19 ]
[ 1515.883, 19 ]
[ 1533.045, 19 ]

Chromium 45.0.2438.0
========

render
[ startTime,  frameNumber ]
[ 1144.354, 2 ]
[ 1161.020, 2 ]
composite"
[ startTime,  frameNumber ]
[ 1144.354, 2 ]
[ 1161.020, 2 ]
[ 1177.686, 2 ]
[ 1194.369, 2 ]
[ 1211.034, 2 ]
[ 1227.717, 2 ]
[ 1244.393, 2 ]
[ 1261.053, 2 ]
[ 1277.704, 2 ] 
[ 1294.382, 2 ]
[ 1311.052, 2 ]
[ 1327.724, 2 ]
[ 1344.380, 2 ]
[ 1361.048, 2 ]
[ 1377.724, 2 ]
[ 1394.395, 2 ]
[ 1411.032, 2 ]
[ 1427.726, 2 ]
[ 1444.381, 2 ]
[ 1461.048, 2 ]
[ 1477.725, 2 ] 
[ 1494.367, 2 ]
[ 1511.055, 2 ]
[ 1527.703, 2 ]
[ 1544.386, 2 ]
[ 1561.058, 2 ]

As you can see there are some differences:

1. Firefox starts compositor events from a subsequent frame but Chromium starts it in the same frame.
2. Chromium renders something in the second frame.
Comment 15 User image Ilya Grigorik 2015-09-23 14:19:42 PDT
Note that the spec has been updated to use the "slow-only" model:
- background: https://github.com/w3c/frame-timing/pull/48
- new draft: https://w3c.github.io/frame-timing/

Note You need to log in before you can comment on or make changes to this bug.