1518479 - Add Firefox FirstContentfulPaint measurement to raptor tests

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P1)

Description

[Randell Jesup [:jesup] (needinfo me)]

Attachment: raptor.patch

Note: FCP is not yet spec-compliant (and is behind a pref); it's implemented similarly to firstNonBlankPaint and isn't in the PerfEntries list yet.
It's behind a pref.

Comment 1

[Robert Wood [:rwood]]

Cool thanks :jesup! NI'ing myself to review this.

Comment 2

[Robert Wood [:rwood]]

Thank you :jesup, appreciated, I will take your patch and go from there.

Note: this is for adding the fcp measurement for Firefox; we already have fcp being measured on google Chromium in production.

Comment 3

[Robert Wood [:rwood]]

Attachment: Bug 1518479 - Add Firefox FirstContentfulPaint measurement to raptor tests (on behalf of :jesup); r?davehunt

Comment 4

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f82db83025550794bbf50d17c77735de9467cd2b

Comment 5

[Robert Wood [:rwood]]

(In reply to Robert Wood [:rwood] from comment #4)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f82db83025550794bbf50d17c77735de9467cd2b

:jesup, it looks like first-contentful-paint on Firefox (window.performance.timing.timeToContentfulPaint) is returning exactly the same value as first-non-blank-paint (window.performance.timing.timeToNonBlankPaint;). Is that to be expected (is it not fully implemented yet or something)?

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

rwood: often FNBP and FCP are the same. In fact, there's a bug where FNBP fires too early, before FCP, when it should fire at the same time as FCP.

They should not be the same in every case; a profile should show markers for FCP and FNBP

Comment 7

[Robert Wood [:rwood]]

(In reply to Randell Jesup [:jesup] from comment #6)

rwood: often FNBP and FCP are the same. In fact, there's a bug where FNBP fires too early, before FCP, when it should fire at the same time as FCP.

They should not be the same in every case; a profile should show markers for FCP and FNBP

Thanks :jesup - cool yes I do see markers in a profile [1] for both FCP and FNBP. Will go ahead and land this.

[1] https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FCswAsJkRTlK4OajcSOTzMg%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_raptor-tp6-google-firefox.zip/marker-chart/?file=profile_raptor-tp6-google-firefox%2Fraptor-tp6-google-firefox_pagecycle_1.profile&globalTrackOrder=0-1-2-3-4-5&hiddenGlobalTracks=2-3-4&localTrackOrderByPid=1316-1-0~4592-0~6460-0~6884-0~&markerSearch=&thread=7&v=3

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ed73c6c4e5d5
Add Firefox FirstContentfulPaint measurement to raptor tests (on behalf of :jesup); r=davehunt

Comment 9

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/ed73c6c4e5d5

Comment 10

[Florin Strugariu [:Bebe]]

Performance Improvement

== Change summary for alert #18791 (as of Fri, 18 Jan 2019 10:48:47 GMT) ==

Improvements:

30% raptor-tp6-microsoft-firefox windows7-32 opt 1,499.18 -> 1,051.19
26% raptor-tp6-microsoft-firefox linux64 pgo 1,441.61 -> 1,073.51
24% raptor-tp6-reddit-firefox osx-10-10 opt 3,242.14 -> 2,478.40
23% raptor-tp6-reddit-firefox linux64 opt 1,036.41 -> 801.66
23% raptor-tp6-reddit-firefox linux64-qr opt 1,103.71 -> 854.68
22% raptor-tp6-reddit-firefox windows10-64 opt 1,120.79 -> 869.80
22% raptor-tp6-reddit-firefox linux64 pgo 957.28 -> 743.44
22% raptor-tp6-reddit-firefox windows10-64-qr opt 1,113.19 -> 864.85
22% raptor-tp6-reddit-firefox windows10-64 pgo 1,010.77 -> 789.71
22% raptor-tp6-yandex-firefox osx-10-10 opt 390.53 -> 305.17
22% raptor-tp6-reddit-firefox windows7-32 opt 1,061.26 -> 832.18
20% raptor-tp6-reddit-firefox windows7-32 pgo 958.25 -> 762.28
20% raptor-tp6-yandex-firefox windows10-64 opt 209.04 -> 168.12
19% raptor-tp6-yandex-firefox linux64 pgo 195.61 -> 157.93
19% raptor-tp6-yandex-firefox windows7-32 opt 205.03 -> 165.60
19% raptor-tp6-yandex-firefox windows7-32 pgo 187.79 -> 151.98
19% raptor-tp6-microsoft-firefox osx-10-10 opt 2,756.58 -> 2,237.00
18% raptor-tp6-yandex-firefox windows10-64 pgo 189.89 -> 156.37
17% raptor-tp6-yandex-firefox windows10-64-qr opt 192.97 -> 159.71
17% raptor-tp6-imdb-firefox osx-10-10 opt 479.91 -> 397.95
16% raptor-tp6-imdb-firefox linux64 pgo 338.53 -> 282.70
16% raptor-tp6-imdb-firefox linux64 opt 363.62 -> 304.34
16% raptor-tp6-sheets-firefox windows7-32 opt 515.55 -> 432.46
16% raptor-tp6-yahoo-news-firefox linux64 opt 560.60 -> 470.66
16% raptor-tp6-yahoo-news-firefox osx-10-10 opt 1,349.98 -> 1,135.72
13% raptor-tp6-imdb-firefox linux64-qr opt 367.61 -> 318.32
13% raptor-tp6-amazon-firefox osx-10-10 opt 1,344.91 -> 1,173.29
12% raptor-tp6-imgur-firefox osx-10-10 opt 1,069.27 -> 936.24
12% raptor-tp6-yahoo-news-firefox linux64 pgo 492.50 -> 432.55
12% raptor-tp6-google-firefox osx-10-10 opt 405.37 -> 356.28
12% raptor-tp6-sheets-firefox osx-10-10 opt 1,169.40 -> 1,032.05
12% raptor-tp6-yahoo-news-firefox windows10-64-qr opt 415.16 -> 366.93
11% raptor-tp6-yahoo-news-firefox windows10-64 opt 411.81 -> 365.71
11% raptor-tp6-amazon-firefox windows10-64 opt 536.37 -> 479.24
11% raptor-tp6-amazon-firefox linux64 opt 539.37 -> 481.99
11% raptor-tp6-amazon-firefox windows7-32 pgo 449.39 -> 401.84
11% raptor-tp6-amazon-firefox windows7-32 opt 500.40 -> 447.46
10% raptor-tp6-amazon-firefox linux64 pgo 482.20 -> 433.17
10% raptor-tp6-amazon-firefox linux64-qr opt 575.79 -> 520.79
9% raptor-tp6-apple-firefox linux64 opt 512.49 -> 465.57
9% raptor-tp6-yahoo-news-firefox windows7-32 opt 394.41 -> 358.48
9% raptor-tp6-youtube-firefox linux64-qr opt 459.44 -> 418.65
9% raptor-tp6-youtube-firefox linux64 opt 430.78 -> 393.65
9% raptor-tp6-apple-firefox windows7-32 opt 437.78 -> 400.48
8% raptor-tp6-slides-firefox linux64-qr opt 1,168.05 -> 1,072.70
8% raptor-tp6-apple-firefox osx-10-10 opt 896.86 -> 824.75
8% raptor-tp6-apple-firefox windows10-64 opt 439.51 -> 404.90
8% raptor-tp6-slides-firefox windows10-64 pgo 1,112.19 -> 1,026.85
7% raptor-tp6-slides-firefox windows7-32 pgo 1,035.68 -> 958.67
7% raptor-tp6-apple-firefox windows10-64-qr opt 438.75 -> 406.45
7% raptor-tp6-youtube-firefox osx-10-10 opt 1,084.69 -> 1,005.81
7% raptor-tp6-apple-firefox linux64 pgo 482.22 -> 447.53
7% raptor-tp6-facebook-firefox windows7-32 pgo 413.14 -> 384.33
7% raptor-tp6-slides-firefox linux64 opt 1,078.70 -> 1,004.30
7% raptor-tp6-apple-firefox linux64-qr opt 530.67 -> 495.06
7% raptor-tp6-facebook-firefox windows7-32 opt 433.39 -> 405.21
6% raptor-tp6-slides-firefox linux64 pgo 993.56 -> 930.43
6% raptor-tp6-wikia-firefox linux64 opt 208.63 -> 195.68
6% raptor-tp6-facebook-firefox windows10-64-qr opt 457.06 -> 429.26
6% raptor-tp6-facebook-firefox linux64 opt 438.13 -> 411.84
6% raptor-tp6-facebook-firefox windows10-64 pgo 417.54 -> 393.02
6% raptor-tp6-facebook-firefox linux64 pgo 406.93 -> 383.52
6% raptor-tp6-facebook-firefox linux64-qr opt 462.20 -> 435.78
6% raptor-tp6-facebook-firefox windows10-64 opt 442.70 -> 417.53
6% raptor-tp6-google-firefox windows7-32 pgo 151.74 -> 143.21
6% raptor-tp6-google-firefox linux64-qr opt 184.82 -> 174.55
5% raptor-tp6-docs-firefox windows7-32 pgo 1,450.89 -> 1,373.43
5% raptor-tp6-google-firefox linux64 pgo 167.18 -> 158.26
5% raptor-tp6-docs-firefox windows10-64 pgo 1,513.81 -> 1,434.00
5% raptor-tp6-docs-firefox windows10-64 opt 1,556.09 -> 1,476.49
5% raptor-tp6-google-firefox windows10-64-qr opt 164.20 -> 155.97
5% raptor-tp6-google-firefox windows10-64 pgo 151.29 -> 144.16
5% raptor-tp6-google-firefox windows10-64 opt 164.60 -> 157.03
5% raptor-tp6-docs-firefox windows7-32 opt 1,497.76 -> 1,429.11

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=18791