1543853 - Evaluate using stylebench for PGO training

Status: NEW

(Firefox Build System :: General, enhancement)

Description

[Eric Rahm [:erahm]]

We would like to evaluate using the stylebench perf test for PGO training. This exercises an extensive rust code base used by the style system and will be useful once we have rust PGO enabled.

Work can start by building on top of the initial patches for bug 1437452 that enable rust PGO on Linux.

Comment 1

[(Away)]

Preliminary testing shows a 3-4% improvement from doing this: https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&newProject=try&newRevision=9de5cc6ad29ca599d9eae0cb44a4d64c63eb7ad3&framework=10&originalRevision=86d7a5dbe24477fefa9f7a43b8e329566c97e382

Comment 2

[(Away)]

Attachment: Add StyleBench to PGO training

Now that we have Rust PGO, training on this set can actually make a difference (and in fact may be needed in order to see a difference from Rust PGO at all).

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ebbc8a9e9737
Add StyleBench to PGO training r=firefox-build-system-reviewers,rstewart

Comment 4

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/ebbc8a9e9737

Comment 5

[Ryan VanderMeulen [:RyanVM]]

Backed out per bug 1603482 comment 4.
https://hg.mozilla.org/integration/autoland/rev/a78616dc94d07b725becb47befa69f1f5d7c5298

Comment 6

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backout merged https://hg.mozilla.org/mozilla-central/rev/a78616dc94d0

Comment 7

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Backout improvement:
== Change summary for alert #24510 (as of Tue, 17 Dec 2019 18:59:34 GMT) ==

Improvements:

6% perf_reftest_singletons getElementById-1.html windows10-64-shippable-qr opt e10s stylo 54.95 -> 51.47
6% perf_reftest_singletons getElementById-1.html windows10-64-shippable opt e10s stylo 54.80 -> 51.34

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24510

Comment 8

[(Away)]

I attempted to revisit this using just one iteration of StyleBench (vs its default of 10): https://hg.mozilla.org/try/rev/7bfca42fd6f7cf34a264cdb8ef3a866fdd8eb977

Perf results showed a low-confidence 1.29% improvement on StyleBench, and mixed results on the reftest_singletons that had previously regressed. At this point I don't really have much enthusiasm to pursue this further. I'm not outright resolving the bug though because maybe things could change in the future.

Comment 9

[Mayank Bansal]

Is this bug still relevant?