1869835 - Add support for ETW events from Profiler markers

Status: RESOLVED FIXED

(Core :: Gecko Profiler, enhancement)

Description

[Bas Schouten (:bas.schouten)]

Currently getting important markers into ETW is a pain and goes through text files. We can use ETW tracelogging to expose these directly.

Comment 1

[Markus Stange [:mstange]]

Context: We're using xperf to profile Firefox in situations where the Gecko profiler isn't sufficient. We process the etl files with etw-gecko. To get UserTiming markers into the profile, we added a custom hack with an external text file in bug 1835513. This hack is annoying to use because you have to find out the pid of the process and pass a separate argument to the converter. It would be better to expose UserTiming markers using native ETW trace events.

Comment 2

[Bas Schouten (:bas.schouten)]

Attachment: Bug 1869835 - Part 1: Add code to emit ETW markers on Windows when a trace logger is enabled. r=canaltinova

This adds code to emit ETW markers on windows. It supports all current
markers through a generic marker emitting schema. It also supports
additional payload and filtering for any markers that are updated to a
new compile-time marker schema system. Because of our abstraction layer
and cross-platform nature we have to manually create the TraceLogging
structs used and can't rely on their macro's, but this should give us a
very flexible and performant marker implementation on ETW.

Comment 3

[Bas Schouten (:bas.schouten)]

Attachment: Bug 1869835 - Part 2: Do some code deduplication and convert UserTimingMarker to the new schema. r=mstange

This will allow ETW to collect the additional payload on the UserTiming
markers. As well as allow the provider to filter on these markers in
order to minimize overhead from other markers that don't need to be
collected.

Depends on D196331

Comment 4

[Bas Schouten (:bas.schouten)]

Attachment: Bug 1869835 - Part 3: Convert CC/GC markers to the new MarkerSchema so it gets full ETW support. r=smaug

Depends on D196332

Comment 5

[Bas Schouten (:bas.schouten)]

Attachment: Bug 1869835 - Part 4: Convert TaskMarker to the new MarkerSchema so it gets full ETW support. r=smaug

Depends on D196374

Comment 6

[Bas Schouten (:bas.schouten)]

Always hard to test this locally. But I gave it a bit of a shot, here's performance numbers for Sp3.

Nothing enabled: { 14.9, 14.8, 14.46, 14.64, 14.4 } Mean: 14.64
All enabled: { 14.4 14.3, 14.0, 14.04, 14.1 } (Collected 2 million events per run) Mean: 14.17 (~3% slower)
Runnables/CC/User: { 14.5, 14.2, 14.3, 14.3, 14.4 } (Collected ~160k events per run) Mean: 14.34 (~2% slower)
CC/User only: { 14.5, 14.6, 14.4, 14.3, 14.4 } (Collected ~5000 events per run) Mean: 14.44 (~1.5% slower)

Some of this may be noise, but there does seem to be a bit of a pattern.

Comment 7

[Bas Schouten (:bas.schouten)]

Good news.

I had an error in my code which caused us to write out gecko events to the buffer when ETW event tracing was running. I addressed that and the perf numbers look better:

Nothing enabled: { 14.5, 14.6, 14.7, 14.7, 14.7 } Mean: 14.64
All enabled: { 14.5, 14.5, 14.5, 14.4, 14.33 } Mean: 14.45 (~1.5% slower)
All but generic: { 14.9, 14.54, 14.7, 14.6, 14.5 } Mean: 14.65 (Noise)

This means it's feasible right now to collect runnable/cc/gc and user timing markers without a meaningful impact at runtime using ETW as far as I can tell.

Comment 8

[Bas Schouten (:bas.schouten)]

Attachment: Bug 1869835 - Part 5: Update profiler documentation to match the new standard for defining markers. r=canaltinova

Depends on D196375

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f4ae79c39bbf
Part 1: Add code to emit ETW markers on Windows when a trace logger is enabled. r=canaltinova,profiler-reviewers
https://hg.mozilla.org/integration/autoland/rev/2154f1cbb973
Part 2: Do some code deduplication and convert UserTimingMarker to the new schema. r=mstange,profiler-reviewers
https://hg.mozilla.org/integration/autoland/rev/983b5d3e66d7
Part 3: Convert CC/GC markers to the new MarkerSchema so it gets full ETW support. r=smaug
https://hg.mozilla.org/integration/autoland/rev/a7cafe432d21
Part 4: Convert TaskMarker to the new MarkerSchema so it gets full ETW support. r=smaug,canaltinova
https://hg.mozilla.org/integration/autoland/rev/f1ff82c038af
Part 5: Update profiler documentation to match the new standard for defining markers. r=canaltinova,profiler-reviewers
https://hg.mozilla.org/integration/autoland/rev/592d811ee495
apply code formatting via Lando

Comment 10

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/f4ae79c39bbf
https://hg.mozilla.org/mozilla-central/rev/2154f1cbb973
https://hg.mozilla.org/mozilla-central/rev/983b5d3e66d7
https://hg.mozilla.org/mozilla-central/rev/a7cafe432d21
https://hg.mozilla.org/mozilla-central/rev/f1ff82c038af
https://hg.mozilla.org/mozilla-central/rev/592d811ee495