1567236 - Gecko Profiler on Windows only shows pseudo-stacks and leaf functions for C++

Status: RESOLVED FIXED

(Core :: Gecko Profiler, defect, P1)

Description

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

There's a recent regression that a profile in the Gecko profiler on Windows nightly (and try) builds only shows leaf functions and pseudo-stack markers for C++ profiling and doesn't show the full C++ stack.

I'm currently running mozregression and will have a regression range shortly, though so far it's down to this range on mozilla-central.

Comment 1

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

final regression range from mozregression blames bug 1557282

Comment 2

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Steps to reproduce:

1. go to https://profiler.firefox.com/
2. click the button to install the addon (if there's no button then it's already installed) and accept the prompts for installation
3. press Ctrl+Shift+1 to start profiling
4. do something (Scroll a page?)
5. press Ctrl+Shift+2 to show the profile
6. look at the profile

Expected results: deep stacks

Actual results: shallow pseudo-stacks plus leaf functions for C++

Comment 3

[Markus Stange [:mstange]]

The entry point for native stack walking on Windows is this call to RtlVirtualUnwind.

Comment 4

[Bob Owen (:bobowen)]

(In reply to Markus Stange [:mstange] from comment #3)

The entry point for native stack walking on Windows is this call to RtlVirtualUnwind.

I take it the stack sampling is done from within the process, not from another process?

Comment 5

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

(In reply to Bob Owen (:bobowen) from comment #4)

I think I can answer that now:

(In reply to Markus Stange [:mstange] from comment #3)

The entry point for native stack walking on Windows is this call to RtlVirtualUnwind.

I take it the stack sampling is done from within the process, not from another process?

That's right, the Profiler's Sampler thread stops each thread to be sampled (in the same process) before walking their stack.

Comment 6

[Bob Owen (:bobowen)]

OK thanks, I'm going to drop the sandbox level to turn this off, until I can investigate.
There was a performance regression as well, which I guess might have a similar root cause.
I think we could live with that, but I don't want to mess up profiling while I try and find the time to look at this.

Comment 7

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Thank you Bob, much appreciated.

Sorry we didn't have tests that would have caught this earlier. I've opened bug 1567390 for that.

Comment 8

[Bob Owen (:bobowen)]

Attachment: Bug 1567236: Drop Windows content sandbox level back to 5 on Nightly. r=gcp!

This is because of profiling and performance regressions, which we don't want
to live with while we investigate.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/eb7f4d56f54b
Drop Windows content sandbox level back to 5 on Nightly. r=gcp

Comment 10

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/eb7f4d56f54b

Comment 11

[Florin Strugariu [:Bebe]]

== Change summary for alert #21999 (as of Fri, 19 Jul 2019 19:53:09 GMT) ==

Improvements:

13% raptor-tp6-amazon-firefox-cold fcp windows7-32-shippable opt 489.08 -> 424.75
12% raptor-tp6-amazon-firefox-cold fcp windows10-64-shippable-qr opt 488.83 -> 428.17
12% raptor-tp6-amazon-firefox-cold windows7-32-shippable opt 568.70 -> 501.15
12% raptor-tp6-google-firefox-cold fcp windows7-32-shippable opt 359.58 -> 317.00
11% raptor-tp6-amazon-firefox-cold fcp windows10-64-shippable opt 473.17 -> 419.75
10% raptor-tp6-google-firefox-cold fcp windows10-64-shippable-qr opt 327.67 -> 293.75
10% raptor-tp6-amazon-firefox-cold windows10-64-shippable opt 546.03 -> 489.55
10% raptor-tp6-google-firefox-cold windows10-64-shippable-qr opt 348.89 -> 315.02
9% raptor-tp6-google-firefox-cold fcp windows10-64-shippable opt 347.83 -> 315.17
9% raptor-tp6-google-firefox-cold windows7-32-shippable opt 342.81 -> 313.46
9% raptor-tp6-amazon-firefox-cold loadtime windows10-64-shippable opt 759.62 -> 694.75
5% raptor-tp6-google-firefox-cold windows10-64-shippable opt 335.47 -> 318.14
4% raptor-tp6-facebook-firefox-cold fcp windows7-32-shippable opt 385.92 -> 369.25
4% raptor-tp6-facebook-firefox-cold windows10-64-shippable-qr opt 427.80 -> 409.98
4% raptor-tp6-facebook-firefox-cold windows7-32-shippable opt 445.86 -> 428.08
4% raptor-tp6-facebook-firefox-cold fcp windows10-64-shippable-qr opt 354.83 -> 341.25
4% raptor-tp6-facebook-firefox-cold fcp windows10-64-shippable opt 376.46 -> 362.25
3% raptor-tp6-facebook-firefox-cold windows10-64-shippable opt 437.98 -> 422.68
3% raptor-tp6-facebook-firefox-cold loadtime windows7-32-shippable opt 862.25 -> 835.33

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=21999

Comment 12

[Ionuț Goldan [:igoldan]]

Noticed Talos improvements:

== Change summary for alert #22008 (as of Fri, 19 Jul 2019 23:36:24 GMT) ==

Improvements:

3% startup_about_home_paint windows10-64-shippable opt e10s stylo 661.46 -> 638.92

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=22008