1663424 - mozilla-central / nightly PGO builds do not get profiling data from non-webcontent child processes

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[:Gijs (he/him)]

It looks like child processes are not able to output profiling data, despite the profiling script disabling the sandbox. Output on linux, mac and windows includes variations of the following error messages:

LLVM Profile Error: Failed to write file "default_582_random_23848062398498425_0.profraw": Permission denied

LLVM Profile Error: Failed to write file "default_6216_random_15823529472270747130_0.profraw": Operation not permitted

LLVM Profile Error: Failed to write file "default_476_random_12898274460901853933_0.profraw": No error

It's not clear to me what the consequences here are, but I imagine they're not good? I'm also not 100% sure how to investigate... :dmajor or :froydnj, can you help? My changes in bug 1196094 originally tripped up talos at various points, so I'd be surprised if this didn't have a performance impact...

It would appear these messages have been here since at least May (unsure exactly how far back our logs/artifact retention goes and if we can find out when this changed), so if it's a regression it's not a recent one, and it's not clear if there is/was a corresponding change in talos or other perf test output at any point.

Comment 1

[:Gijs (he/him)]

Meant to needinfo for comment #0, oops...

Comment 2

[Mike Hommey [:glandium]]

I was wondering if it could be related to the upgrade to clang 11, but it turns out it's not: the error messages are there in the logs before the landing of the clang upgrade. It would be good to find when they appeared first.

Comment 3

[Mike Hommey [:glandium]]

It's there in the oldest log I could find on mozilla-central: https://firefoxci.taskcluster-artifacts.net/DszLDlyoQSmMA2H1IL5zog/0/public/build/profile-run-2.log

Comment 4

[(Away)]

Does it still complain if you set all of the following?

MOZ_DISABLE_CONTENT_SANDBOX
MOZ_DISABLE_RDD_SANDBOX
MOZ_DISABLE_SOCKET_PROCESS_SANDBOX
MOZ_DISABLE_GPU_SANDBOX
MOZ_DISABLE_GMP_SANDBOX
MOZ_DISABLE_NPAPI_SANDBOX
MOZ_DISABLE_VR_SANDBOX

Comment 5

[:Gijs (he/him)]

(In reply to :dmajor from comment #4)

Does it still complain if you set all of the following?

MOZ_DISABLE_CONTENT_SANDBOX
MOZ_DISABLE_RDD_SANDBOX
MOZ_DISABLE_SOCKET_PROCESS_SANDBOX
MOZ_DISABLE_GPU_SANDBOX
MOZ_DISABLE_GMP_SANDBOX
MOZ_DISABLE_NPAPI_SANDBOX
MOZ_DISABLE_VR_SANDBOX

Good point... Trypush: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f65fed9fdf48039307c2a1cc0ccb0175c0fa43ce

I did already go and check, and these same warnings aren't showing up on beta and release builds, so that might lend weight to your suggestion (and is good because it means whatever it is likely isn't affecting beta/release users...).

Comment 6

[:Gijs (he/him)]

Yup, https://firefoxci.taskcluster-artifacts.net/AA0_viN6QaGtlUiTS8scdQ/0/public/build/profile-run-2.log does not show these errors anymore.

Comment 7

[:Gijs (he/him)]

Attachment: Bug 1663424 - fix pgo file writing from non-webcontent child processes in PROFILE_GENERATE runs, r?dmajor

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7d8764be7899
fix pgo file writing from non-webcontent child processes in PROFILE_GENERATE runs, r=dmajor

Comment 9

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/7d8764be7899