1746325 - Perma l10n_cross_channel PermissionError: [Errno 13] Permission denied: '/builds/worker/.mozbuild/srcdirs'

Status: RESOLVED FIXED

(Firefox Build System :: Mach Core, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: mlaza [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=361410983&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LXdrx-hCSmKlc8l5M3hefA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2021-12-15T23:03:46.297Z] + ./mach l10n-cross-channel -o /builds/worker/artifacts/outgoing.diff --attempts 5 --ssh-secret project/releng/gecko/build/level-3/l10n-cross-channel-quarantine-ssh prep create push
[task 2021-12-15T23:03:47.217Z] Creating local state directory: /builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69
[task 2021-12-15T23:03:47.217Z] Error running mach:
[task 2021-12-15T23:03:47.217Z] 
[task 2021-12-15T23:03:47.217Z]     ['l10n-cross-channel', '-o', '/builds/worker/artifacts/outgoing.diff', '--attempts', '5', '--ssh-secret', 'project/releng/gecko/build/level-3/l10n-cross-channel-quarantine-ssh', 'prep', 'create', 'push']
[task 2021-12-15T23:03:47.217Z] 
[task 2021-12-15T23:03:47.217Z] The error occurred in code that was called by the mach command. This is either
[task 2021-12-15T23:03:47.217Z] a bug in the called code itself or in the way that mach is calling it.
[task 2021-12-15T23:03:47.217Z] You can invoke |./mach busted| to check if this issue is already on file. If it
[task 2021-12-15T23:03:47.217Z] isn't, please use |./mach busted file l10n-cross-channel| to report it. If |./mach busted| is
[task 2021-12-15T23:03:47.217Z] misbehaving, you can also inspect the dependencies of bug 1543241.
[task 2021-12-15T23:03:47.217Z] 
[task 2021-12-15T23:03:47.217Z] If filing a bug, please include the full output of mach, including this error
[task 2021-12-15T23:03:47.217Z] message.
[task 2021-12-15T23:03:47.217Z] 
[task 2021-12-15T23:03:47.217Z] The details of the failure are as follows:
[task 2021-12-15T23:03:47.217Z] 
[task 2021-12-15T23:03:47.217Z] PermissionError: [Errno 13] Permission denied: '/builds/worker/.mozbuild/srcdirs'
[task 2021-12-15T23:03:47.217Z] 
[task 2021-12-15T23:03:47.217Z]   File "/builds/worker/checkouts/gecko/tools/compare-locales/mach_commands.py", line 174, in cross_channel
[task 2021-12-15T23:03:47.217Z]     command_context.activate_virtualenv()
[task 2021-12-15T23:03:47.217Z]   File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/base.py", line 876, in activate_virtualenv
[task 2021-12-15T23:03:47.217Z]     self.virtualenv_manager.activate()
[task 2021-12-15T23:03:47.217Z]   File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/base.py", line 298, in virtualenv_manager
[task 2021-12-15T23:03:47.217Z]     get_state_dir(specific_to_topsrcdir=True, topsrcdir=self.topsrcdir),
[task 2021-12-15T23:03:47.217Z]   File "/builds/worker/checkouts/gecko/python/mach/mach/util.py", line 79, in get_state_dir
[task 2021-12-15T23:03:47.217Z]     os.makedirs(state_dir, mode=0o770)
[task 2021-12-15T23:03:47.217Z]   File "/usr/lib/python3.7/os.py", line 211, in makedirs
[task 2021-12-15T23:03:47.217Z]     makedirs(head, exist_ok=exist_ok)
[task 2021-12-15T23:03:47.217Z]   File "/usr/lib/python3.7/os.py", line 221, in makedirs
[task 2021-12-15T23:03:47.217Z]     mkdir(name, mode)
[taskcluster 2021-12-15 23:03:47.648Z] === Task Finished ===
[taskcluster 2021-12-15 23:03:47.804Z] Artifact "public/build" not found at "/builds/worker/artifacts"
[taskcluster 2021-12-15 23:03:47.867Z] Unsuccessful task run with exit code: 1 completed in 21.908 seconds

Comment 1

[Mike Hommey [:glandium]]

Likely a regression from bug 1745508.

Comment 2

[Andreea Pavel [:apavel]]

Hi Mitchell, please take a look when you have the time.
Thank you.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1745508

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1746325: Defer/avoid creation of scoped state dir for command sites

In CI, we sometimes don't have permissions to create a scoped state dir.
Additionally, the current behaviour for resolving the path to a scoped
state dir will also attempt to create it if it doesn't exist.

There's likely a more eloquent solution, but the short-term fix is to
have sites defer the resolution of the state dir until they know they
need it.

Comment 5

[Intermittent Failures Robot]

12 failures in 3263 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 12

Platform and build breakdown:

• firefox-release: 12
  • opt: 12

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1746325&startday=2021-12-13&endday=2021-12-19&tree=all

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1d325bff960d
Defer/avoid creation of scoped state dir for command sites r=ahal

Comment 7

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/1d325bff960d

Comment 8

[Intermittent Failures Robot]

1 failures in 2286 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• firefox-release: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1746325&startday=2021-12-20&endday=2021-12-26&tree=all