Closed Bug 1721522 Opened 3 years ago Closed 3 years ago

Perma [Tier 2] PermissionError: [WinError 5] Access is denied: 'z:/task_1626836235/workspace/obj-build\\_virtualenvs\\common\\Scripts\\python.exe'

Categories

(Firefox Build System :: Mach Core, defect, P5)

defect

Tracking

(firefox-esr78 unaffected, firefox90 unaffected, firefox91 unaffected, firefox92 fixed)

RESOLVED FIXED
92 Branch
Tracking Status
firefox-esr78 --- unaffected
firefox90 --- unaffected
firefox91 --- unaffected
firefox92 --- fixed

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=345839077&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/SBJ2tXIRTfi8zUxXFflj1w/runs/0/artifacts/public/logs/live_backing.log


INFO -  created virtual environment CPython3.6.5.final.0-64 in 211ms
[task 2021-07-21T03:07:52.093Z] 03:07:52     INFO -    creator CPython3Windows(dest=Z:\task_1626836235\workspace\obj-build\_virtualenvs\common, clear=False, no_vcs_ignore=False, global=False)
[task 2021-07-21T03:07:52.094Z] 03:07:52     INFO -    seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=Z:\task_1626836235\AppData\Local\pypa\virtualenv)
[task 2021-07-21T03:07:52.094Z] 03:07:52     INFO -      added seed packages: pip==20.3.1, setuptools==51.0.0, wheel==0.36.1
[task 2021-07-21T03:07:52.094Z] 03:07:52     INFO -    activators BashActivator,BatchActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
[task 2021-07-21T03:07:52.228Z] 03:07:52     INFO -  Re-executing in the virtualenv
[task 2021-07-21T03:07:52.640Z] 03:07:52     INFO -  Creating Python 3 environment
[task 2021-07-21T03:07:52.674Z] 03:07:52    ERROR -  Traceback (most recent call last):
[task 2021-07-21T03:07:52.674Z] 03:07:52     INFO -    File "Z:/task_1626836235/build/src\configure.py", line 229, in <module>
[task 2021-07-21T03:07:52.675Z] 03:07:52     INFO -      sys.exit(main(sys.argv))
[task 2021-07-21T03:07:52.675Z] 03:07:52     INFO -    File "Z:/task_1626836235/build/src\configure.py", line 55, in main
[task 2021-07-21T03:07:52.675Z] 03:07:52     INFO -      sandbox.run(os.path.join(os.path.dirname(__file__), "moz.configure"))
[task 2021-07-21T03:07:52.675Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 497, in run
[task 2021-07-21T03:07:52.675Z] 03:07:52     INFO -      self.include_file(path)
[task 2021-07-21T03:07:52.675Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 488, in include_file
[task 2021-07-21T03:07:52.675Z] 03:07:52     INFO -      exec_(code, self)
[task 2021-07-21T03:07:52.676Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\util.py", line 56, in exec_
[task 2021-07-21T03:07:52.676Z] 03:07:52     INFO -      exec(object, globals, locals)
[task 2021-07-21T03:07:52.676Z] 03:07:52     INFO -    File "Z:/task_1626836235/build/src/moz.configure", line 7, in <module>
[task 2021-07-21T03:07:52.676Z] 03:07:52     INFO -      include("build/moz.configure/init.configure")
[task 2021-07-21T03:07:52.676Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 848, in include_impl
[task 2021-07-21T03:07:52.676Z] 03:07:52     INFO -      self.include_file(what)
[task 2021-07-21T03:07:52.677Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 488, in include_file
[task 2021-07-21T03:07:52.677Z] 03:07:52     INFO -      exec_(code, self)
INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\util.py", line 56, in exec_
[task 2021-07-21T03:07:52.677Z] 03:07:52     INFO -      exec(object, globals, locals)
[task 2021-07-21T03:07:52.677Z] 03:07:52     INFO -    File "Z:/task_1626836235/build/src/build/moz.configure/init.configure", line 233, in <module>
[task 2021-07-21T03:07:52.677Z] 03:07:52     INFO -      @imports(_from="six", _import="ensure_text")
[task 2021-07-21T03:07:52.678Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 831, in decorator
[task 2021-07-21T03:07:52.678Z] 03:07:52     INFO -      depends = DependsFunction(self, func, dependencies, when=when)
[task 2021-07-21T03:07:52.678Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 132, in __init__
[task 2021-07-21T03:07:52.678Z] 03:07:52     INFO -      sandbox._value_for(self)
[task 2021-07-21T03:07:52.678Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 609, in _value_for
[task 2021-07-21T03:07:52.678Z] 03:07:52     INFO -      return self._value_for_depends(obj)
[task 2021-07-21T03:07:52.679Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\util.py", line 1050, in method_call
[task 2021-07-21T03:07:52.679Z] 03:07:52     INFO -      cache[args] = self.func(instance, *args)
[task 2021-07-21T03:07:52.679Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 618, in _value_for_depends
[task 2021-07-21T03:07:52.679Z] 03:07:52     INFO -      value = obj.result()
[task 2021-07-21T03:07:52.679Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\util.py", line 1050, in method_call
[task 2021-07-21T03:07:52.679Z] 03:07:52     INFO -      cache[args] = self.func(instance, *args)
[task 2021-07-21T03:07:52.680Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 157, in result
[task 2021-07-21T03:07:52.680Z] 03:07:52     INFO -      return self._func(*resolved_args)
[task 2021-07-21T03:07:52.680Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 1287, in wrapped
[task 2021-07-21T03:07:52.680Z] 03:07:52     INFO -      return new_func(*args, **kwargs)
[task 2021-07-21T03:07:52.680Z] 03:07:52     INFO -    File "Z:/task_1626836235/build/src/build/moz.configure/init.configure", line 345, in virtualenv_python3
[task 2021-07-21T03:07:52.680Z] 03:07:52     INFO -      manager.build(python)
[task 2021-07-21T03:07:52.681Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\virtualenv.py", line 395, in build
[task 2021-07-21T03:07:52.681Z] 03:07:52     INFO -      self.create(python)
[task 2021-07-21T03:07:52.681Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python\mozbuild\mozbuild\virtualenv.py", line 260, in create
[task 2021-07-21T03:07:52.681Z] 03:07:52     INFO -      shutil.rmtree(self.virtualenv_root)
[task 2021-07-21T03:07:52.681Z] 03:07:52     INFO -    File "c:\mozilla-build\python3\lib\shutil.py", line 494, in rmtree
[task 2021-07-21T03:07:52.681Z] 03:07:52     INFO -      return _rmtree_unsafe(path, onerror)
[task 2021-07-21T03:07:52.681Z] 03:07:52     INFO -    File "c:\mozilla-build\python3\lib\shutil.py", line 384, in _rmtree_unsafe
[task 2021-07-21T03:07:52.682Z] 03:07:52     INFO -      _rmtree_unsafe(fullname, onerror)
[task 2021-07-21T03:07:52.682Z] 03:07:52     INFO -    File "c:\mozilla-build\python3\lib\shutil.py", line 389, in _rmtree_unsafe
[task 2021-07-21T03:07:52.682Z] 03:07:52     INFO -      onerror(os.unlink, fullname, sys.exc_info())
[task 2021-07-21T03:07:52.682Z] 03:07:52     INFO -    File "c:\mozilla-build\python3\lib\shutil.py", line 387, in _rmtree_unsafe
[task 2021-07-21T03:07:52.682Z] 03:07:52     INFO -      os.unlink(fullname)
[task 2021-07-21T03:07:52.682Z] 03:07:52     INFO -  PermissionError: [WinError 5] Access is denied: 'z:/task_1626836235/workspace/obj-build\\_virtualenvs\\common\\Scripts\\python.exe'
[task 2021-07-21T03:07:52.713Z] 03:07:52     INFO -   Config object not found by mach.
[task 2021-07-21T03:07:52.713Z] 03:07:52     INFO -  Error running mach:
INFO -      ['--log-no-times', 'build', '-v']
[task 2021-07-21T03:07:52.714Z] 03:07:52     INFO -  The error occurred in code that was called by the mach command. This is either
[task 2021-07-21T03:07:52.714Z] 03:07:52     INFO -  a bug in the called code itself or in the way that mach is calling it.
[task 2021-07-21T03:07:52.714Z] 03:07:52     INFO -  You can invoke |./mach busted| to check if this issue is already on file. If it
[task 2021-07-21T03:07:52.714Z] 03:07:52     INFO -  isn't, please use |./mach busted file build| to report it. If |./mach busted| is
[task 2021-07-21T03:07:52.714Z] 03:07:52     INFO -  misbehaving, you can also inspect the dependencies of bug 1543241.
[task 2021-07-21T03:07:52.714Z] 03:07:52     INFO -  If filing a bug, please include the full output of mach, including this error
[task 2021-07-21T03:07:52.714Z] 03:07:52     INFO -  message.
[task 2021-07-21T03:07:52.715Z] 03:07:52     INFO -  The details of the failure are as follows:
[task 2021-07-21T03:07:52.715Z] 03:07:52     INFO -  Exception: Process executed with non-0 exit code 1: ['c:\\mozilla-build\\python3\\python3.exe', 'Z:/task_1626836235/build/src\\configure.py']
[task 2021-07-21T03:07:52.715Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python/mozbuild/mozbuild/build_commands.py", line 161, in build
[task 2021-07-21T03:07:52.715Z] 03:07:52     INFO -      append_env=append_env,
[task 2021-07-21T03:07:52.715Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python/mozbuild\mozbuild\controller\building.py", line 1148, in build
[task 2021-07-21T03:07:52.715Z] 03:07:52     INFO -      append_env=append_env,
[task 2021-07-21T03:07:52.715Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python/mozbuild\mozbuild\controller\building.py", line 1531, in configure
[task 2021-07-21T03:07:52.715Z] 03:07:52     INFO -      append_env=append_env,
[task 2021-07-21T03:07:52.716Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python/mozbuild\mozbuild\base.py", line 845, in _run_command_in_objdir
[task 2021-07-21T03:07:52.716Z] 03:07:52     INFO -      return self.run_process(cwd=self.topobjdir, **args)
[task 2021-07-21T03:07:52.716Z] 03:07:52     INFO -    File "Z:\task_1626836235\build\src\python/mach\mach\mixin\process.py", line 175, in run_process
[task 2021-07-21T03:07:52.716Z] 03:07:52     INFO -      "Process executed with non-0 exit code %d: %s" % (status, args)
[task 2021-07-21T03:07:52.768Z] 03:07:52    ERROR - Return code: 1
[task 2021-07-21T03:07:52.768Z] 03:07:52  WARNING - setting return code to 2
[task 2021-07-21T03:07:52.768Z] 03:07:52    FATAL - 'mach build -v' did not run successfully. Please check log for errors.
[task 2021-07-21T03:07:52.768Z] 03:07:52    FATAL - Running post_fatal callback...
[task 2021-07-21T03:07:52.768Z] 03:07:52    FATAL - Exiting -1

@Mitchell, could you please take a look at these build bustages?
They seem to start from your push

Flags: needinfo?(mhentges)

Please backout, these failures mean local builds will fail too, and that will block people when this merges to central.

Flags: needinfo?(mhentges) → needinfo?(smolnar)
Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(smolnar)
Resolution: --- → FIXED

I'll resolve this in the regressing bug. Thanks!

The issue seems to be that, after creating and activating the common virtualenv, the "pth" items don't line up due to drive letter casing (uppercase vs lowercase).
By making the pth check case-agnostic, the issue is resolved.

Target Milestone: --- → 92 Branch
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.