1673914 - Perma [Tier2] Exception: Error populating virtualenv.

Status: RESOLVED FIXED

(Testing :: General, defect, P5)

Description

[Narcis Beleuzu [:NarcisB]]

Parsed log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=320031079&repo=autoland&lineNumber=2359
Full log: https://firefoxci.taskcluster-artifacts.net/YLhbnIL4TvCFw6IPJitZ3A/0/public/logs/live_backing.log

Log snippet:

[fetches 2020-10-28T15:22:05.802Z] PERFHERDER_DATA: {"suites": [{"name": "fetch_content", "lowerIsBetter": true, "subtests": [], "shouldAlert": false, "value": 3.247064540000011}], "framework": {"name": "build_metrics"}}
[fetches 2020-10-28T15:22:05.816Z] finished fetching artifacts
[task 2020-10-28T15:22:05.816Z] executing ['bash', '-cx', './mach jsshell-bench --perfherder=sm --binary=$MOZ_FETCHES_DIR/$JSSHELL six-speed']in checkouts/gecko
[task 2020-10-28T15:22:05.818Z] + ./mach jsshell-bench --perfherder=sm --binary=/home/cltbld/tasks/task_1603898477/fetches/js six-speed
[task 2020-10-28T15:22:06.904Z] created virtual environment CPython3.5.2.final.0-64 in 213ms
[task 2020-10-28T15:22:06.904Z]   creator CPython3Posix(dest=/home/cltbld/tasks/task_1603898477/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3, clear=False, global=False)
[task 2020-10-28T15:22:06.904Z]   seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/cltbld/.local/share/virtualenv)
[task 2020-10-28T15:22:06.904Z]     added seed packages: pip==20.2.2, setuptools==49.6.0, wheel==0.35.1
[task 2020-10-28T15:22:06.904Z]   activators CShellActivator,FishActivator,XonshActivator,PythonActivator,PowerShellActivator,BashActivator
[task 2020-10-28T15:22:06.978Z] One of the following Python versions are required:
[task 2020-10-28T15:22:06.978Z] * Python 2.7.3 or greater
[task 2020-10-28T15:22:06.978Z] * Python 3.6.0 or greater
[task 2020-10-28T15:22:06.978Z] You are running Python 3.5.2.
[task 2020-10-28T15:22:06.978Z] Run |mach bootstrap| to ensure your system is up to date.
[task 2020-10-28T15:22:06.978Z] 
[task 2020-10-28T15:22:06.978Z] If you still receive this error, your shell environment is likely detecting
[task 2020-10-28T15:22:06.978Z] another Python version. Ensure a modern Python can be found in the paths
[task 2020-10-28T15:22:06.978Z] defined by the $PATH environment variable and try again.
[task 2020-10-28T15:22:06.983Z] Error running mach:
[task 2020-10-28T15:22:06.983Z] 
[task 2020-10-28T15:22:06.983Z]     ['jsshell-bench', '--perfherder=sm', '--binary=/home/cltbld/tasks/task_1603898477/fetches/js', 'six-speed']
[task 2020-10-28T15:22:06.983Z] 
[task 2020-10-28T15:22:06.983Z] The error occurred in code that was called by the mach command. This is either
[task 2020-10-28T15:22:06.983Z] a bug in the called code itself or in the way that mach is calling it.
[task 2020-10-28T15:22:06.983Z] You can invoke |./mach busted| to check if this issue is already on file. If it
[task 2020-10-28T15:22:06.983Z] isn't, please use |./mach busted file jsshell-bench| to report it. If |./mach busted| is
[task 2020-10-28T15:22:06.983Z] misbehaving, you can also inspect the dependencies of bug 1543241.
[task 2020-10-28T15:22:06.983Z] 
[task 2020-10-28T15:22:06.983Z] If filing a bug, please include the full output of mach, including this error
[task 2020-10-28T15:22:06.983Z] message.
[task 2020-10-28T15:22:06.983Z] 
[task 2020-10-28T15:22:06.983Z] The details of the failure are as follows:
[task 2020-10-28T15:22:06.983Z] 
[task 2020-10-28T15:22:06.983Z] Exception: Error populating virtualenv.
[task 2020-10-28T15:22:06.983Z] 
[task 2020-10-28T15:22:06.983Z]   File "/home/cltbld/tasks/task_1603898477/checkouts/gecko/testing/mach_commands.py", line 675, in run_jsshelltests
[task 2020-10-28T15:22:06.983Z]     self.activate_virtualenv()
[task 2020-10-28T15:22:06.983Z]   File "/home/cltbld/tasks/task_1603898477/checkouts/gecko/python/mozbuild/mozbuild/base.py", line 911, in activate_virtualenv
[task 2020-10-28T15:22:06.983Z]     self.virtualenv_manager.ensure()
[task 2020-10-28T15:22:06.983Z]   File "/home/cltbld/tasks/task_1603898477/checkouts/gecko/python/mozbuild/mozbuild/virtualenv.py", line 230, in ensure
[task 2020-10-28T15:22:06.983Z]     return self.build(python)
[task 2020-10-28T15:22:06.983Z]   File "/home/cltbld/tasks/task_1603898477/checkouts/gecko/python/mozbuild/mozbuild/virtualenv.py", line 546, in build
[task 2020-10-28T15:22:06.983Z]     raise Exception("Error populating virtualenv.")
[taskcluster 2020-10-28T15:22:07.029Z]    Exit Code: 1
[taskcluster 2020-10-28T15:22:07.029Z]    User Time: 2m10.6s
[taskcluster 2020-10-28T15:22:07.029Z]  Kernel Time: 27.64s
[taskcluster 2020-10-28T15:22:07.029Z]    Wall Time: 48.50920873s
[taskcluster 2020-10-28T15:22:07.029Z]       Result: FAILED
[taskcluster 2020-10-28T15:22:07.029Z] === Task Finished ===
[taskcluster 2020-10-28T15:22:07.029Z] Task Duration: 48.509879943s
[taskcluster 2020-10-28T15:22:07.029Z] [mounts] Preserving cache: Moving "/home/cltbld/tasks/task_1603898477/checkouts" to "/home/cltbld/caches/GwVmquzhREi9FZHItQXy2w"
[taskcluster:error] exit status 1

Comment 1

[Cosmin Sabou [:CosminS]]

Looks like this is from Bug 1638970. https://hg.mozilla.org/integration/autoland/rev/86e3c90d221f5e2beaa75ad5be09433525d4ea76
Ben, could you have a look over these jsshell-bench permafails? Thank you.

Comment 2

[[e-mail] (:bhearsum)]

(In reply to Cosmin Sabou [:CosminS] from comment #1)

Looks like this is from Bug 1638970. https://hg.mozilla.org/integration/autoland/rev/86e3c90d221f5e2beaa75ad5be09433525d4ea76
Ben, could you have a look over these jsshell-bench permafails? Thank you.

Back me out, please. I didn't realize we were running python 3.5(!) - which clearly isn't supported here.

Comment 3

[Cosmin Sabou [:CosminS]]

Fixed by backing out bug 1638970: https://hg.mozilla.org/integration/autoland/rev/9d5406dbcbcfe4904d028904790e5aa7e7b5efbd

Comment 4

[Intermittent Failures Robot]

47 failures in 863 pushes (0.054 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 40
• mozilla-central: 7

Platform breakdown:

• linux64-shippable: 47

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1673914&startday=2020-10-28&endday=2020-10-28&tree=all

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1638970

Comment 6

[Intermittent Failures Robot]

47 failures in 4275 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #19 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 40
• mozilla-central: 7

Platform breakdown:

• linux64-shippable: 47

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1673914&startday=2020-10-26&endday=2020-11-01&tree=all