Closed Bug 1557336 Opened 1 year ago Closed 1 year ago

Permy macOS-py2(ml) ValueError: unknown locale: UTF-8

Categories

(Firefox Build System :: General, defect, P5)

defect

Tracking

(firefox-esr60 unaffected, firefox67 unaffected, firefox68 fixed, firefox69 fixed)

RESOLVED FIXED
mozilla69
Tracking Status
firefox-esr60 --- unaffected
firefox67 --- unaffected
firefox68 --- fixed
firefox69 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: glandium)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Attachments

(1 file)

Filed by: nbeleuzu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=250345312&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/IVtapO-iTv6Q2A07f3Nozg/runs/0/artifacts/public/logs/live_backing.log


[task 2019-06-06T14:13:53.165Z] File "/Users/cltbld/tasks/task_1559830198/checkouts/gecko/obj-x86_64-apple-darwin14.5.0/_virtualenvs/init/lib/python2.7/site-packages/pipenv/patched/pew/_utils.py", line 25, in <module>
[task 2019-06-06T14:13:53.165Z] encoding = locale.getlocale()[1] or 'ascii'
[task 2019-06-06T14:13:53.165Z] File "/Users/cltbld/tasks/task_1559830198/checkouts/gecko/obj-x86_64-apple-darwin14.5.0/_virtualenvs/init/lib/python2.7/locale.py", line 562, in getlocale
[task 2019-06-06T14:13:53.165Z] return _parse_localename(localename)
[task 2019-06-06T14:13:53.165Z] File "/Users/cltbld/tasks/task_1559830198/checkouts/gecko/obj-x86_64-apple-darwin14.5.0/_virtualenvs/init/lib/python2.7/locale.py", line 475, in _parse_localename
[task 2019-06-06T14:13:53.165Z] raise ValueError, 'unknown locale: %s' % localename
[task 2019-06-06T14:13:53.165Z] ValueError: unknown locale: UTF-8
[task 2019-06-06T14:13:53.190Z] Error running mach:
[task 2019-06-06T14:13:53.190Z]
[task 2019-06-06T14:13:53.190Z] ['python-test', '--python', '2', '--subsuite', 'mozlint']
[task 2019-06-06T14:13:53.190Z]
[task 2019-06-06T14:13:53.190Z] The error occurred in code that was called by the mach command. This is either
[task 2019-06-06T14:13:53.190Z] a bug in the called code itself or in the way that mach is calling it.
[task 2019-06-06T14:13:53.190Z] You can invoke |./mach busted| to check if this issue is already on file. If it
[task 2019-06-06T14:13:53.190Z] isn't, please use |./mach busted file| to report it. If |./mach busted| is
[task 2019-06-06T14:13:53.190Z] misbehaving, you can also inspect the dependencies of bug 1543241.
[task 2019-06-06T14:13:53.190Z]
[task 2019-06-06T14:13:53.190Z] If filing a bug, please include the full output of mach, including this error
[task 2019-06-06T14:13:53.190Z] message.
[task 2019-06-06T14:13:53.190Z]
[task 2019-06-06T14:13:53.190Z] The details of the failure are as follows:
[task 2019-06-06T14:13:53.190Z]
[task 2019-06-06T14:13:53.190Z] CalledProcessError: Command '[u'/Users/cltbld/tasks/task_1559830198/checkouts/gecko/obj-x86_64-apple-darwin14.5.0/_virtualenvs/init/bin/pipenv', u'--python', '2']' returned non-zero exit status 1
[task 2019-06-06T14:13:53.190Z]
[task 2019-06-06T14:13:53.190Z] File "/Users/cltbld/tasks/task_1559830198/checkouts/gecko/python/mach_commands.py", line 99, in python_test
[task 2019-06-06T14:13:53.190Z] return self.run_python_tests(*args, **kwargs)
[task 2019-06-06T14:13:53.190Z] File "/Users/cltbld/tasks/task_1559830198/checkouts/gecko/python/mach_commands.py", line 113, in run_python_tests
[task 2019-06-06T14:13:53.190Z] self.activate_pipenv(pipfile=None, populate=True, python=python)
[task 2019-06-06T14:13:53.190Z] File "/Users/cltbld/tasks/task_1559830198/checkouts/gecko/python/mozbuild/mozbuild/base.py", line 854, in activate_pipenv
[task 2019-06-06T14:13:53.190Z] self.virtualenv_manager.activate_pipenv(pipfile, populate, python)
[task 2019-06-06T14:13:53.190Z] File "/Users/cltbld/tasks/task_1559830198/checkouts/gecko/python/mozbuild/mozbuild/virtualenv.py", line 628, in activate_pipenv
[task 2019-06-06T14:13:53.190Z] self.virtualenv_root = ensure_venv()
[task 2019-06-06T14:13:53.190Z] File "/Users/cltbld/tasks/task_1559830198/checkouts/gecko/python/mozbuild/mozbuild/virtualenv.py", line 609, in ensure_venv
[task 2019-06-06T14:13:53.190Z] env=env)
[task 2019-06-06T14:13:53.190Z] File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 540, in check_call
[task 2019-06-06T14:13:53.190Z] raise CalledProcessError(retcode, cmd)
[taskcluster 2019-06-06T14:13:53.246Z] Exit Code: 1
[taskcluster 2019-06-06T14:13:53.246Z] User Time: 4m13.634299s
[taskcluster 2019-06-06T14:13:53.246Z] Kernel Time: 2m29.373435s
[taskcluster 2019-06-06T14:13:53.246Z] Wall Time: 3m43.890867499s
[taskcluster 2019-06-06T14:13:53.246Z] Result: FAILED
[taskcluster 2019-06-06T14:13:53.246Z] === Task Finished ===
[taskcluster 2019-06-06T14:13:53.246Z] Task Duration: 3m43.8953455s
[taskcluster 2019-06-06T14:13:53.246Z] [mounts] Preserving cache: Moving "/Users/cltbld/tasks/task_1559830198/checkouts" to "/Users/cltbld/caches/ElRMXJN9TC-S-97w-tcJWg"
[taskcluster:error] exit status 1

Seems to be a regression from bug 1534578. Failing job is tier 2. Please take a look when you find time for it.

Flags: needinfo?(mh+mozilla)
Regressed by: 1534578
Summary: Intermittent ValueError: unknown locale: UTF-8 → Permy macOS-py2(ml) ValueError: unknown locale: UTF-8

For some reason, python 3.7 sets LC_CTYPE to UTF-8 and passes it down. That didn't happen with python 3.6 at all (LC_CTYPE was not set). There's probably a legitimate reason why python 3.7 sets LC_CTYPE, but UTF-8 is clearly a broken value.

log of running env wrapped with python 3.6:
https://tools.taskcluster.net/groups/NS1xFV8ATj6FI1vtzyRZaA/tasks/NS1xFV8ATj6FI1vtzyRZaA/runs/0/logs/public%2Flogs%2Flive_backing.log

log of running env wrapped with python 3.7:
https://tools.taskcluster.net/groups/Rj4BvBhoRnGn0-mzTrpaBQ/tasks/Rj4BvBhoRnGn0-mzTrpaBQ/runs/0/logs/public%2Flogs%2Flive_backing.log

Flags: needinfo?(mh+mozilla) → needinfo?(dcrisan)

Looks related to PEP 538.

It is PEP 538, and setting PYTHONCOERCECLOCALE to 0 prevents python from setting the environment variable. That has some other side effects that should be double-checked, though.

Flags: needinfo?(dcrisan)
Flags: needinfo?(mh+mozilla)
Assignee: nobody → mh+mozilla
Flags: needinfo?(mh+mozilla)
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/86f8d38c42c9
Unset LC_CTYPE when executing pipenv. r=chmanchester
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla69
Whiteboard: [checkin-needed-beta]
You need to log in before you can comment on or make changes to this bug.