1599744 - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Emilio Cobos Álvarez (:emilio)]

Configure fails tody with:

 0:00.44 /usr/bin/gmake -f client.mk -s configure
 0:00.45 cd /home/emilio/src/moz/gecko/obj-debug-static-analysis
 0:00.45 /home/emilio/src/moz/gecko/configure
 0:00.59 Creating Python 2 environment
 0:00.59 Traceback (most recent call last):
 0:00.59   File "/home/emilio/src/moz/gecko/configure.py", line 170, in <module>
 0:00.59     sys.exit(main(sys.argv))
 0:00.59   File "/home/emilio/src/moz/gecko/configure.py", line 46, in main
 0:00.59     sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
 0:00.59   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 446, in run
 0:00.60     self.include_file(path)
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 437, in include_file
 0:00.60     exec_(code, self)
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/util.py", line 51, in exec_
 0:00.60     exec(object, globals, locals)
 0:00.60   File "/home/emilio/src/moz/gecko/moz.configure", line 7, in <module>
 0:00.60     include('build/moz.configure/init.configure')
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 769, in include_impl
 0:00.60     self.include_file(what)
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 437, in include_file
 0:00.60     exec_(code, self)
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/util.py", line 51, in exec_
 0:00.60     exec(object, globals, locals)
 0:00.60   File "/home/emilio/src/moz/gecko/build/moz.configure/init.configure", line 213, in <module>
 0:00.60     @imports(_from='mozbuild.virtualenv', _import='PY2')
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 752, in decorator
 0:00.60     depends = DependsFunction(self, func, dependencies, when=when)
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 129, in __init__
 0:00.60     sandbox._value_for(self)
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 543, in _value_for
 0:00.60     return self._value_for_depends(obj)
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/util.py", line 997, in method_call
 0:00.60     cache[args] = self.func(instance, *args)
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 552, in _value_for_depends
 0:00.60     value = obj.result()
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/util.py", line 997, in method_call
 0:00.60     cache[args] = self.func(instance, *args)
 0:00.60   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 155, in result
 0:00.61     return self._func(*resolved_args)
 0:00.61   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 1152, in wrapped
 0:00.61     return new_func(*args, **kwargs)
 0:00.61   File "/home/emilio/src/moz/gecko/build/moz.configure/init.configure", line 279, in virtualenv_python2
 0:00.61     manager.build(python)
 0:00.61   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/virtualenv.py", line 493, in build
 0:00.61     self.create(python)
 0:00.61   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/virtualenv.py", line 228, in create
 0:00.61     env=ensure_subprocess_env(os.environ))
 0:00.61   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/virtualenv.py", line 727, in ensure_subprocess_env
 0:00.61     return {ensure(k, encoding): ensure(v, encoding) for k, v in env.iteritems()}
 0:00.61   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/virtualenv.py", line 727, in <dictcomp>
 0:00.61     return {ensure(k, encoding): ensure(v, encoding) for k, v in env.iteritems()}
 0:00.61   File "/home/emilio/src/moz/gecko/python/mozbuild/mozbuild/virtualenv.py", line 715, in ensure_bytes
 0:00.61     return value.encode(encoding)
 0:00.61 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

This is because my $PROMPT has non-ascii characters, but it's a really recent regression.

Comment 1

[Emilio Cobos Álvarez (:emilio)]

Manual bisecting shows this is a regression from bug 1590745.

Comment 2

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1599744 - Ignore more errors in python2 virtualenv stuff. r=mars,#build

Unclear if this is the right thing to do, but this works for me and this is
python2 code only.

Comment 3

[Emilio Cobos Álvarez (:emilio)]

Minimal repro is:

$ python2
value = "╭"
value.encode('utf-8')

Comment 4

[Māris Fogels [:mars] (please needinfo)]

I'm looking into it.

A second repo:

$ ./mach clobber
$ env PROMPT=╭ ./mach configure

Comment 5

[Sylvestre Ledru [:Sylvestre]]

Is it possible to write a test to make sure it won't happen again?

Comment 6

[Māris Fogels [:mars] (please needinfo)]

Attachment: Bug 1599744 - Ensure unicode env vars are handled by virtualenv r?#build

Ensure that environment variables with unicode values are handled correctly
when setting up a virtualenv.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cac335eb3140
Ensure unicode env vars are handled by virtualenv r=ahal

Comment 8

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/cac335eb3140

Comment 9

[Intermittent Failures Robot]

1 failures in 3445 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr91: 1

Platform and build breakdown:

• android-hw-p2-8-0-arm7: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1599744&startday=2022-01-24&endday=2022-01-30&tree=all