1585258 - TypeError: environment can only contain strings

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Building on Win10 ends with the following error:

0:52.78 Adding make options from c:\src\mozilla.org\mozilla-central2\.mozconfig
    AUTOCLOBBER=1
    MOZ_OBJDIR=c:/src/mozilla.org/mozilla-central2/obj-x86_64-pc-mingw32
    OBJDIR=c:/src/mozilla.org/mozilla-central2/obj-x86_64-pc-mingw32
    FOUND_MOZCONFIG=c:/src/mozilla.org/mozilla-central2/.mozconfig
    export FOUND_MOZCONFIG
 0:52.82 c:\mozilla-build\bin\mozmake.EXE -f client.mk -s
 0:55.64 Error running mach:
 0:55.64     ['--log-no-times', 'artifact', 'install']
 0:55.64 The error occurred in code that was called by the mach command. This is either
 0:55.64 a bug in the called code itself or in the way that mach is calling it.
 0:55.64 You can invoke |./mach busted| to check if this issue is already on file. If it
 0:55.64 isn't, please use |./mach busted file| to report it. If |./mach busted| is
 0:55.64 misbehaving, you can also inspect the dependencies of bug 1543241.
 0:55.65 If filing a bug, please include the full output of mach, including this error
 0:55.65 message.
 0:55.65 The details of the failure are as follows:
 0:55.65 TypeError: environment can only contain strings
 0:55.65   File "c:/src/mozilla.org/mozilla-central2\python/mozbuild/mozbuild/artifact_commands.py", line 143, in artifact_install
 0:55.65     return artifacts.install_from(source, distdir or self.distdir)
 0:55.65   File "c:/src/mozilla.org/mozilla-central2\python/mozbuild\mozbuild\artifacts.py", line 1259, in install_from
 0:55.65     return self.install_from_recent(distdir)
 0:55.65   File "c:/src/mozilla.org/mozilla-central2\python/mozbuild\mozbuild\artifacts.py", line 1179, in install_from_recent
 0:55.65     return self._install_from_hg_pushheads(hg_pushheads, distdir)
 0:55.66   File "c:/src/mozilla.org/mozilla-central2\python/mozbuild\mozbuild\artifacts.py", line 1158, in _install_from_hg_pushheads
 0:55.66     for trees, hg_hash in hg_pushheads:
 0:55.66   File "c:/src/mozilla.org/mozilla-central2\python/mozbuild\mozbuild\artifacts.py", line 1037, in _find_pushheads
 0:55.66     last_revs = self._get_recent_public_revisions()
 0:55.66   File "c:/src/mozilla.org/mozilla-central2\python/mozbuild\mozbuild\artifacts.py", line 998, in _get_recent_public_revisions
 0:55.66     cwd=self._topsrcdir).splitlines()
 0:55.66   File "c:/src/mozilla.org/mozilla-central2\python/mozbuild\mozbuild\artifacts.py", line 887, in run_hg
 0:55.66     **kwargs)
 0:55.66   File "c:\mozilla-build\python\lib\subprocess.py", line 216, in check_output
 0:55.66     process = Popen(stdout=PIPE, *popenargs, **kwargs)
 0:55.66   File "c:\mozilla-build\python\lib\subprocess.py", line 394, in __init__
 0:55.67     errread, errwrite)
 0:55.67   File "c:\mozilla-build\python\lib\subprocess.py", line 644, in _execute_child
 0:55.67     startupinfo)
 0:55.67 mozmake.EXE[3]: *** [Makefile:159: recurse_artifact] Error 1
 0:55.67 mozmake.EXE[2]: *** [c:/src/mozilla.org/mozilla-central2/config/recurse.mk:34: artifact] Error 2
 0:55.67 mozmake.EXE[1]: *** [c:/src/mozilla.org/mozilla-central2/config/rules.mk:389: default] Error 2
 0:55.68 mozmake.EXE: *** [client.mk:125: build] Error 2
 0:55.70 0 compiler warnings present.

Honza

Comment 1

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Bug 1578198 sounds similar.

Honza

Comment 2

[Marco Bonardo [:mak]]

To workaround the problem try backing out bug 1583234 (changeset e72742f591f8997ac73863ad321f66a353923076) locally.

Comment 3

[Chris Manchester (limited bugmail, email directly)]

Andrew, what's the way forward here, do we need to call ensure_subprocess_env whenever we touch environment variables now?

Comment 4

[Andrew Halberstadt [:ahal]]

Yes calling that method should fix this. Though this issue predates the Python 3 migration and is why we historically defined env like env[b"FOO"] = b"bar".

Details: With Python 2 certain Windows environments (but not all) require env to be bytes in the subprocess module. But on Python 3 passing bytes into subprocess is an error. So ensure_subprocess_env will convert the environment into the proper format depending on Python version.

Comment 5

[Chris Manchester (limited bugmail, email directly)]

Attachment: Bug 1585258 - Call `ensure_subprocess_env` when setting environment variables during an artifact build.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/388f454a9d3d
Call `ensure_subprocess_env` when setting environment variables during an artifact build. r=ahal

Comment 7

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/388f454a9d3d