Closed Bug 810435 Opened 7 years ago Closed 7 years ago

mach xpcshell-test fails with TypeError: environment can only contain strings

Categories

(Testing :: XPCShell Harness, defect)

x86_64
Windows 7
defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla19

People

(Reporter: mak, Assigned: gps)

Details

(Whiteboard: [mach])

Attachments

(1 file)

I have this failure when trying to run xpcshell-tests, environment is win7 x64 and the common mozilla-build

TypeError: environment can only contain strings

  File "c:\mozilla\mozilla-inbound\testing/xpcshell/mach_commands.py", line 137, in run_xpcshell_test
    xpcshell.run_test(**params)
  File "c:\mozilla\mozilla-inbound\testing/xpcshell/mach_commands.py", line 65, in run_test
    self._run_xpcshell_harness(**args)
  File "c:\mozilla\mozilla-inbound\testing/xpcshell/mach_commands.py", line 111, in _run_xpcshell_harness
    xpcshell.runTests(**args)
  File "c:\mozilla\mozilla-inbound\testing/xpcshell\runxpcshelltests.py", line 801, in runTests
    stdout=pStdout, stderr=pStderr, env=self.env, cwd=testdir)
  File "c:\mozilla\mozilla-inbound\testing/xpcshell\runxpcshelltests.py", line 310, in launchProcess
    env=env, cwd=cwd)
  File "c:\mozilla\mozilla-build\python\lib\subprocess.py", line 679, in __init__
    errread, errwrite)
  File "c:\mozilla\mozilla-build\python\lib\subprocess.py", line 893, in _execute_child
    startupinfo)

I tried to figure out where this comes from, what looks like is that here os.path.join generates a unicode string for 'xpcshell'

http://mxr.mozilla.org/mozilla-central/source/testing/xpcshell/mach_commands.py#82

then here we use xpcshell as xrePath

http://mxr.mozilla.org/mozilla-central/source/testing/xpcshell/runxpcshelltests.py#117

then here we add xrePath to the current $PATH and set it to env['PATH'] that is now unicode

http://mxr.mozilla.org/mozilla-central/source/testing/xpcshell/runxpcshelltests.py#139

finally we pass env to Popen, that complains

http://mxr.mozilla.org/mozilla-central/source/testing/xpcshell/runxpcshelltests.py#309

My workaround was just to:

'xpcshell': os.path.join(self.bindir, 'xpcshell').encode('utf-8'),

but I suppose there is a better solution.
Nice detective work!

I hate this Python bug so much. The problem magically goes away in Python 2.7.3. Since you are on Windows, I'm guessing you are using the Python provided by MozillaBuild, which is Python 2.7.2.

The mach command should likely convert the unicode strings to regular strings before it passes them into runxpcshelltests.py. I think this is the preferred solution because runxpcshelltests.py is not Unicode aware and mach is.
Yes I use that version of python in mozilla-build.
Actually my workaround works only for a full harness, single tests is even worse (there's likely other data passed through with wrong encoding).
This is necessary evil, I reckon. Once MozillaBuild ships with Python 2.7.3, I may change the mach driver to enforce that point release so we don't have to workaround these unicode bugs.
Assignee: nobody → gps
Status: NEW → ASSIGNED
Attachment #681358 - Flags: review?(jhammel)
Component: mach → XPCShell Harness
Product: Core → Testing
Whiteboard: [mach]
Comment on attachment 681358 [details] [diff] [review]
Convert unicode to str

+        for k in args:
+            v = args[k]

I generally do `for k, v in args.items()`
Attachment #681358 - Flags: review?(jhammel) → review+
(In reply to Jeff Hammel [:jhammel] from comment #4)
> Comment on attachment 681358 [details] [diff] [review]
> Convert unicode to str
> 
> +        for k in args:
> +            v = args[k]
> 
> I generally do `for k, v in args.items()`

This is what happens when I write JavaScript for a week. Ugh.
https://hg.mozilla.org/mozilla-central/rev/7fe91eedc16c
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla19
You need to log in before you can comment on or make changes to this bug.