1637271 - ./mach try --preset=fpush-linux-android throws python error

Status: RESOLVED FIXED

(Developer Infrastructure :: Try, defect)

Description

[Kartikaya Gupta (email:[e-mail])]

This is a recent regression:

$ ./mach try --preset=fpush-linux-android
No existing task history to clean up.
Error running mach:

    ['try', '--preset=fpush-linux-android']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

TypeError: the JSON object must be str, not 'bytes'

  File "/home/kats/zspace/gecko-wr/tools/tryselect/mach_commands.py", line 207, in try_default
    'try', subcommand=sub, context=self._mach_context, argv=argv, **kwargs)
  File "/home/kats/zspace/gecko-wr/python/mach/mach/registrar.py", line 152, in dispatch
    return self._run_command_handler(handler, context=context, **kwargs)
  File "/home/kats/zspace/gecko-wr/python/mach/mach/registrar.py", line 109, in _run_command_handler
    result = fn(**kwargs)
  File "/home/kats/zspace/gecko-wr/tools/tryselect/mach_commands.py", line 308, in try_fuzzy
    return self.run(**kwargs)
  File "/home/kats/zspace/gecko-wr/tools/tryselect/mach_commands.py", line 178, in run
    return mod.run(**kwargs)
  File "/home/kats/zspace/gecko-wr/tools/tryselect/selectors/fuzzy.py", line 395, in run
    closed_tree=closed_tree)
  File "/home/kats/zspace/gecko-wr/tools/tryselect/push.py", line 147, in push_to_try
    display_push_estimates(try_task_config)
  File "/home/kats/zspace/gecko-wr/tools/tryselect/push.py", line 121, in display_push_estimates
    download_task_history_data(cache_dir=cache_dir)
  File "/home/kats/zspace/gecko-wr/tools/tryselect/util/manage_estimates.py", line 72, in download_task_history_data
    duration_data.append(json.loads(line))
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))

Comment 1

[Kartikaya Gupta (email:[e-mail])]

Workaround: https://hg.mozilla.org/try/rev/0a645c1aeea3652817a29fe71edd16b87fa8980c

Comment 2

[Toshihito Kikuchi [:toshi]]

I'm hitting the same error when running "mach try" on Windows.

$ mach try syntax -b do -p win32,win64,win32-mingwclang,win64-mingwclang,linux32,linux64 -u cppunit,mochitest -t none
['D:\\mozilla-build\\/msys/bin/sh.exe', 'u:/mz/mozilla-central/python/mozbuild/mozbuild/mozconfig_loader', 'u:/mz/mozilla-central', 'U:/mz/mozconfig', 'd:\\mozilla-build\\python3\\python3.exe', 'u:/mz/mozilla-central/python/mozbuild/mozbuild/action/dump_env.py']
('d:\\mozilla-build\\python\\Scripts\\hg.EXE', b'status', b'--modified', b'--added', b'--removed', b'--deleted')
Error running mach:

    ['try', 'syntax', '-b', 'do', '-p', 'win32,win64,win32-mingwclang,win64-mingwclang,linux32,linux64', '-u', 'cppunit,mochitest', '-t', 'none']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

TypeError: a bytes-like object is required, not 'str'

  File "u:\mz\mozilla-central\tools/tryselect/mach_commands.py", line 417, in try_syntax
    return self.run(**kwargs)
  File "u:\mz\mozilla-central\tools/tryselect/mach_commands.py", line 178, in run
    return mod.run(**kwargs)
  File "u:\mz\mozilla-central\tools\tryselect\selectors\syntax.py", line 614, in run
    return at.run(**kwargs)
  File "u:\mz\mozilla-central\tools\tryselect\selectors\syntax.py", line 609, in run
    closed_tree=kwargs["closed_tree"])
  File "u:\mz\mozilla-central\tools\tryselect\push.py", line 144, in push_to_try
    check_working_directory(push)
  File "u:\mz\mozilla-central\tools\tryselect\push.py", line 83, in check_working_directory
    if not vcs.working_directory_clean():
  File "u:\mz\mozilla-central\python/mozversioncontrol\mozversioncontrol\__init__.py", line 412, in working_directory_clean
    return not len(self._run(*args).strip())
  File "u:\mz\mozilla-central\python/mozversioncontrol\mozversioncontrol\__init__.py", line 318, in _run
    return super(HgRepository, self)._run(*args, **runargs)
  File "u:\mz\mozilla-central\python/mozversioncontrol\mozversioncontrol\__init__.py", line 115, in _run
    universal_newlines=True)
  File "d:\mozilla-build\python3\lib\subprocess.py", line 395, in check_output
    **kwargs).stdout
  File "d:\mozilla-build\python3\lib\subprocess.py", line 475, in run
    with Popen(*popenargs, **kwargs) as process:
  File "d:\mozilla-build\python3\lib\subprocess.py", line 778, in __init__
    restore_signals, start_new_session)
  File "d:\mozilla-build\python3\lib\subprocess.py", line 1122, in _execute_child
    args = list2cmdline(args)
  File "d:\mozilla-build\python3\lib\subprocess.py", line 533, in list2cmdline
    needquote = (" " in arg) or ("\t" in arg) or not arg

Comment 3

[Andrew Halberstadt [:ahal]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #1)

Workaround: https://hg.mozilla.org/try/rev/0a645c1aeea3652817a29fe71edd16b87fa8980c

I'd accept this patch, Kats want to throw it up on phabricator? Or want me to do it?

Comment 4

[Andrew Halberstadt [:ahal]]

(In reply to Toshihito Kikuchi [:toshi] from comment #2)

I'm hitting the same error when running "mach try" on Windows.

Note this is a different error, though also caused by the switch to Python 3. Filed bug 1637305 to track it.

Comment 6

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1637271 - Avoid passing a bytes obj to json.loads with Python3. r?ahal

Comment 7

[Kartikaya Gupta (email:[e-mail])]

Done

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/66d1d4f318c2
Avoid passing a bytes obj to json.loads with Python3. r=ahal

Comment 10

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/66d1d4f318c2

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1632429