1637232 - ./mach try release ... --migration fails in tools/tryselect/push.py :: display_push_estimates

Status: RESOLVED DUPLICATE of bug 1637271

(Developer Infrastructure :: Try, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This is in a VirtualBox with Ubuntu 16.04 LTS, haven't tested other setups:

$ ./mach try release -v 78.0b12 --tasks release-sim --migration central-to-beta --migration central-to-beta --migration early-to-late-beta --no-push
No existing task history to clean up.
Error running mach:

    ['try', 'release', '-v', '78.0b12', '--tasks', 'release-sim', '--migration', 'central-to-beta', '--migration', 'central-to-beta', '--migration', 'early-to-late-beta', '--no-push']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

TypeError: the JSON object must be str, not 'bytes'

  File "/home/user/mozilla/mozilla-unified-trunk/tools/tryselect/mach_commands.py", line 435, in try_release
    return self.run(**kwargs)
  File "/home/user/mozilla/mozilla-unified-trunk/tools/tryselect/mach_commands.py", line 178, in run
    return mod.run(**kwargs)
  File "/home/user/mozilla/mozilla-unified-trunk/tools/tryselect/selectors/release.py", line 126, in run
    files_to_change=files_to_change,
  File "/home/user/mozilla/mozilla-unified-trunk/tools/tryselect/push.py", line 147, in push_to_try
    display_push_estimates(try_task_config)
  File "/home/user/mozilla/mozilla-unified-trunk/tools/tryselect/push.py", line 121, in display_push_estimates
    download_task_history_data(cache_dir=cache_dir)
  File "/home/user/mozilla/mozilla-unified-trunk/tools/tryselect/util/manage_estimates.py", line 72, in download_task_history_data
    duration_data.append(json.loads(line))
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))

Directly exiting display_push_estimates by adding a return at the start lets the push to Try succeed. Before this change, the estimation for these Try pushes was bogus (0 seconds).

Comment 1

[Andrew Halberstadt [:ahal]]

If they were bogus before, you should add release to this list:
https://searchfox.org/mozilla-central/source/tools/tryselect/push.py#145

Though this may indicate a problem with the estimates code on Python 3 that should be fixed in addition.

Comment 2

[Simon Fraser [:sfraser] ⌚️GMT]

./mach try release will show 0 duration as mach doesn't know which tasks are part of a release, that's decided in the decision task. We could take the task graph generation further - optimise, filter and so on, but it's a lot of extra local resources

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

Seems to require an older version of python. It won't repro with 3.8, but will with 3.5.9

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1637232: Don't show time estimates for |./mach try release| r?rstewart

Mach doesn't know which tasks are part of a release - that's decided in the decision task. Instead of
showing a bogus estimation, we shouldn't show one at all.

Comment 5

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1637232: ensure JSON string is text r?rstewart

Older versions of Python 3 require that a text string (not bytes) is provided to json.loads(...).

Comment 6

[Phabricator Automation]

Comment on attachment 9147704 [details]
Bug 1637232: Don't show time estimates for |./mach try release| r?rstewart

Revision D74904 was moved to bug 1637354. Setting attachment 9147704 [details] to obsolete.

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1632429

Comment 9

[Andrew Halberstadt [:ahal]]

Not sure why phabricator automation re-opened this...