Closed Bug 1717427 Opened 3 years ago Closed 3 years ago

`mach try fuzzy` fails: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Categories

(Developer Infrastructure :: Try, defect, P2)

Tracking

(firefox91 fixed)

RESOLVED FIXED
91 Branch
Tracking Status
firefox91 --- fixed

People

(Reporter: jwatt, Assigned: ahal)

References

(Blocks 1 open bug)

Details

(Whiteboard: [dev-prod])

Attachments

(2 files)

$ ./mach try fuzzy
No existing task history to clean up.
Error running mach:

    ['try', 'fuzzy']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file try| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

  File "/Users/jwatt/repos/mozilla/u/tools/tryselect/mach_commands.py", line 334, in try_fuzzy
    return self.run(**kwargs)
  File "/Users/jwatt/repos/mozilla/u/tools/tryselect/mach_commands.py", line 194, in run
    return mod.run(**kwargs)
  File "/Users/jwatt/repos/mozilla/u/tools/tryselect/selectors/fuzzy.py", line 448, in run
    return push_to_try(
  File "/Users/jwatt/repos/mozilla/u/tools/tryselect/push.py", line 174, in push_to_try
    display_push_estimates(try_task_config)
  File "/Users/jwatt/repos/mozilla/u/tools/tryselect/push.py", line 137, in display_push_estimates
    download_task_history_data(cache_dir=cache_dir)
  File "/Users/jwatt/repos/mozilla/u/tools/tryselect/util/manage_estimates.py", line 83, in download_task_history_data
    duration_data.append(json.loads(line))
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None

It seems this happens trying to load https://storage.googleapis.com/mozilla-mach-data/task_duration_history.json which when loaded directly returns an XML file containing:

<Error>
<Code>AccessDenied</Code>
<Message>Access denied.</Message>
<Details>
We're sorry, but this service is not available in your location
</Details>
</Error>

It looks like the current exit point of the VPN I'm using got blacklisted by Google, or something like that.

Maybe the code could be modified to check the Content-Type returned by request and give a better error message?

Priority: -- → P2
Whiteboard: [dev-prod]

I think estimates should never cause mach try to error out, at worst it should print a warning.

See Also: → 1606057
See Also: → 1676080, 1661901

A failure displaying estimates should never cause the try push to fail, as it
is entirely optional. This adds a blanket except to catch any exception that
was mishandled inside the estimates submodule.

It still displays a traceback as any failures displayed here should still
legitimately be fixed or have smoother error handling.

Assignee: nobody → ahal
Status: NEW → ASSIGNED

These requests had except blocks to catch all requests exceptions. But because
we weren't calling 'raise_for_status', exceptions were never raised, resulting
in bizarre error tracebacks later on.

Depends on D118356

Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d318b2845c40
[tryselect] Don't allow 'display_estimates' to fail the whole |mach try| run, r=mhentges
https://hg.mozilla.org/integration/autoland/rev/cd15a0ea4cbe
[tryselect] Use 'raise_for_status' when downloading estimates data, r=mhentges
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 91 Branch
Product: Firefox Build System → Developer Infrastructure
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: