Open Bug 2023776 Opened 8 months ago Updated 1 month ago

mach vendor rust lint job Perma on certain pushes UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 62352: invalid start byte

Categories

(Developer Infrastructure :: Mach Vendor & Updatebot, defect, P5)

defect

Tracking

(Not tracked)

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure)

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=554054084&repo=autoland&task=W42SdLhHSL2J2jqDDjhhWw.0
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/W42SdLhHSL2J2jqDDjhhWw/runs/0/artifacts/public/logs/live_backing.log


[task 2026-03-17T02:54:05.300+00:00]    Vendoring zerovec v0.11.4 (/builds/worker/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/zerovec-0.11.4) to /builds/worker/checkouts/gecko/third_party/rust/zerovec
[task 2026-03-17T02:54:05.308+00:00]    Vendoring zerovec-derive v0.11.1 (/builds/worker/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/zerovec-derive-0.11.1) to /builds/worker/checkouts/gecko/third_party/rust/zerovec-derive
[task 2026-03-17T02:54:05.311+00:00]    Vendoring zlib-rs v0.6.3 (/builds/worker/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/zlib-rs-0.6.3) to /builds/worker/checkouts/gecko/third_party/rust/zlib-rs
[task 2026-03-17T02:54:05.319+00:00]    Vendoring zmij v1.0.20 (/builds/worker/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/zmij-1.0.20) to /builds/worker/checkouts/gecko/third_party/rust/zmij
[task 2026-03-17T02:54:05.335+00:00] To use vendored sources, add this to your .cargo/config.toml for this project:
[task 2026-03-17T02:54:05.335+00:00] 
[task 2026-03-17T02:54:24.277+00:00] Creating default state directory: /builds/worker/.mozbuild
[task 2026-03-17T02:54:24.277+00:00] Running "pip check" to verify compatibility between the system Python and the "mach" site.
[task 2026-03-17T02:54:24.277+00:00] Creating local state directory: /builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69
[task 2026-03-17T02:54:24.277+00:00] Running "pip check" to verify compatibility between the system Python and the "common" site.
[task 2026-03-17T02:54:24.277+00:00] Creating the 'common' site at /builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69/_virtualenvs/common
[task 2026-03-17T02:54:24.277+00:00] Error running mach:
[task 2026-03-17T02:54:24.277+00:00] 
[task 2026-03-17T02:54:24.277+00:00]     mach vendor rust --issues-json /builds/worker/issues.json --vcs-diff /builds/worker/vcs-diff.diff
[task 2026-03-17T02:54:24.277+00:00] 
[task 2026-03-17T02:54:24.277+00:00] The error occurred in code that was called by the mach command. This is either
[task 2026-03-17T02:54:24.277+00:00] a bug in the called code itself or in the way that mach is calling it.
[task 2026-03-17T02:54:24.277+00:00] You can invoke ``./mach busted`` to check if this issue is already on file. If it
[task 2026-03-17T02:54:24.277+00:00] isn't, please use ``./mach busted file vendor`` to report it. If ``./mach busted`` is
[task 2026-03-17T02:54:24.277+00:00] misbehaving, you can also inspect the dependencies of bug 1543241.
[task 2026-03-17T02:54:24.277+00:00] 
[task 2026-03-17T02:54:24.277+00:00] If filing a bug, please include the full output of mach, including this error
[task 2026-03-17T02:54:24.277+00:00] message.
[task 2026-03-17T02:54:24.277+00:00] 
[task 2026-03-17T02:54:24.277+00:00] The details of the failure are as follows:
[task 2026-03-17T02:54:24.277+00:00] 
[task 2026-03-17T02:54:24.277+00:00] Traceback (most recent call last):
[task 2026-03-17T02:54:24.277+00:00]   File "/builds/worker/checkouts/gecko/python/mach/mach/main.py", line 416, in _run
[task 2026-03-17T02:54:24.277+00:00]     return Registrar._run_command_handler(
[task 2026-03-17T02:54:24.277+00:00]   File "/builds/worker/checkouts/gecko/python/mach/mach/registrar.py", line 124, in _run_command_handler
[task 2026-03-17T02:54:24.277+00:00]     result = fn(instance, **kwargs)
[task 2026-03-17T02:54:24.277+00:00]   File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/vendor/mach_commands.py", line 226, in vendor_rust
[task 2026-03-17T02:54:24.277+00:00]     shutil.copyfileobj(vendor_command.generate_diff_stream(), fh)
[task 2026-03-17T02:54:24.277+00:00]   File "/builds/worker/fetches/python/lib/python3.9/shutil.py", line 205, in copyfileobj
[task 2026-03-17T02:54:24.277+00:00]     buf = fsrc_read(length)
[task 2026-03-17T02:54:24.277+00:00]   File "/builds/worker/fetches/python/lib/python3.9/codecs.py", line 322, in decode
[task 2026-03-17T02:54:24.277+00:00]     (result, consumed) = self._buffer_decode(data, self.errors, final)
[task 2026-03-17T02:54:24.277+00:00] UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 62352: invalid start byte
[taskcluster 2026-03-17T02:54:26.994Z]                        Exit Code: 1
[taskcluster 2026-03-17T02:54:26.994Z]                        User Time: 39.792ms
[taskcluster 2026-03-17T02:54:26.994Z]                      Kernel Time: 63.909ms
[taskcluster 2026-03-17T02:54:26.994Z]                        Wall Time: 4m38.483430018s
[taskcluster 2026-03-17T02:54:26.994Z]  Average Available System Memory: 28.81 GiB
[taskcluster 2026-03-17T02:54:26.994Z]       Average System Memory Used: 2.53 GiB
[taskcluster 2026-03-17T02:54:26.994Z]          Peak System Memory Used: 4.93 GiB
[taskcluster 2026-03-17T02:54:26.994Z]              Total System Memory: 31.34 GiB
[taskcluster 2026-03-17T02:54:26.994Z]                           Result: FAILED
[taskcluster 2026-03-17T02:54:26.994Z] === Task Finished ===
[taskcluster 2026-03-17T02:54:26.994Z] Task Duration: 4m38.48390631s
[taskcluster 2026-03-17T02:54:29.146Z] [mounts] Preserving cache: Moving "/home/task_177371576483916/cache0" to "/home/generic-worker/caches/O4AqtOrFTje29PU3gy6acQ"
[taskcluster:error] exit status 1

See Also: → 2023513

This failed also on a backout here as we thought it is tree related.
You can see the behavior, on initial land orange, on the backout orange and in between (jobs were added later on) is green.

[task 2026-03-17T02:54:24.277+00:00] UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 62352: invalid start byte

FWIW, the way we fixed extremely similar issues in other codebase was to pass errors="surrogateescape" to the encode and decode calls.

https://docs.python.org/3/library/codecs.html#error-handlers:~:text=surrogateescape

Blocks: 2023197
Summary: Perma on certain pushes UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 62352: invalid start byte → mach vendor rust lint job Perma on certain pushes UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 62352: invalid start byte
You need to log in before you can comment on or make changes to this bug.