1975187 - Unable to submit try patches involving non-UTF8 encodable bytes

Status: NEW

(Conduit :: Lando, defect)

Description

[James Graham [:jgraham]]

./mach try for a specific set of patches I have ends up with

Submitting patches to Lando.
abort: error submitting patches to Lando.
Patch does not match expected format `git-format-patch`: 'utf-8' codec can't decode byte 0x88 in position 63919418: invalid start byte

The patches contain some files with bytes that can't be encoded as UTF-8. This kind of thing is not very common, but also not that rare for e.g. test data.

On the Firefox side we correctly encode the entire patch as base64 before embedding it into the JSON object that gets submitted. However on the lando side we have decode_json_patch_to_text which makes the incorrect assumption that any patch must have a UTF8 string representation.

Unfortunately the assumption that we're working with string data rather than arbitrary bytes is then embedded quite deeply in the rest of the code. Both GitPatchHelper and (for consistency) HgPatchHelper also need to be updated to ensure that they allow at least the changes in a patch (if not the message or author) to be expressed as bytes. Then Revision also needs to be updated, which unfortunately looks like a schema change.

I don't know if this is fixed in the non-legacy backend, but if not it's a hard blocker for landing specific patches.

Comment 1

[Zeid [:zeid]]

Side note: try pushes are still going through the legacy backend (i.e., this repo). There are some differences between the legacy and new backend although the latter was originally ported from the former. Likely the part of the code that is affecting this is very similar in both.