1699457 - Mach Vendor fails to pull from some git repos

Status: RESOLVED DUPLICATE of bug 1714244

(Developer Infrastructure :: Mach Vendor & Updatebot, defect, P2)

Description

[June Wilde (she/her) [:jewilde]]

Right now we assume that the default branch in the upstream repo for third party libraries that updatebot can update is 'master', which is not always true.

We should either see if we can detect what the default branch is or have a field in moz.yaml files for manually setting what branch to check.

Comment 1

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

https://searchfox.org/mozilla-central/rev/200bfc652c5128011e7725fc7c201eb125d453e7/python/mozbuild/mozbuild/vendor/moz_yaml.py#367 ?

It's not used for vendoring tasks right now I don't think (we can put that bug in the github repo); but it's there and is used for commit alert tasks...

Comment 2

[June Wilde (she/her) [:jewilde]]

Sorry, I forgot to add the error I was actually seeing, and also what I wrote down for the bug was slightly wrong :S
When running ./mach vendor media/libpng/moz.yaml with an upstream github repo set that doesn't have a master branch I get:

╰─$ ./mach vendor media/libpng/moz.yaml
 0:01.12 Starting new HTTPS connection (1): api.github.com
Error running mach:

    ['vendor', 'media/libpng/moz.yaml']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file vendor| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://api.github.com/repos/metalcanine/libpng/commits/master

  File "/Users/jewilde/Source/holly/python/mozbuild/mozbuild/vendor/mach_commands.py", line 91, in vendor
    vendor_command.vendor(library, manifest, revision, check_for_update)
  File "/Users/jewilde/Source/holly/python/mozbuild/mozbuild/vendor/vendor_manifest.py", line 33, in vendor
    commit, timestamp = self.source_host.upstream_commit(revision)
  File "/Users/jewilde/Source/holly/python/mozbuild/mozbuild/vendor/host_github.py", line 22, in upstream_commit
    req.raise_for_status()
  File "/Users/jewilde/Source/holly/third_party/python/requests/requests/models.py", line 840, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

Sentry event ID: b128e31efa4841b5a96ba626e0d38360
Sentry is attempting to send 0 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit

Which I think stems from https://searchfox.org/mozilla-central/source/python/mozbuild/mozbuild/vendor/mach_commands.py#86 and also what you pointed out of us not using that moz.yaml field for vendor tasks.

I did this with the patches to add libpng to updatebot's repos checked out, but I don't think that those are affecting this at all (i think)

Comment 3

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Ahha okay; yeah then I think we need to update both that bit of code and the vendoring bit of updatebot to use the branch specified in the moz.yaml

Comment 4

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Re-assigning bugs to the new Mach Vendor and Updatebot Component. Guid for this change: ade7b229-ec85-41f8-bafe-9d4451bfeed1

Comment 5

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

A mass updatebot change inadvertently added a security group to bugs. Removing it. Guid for this change: 09d63db4-11a1-4820-a398-d718f112f885