1047501 - Make b2g-manifests less fragile to changes which require mirroring

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Pete Moore [:pmoore][:pete]]

In light of e.g. https://bugzilla.mozilla.org/show_bug.cgi?id=1029342#c9 it is clear that making changes to b2g-manifest that involve pulling in new repos are risky:
  * there is no obvious way to test the changes, you just have to land, and hope you did it right
  * the mappings e.g. in http://hg.mozilla.org/build/mozharness/file/26e71e92141f/configs/b2g_bumper/master.py#l72 are outside of b2g-manifest repo, are different per branch, and not immediately obvious
  * devs don't always know that releng infrastructure is using git mirrors
  * devs might not realise which "remote" is being used (although they can derive this from close inspection of b2g-manifest code) so may request mirroring from wrong source repo (as happened in bug 1029342)

This bug aims to simplify this process, and implement steps to reduce chances of similar failures in future.

Comment 1

[Michael Wu [:mwu]]

The manifest changes really aren't risky *at all*. They're fully testable locally. The real problem comes with incorrect mirror requests. Feel free to get review from me or someone familiar with the source repo if you want additional assurance.

Comment 2

[Pete Moore [:pmoore][:pete]]

Hi Michael,

That's appreciated. The only gap I see there is if a dev doesn't know they need to make a mirror request - this was the first problem we had here. So maybe it slips through (as before) without you or us knowing, because we are out-of-the-loop.

I agree the manifest changes can be tested locally from source repos, but there is not a way to test pulling from mirrors, as far as I know, that pulls in the exact configs we have in mozharness with the source->mirror url path mappings (http://hg.mozilla.org/build/mozharness/file/26e71e92141f/configs/b2g_bumper/master.py#l72) - let me know if I am wrong about this.

Pete

Comment 3

[Michael Wu [:mwu]]

(In reply to Pete Moore [:pete][:pmoore] from comment #2)
> That's appreciated. The only gap I see there is if a dev doesn't know they
> need to make a mirror request - this was the first problem we had here. So
> maybe it slips through (as before) without you or us knowing, because we are
> out-of-the-loop.
> 

I always tell people when they need to get a mirror before merging. If people are getting reviews for manifest changes from other people.. well they probably shouldn't for Tier 1 devices visible on tbpl.

> I agree the manifest changes can be tested locally from source repos, but
> there is not a way to test pulling from mirrors, as far as I know, that
> pulls in the exact configs we have in mozharness with the source->mirror url
> path mappings
> (http://hg.mozilla.org/build/mozharness/file/26e71e92141f/configs/b2g_bumper/
> master.py#l72) - let me know if I am wrong about this.
> 

The mirror part is more difficult to test. I don't know of an easy way to fix this though, other than more thorough reviews.

Comment 4

[Michael Wu [:mwu]]

Actually, the best way to fix the mirror request part is to automate it. It's possible to create a script to automatically determine missing mirrors after a manifest update and generate mirror requests.

Comment 5

[Hal Wine [:hwine] use NI!]

While there's a bug for automation, it won't address problems like bug 1029342. There the issue is the dev doesn't know where the proper upstream repository is located. Hmm bug 1031378 is all I find atm.

Based on this exchange, it sounds like a reasonable short term process is to ask :mwu to approve all non-CAF mirror requests.

:mwu - does that sound okay? And who can cover if you're on PTO?

Comment 6

[Michael Wu [:mwu]]

For the short term, that's fine. If something is really urgent, I think John Ford can cover for me.

By automation, I mean that all requests should be generated using a script. The dev doesn't need to know where the right upstream is - the script figures it out.

Comment 7

[Hal Wine [:hwine] use NI!]

(In reply to Michael Wu [:mwu] from comment #6)
> By automation, I mean that all requests should be generated using a script.
> The dev doesn't need to know where the right upstream is - the script
> figures it out.

I'm confused by this. I thought the use case we are discussing is "as a dev, I have a new upstream repo I want to make part of the build process".

Given the upstream repo URL, it's clear to me how to set up mirroring that works with all the magic bits of the releng environment.

However, I do expect the dev to have the knowledge of _which_ upstream library is needed/preferred for the particular device(s).

If it is safe to assume that you're talking about some b2g automation somewhere that can (eventually) provide releng with the upstream URL, then I'm unconfused :)