1311001 - Make shipit the source of truth for watershed information

Status: NEW

(Release Engineering :: Applications: Shipit, defect, P3)

Description

[Chris AtLee [:catlee]]

We need a canonical place to describe the various update watersheds, and I think that shipit is the natural place for that to live.

Comment 1

[[e-mail] (:bhearsum)]

Assuming we can limit this to true watersheds (that is, releases that users must route through before getting the latest release), the format for this should be quite simple. I had a look over the watersheds currently in the system and it looks like we have some all-platform watersheds and some platform specific ones. I couldn't find any locale-specific watersheds.

An endpoint that returrns data like this should provide what we need:
[
    {
        "watershed_release": "Firefox-12.0-build1",
        "platforms": [],
        "comment": "watershed for new-style build targets - bug xxxxxxxx"
    },
    {
        "watershed_release": "Firefox-45.0.2-build1",
        "platforms": ["Linux_x86-gcc3", "Linux_x86_64-gcc3"],
        "comment": "add GTK 3 support to Linux update URLs - bug xxxxxxxx"
    }
]

Technically, these watersheds are channel-specific as well. I think we should have one data structure per channel (which maps to a branch in ship-it) to handle this. So maybe we'd end with endpoints like:
/watersheds?branch=releases/mozilla-release. I'd also suggest that we simply make this a static file to begin with - watersheds are relative infrequent, and I don't think it's worth the investment upfront to even build CRUD ui for manipulating them.

I want to explicitly state that this idea does _not_ handle the following things that some people may think of as watersheds:
- Updating users to their last supported release before deprecating them
- OS deprecation
- Blocking users who can't update for some reason (bad jaws, websense, etc.)
- 32 -> 64-bit migration

If it's important to handle these we'll need to fiddle with the data format a bit.

Comment 2

[[e-mail] (:bhearsum)]

Sylvestre, I see you were heavily involved with the new product-details.mozilla.org. I was thinking that publishing watersheds at https://product-details.mozilla.org/1.0/firefox_watersheds.json might be a good place -- they're useful to automation, but probably useful to publish more generally, too. Do you have any thoughts on this?

Comment 3

[Sylvestre Ledru [:Sylvestre]]

I agree that we should publish these information on p-d.m.o
(and this is a great idea)

Comment 4

[[e-mail] (:bhearsum)]

Chris, what do you think of comment #1? Do you think we need to include some of the things I've explicitly excluded? Do we need more specifity (like versions)? What do you think of the format?

Anyone else is welcome to chime in as well.

Comment 5

[[e-mail] (:bhearsum)]

Attachment: firefox_watersheds.json strawman

This is more or less the format I proposed, except I stopped explicitly referencing Balrog releases, because for certain watersheds (like 56.0 on the release channel), it takes multiple rules/releases to express them.

Comment 6

[Chris AtLee [:catlee]]

What are the intended use cases of this data?

If we want to consume this information from automated tools, then I think the "watershed_release" field needs to be better defined. I like having the buildid in there, especially for nightly/old-aurora.

What is your concern with 56.0 and multiple rules/releases?

Comment 7

[[e-mail] (:bhearsum)]

(In reply to Chris AtLee [:catlee] from comment #6)
> What are the intended use cases of this data?

The specific use case right now is "find the last watershed by platform and channel".

> If we want to consume this information from automated tools, then I think
> the "watershed_release" field needs to be better defined. I like having the
> buildid in there, especially for nightly/old-aurora.

Which tools are you thinking of?

Update & final verify just need to know what the last watershed is (to know the oldest release to test the current release with).

If we want to write something that, eg: verifies that Balrog correctly implements watersheds, you're right that we need buildids to do that. If Ship It ends up as the source of truth for them, I think it makes a lot of sense to add them here. If it doesn't, I'd argue that such a tool should grab it from whatever the source of truth for buildids is.

I can also see the argument for publishing most, or possibly all, of the information that Ship It has about a release in the watersheds endpoint. Eg: revision, locales.

> What is your concern with 56.0 and multiple rules/releases?

It feels like this is just a byproduct of limitations of Balrog. If I'm a tool or human that cares about "Which versions does Firefox watershed at?", the fact that we need multiple rules in Balrog due to something like lzma vs bz2 isn't something that I care about - it's a Balrog implementation detail.

If we were to add multiple entries per version in the watersheds export, how do we express it? 

Do we s/watershed_release/watershed_releases/? How do you know which one applies to platform X?

Or do we make one entry per Balrog rule? Doesn't that increase complexity for consumers?

Comment 8

[Rail Aliiev [:rail]]

I have some questions:

1) what do we maintain the data? I assume this is going to be a manual process in the beginning. Probably via releasewarrior.

2) Can we use these data to test Balrog rules?

3) Should we mention the target version in the rules. For example, "30.0.1 should update to 39.0", "39.0 should update to latest". This is kind of related to 2)

Comment 9

[[e-mail] (:bhearsum)]

This got descoped for bug 1398799 - I won't be looking at it.