move partner configs into parameters, populated by rr3

RESOLVED FIXED

Status

enhancement
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: aki, Assigned: aki)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments, 2 obsolete attachments)

(Assignee)

Description

a year ago
Currently, the workflow on maple is

ship-it -> rr3 -> action task

The action task populates the graph. The partner taskgraph logic hits the github partner repo, and creates the taskgraph config for the partner repacks.

Instead, we're thinking that rr3 should populate a partner_config parameter in the action task. The taskgraph logic can reference this. For out-of-cycle partner repacks, we're thinking mkaply may use ship-it-v2, which would still use rr3. (At some point rr3 could turn into a hook, but that's a separate bug, and still works in this model, as long as the hook has access to the github token.)

It's essentially the same, but this is much better for testing purposes, since we can populate the partner repack config with known values, rather than having to hit a github url.
(Assignee)

Comment 1

a year ago
Attachment #8965142 - Flags: review?(rail)
Attachment #8965142 - Flags: review?(rail) → review+
(Assignee)

Comment 3

a year ago
(Assignee)

Comment 4

a year ago
Posted patch puppet-partner-urls.diff (obsolete) — Splinter Review
I was wanting to use the live partner configs on maple, but it looks like something's broken in chunking. Once we fix that, we should test against live partner configs until we're confident in end-to-end testing.
Attachment #8965849 - Flags: review?(rail)
(Assignee)

Comment 5

a year ago
Attachment #8965849 - Attachment is obsolete: true
Attachment #8965849 - Flags: review?(rail)
Attachment #8965853 - Flags: review?(rail)
(Assignee)

Comment 6

a year ago
Attachment #8965808 - Attachment is obsolete: true
Attachment #8965854 - Flags: review?(rail)
Attachment #8965853 - Flags: review?(rail) → review+
Comment on attachment 8965854 [details] [diff] [review]
rr3_partner_config.diff

Review of attachment 8965854 [details] [diff] [review]:
-----------------------------------------------------------------

::: lib/python/kickoff/partners.py
@@ +16,5 @@
> +GITHUB_API_ENDPOINT = "https://api.github.com/graphql"
> +
> +"""
> +LOGIN_QUERY, MANIFEST_QUERY, and REPACK_CFG_QUERY are all written to the Github v4 API,
> +which users GraphQL. See https://developer.github.com/v4/

Wow! Great job!

@@ +199,5 @@
> +    return partners
> +
> +
> +def parse_config(data):
> +    """ Parse a single repack.cfg file into a python dictionary.

We should switch to YAML at some point :)
Attachment #8965854 - Flags: review?(rail) → review+
(Assignee)

Comment 8

a year ago
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #7)
> ::: lib/python/kickoff/partners.py
> @@ +16,5 @@
> > +GITHUB_API_ENDPOINT = "https://api.github.com/graphql"
> > +
> > +"""
> > +LOGIN_QUERY, MANIFEST_QUERY, and REPACK_CFG_QUERY are all written to the Github v4 API,
> > +which users GraphQL. See https://developer.github.com/v4/
> 
> Wow! Great job!

This was all Nick :)
Followup to fix version comparison, as str >= int is always True in py2, p=aki, r=nthomas on IRC
https://hg.mozilla.org/build/tools/rev/06a71a5fe369

Followup to fix extraneous empty dictionary for sub configs, p=nthomas, r=aki on IRC
https://hg.mozilla.org/build/tools/rev/5641e7cf9ea442240b5b2d8eed47a1e08293bec2
(Assignee)

Comment 13

a year ago
Fixed on maple. The in-tree portions will probably land as one big patchset.
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.