1165980 - Add route template in tree

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Michael Shal [:mshal]]

Right now taskcluster builds use one set of routes to upload artifacts, while buildbot/mozharness builds use another set of routes. It would be helpful if both of these systems used the same set of routes, pulled from a common file in tree. lightsofapollo suggested something like:

gecko.v1.{{builder_name}}.{{os}}
gecko.v2.imakenamespacesupallday.{{builder_name}}.{{os}}

(not exactly this obviously, but you get the idea :)

TC/mozharness could then read from this file and use that when specifying the routes for the task.

We still need to figure out what exactly the file should look like, and how that will map to both buildbot and taskcluster's notions of what a product, platform, branch, etc are.

I'd like to avoid changing the platform names and such in buildbot-configs if possible, but we can look into this if necessary.

Comment 1

[James Lal [:lightsofapollo]]

JSON is probably best for this how about:

{
  "routes": [
    "gecko.v1.{{builer_name}}..."
  ],

  "builders": {
    "windows32_gecko": {
      "name": "b2g-desktop",
      "os": "windows",
      "arch": "32"
    }
  }
}

---

I have a couple of goals here:

1. I want to map buildernames to something more understandable by machines (and humans).
2. JSON is easier to parse then YAML (python just works)
3. We might need to change the template format?

I have a few variables I care about in particular:

 - buildername (the original builder name this _may_ not exist)
 - name (normalized name of the build)
 - os (windows, mac, linux)
 - arch (i686, etc...)

For this to play nice with TC and vice versa it would be ideal if the routes are only used to create indexes if all the variables are available. In particular this is to support b2g-desktop mac/windows (or mulet of the same) so these can keep their builder names (and route them) but also support the nicer naming format.

Comment 2

[Michael Shal [:mshal]]

This is done as part of bug 1133074.