Closed Bug 1525946 Opened 7 years ago Closed 7 years ago

Upgrade nss to use a .taskcluster.yml that can be embedded in a hook

Categories

(Taskcluster :: General, enhancement)

Product:
Taskcluster
Component:
General
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dustin, Assigned: dustin)

References

Details

Attachments

(2 files)

Bug 1525946 - get hg commit message from hg
47 bytes, text/x-phabricator-request
Details | Review
Bug 1525946 - update .taskcluster.yml to modern standards
47 bytes, text/x-phabricator-request
Details | Review

The current .taskcluster.yml is a "version 0". When we turn off mozilla-taskcluster, that format will no longer be supported.

Instead, we'll need to create a .taskcluster.yml that uses JSON-e. The other change required is for the decision task to fetch the commit comment directly from hg, rather than from the decision task template. That will require regenerating and pushing the decision task docker image (for which I'll require a little help, since I don't have push permission to that docker repo).

:mt, can you review these changes and help me with pushing?

(This is in support of bug 1204891, where we're turning off the ancient mozilla-taskcluster service)

Bug 1525946 - update .taskcluster.yml to modern standards

Bug 1525946 - update .taskcluster.yml to modern standards

Attachment #9042250 - Attachment description: Bug 1525946 - get hg commit message from hg → Bug 1525946 - update .taskcluster.yml to modern standards

I'm not sure how to test these! I don't have permission to push to nss-try (I'm not in scm_nss).

..and anyway, I'll need some help building that decision docker image and pushing it.

https://treeherder.mozilla.org/#/jobs?repo=nss-try&resultStatus=testfailed%2Cbusted%2Cexception%2Cusercancel%2Crunning%2Cpending%2Crunnable&classifiedState=unclassified&revision=dd0e499176d311f8ac9891c3a1d2043396fb2031

The changes to the JS are fine, but I have no capacity, other than just trusting you, on the .yml changes. It looks sane, but I don't know what I'm supposed to be looking for. Can you ask someone more familiar with taskcluster configuration to review that change?

When you get a LGTM, n-i me and I'll land these.

The try run appears to have done nothing at all. Do you have a way of testing this locally that might allow you to make progress without me? I'm happy to forward things on to nss-try, but the round trip time is a little long.

try push:
https://treeherder.mozilla.org/#/jobs?repo=nss-try&revision=dd0e499176d311f8ac9891c3a1d2043396fb2031

From the m-t logs:

Client ID project/taskcluster/mozilla-taskcluster/production-2 does not have sufficient scopes and are missing the following scopes:

{
  "AnyOf": [
    {
      "AllOf": [
        "queue:scheduler-id:gecko-level-1",
        {
          "AnyOf": [
            "queue:create-task:highest:aws-provisioner-v1/gecko-decision",
            "queue:create-task:very-high:aws-provisioner-v1/gecko-decision",
            "queue:create-task:high:aws-provisioner-v1/gecko-decision",
            "queue:create-task:medium:aws-provisioner-v1/gecko-decision",
            "queue:create-task:low:aws-provisioner-v1/gecko-decision",
            "queue:create-task:very-low:aws-provisioner-v1/gecko-decision",
            "queue:create-task:lowest:aws-provisioner-v1/gecko-decision"
          ]
        }
      ]
    },
    {
      "AnyOf": [
        "queue:create-task:aws-provisioner-v1/gecko-decision",
        {
          "AllOf": [
            "queue:define-task:aws-provisioner-v1/gecko-decision",
            "queue:task-group-id:gecko-level-1/Tg_K_HAgSiK-uJetTK0vuQ",
            "queue:schedule-task:gecko-level-1/Tg_K_HAgSiK-uJetTK0vuQ/Tg_K_HAgSiK-uJetTK0vuQ"
          ]
        }
      ]
    }
  ]
}

You have the scopes:

[
  "assume:repo:hg.mozilla.org/projects/nss-try:branch:default",
  "docker-worker:feature:allowPtrace",
  "queue:create-task:aws-provisioner-v1/ami-test-win2012r2",
  "queue:create-task:aws-provisioner-v1/gecko-1-b-win*",
  "queue:create-task:aws-provisioner-v1/hg-worker",
  "queue:create-task:aws-provisioner-v1/nss-*",
  "queue:create-task:aws-provisioner-v1/win2012r2",
  "queue:create-task:localprovisioner/nss-aarch64",
  "queue:create-task:localprovisioner/nss-macos-10-12",
  "queue:route:index.docker.images.v1.nss-try.*",
  "queue:route:notify.email.martin.thomson@gmail.com.*",
  "queue:route:tc-treeherder-stage.nss-try.*",
  "queue:route:tc-treeherder-stage.v2.nss-try.*",
  "queue:route:tc-treeherder.nss-try.*",
  "queue:route:tc-treeherder.v2.nss-try.*",
  "scheduler:create-task-graph",
  "scheduler:extend-task-graph:*",
  "secrets:get:garbage/*",
  "secrets:set:garbage/*"
]

This request requires you to satisfy this scope expression:

{
  "AllOf": [
    "assume:repo:hg.mozilla.org/projects/nss-try:branch:default",
    "queue:route:notify.email.martin.thomson@gmail.com.*",
    "queue:route:tc-treeherder.v2.nss-try.dd0e499176d311f8ac9891c3a1d2043396fb2031.3411",
    "queue:route:notify.email.martin.thomson@gmail.com.on-failed",
    {
      "AnyOf": [
        {
          "AllOf": [
            "queue:scheduler-id:gecko-level-1",
            {
              "AnyOf": [
                "queue:create-task:highest:aws-provisioner-v1/gecko-decision",
                "queue:create-task:very-high:aws-provisioner-v1/gecko-decision",
                "queue:create-task:high:aws-provisioner-v1/gecko-decision",
                "queue:create-task:medium:aws-provisioner-v1/gecko-decision",
                "queue:create-task:low:aws-provisioner-v1/gecko-decision",
                "queue:create-task:very-low:aws-provisioner-v1/gecko-decision",
                "queue:create-task:lowest:aws-provisioner-v1/gecko-decision"
              ]
            }
          ]
        },
        {
          "AnyOf": [
            "queue:create-task:aws-provisioner-v1/gecko-decision",
            {
              "AllOf": [
                "queue:define-task:aws-provisioner-v1/gecko-decision",
                "queue:task-group-id:gecko-level-1/Tg_K_HAgSiK-uJetTK0vuQ",
                "queue:schedule-task:gecko-level-1/Tg_K_HAgSiK-uJetTK0vuQ/Tg_K_HAgSiK-uJetTK0vuQ"
              ]
            }
          ]
        }
      ]
    }
  ]
}
  • method: createTask
  • errorCode: InsufficientScopes
  • statusCode: 403
  • time: 2019-02-08T03:55:59.786Z

I'll poke at that. There's no great way to test this locally since it depends on contextual things like the scopes for the push.

Oh, that's the fake error task. Just below that:

YAMLException: bad indentation of a sequence entry at line 13, column 31

The fix:
https://phabricator.services.mozilla.com/D19042?vs=59715&id=60061&whitespace=ignore-most#toc

I'd be most comfortable trying another try push. I know they're annoying! If you paste the link here or ping me with it I'm happy to watch it and debug if it fails or make a note here if it succeeds.

Martin, any chance we could do another try push to check this, and land if it's OK?

Flags: needinfo?(martin.thomson)

Thanks -- I see the errors in the service logs.

OK! I got a successful push!

https://treeherder.mozilla.org/#/jobs?repo=nss-try&resultStatus=superseded%2Ctestfailed%2Cbusted%2Cexception%2Csuccess%2Cretry%2Cusercancel%2Crunning%2Cpending%2Crunnable&classifiedState=unclassified&revision=3a3c4c1a01bed0e0022dd526d8b3247fad3e94f6

That's still using a docker image under my hub account (djmitche/nss-decision:0.0.3). That's fine for whatever duration is necessary, but once we find someone with access to the nssdev account, we can just pull from my account and push to that one, or rebuild a new image.

Martin -- I think this is ready to land.. I'm not sure how best to do that?

Flags: needinfo?(martin.thomson)

https://hg.mozilla.org/projects/nss/rev/57bad0ae437f32cbb69de1851aea0d2f00212c3b
https://hg.mozilla.org/projects/nss/rev/731692f63b45297a302c7d847fcf742321750531

Thanks for doing the work of upgrading these. This wouldn't have been easy for any of us to do.

Flags: needinfo?(martin.thomson)

Rock on, thanks! I think the TC team is going to be a bit more "forward-engaged" for a while as we bring users like NSS over some of the bumps involved in new TC developments. So, we're happy to help.

This seems to have "stuck" :)

Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
See Also: → 1546925
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: