1607026 - taskcluster.optionsFromEnvironment() in tc python client could consider TASKCLUSTER_PROXY_URL in determining workable options

Status: NEW

(Taskcluster :: General, enhancement)

Description

[Rob Thijssen [:grenade (EET/UTC+0300)]]

i created a new worker pool in the firefox-ci tc deployment called relops/decision. much of the definition (including ami ids) was just copied from gecko-1/decision.

when i trigger a task using the task creator and the following task definition, the task fails complaining of insufficient scopes. it is unclear to me which scopes are required and the tasks own exception log does not specify which scopes are unsatisfied. the task calls the tc auth api using the tc python client and logs the response from auth.currentScopes() as {'scopes': []} which suggests to me that the scopes defined in the task definition are not being propagated to the task.

provisionerId: relops
workerType: decision
schedulerId: '-'
taskGroupId: XmqTRuybRO65ECPOF5fuUQ
dependencies: []
requires: all-completed
routes: []
priority: low
retries: 5
created: '2020-01-04T17:41:00.820Z'
deadline: '2020-01-07T17:41:00.820Z'
expires: '2021-01-07T17:41:00.820Z'
scopes:
  - 'generic-worker:os-group:relops/win2019/Administrators'
  - 'generic-worker:run-as-administrator:relops/*'
  - 'queue:create-task:highest:relops/*'
  - 'queue:create-task:very-high:relops/*'
  - 'queue:create-task:high:relops/*'
  - 'queue:create-task:medium:relops/*'
  - 'queue:create-task:low:relops/*'
  - 'queue:route:index.project.relops.cloud-image-builder.*'
  - 'queue:scheduler-id:-'
  - 'secrets:get:project/relops/image-builder/dev'
payload:
  maxRunTime: 600
  command:
    - /bin/bash
    - '--login'
    - '-c'
    - >-
      git clone https://github.com/grenade/cloud-image-builder.git && pip
      install azure boto3 pyyaml slugid taskcluster urllib3 && python
      cloud-image-builder/ci/create-image-build-tasks.py
  artifacts: {}
  features:
    taskclusterProxy: true
  env:
    GITHUB_HEAD_SHA: 5e18c2728c07ae378bdb2ba18c082e1db3e25ef2
    TASKCLUSTER_PROXY_URL: 'http://localhost/'
  image: python
metadata:
  name: '00 :: decision task'
  description: >-
    determine which windows cloud images should be built, where they should be
    deployed and trigger appropriate build tasks for the same
  owner: [e-mail]
  source: 'https://github.com/grenade/cloud-image-builder'
tags: {}
extra: {}

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

From Slack, it looks like your task is not using TASKCLUSTER_PROXY_URL. The worker sets it for you -- it's up to the task implementation to use it. In Python that would be something like taskcluster.Queue({"rootUrl": os.environ['TASKCLUSTER_PROXY_URL']}).

Comment 2

[Rob Thijssen [:grenade (EET/UTC+0300)]]

(In reply to Dustin J. Mitchell [:dustin] (he/him) from comment #1)

From Slack, it looks like your task is not using TASKCLUSTER_PROXY_URL. The worker sets it for you -- it's up to the task implementation to use it. In Python that would be something like taskcluster.Queue({"rootUrl": os.environ['TASKCLUSTER_PROXY_URL']}).

thank you! that was indeed the problem.

i was using taskcluster.Queue(taskcluster.optionsFromEnvironment()) and assuming that optionsFromEnvironment would get the options it needs from the environment.