1606002 - "Invalid Credentials" talking to Google in worker scanner

Status: NEW

(Taskcluster :: Services, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

This appears to occur sporadically in both community and firefox-ci. It started in 24.0.1 and continues in 24.1.1. It is 100% from worker-scanner.

https://sentry.prod.mozaws.net/operations/taskcluster-firefox-ci/issues/6871158/events/36884357/

Error: Invalid Credentials
  File "/app/node_modules/gaxios/build/src/gaxios.js", line 70, col 23, in Gaxios.request
    throw new common_1.GaxiosError(`Request failed with status code ${translatedResponse.status}`, opts, translatedResponse);
  ?, in runMicrotasks
  File "internal/process/task_queues.js", line 93, col 5, in processTicksAndRejections
  File "/app/node_modules/google-auth-library/build/src/auth/oauth2client.js", line 346, col 18, in async JWT.requestAsync
    r2 = await this.transporter.request(opts);
  File "/app/node_modules/p-queue/dist/index.js", line 157, col 29, in async run
    resolve(await operation);

https://sentry.prod.mozaws.net/operations/taskcluster-community/issues/6836911/?query=is:unresolved

Error: Invalid Credentials
  File "/app/node_modules/gaxios/build/src/gaxios.js", line 70, col 23, in Gaxios.request
    throw new common_1.GaxiosError(`Request failed with status code ${translatedResponse.status}`, opts, translatedResponse);
  File "internal/process/task_queues.js", line 93, col 5, in processTicksAndRejections
  File "/app/node_modules/google-auth-library/build/src/auth/oauth2client.js", line 346, col 18, in async JWT.requestAsync
    r2 = await this.transporter.request(opts);
  File "/app/node_modules/p-queue/dist/index.js", line 157, col 29, in async run
    resolve(await operation);