1228228 - Mulet/B2G desktop tests all failing in taskcluster-npm-cache-get, keeping all trees closed

Status: RESOLVED FIXED

(Taskcluster :: General, defect)

Description

[Phil Ringnalda (:philor)]

After the Azure fun, everything seems to be back to normal except that every Mulet and B2G desktop test is failing like https://treeherder.mozilla.org/logviewer.html#?job_id=17853192&repo=mozilla-inbound

 05:37:32     INFO - Running command: ['taskcluster-npm-cache-get', '--namespace', 'gaia.npm_cache', 'package.json'] in /home/worker/gaia
 05:37:32     INFO - Copy/paste: taskcluster-npm-cache-get --namespace gaia.npm_cache package.json
 05:37:32     INFO - Calling ['taskcluster-npm-cache-get', '--namespace', 'gaia.npm_cache', 'package.json'] with output_timeout 300
 05:37:41     INFO -  Something is wrong...
 05:37:41     INFO -  /usr/local/lib/node_modules/taskcluster-npm-cache/build/bin/taskcluster-npm-cache-get.js:186
 05:37:41     INFO -      throw e;
 05:37:41     INFO -      ^
 05:37:41     INFO -  ReferenceError: e is not defined
 05:37:41     INFO -      at main$ (/usr/local/lib/node_modules/taskcluster-npm-cache/build/bin/taskcluster-npm-cache-get.js:135:15)
 05:37:41     INFO -      at tryCatch (/usr/local/lib/node_modules/taskcluster-npm-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:61:40)
 05:37:41     INFO -      at GeneratorFunctionPrototype.invoke [as _invoke] (/usr/local/lib/node_modules/taskcluster-npm-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:328:22)
 05:37:41     INFO -      at GeneratorFunctionPrototype.prototype.(anonymous function) [as throw] (/usr/local/lib/node_modules/taskcluster-npm-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:94:21)
 05:37:41     INFO -      at GeneratorFunctionPrototype.invoke (/usr/local/lib/node_modules/taskcluster-npm-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:136:37)
 05:37:41     INFO -      at run (/usr/local/lib/node_modules/taskcluster-npm-cache/node_modules/babel/node_modules/babel-core/node_modules/core-js/modules/es6.promise.js:104:47)
 05:37:41     INFO -      at /usr/local/lib/node_modules/taskcluster-npm-cache/node_modules/babel/node_modules/babel-core/node_modules/core-js/modules/es6.promise.js:115:28
 05:37:41     INFO -      at flush (/usr/local/lib/node_modules/taskcluster-npm-cache/node_modules/babel/node_modules/babel-core/node_modules/core-js/modules/$.microtask.js:19:5)
 05:37:41     INFO -      at doNTCallback0 (node.js:419:9)
 05:37:41     INFO -      at process._tickCallback (node.js:348:13)
 05:37:41    ERROR - Return code: 1
etc.

Comment 1

[Phil Ringnalda (:philor)]

garndt restarted something, "the index" maybe, and cleared it up.

Comment 2

[Greg Arndt [:garndt]]

After Azure table storage came back up, the index looks like it hit some memory limits when processing all the requests afterwards resulting in 500 errors being returned.  Restarting it seemed to clear out old requests, and everything started working again.