948727 - npm mirror script fails occasionally

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

I'm getting cronspam emails about these, but don't have any particular action to take.  From today:

--

/data/releng/src/npm-mirror/node_modules/npm-mirror/bin/npm-mirror:117
      throw e;
            ^
Error: Bad status 500 for http://registry.npmjs.org/from
    at Object.<anonymous> (/data/releng/src/npm-mirror/node_modules/npm-mirror/lib/download.js:54:11)
    at ClientRequest.g (events.js:175:14)
    at ClientRequest.EventEmitter.emit (events.js:95:17)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (http.js:1689:21)
    at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:120:23)
    at Socket.socketOnData [as ondata] (http.js:1584:20)
    at TCP.onread (net.js:525:27)

--

Error: getaddrinfo ENOTFOUND
    at errnoException (dns.js:37:11)
    at Object.onanswer [as oncomplete] (dns.js:124:16)

--

Error: socket hang up
    at createHangUpError (http.js:1473:15)
    at Socket.socketOnEnd [as onend] (http.js:1569:23)
    at Socket.g (events.js:175:14)
    at Socket.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
    at process._tickCallback (node.js:415:13)

--

I assume all of these are transient, but I don't think it's a good idea to send this to /dev/null as we might then miss a more permanent failure condition.

A human staring at several cronspam per day to distinguish "harmless" from "harmful" doesn't scale.  How can we monitor this service for success automatically?

Comment 1

[Gareth Aye [:gaye] (back from PTO)]

So the npm registry that we're mirroring packages from was having some issues. I think they're mostly resolved now, but I'd like to purge the existing packages to make way for a full resync since there are some packages with bad shas in the mirror right now.

Feel free to send the cronspam to me at [e-mail].

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Done and done.  Thanks!