Closed Bug 1237310 Opened 8 years ago Closed 6 years ago

[purge-cache] Add API stability levels for purge-cache

Categories

(Taskcluster :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jonasfj, Assigned: pmoore, Mentored)

References

Details

Attachments

(1 file)

      No description provided.
The intent here is to add the "stability" option to every API method in the taskcluster-purge-cache service.  See https://github.com/taskcluster/taskcluster-lib-api for details on this option.

All API methods in this service should be marked as "stable".
Mentor: dustin
hi, I'm an outreachy intern applicant. I am unclear how I should test my changes locally, should I be using the babel-node package to run the server locally?
Assignee: nobody → lizardwalk5
Hi, good to hear from you after our irc conversation last week.

Yes, running

  babel-node bin/server.js test

should do the trick for you.
(In reply to Dustin J. Mitchell [:dustin] from comment #3)
>   babel-node bin/server.js test
> 
> should do the trick for you.

thanks. when I try that, after a few moments my regular console prompt appears without any messages that the server is running (also I get no response in the browser for http://localhost:60415/). I have tried to run it with the test.js pulse username/password both uncommented and commented. but that does not seem to make a difference.

should I need to change anything in the defaults.js or the test.js file?
(In reply to Carol Chung[:lizardwalk5] from comment #4)
> (In reply to Dustin J. Mitchell [:dustin] from comment #3)
> >   babel-node bin/server.js test
> > 
> > should do the trick for you.
> 
> thanks. when I try that, after a few moments my regular console prompt
> appears without any messages that the server is running (also I get no
> response in the browser for http://localhost:60415/). I have tried to run it
> with the test.js pulse username/password both uncommented and commented. but
> that does not seem to make a difference.
> 
> should I need to change anything in the defaults.js or the test.js file?

I should clarify is that the reason why I ask is that I felt I should run the existing tests before submitting a PR in github. I made the api changes and I don't think the tests will actually verify that my  changes were correct but thought I should verify not messing up anything.
You're absolutely right, so let's get things working for you.

The lack of output was from using debug(..) instead of console.log(..).  I submitted a pull request here:
  https://github.com/taskcluster/taskcluster-purge-cache/pull/4

Actually, to run the tests, you'd want to use `npm test`, which fails in a similar way to the below.

In general, when node things misbehave, it can help to run with DEBUG=*:

  DEBUG=* babel-node bin/server.js test

to see what is going on.  In this case,

Failed to start server, err: AssertionError: options.connectionString is missing, as JSON: {"name":"AssertionError","expected":true,"operator":"==","message":"options.connectionString is missing","generatedMessage":false} AssertionError: options.connectionString is missing
    at new Influx (/home/dustin/p/taskcluster-purge-cache/node_modules/taskcluster-base/lib/stats.js:41:3)
    at launch$ (/home/dustin/p/taskcluster-purge-cache/bin/server.js:31:5)
    at tryCatch (/home/dustin/p/taskcluster-purge-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:61:40)
    at GeneratorFunctionPrototype.invoke [as _invoke] (/home/dustin/p/taskcluster-purge-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:328:22)
    at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (/home/dustin/p/taskcluster-purge-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:94:21)
    at invoke (/home/dustin/p/taskcluster-purge-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:136:37)
    at callInvokeWithMethodAndArg (/home/dustin/p/taskcluster-purge-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:172:16)
    at previousPromise (/home/dustin/p/taskcluster-purge-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:194:19)
    at new Promise (/home/dustin/p/taskcluster-purge-cache/node_modules/babel/node_modules/babel-core/node_modules/core-js/modules/es6.promise.js:197:7)
    at AsyncIterator.enqueue (/home/dustin/p/taskcluster-purge-cache/node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:193:17)

which is due to the lack of test credentials.  Jonas should be able to get you a copy of the credentials file.  It'd be nice to fix things so that we don't need credentials!
Flags: needinfo?(jopsen)
My config file is:

taskcluster-purge-cache.conf.json:
{
  "pulse": {
    "username":           "jonasfj-debug",
    "password":           "<secret>"
  },
  "influx": {
    "connectionString":   "https://u:p@forgetaboutthis.com:80/nothing"
  }
}

The url: "https://u:p@forgetaboutthis.com:80/nothing"
won't work, but the stats part of this that submits to influxdb won't complain if it doesn't :)
It'll write something to log, but just ignore that... Out influxdb is overrun anyways.
So submitting to it might not always work.

for pulse credentials go to:
  https://pulseguardian.mozilla.org/
And create a username and password for your own pulse user.
Then use that.
Flags: needinfo?(jopsen)
thanks all. I updated the pulse credentials and got further. currently when I run npm test, I get a couple of errors.

1)  "before all" hook:
     Error: connect ECONNREFUSED
      at errnoException (net.js:863:11)
      at Object.afterConnect [as oncomplete] (net.js:854:19)

  2)  "after all" hook: callee$0$0:
     TypeError: Cannot call method 'terminate' of null
      at Object.callee$0$0$ (test/helper.js:112:19)
      at tryCatch (node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:61:40)
      at GeneratorFunctionPrototype.invoke [as _invoke] (node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:328:22)
      at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:94:21)
      at invoke (node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:136:37)
      at callInvokeWithMethodAndArg (node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:172:16)
      at previousPromise (node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:194:19)
      at new Promise (node_modules/babel/node_modules/babel-core/node_modules/core-js/modules/es6.promise.js:197:7)
      at AsyncIterator.enqueue (node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:193:17)
      at AsyncIterator.prototype.(anonymous function) [as next] (node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:94:21)
      at Object.runtime.async (node_modules/babel/node_modules/babel-core/node_modules/regenerator/runtime.js:215:14)
      at Context.callee$0$0 (test/helper.js:110:13)
      at node_modules/promise/lib/core.js:33:15
      at flush (node_modules/promise/node_modules/asap/asap.js:27:13)

... I assume that the ECONNREFUSED means that the test is not able to connect to a taskcluster-purge-cache server. that's why I was trying to run a local server but when I try, DEBUG=* babel-node bin/server.js test, I also get an ECONNREFUSED error.

Failed to start server, err: Error: connect ECONNREFUSED, as JSON: {"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"} Error: connect ECONNREFUSED
    at exports._errnoException (util.js:746:11)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1012:19)

I'm using node v0.12.12. should it be higher?
am checking my firewall settings.
Or it sounds as if something isn't listening on the port that it is supposed to be listening.

It could also be an error from pulse... There is two things going on in this component:
1) it keeps a persistent connection to pulse (a rabbitMQ server)
2) it listens for http requests (on configured port) and sends a message on pulse.
This is a very tiny service, and we haven't kept the code clean or up-to-date, I recommend
inserting some simple console.log lines and see how far you get.

Try to run with DEBUG=* that might give you more insights.
If on windows with system firewalls I'm not sure how that plays in.
(In reply to Jonas Finnemann Jensen (:jonasfj) from comment #10)
> it gets to the launch() but then it errors out.

do I need these values in defaults.js?
  aws accessKeyId
  aws secretAccessKey

they are now undefined.
I'm able to run the taskcluster-purge-cache server now. however I still get the ECONNREFUSED error (with a more recent version of node I get):

Potentially unhandled rejection [1] Error: connect ECONNREFUSED 127.0.0.1:5671
    at Object.exports._errnoException (util.js:856:11)
    at exports._exceptionWithHostPort (util.js:879:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1057:14)

does this mean I need to be running a local rabbitMQ server?
 
I'm also seeing an additional error:

base:stats Failed to send to influxdb, err: Error: write EPROTO 140735178904320:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:794:
(In reply to Carol Chung[:lizardwalk5] from comment #12)
> I'm able to run the taskcluster-purge-cache server now. however I still get
> the ECONNREFUSED error (with a more recent version of node I get):
> 
> Potentially unhandled rejection [1] Error: connect ECONNREFUSED
> 127.0.0.1:5671
>     at Object.exports._errnoException (util.js:856:11)
>     at exports._exceptionWithHostPort (util.js:879:20)
>     at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1057:14)

I tried running a local rabbitMQ server but still saw the ECONNREFUSED error from the taskcluster-purge-cache server. I think it's actually an issue with my ISP as I was seeing connectivity irregularities today.
It looks like it's trying to connect to 127.0.0.1, which is "localhost", so pointing to your computer.  You're right to deduce that 5671 is the port for RabbitMQ, which is what Pulse is.  But for Pulse it should be connecting to pulse.mozilla.org, not localhost.  

Also, the "Potentially unhandled rejection" indicates a programming error -- and not one in the changes you have made.

The failure to send to influxdb should not be fatal, and can be ignored.

To summarize, I think purge-cache needs some work before it can be properly tested at home.  Still, since you've completed the work on the patch for this bug, please go ahead and submit a pull request and we will get it merged.
I think this PR should do it.
Assignee: lizardwalk5 → pmoore
Status: NEW → ASSIGNED
https://docs.taskcluster.net/reference/core/purge-cache/references/api
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: