API Rate Limit Exceeded for for requests to https://api.github.com



6 years ago
5 years ago


(Reporter: jgriffin, Assigned: cransom)




(1 attachment)



6 years ago
Autolog (http://brasstacks.mozilla.com/autolog/) performs queries against github for the B2G project.  Currently, these are failing with this message from github:

curl https://api.github.com/repos/mozilla/releases-mozilla-central/git/commits/
  "message": "API Rate Limit Exceeded for"

What is  Brasstacks is  Can we figure out what the source of these excessive queries is?

Comment 1

6 years ago
Lawrence, Dietrich, any ideas on what could be automatically hitting github from that IP address?

Hwine, could this be one of the releng mirrors?

Comment 2

6 years ago
I'll be working on this momentarily. The source address is a nat address on the scl3 firewall so it could be any number of machines.
The only code that I know hits github is the daily pull of basecamp+ github issues to generate the csv files that are used in the basecamp dashboard.

Comment 4

6 years ago
Created attachment 672814 [details]
traffic seen for brasstacks to api.github.com

Just a quick note to understand the traffic flow - 
For brasstacks, incoming traffic is via a load balancer for external 80/443 traffic and outgoing traffic is being natted to the general NAT address (.162).  This is why you are seeing connections from brasstacks coming from a different adress than what it is addressed as externally.

I looked at netflow data and the only host hitting api.github.com from the NAT address is brasstacks.

I attached the traffic seen from the firewall for timestamp comparisons, stamps are in pacific time.  Let me know how you want to proceed.


6 years ago
Assignee: network-operations → cransom

Comment 5

6 years ago
Perhaps autolog is getting more hits than it has in the past.  If that's the issue, the only way around it that I can think of is to implement a local cache of the github queries.

Comment 6

6 years ago
this request is complete from the netops end.
Last Resolved: 6 years ago
Resolution: --- → FIXED
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.