Closed Bug 922092 Opened 11 years ago Closed 10 years ago

Orangefactor API returning 500 ISE errors again

Categories

(Tree Management Graveyard :: OrangeFactor, defect)

Product:
Tree Management Graveyard
Component:
OrangeFactor
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: emorley, Unassigned)

Details

[root@brasstacks1.dmz.scl3 ~]# /etc/init.d/orangefactor stop; /etc/init.d/orangefactor st art
stopping orangefactor                                      [  OK  ]
starting orangefactorspawn-fcgi: child spawned successfully: PID: 11484
                                                           [  OK  ]

http://brasstacks.mozilla.com/orangefactor/api/count?startday=2013-09-23&endday=2013-09-30&tree=trunk 
-> works
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Broken again:
18:02:33.756 GET http://brasstacks.mozilla.com/orangefactor/api/count [HTTP/1.1 500 Internal Server Error 1111ms]
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Restarted again but we should leave this open to investigate.

Here's an error I've seen a few times in the nginx error logs. Not sure what it means exactly, and it looks cut off.

2013/09/30 10:12:18 [error] 21909#0: *29507488 FastCGI sent in stderr: "Traceback (most recent call last):
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/web/application.py", line 237, in process
    return self.handle()
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/web/application.py", line 228, in handle
    return self._delegate(fn, self.fvars, args)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/web/application.py", line 409, in _delegate
    return handle_class(cls)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/web/application.py", line 385, in handle_class
    return tocall(*args)
  File "/home/webtools/apps/orangefactor/src/orangefactor/server/handlers.py", line 169, in GET
    results = json.dumps(self._GET(params, body))
  File "/home/webtools/apps/orangefactor/src/orangefactor/server/handlers.py", line 443, in _GET
    return BugData._GET(self, params, True)
  File "/home/webtools/apps/orangefactor/src/orangefactor/server/handlers.py", line 382, in _GET
    orange_bugs = bzcache.get_bugs(list(buglist))
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/bzcache-0.1.0-py2.6.egg/bzcache/bzcache.py", line 86, in get_bugs
    doc_type=[self.doc_type])
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/mozautoeslib-0.1.1-py2.6.egg/mozautoeslib/eslib.py", line 186, in query
    doc_types=self.doc_type)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/pyes-0.15.0-py2.6.egg/pyes/es.py", line 793, in count
    return self._query_call("_count", query, indexes, doc_types, **query_params)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/pyes-0.15.0-py2.6.egg/pyes/es.py", line 246, in _query_call
    response = self._send_request('GET', path, body, querystring_args)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/pyes-0.15.0-py2.6.egg/pyes/es.py", line 218, in _send_request
    raise_if_error(response.status, decoded)
  File "/home/
Just had to restart again.
I don't think we've had an outage since comment #4, so closing this WORKSFORME since we didn't ever actually figure it out afaik.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 10 years ago
Resolution: --- → WORKSFORME
Product: Testing → Tree Management
Product: Tree Management → Tree Management Graveyard
You need to log in before you can comment on or make changes to this bug.