Closed Bug 914063 Opened 9 years ago Closed 9 years ago

OrangeFactor stuck on 'Loading...' due to API giving 500 ISEs

Categories

(Tree Management Graveyard :: OrangeFactor, defect)

defect
Not set
critical

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Unassigned)

References

Details

Attachments

(1 file)

http://brasstacks.mozilla.com/orangefactor/ -> Stuck on "Loading..."

Webconsole:
[10:07:29.509] GET http://brasstacks.mozilla.com/orangefactor/api/count?startday=2013-09-02&endday=2013-09-09&tree=trunk [HTTP/1.1 500 Internal Server Error 986ms]



[root@brasstacks1.dmz.scl3 ~]# tail /var/log/nginx/error.log -n 50

...

2013/09/09 02:09:04 [error] 10624#0: *28733089 FastCGI sent in stderr: "Traceback (most recent call last):
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/web/application.py", line 237, in process
    return self.handle()
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/web/application.py", line 228, in handle
    return self._delegate(fn, self.fvars, args)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/web/application.py", line 409, in _delegate
    return handle_class(cls)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/web/application.py", line 385, in handle_class
    return tocall(*args)
  File "/home/webtools/apps/orangefactor/src/orangefactor/server/handlers.py", line 169, in GET
    results = json.dumps(self._GET(params, body))
  File "/home/webtools/apps/orangefactor/src/orangefactor/server/handlers.py", line 443, in _GET
    return BugData._GET(self, params, True)
  File "/home/webtools/apps/orangefactor/src/orangefactor/server/handlers.py", line 382, in _GET
    orange_bugs = bzcache.get_bugs(list(buglist))
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/bzcache-0.1.0-py2.6.egg/bzcache/bzcache.py", line 86, in get_bugs
    doc_type=[self.doc_type])
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/mozautoeslib-0.1.1-py2.6.egg/mozautoeslib/eslib.py", line 186, in query
    doc_types=self.doc_type)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/pyes-0.15.0-py2.6.egg/pyes/es.py", line 793, in count
    return self._query_call("_count", query, indexes, doc_types, **query_params)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/pyes-0.15.0-py2.6.egg/pyes/es.py", line 246, in _query_call
    response = self._send_request('GET', path, body, querystring_args)
  File "/home/webtools/apps/orangefactor/lib/python2.6/site-packages/pyes-0.15.0-py2.6.egg/pyes/es.py", line 218, in _send_request
    raise_if_error(response.status, decoded)
  File "/home/
2013/09/09 02:09:04 [error] 10624#0: *28733091 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.22.74.209, server: brasstacks1.dmz.scl3.mozilla.com, request: "GET /favicon.ico HTTP/1.1", host: "brasstacks.mozilla.com"
[root@brasstacks1.dmz.scl3 ~]# /etc/init.d/nginx reload
Reloading nginx:                                           [  OK  ]

[root@brasstacks1.dmz.scl3 ~]# /etc/init.d/orangefactor stop; /etc/init.d/orangefactor st art
stopping orangefactor                                      [FAILED]
starting orangefactorspawn-fcgi: child spawned successfully: PID: 21921
                                                           [  OK  ]

[root@brasstacks1.dmz.scl3 ~]# /etc/init.d/orangefactor stop; /etc/init.d/orangefactor st art
stopping orangefactor                                      [FAILED]
starting orangefactorspawn-fcgi: child spawned successfully: PID: 21937
                                                           [  OK  ]
Attached file nginx.conf
Where are the orangefactor entries here?
After reloading OF, the 500 has changed to a 502:

[10:27:38.251] GET http://brasstacks.mozilla.com/orangefactor/api/count?startday=2013-09-02&endday=2013-09-09&tree=trunk [HTTP/1.1 502 Bad Gateway 172ms]
The failure on restart was due to the config file being moved rather than copied as part of bug 912505 (only became an issue once OF was restarted in comment 1).

However with that fixed, brasstacks is still unable to connect to ES (presuming this is the cause of the 500 in comment 0).
Depends on: 914192
Seems that the ES cluster is down or otherwise not reachable.  That may have caused the crash of OF in the first place (restarting it is when I realized the config file had gone missing).  Filed bug 914192.
Working now that bug 914192 is fixed :-)
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Product: Testing → Tree Management
Product: Tree Management → Tree Management Graveyard
You need to log in before you can comment on or make changes to this bug.