914856 - Need error logs for production server running Source website

Status: RESOLVED INACTIVE

(Mozilla Labs :: General, defect)

Description

[Ryan Pitts]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:23.0) Gecko/20100101 Firefox/23.0 (Beta/Release)
Build ID: 20130814063812

Steps to reproduce:

A user reported a bug with the site search on Source. This query throws a 500 error:

http://source.mozillaopennews.org/en-US/search/?q=data

Other queries seem to be behaving normally. E.g.:

http://source.mozillaopennews.org/en-US/search/?q=django

And the problem query seems to work fine on our staging server:

https://source-dev.mozillalabs.com/en-US/search/?q=data

The query on our production server is throwing a 500 error, and I assume there might be a character encoding issue involved, inside a recent article that doesn't exist on our staging server. But I can't get to any error logs for the server source-prod1.vm.labs.scl3.mozilla.com, so I also can't tell whether this error might be happening at the elasticsearch level.

If it's possible to get some help tracking down the actual error that's being thrown, that would be excellent.


Actual results:

 


Expected results:

Comment 1

[C. Liang [:cyliang]]

Ryan:

I don't know if this helps, but I believe that you do have root access on the appropriate server (source-prod1.vm.labs.scl3.mozilla.com).  So, you may just need to exercise sudo. =)  The Apache log files are in /var/log/httpd/source.mozillaopennews.org and the Elasticsearch ones are in /var/log/elasticsearch.

Comment 2

[Ryan Pitts]

Update: I have the Source search engine working properly again, total trial & error method to find the record that was causing the problem. But if possible, I'd still like to find out what error was being thrown so I can make any code changes necessary to avoid this in the future.

Comment 3

[Ryan Pitts]

I went looking there, but I was getting a permission denied when trying to cd into the httpd directory. Is that normal?

Comment 4

[C. Liang [:cyliang]]

[cliang@source-prod1 ~]$ ls -ald /var/log/httpd
drwx------ 4 root root 4096 Sep  8 03:36 /var/log/httpd

Since the directory is owned by root and is only readable as the owner, yes, you will get a "permission denied" error unless you access the directory as root. =)  You can always 'sudo ls <directory>' or, if you are going to be doing a lot of poking about at files, become the root user ('sudo su - root').

Comment 5

[Ryan Pitts]

Ahh, that makes sense now. Thank you! My devops chops are still a work in progress ;) I poked around in those logs and didn't see any errors that jumped out at me as being related to the search issue described above. Remains a mystery.

Comment 6

[Sylvestre Ledru [:Sylvestre]]

Closing all inactive bugs in Mozilla labs :: *
Please reopen if this are still relevant.