910855 - Searches containing non-english characters are causing the MakeAPI to 500

Status: RESOLVED FIXED

(Webmaker Graveyard :: Login, defect)

Description

[Chris DeCairos (:cade)]

Sample URL: https://makeapi.webmaker.org/api/20130724/make/search?or=1&description=%D8%A7%D9%84%D9%81%D9%8A%D8%B3%20%D8%A8%D9%83&user=%D8%A7%D9%84%D9%81%D9%8A%D8%B3%20%D8%A8%D9%83&title=%D8%A7%D9%84%D9%81%D9%8A%D8%B3%20%D8%A8%D9%83&tags=and%2C%D8%A7%D9%84%D9%81%D9%8A%D8%B3%20%D8%A8%D9%83&limit=12&sortByField=createdAt%2Cdesc&page=1

I'm not sure how to fix this in Elastic Search. Will have to investigate.

Comment 1

[Chris DeCairos (:cade)]

I'm seeing this error: 

{
  "error": {
    "code": "ER_CANT_AGGREGATE_",
    "index": 0
  },
  "code": 500
}

research time!

Comment 2

[Chris DeCairos (:cade)]

Traced this to the login server:

"GET /user/الفيس%20بك HTTP/1.1" 500 69 "-" "-"

Comment 3

[Chris DeCairos (:cade)]

possible solution:

`Sequelize.STRING + ' CHARSET utf8 COLLATE utf8_unicode_ci'`

to the "username" column. This would require a migration script.

Other than that, make an effort to not pass these kinds of strings into that system.

Comment 4

[Jon Buckley [:jbuck]]

Waitwaitwait. You're saying that the login server doesn't support utf8? Lets fix that up!

Comment 5

[Jon Buckley [:jbuck]]

Nice catch Chris, I've changed the login db on prod and staging to be utf8: https://webmaker.org/en-US/search?type=title&q=%D8%A7%D9%84%D9%81%D9%8A%D8%B3+%D8%A8%D9%83

Patch incoming to make sure that this doesn't happen on local dev too

Comment 6

[Jon Buckley [:jbuck]]

Attachment: https://github.com/mozilla/login.webmaker.org/pull/149

Comment 7

[Chris DeCairos (:cade)]

Comment on attachment 797858 [details] [review]
https://github.com/mozilla/login.webmaker.org/pull/149

Thanks for getting on this one! R+

Comment 8

[[github robot]]

Commit pushed to master at https://github.com/mozilla/login.webmaker.org

https://github.com/mozilla/login.webmaker.org/commit/aa1d408feaf03d7aa8a4607004fba26f6ad6bbbe
Fix bug 910855 - Force utf8 on table creation