1289574 - Bugscache can't handle unicode characters in summary or summaries of length > 255

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect)

Description

[William Lachance (:wlach)]

When trying to import data from bugzilla into bugscache locally, it generates an exception because of unicode characters in some bug summaries (e.g. bug 1244876). We should fix this.

I've also been working on making the bugscache easier to test, I'll roll that work into this bug.

Comment 1

[James Graham [:jgraham]]

Welcome to the wonderful world of unicode in MySQL.

https://bugzilla.mozilla.org/show_bug.cgi?id=1275425 has some useful information.

Comment 2

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1289574 > mozilla:master

Comment 3

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/67438475a1a75399bf141867669293a662211628
Bug 1289574 - Make failing to import a bug into bugscache non-fatal

https://github.com/mozilla/treeherder/commit/8f4d611292f1c8a0a68e241a2893d2a75ac81784
Bug 1289574 - Add a management command to update bugscache

https://github.com/mozilla/treeherder/commit/ef6833891f1f9e0175a30b1a4bc9e128aed1ed67
Bug 1289574 - Handle various edge cases when importing bugs

* UTF-8 characters in summary
* Summary exceeds 255 characters

https://github.com/mozilla/treeherder/commit/9e6f92e60fa9e7c59fe0740f91fc2120d34bc739
Merge pull request #1732 from wlach/1289574

Bug 1289574 - Handle various edge cases with bugzilla mirroring