Closed Bug 822756 Opened 7 years ago Closed 7 years ago

bzexport mangles attachment descriptions with unicode chars like ♥

Categories

(Developer Services :: Mercurial: bzexport, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: sfink, Assigned: njn)

Details

Attachments

(2 files)

Related to bug 768342, but that fix does not fix this.

If I create a patch whose description contains the unicode character ♥, then the bug title will be correctly encoded and show the right character, but the patch description will be mangled.

Warning: this bug is being posted via bzexport. Let's see what happens...
Gerv, can you take a look at this from the bzAPI end?

When I send the following over openssl s_client, it creates a bug with a correct title:

POST /test/1.0/bug?userid=123456&cookie=xxXXxXxxx HTTP/1.1
Accept-Encoding: identity
Content-Length: 208
Host: api-dev.bugzilla.mozilla.org
Accept: application/json
User-Agent: Python-urllib/2.7
Connection: close
Content-Type: application/json

{"product": "FoodReplicator", "component": "Salt", "comments": [{"text": null}], "summary": "unicode\u2665", "platform": "All", "version": "1.0", "assigned_to": {"name": "sfink@mozilla.com"}, "op_sys": "All"}

However, when I try to attach to an existing bug using a very similar POST, the character gets horribly mangled:

POST /test/1.0/bug/11387/attachment?userid=123456&cookie=XXxXXXxxxX HTTP/1.1
Accept-Encoding: identity
Content-Length: 566
Host: api-dev.bugzilla.mozilla.org
Accept: application/json
User-Agent: Python-urllib/2.7
Connection: close
Content-Type: application/json

{"description": "unicode\u2665", "encoding": "base64", "file_name": "tip", "is_patch": true, "content_type": "text/plain", "data": "IyBIRyBjaGFuZ2VzZXQgcGF0Y2gKIyBVc2VyIFN0ZXZlIEZpbmsgPHNmaW5rQG1vemlsbGEuY29tPgojIERhdGUgMTM1NTgwOTMyMiAyODgwMAojIE5vZGUgSUQgN2I2ZmFiY2QwZWNlYWNiNjJlYzAzMmMxMThmZTY0NjY3NDE5YzVhYQojIFBhcmVudCAgMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMAp1bmljb2Rl4pmlCgpkaWZmIC0tZ2l0IGEvUkVBRE1FLnR4dCBiL1JFQURNRS50eHQKbmV3IGZpbGUgbW9kZSAxMDA2NDQKLS0tIC9kZXYvbnVsbAorKysgYi9SRUFETUUudHh0CkBAIC0wLDAgKzEsMSBAQAorV2UgbG92ZSB1bmljb2Rl4pmlCg=="}

See bug 810196 comment 1 for an example on this server, or https://landfill.bugzilla.org/bzapi_sandbox/show_bug.cgi?id=11387 for a bug created with the above POSTs.
Note that the attachment description field *can* store unicode properly. I used the UI to change the description on the attachment in https://landfill.bugzilla.org/bzapi_sandbox/show_bug.cgi?id=11385 and it worked fine.
I just added a test for this to my test suite, and it passed locally, but fails on the api-dev server when testing against my Bugzilla install on landfill. So I will need to work out what the difference is.

Gerv
I'm not sure why it worked locally, but the problem was HTTP::Message mangling Perl strings into Latin-1. I've written and committed a patch. I'll update the /latest API endpoint very soon, and you can test and see if things improve. :-)

Gerv
OK, apache has been restarted. sfink: can you retest using /latest?

Thanks,

Gerv
Assignee: nobody → sphink
Looks good, thank you!
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Attached patch njn's test.Splinter Review
I was seeing the problem when I put unicode in the attachment comments, so
here's a test of that.  Here come the special chars:  ├──
Assignee: sphink → n.nethercote
> Here come the special chars:  ├──

\o/

Thanks, Gerv.
Product: Other Applications → Developer Services
You need to log in before you can comment on or make changes to this bug.