Closed Bug 769844 Opened 8 years ago Closed 5 years ago

Special characters in TelemetryPing properties are lost when sent to the server


(Webtools Graveyard :: Telemetry Server, defect)

Not set


(Not tracked)



(Reporter: nicolas, Unassigned)




(2 files)

Attached file Repro
Non-ascii characters in the TelemetryPing payload are lost and replaced with a default character before the data is compressed in gzipCompressString(); UTF-8 characters are not preserved.
Attached patch Proposed patchSplinter Review
Attachment #638047 - Flags: review?(taras.mozilla)
Attachment #638047 - Flags: feedback?(vdjeric)
See related bug 767077
Comment on attachment 638047 [details] [diff] [review]
Proposed patch

Looks, good but let's wait on pushing this until we can test this against Metrics' servers:

(7:53:26 PM) nchaim: proposing to update TelemetryPing to send data in UTF-8 format to the server (bug 769844)
(7:53:26 PM) nchaim: does this need any special considerations on the server side?
(7:59:28 PM) vladan: xstevens: would you know this ^^ type of thing?
(8:02:21 PM) xstevens: yeah…I've never tried sending that type of payload
(8:03:02 PM) xstevens: I use UTF-8 just about everywhere I can think of on my end
(8:03:31 PM) vladan: can we do a test ping and you can let us know if any issues come up?
(8:03:54 PM) vladan: do you have guys have a test server somewhere?
(8:05:41 PM) xstevens: not at the moment
(8:05:57 PM) xstevens: my boxes got blown away by ops this week
(8:06:12 PM) xstevens: I can ask them next week to give me some test machines
(8:17:39 PM) vladan: alright
Attachment #638047 - Flags: feedback?(vdjeric) → feedback+
Comment on attachment 638047 [details] [diff] [review]
Proposed patch

Review of attachment 638047 [details] [diff] [review]:

This looks like it's on the right track, but can we avoid converting the string to UTF-8 twice?

Also, this needs a test that we can round-trip properties and/or values with non-ASCII characters.
Attachment #638047 - Flags: review?(taras.mozilla) → review-
As I understand it, we really only need this to deal with sending in plugin information, since we have no control over what encoding the plugin information is.

However, we've decided that we're not interested in general plugins at this point, just Flash.  So this bug is not worth fixing at this point.
Closed: 7 years ago
OS: Windows 7 → All
Hardware: x86_64 → All
Resolution: --- → INCOMPLETE
I thought this came up when reporting extension IDs, not plugins?
Resolution: INCOMPLETE → ---
(In reply to Vladan Djeric (:vladan) from comment #6)
> I thought this came up when reporting extension IDs, not plugins?

I originally stumbled upon this with plugins (and their basically free form text fields for names etc.).
I'm doing an analysis of Search Engine usage (and the non-English search engines are full of non-English characters) and I'm getting a ton of literal \ufffd instead of properly encoded unicode codepoints. Any hope of getting this worked on soon?
Flags: needinfo?(gfritzsche)
I can reproduce this - it appears to be a problem with the server validation/conversion code.
Flags: needinfo?(gfritzsche)
Component: Telemetry → Telemetry Server
Product: Toolkit → Webtools
This fix has been deployed.
Closed: 7 years ago5 years ago
Resolution: --- → FIXED
Product: Webtools → Webtools Graveyard
You need to log in before you can comment on or make changes to this bug.