1393838 - Redash is not escaping HTML-like strings in query results

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect)

Description

[Kate Ustiuzhanina]

https://sql.telemetry.mozilla.org/queries/24136/source
This link shows lots of payload/pingDiscardedForSize failures contains empty string instead of ping type.

The second problem here, that tmo didn't show this empty strings.

Comment 1

[Kate Ustiuzhanina]

Turns out that keeping "<unknown>" value as a key in map was not the best idea.

Comment 2

[Georg Fritzsche [:gfritzsche]]

We have values like "<unknown>" in the health ping for `pingDiscardedForSize`, which is stored as a map.
See bug 1390104 c0 for the schema.

We suspect that either d2p, parquet or presto can't handle this, resulting in a key of "" in queries.
Do you know which?
How & where can we best handle this?

Comment 3

[Kate Ustiuzhanina]

Attachment: unknown.patch

Comment 4

[Georg Fritzsche [:gfritzsche]]

Comment on attachment 8901263 [details] [diff] [review]
unknown.patch

Review of attachment 8901263 [details] [diff] [review]:
-----------------------------------------------------------------

Cancelling the review until we know what the proper approach is in the pipeline.

Comment 5

[Mike Trinkala [:trink]]

A key of '<unknown>' is fine in the ingestion pipeline and parquet output.

Comment 6

[Georg Fritzsche [:gfritzsche]]

Mark, do you have any ideas on the point of failure here (per comment 2 and comment 5)?

Comment 7

[Mark Reid [:mreid]]

This looks like a re:dash issue where it's trying to render the string "<unknown>" as an HTML tag... if you download the csv you see items like:

id,oversize
some-uuid1,{u'<unknown>': 1}
some-uuid2,{u'<unknown>': 1}
some-uuid3,{u'main': 1}
some-uuid4,{u'<unknown>': 1}

Comment 8

[Rob Miller [:rmiller]]

This issue should now be resolved: https://github.com/mozilla/redash/issues/222