1875816 - Event stream: make non-event metrics accessible in event stream table

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean Platform, task)

Description

[Jan-Erik Rediger [:janerik]]

We should make non-event metrics accessible in the event stream table.
Some apps do set them.
Luckily it's often the case that they send only one event per ping, so metrics apply as is and are either app or ping lifetime.

After talking with wil: We could merge all metrics into the event extra as JSON.
Would need another UDF, e.g. https://sql.telemetry.mozilla.org/queries/96989/source

Comment 1

[Jan-Erik Rediger [:janerik]]

Back in the proposal we called them out as a separate column.

As a structured column this would require us to list out the schema for each table. That doesn't seem good.
We essentially have two options:

1. Merge all of metrics converted to JSON into event_extra
2. Have a new column metrics, JSON type, and store mozfun.json.to_map(metrics) in there. That way it's easily accessible, same as the event extras

:wil, thoughts?

Comment 2

[Jan-Erik Rediger [:janerik]]

After talking with will we're going with a JSON column metrics, done by TO_JSON(metrics) from the original table.

Comment 3

[Jan-Erik Rediger [:janerik]]

Attachment: [mozilla/bigquery-etl] Bug 1875816 - events stream: Expose additional metrics as a new JSON column (#4926)

Comment 4

[Jan-Erik Rediger [:janerik]]

badboy merged PR [mozilla/bigquery-etl]: Bug 1875816 - events stream: Expose additional metrics as a new JSON column (#4926) in 56a12c7.