Add telemetry-parquet/addons/agg/v1 bucket to re:dash

RESOLVED FIXED

Status

Cloud Services
Metrics: Pipeline
P1
normal
RESOLVED FIXED
9 months ago
9 months ago

People

(Reporter: bmiroglio, Assigned: robotblake)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [SvcOps])

(Reporter)

Description

9 months ago
I created a new dataset that lives in telemetry-parquet/addons/agg/v1. Would like to have this available as a table in presto for dashboarding purposes.
Flags: needinfo?(bimsland)
Whiteboard: [SvcOps]
Blocks: 1255755

Updated

9 months ago
Assignee: nobody → bimsland
Priority: -- → P2
(Reporter)

Updated

9 months ago
Priority: P2 → P1
Probably this table should not be called "agg". Perhaps "addon_aggregates"?
Flags: needinfo?(bmiroglio)
Note we can set the name with parquet2hive, so you won't have to do any work, Ben.
(Reporter)

Comment 3

9 months ago
(In reply to Frank Bertsch [:frank] from comment #1)
> Probably this table should not be called "agg". Perhaps "addon_aggregates"?

Yes, addon_aggregates is a good name--Thanks!
Flags: needinfo?(bmiroglio)
Ben, I tried loading it, and it seems you have 'submission_date_s3' as both a partitioning column and a column in the table. It has to be one or the other, not both. I'd recommend taking it out of the parquet file, and just partition by it.
Flags: needinfo?(bmiroglio)
(Reporter)

Comment 5

9 months ago
(In reply to Frank Bertsch [:frank] from comment #4)
> Ben, I tried loading it, and it seems you have 'submission_date_s3' as both
> a partitioning column and a column in the table. It has to be one or the
> other, not both. I'd recommend taking it out of the parquet file, and just
> partition by it.

Ah right, ok. I'll re-run my script having removed it from the table. Thanks for catching that
Flags: needinfo?(bmiroglio)
Okay, I worked with ben and we have this worked out.
Status: NEW → RESOLVED
Last Resolved: 9 months ago
Flags: needinfo?(bimsland)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.