1333634 - Add telemetry-parquet/addons/agg/v1 bucket to re:dash

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P1)

Description

[Ben Miroglio [:bmiroglio]]

I created a new dataset that lives in telemetry-parquet/addons/agg/v1. Would like to have this available as a table in presto for dashboarding purposes.

Comment 1

[Frank Bertsch [:frank]]

Probably this table should not be called "agg". Perhaps "addon_aggregates"?

Comment 2

[Frank Bertsch [:frank]]

Note we can set the name with parquet2hive, so you won't have to do any work, Ben.

Comment 3

[Ben Miroglio [:bmiroglio]]

(In reply to Frank Bertsch [:frank] from comment #1)
> Probably this table should not be called "agg". Perhaps "addon_aggregates"?

Yes, addon_aggregates is a good name--Thanks!

Comment 4

[Frank Bertsch [:frank]]

Ben, I tried loading it, and it seems you have 'submission_date_s3' as both a partitioning column and a column in the table. It has to be one or the other, not both. I'd recommend taking it out of the parquet file, and just partition by it.

Comment 5

[Ben Miroglio [:bmiroglio]]

(In reply to Frank Bertsch [:frank] from comment #4)
> Ben, I tried loading it, and it seems you have 'submission_date_s3' as both
> a partitioning column and a column in the table. It has to be one or the
> other, not both. I'd recommend taking it out of the parquet file, and just
> partition by it.

Ah right, ok. I'll re-run my script having removed it from the table. Thanks for catching that

Comment 6

[Frank Bertsch [:frank]]

Okay, I worked with ben and we have this worked out.