Open Bug 1313108 Opened 8 years ago Updated 4 years ago

Document Backwards Compatible BigQuery Schema Changes

Categories

(Data Platform and Tools :: Documentation and Knowledge Repo (RTMO), defect, P3)

defect
Points:
2

Tracking

(Not tracked)

People

(Reporter: frank, Unassigned)

References

Details

There are a lot of questions about how we handle schema changes. We should document what schema changes are backwards compatible, any Spark/Presto quirks, how they will handle specific types of changes, and what to do for non backwards compatible changes (mainly, increase versioning).

This will be especially relevant as people generate their own datasets.
Points: --- → 2
Priority: -- → P2
Depends on: 1313182
A lot of the relevant investigation has been done by Frank here:
https://github.com/fbertsch/schema_evolution_exploration
Component: Metrics: Pipeline → Documentation and Knowledge Repo (RTMO)
Product: Cloud Services → Data Platform and Tools
Moving to P3, I don't think we'll be able to get to this in Q2.
Priority: P2 → P3
See Also: → 1338365

We're no longer using parquet directly, but this sort of issue comes up occasionally for bigquery so I think this bug is still valid.

Summary: Document Backwards Compatible Parquet Schema Changes → Document Backwards Compatible BigQuery Schema Changes
You need to log in before you can comment on or make changes to this bug.