Closed Bug 1352521 Opened 7 years ago Closed 6 years ago

Use new Presto parquet reader

Categories

(Data Platform and Tools Graveyard :: Presto, enhancement, P3)

enhancement

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: frank, Assigned: robotblake)

References

Details

(Whiteboard: [DataOps])

This new reader will make schema evolution of ROW types work. See [0].

Update it with the following option: hive.parquet-optimized-reader.enabled=true

[0] https://github.com/fbertsch/schema_evolution_exploration/blob/master/parquet-optimized-reader%3Dtrue/use-column-names%3Dtrue/Schema_Evolution_Exploration.ipynb
Whiteboard: [SvcOps]
Points: 1 → 2
Priority: P1 → P2
Depends on: 1358232
Component: Metrics: Pipeline → Presto
Product: Cloud Services → Data Platform and Tools
This is on hold as it breaks some queries due to what appears to be a bug in upstream Presto. Currently working on a repro and will drop the link in here once I get that bug filed.
Whiteboard: [SvcOps] → [DataOps]
I'm trying to remember if this is still broken (or even a valid bug anymore).
Flags: needinfo?(fbertsch)
Schema evolution of ROW types is still broken. If you haven't started using the new one then we aren't.
Flags: needinfo?(fbertsch)
Priority: P2 → P3
With the new version deployed this is now the default.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Product: Data Platform and Tools → Data Platform and Tools Graveyard
You need to log in before you can comment on or make changes to this bug.