Closed Bug 1542809 Opened 6 years ago Closed 5 years ago

Unknown re:dash error for untrusted module ping (HIVE_CANNOT_OPEN_SPLIT)

Categories

(Data Platform and Tools :: General, defect, P2)

defect
Points:
1

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: jimm, Unassigned)

Details

This started showing up recently, would appreciate some help in understanding what's going wrong.

https://sql.telemetry.mozilla.org/queries/61950/source

Error running query: HIVE_CANNOT_OPEN_SPLIT: Error opening Hive split s3://net-mozaws-prod-us-west-2-pipeline-data/telemetry-untrustedModules-parquet/v2/submission_date_s3=20190313/1552439039_0_ip-172-31-20-140 (offset=0, length=146944): Schema mismatch, metastore schema for row column environment.settings has 14 fields but parquet schema has 13 fields

adding a limit on submission date of 4/1 helped fixed this.

Blake, is there something we can do to fix the schema for the files prior to 4/1?

Flags: needinfo?(bimsland)
Points: --- → 1
Priority: -- → P2

I think this should work in Presto as it does the pathing differently. For Athena there's not much we can do unfortunately without bumping the version.

Flags: needinfo?(bimsland)

This is no longer relevant in the new pipeline.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
Component: Datasets: General → General
You need to log in before you can comment on or make changes to this bug.