Closed Bug 1604910 Opened 6 years ago Closed 6 years ago

Update CI for mozilla-pipeline-schemas with data from BigQuery document sample

Categories

(Data Platform and Tools :: General, enhancement, P2)

Product:
Data Platform and Tools
Component:
General
Type:
enhancement
Points:
2

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: amiyaguchi, Assigned: amiyaguchi)

Details

mozilla-pipeline-schemas currently has CI that should be updated to take advantage of the current ingestion workflow.

In particular, the following things would improve the workflow for updating schemas:

  • Providing validation error rates on a subset of documents sampled from the GCP pipeline using locally hosted infrastructure
  • Show what the generated schemas would look like on PR
  • Provide access to tables generated by the CI process
Assignee: nobody → amiyaguchi
Summary: Update CI for mozilla-pipeline-schemas with workflow from ingestion-beam → Update CI for mozilla-pipeline-schemas with data from BigQuery document sample
Points: --- → 2
Priority: -- → P2
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED

This is released as v1.5.0.

You need to log in before you can comment on or make changes to this bug.