Closed Bug 1826618 Opened 2 years ago Closed 2 years ago

Set up scheduled dummy ETL tasks to identify ETL problems faster

Categories

(Data Platform and Tools :: General, enhancement, P2)

Product:
Data Platform and Tools
Component:
General
Type:
enhancement

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: ascholtz, Assigned: ascholtz)

Details

(Whiteboard: [dataplatform])

Attachments

(1 file)

[mozilla/bigquery-etl] [Bug 1826618] bqetl status checks (#3706)
49 bytes, text/x-github-pull-request
Details | Review

More details are in https://mozilla.slack.com/archives/C01E8GDG80N/p1680633858101559

Often, breaking changes to bigquery-etl are noticed only during nightly ETL runs. To detect them earlier, dummy ETL tasks that are scheduled every 30 minutes could be set up that would alert data engineers much earlier. Dummy tasks for running a SQL query and Python query (and other critical tasks) should be scheduled.

Assignee: nobody → ascholtz
Whiteboard: [data-platform-infra-wg] → [dataplatform]
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: