Migrate Search ETL job to python_mozetl

RESOLVED FIXED

Status

Data Platform and Tools
Datasets: Search
P1
normal
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: Thomas Huelbert, Assigned: amiyaguchi)

Tracking

Details

Attachments

(3 attachments)

(Reporter)

Description

a year ago
placeholder, actual steps (version control, monitoring etc) to be added.
This job would be a good candidate for moving to python_mozetl.

Updated

a year ago
Summary: We need to productionize the search job → Migrate Search ETL job to python_mozetl
(Assignee)

Comment 2

a year ago
Created attachment 8868294 [details] [review]
Bug 1364530 - Add daily search rollups via nbconvert #35
(Assignee)

Comment 3

a year ago
Created attachment 8868324 [details] [review]
Bug 1364530 - Use search rollup from mozetl instead of gist #117
(Assignee)

Comment 4

a year ago
I've added pull requests to the corresponding repositories to add a minimally modified version of the search rollup to mozetl and schedule it on airflow. This brings the job into a repository, making changes easier to review and merge.

This job still needs to hit some subset of the following points:

* Backfill all failed dates from May 8 - May 14/15
* Use the execution date passed by airflow in rollup to enable backfill
* Exercise code paths via unit testing
* Remove custom alerting in lieu of airflow alerts, log relevant information to logfiles/stdout
(Assignee)

Updated

a year ago
Assignee: nobody → amiyaguchi
Points: --- → 2
Priority: -- → P1
(Assignee)

Updated

a year ago
Duplicate of this bug: 1365653
(Assignee)

Comment 6

a year ago
Created attachment 8877849 [details] [review]
Bug 1364530 - Migrate Search ETL #52

Comment 7

a year ago
Shall we close the following bugs as duplicates of this one:

https://bugzilla.mozilla.org/show_bug.cgi?id=1365578
https://bugzilla.mozilla.org/show_bug.cgi?id=1365580

Updated

a year ago
Duplicate of this bug: 1365578

Updated

a year ago
Depends on: 1365580

Updated

a year ago
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.