Closed Bug 1386639 Opened 8 years ago Closed 8 years ago

Backfill monthly search rollups for 201706 with patch from in-content searches

Categories

(Data Platform and Tools :: General, enhancement, P1)

enhancement
Points:
1

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: amiyaguchi, Assigned: amiyaguchi)

Details

Searches for the month of June are inflated due to incontent searches. See related bug 1380002 for the daily backfill.
Assignee: nobody → amiyaguchi
Points: --- → 1
Priority: -- → P1
Data is backfilled for June and July, under s3://net-mozaws-prod-us-west-2-pipeline-analysis/spenrose/search/to_vertica/monthly/processed-2017-*-01.csv The naming convention has changed slightly, but I also want to get rid of the manifest files.
Following up from Anthony's comment- Matt does that mean we have data in vertica now to be able to fetch via Tableau?
Flags: needinfo?(mpressman)
Anthony and I have agreed to get rid of the manifest files. They were cumbersome and added complexity to the process. The only thing that's really necessary is to have a consistent naming convention of the files in order to pull down the correct files for the day being processed. I'll need to modify the current process to remove the logic that pulls the manifest to determine which files to download.
Flags: needinfo?(mpressman)
(In reply to Matt Pressman [:mpressman] from comment #4) > Anthony and I have agreed to get rid of the manifest files. They were > cumbersome and added complexity to the process. The only thing that's really > necessary is to have a consistent naming convention of the files in order to > pull down the correct files for the day being processed. I'll need to modify > the current process to remove the logic that pulls the manifest to determine > which files to download. Thanks Matt for the clarification. This sounds like a sprint work which we can prioritize in our calls.
The bug is not dependent on the state of vertica/tableau, and the data has been backfilled appropriately. 2017-08-22 17:15:12 118578540 processed-2017-06-01.csv 2017-08-22 19:05:04 117934363 processed-2017-07-01.csv
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Component: Datasets: Search → Datasets: General
Component: Datasets: General → General
You need to log in before you can comment on or make changes to this bug.