Backfill monthly search rollups for 201706 with patch from in-content searches

RESOLVED FIXED

Status

Product:
Data Platform and Tools
Component:
Datasets: Search
Importance:
P1
normal
Status:
RESOLVED FIXED
Reported:
a year ago
Modified:
a year ago

People

(Reporter: amiyaguchi, Assigned: amiyaguchi)

Tracking

Details
(Assignee)

Description

•
a year ago 
Searches for the month of June are inflated due to incontent searches. See related bug 1380002 for the daily backfill.
 (Assignee)

Updated

•
a year ago
Assignee: nobody → amiyaguchi
Points: --- → 1
Priority: -- → P1
 (Assignee)

Updated

•
a year ago
Duplicate of this bug: 1392743
 (Assignee)

Comment 2

•
a year ago 
Data is backfilled for June and July, under s3://net-mozaws-prod-us-west-2-pipeline-analysis/spenrose/search/to_vertica/monthly/processed-2017-*-01.csv

The naming convention has changed slightly, but I also want to get rid of the manifest files.

Comment 3

•
a year ago 
Following up from Anthony's comment-

Matt does that mean we have data in vertica now to be able to fetch via Tableau?
Flags: needinfo?(mpressman)

Comment 4

•
a year ago 
Anthony and I have agreed to get rid of the manifest files. They were cumbersome and added complexity to the process. The only thing that's really necessary is to have a consistent naming convention of the files in order to pull down the correct files for the day being processed. I'll need to modify the current process to remove the logic that pulls the manifest to determine which files to download.
Flags: needinfo?(mpressman)

Comment 5

•
a year ago 
(In reply to Matt Pressman [:mpressman] from comment #4)
> Anthony and I have agreed to get rid of the manifest files. They were
> cumbersome and added complexity to the process. The only thing that's really
> necessary is to have a consistent naming convention of the files in order to
> pull down the correct files for the day being processed. I'll need to modify
> the current process to remove the logic that pulls the manifest to determine
> which files to download.

Thanks Matt for the clarification. This sounds like a sprint work which we can prioritize in our calls.
 (Assignee)

Comment 6

•
a year ago 
The bug is not dependent on the state of vertica/tableau, and the data has been backfilled appropriately.

2017-08-22 17:15:12  118578540 processed-2017-06-01.csv
2017-08-22 19:05:04  117934363 processed-2017-07-01.csv
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.