Backfill monthly search rollups for 201706 with patch from in-content searches

RESOLVED FIXED

Status

P1
normal
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: amiyaguchi, Assigned: amiyaguchi)

Tracking

Details

(Assignee)

Description

a year ago
Searches for the month of June are inflated due to incontent searches. See related bug 1380002 for the daily backfill.
(Assignee)

Updated

a year ago
Assignee: nobody → amiyaguchi
Points: --- → 1
Priority: -- → P1
(Assignee)

Updated

a year ago
Duplicate of this bug: 1392743
(Assignee)

Comment 2

a year ago
Data is backfilled for June and July, under s3://net-mozaws-prod-us-west-2-pipeline-analysis/spenrose/search/to_vertica/monthly/processed-2017-*-01.csv

The naming convention has changed slightly, but I also want to get rid of the manifest files.
Following up from Anthony's comment-

Matt does that mean we have data in vertica now to be able to fetch via Tableau?
Flags: needinfo?(mpressman)
Anthony and I have agreed to get rid of the manifest files. They were cumbersome and added complexity to the process. The only thing that's really necessary is to have a consistent naming convention of the files in order to pull down the correct files for the day being processed. I'll need to modify the current process to remove the logic that pulls the manifest to determine which files to download.
Flags: needinfo?(mpressman)
(In reply to Matt Pressman [:mpressman] from comment #4)
> Anthony and I have agreed to get rid of the manifest files. They were
> cumbersome and added complexity to the process. The only thing that's really
> necessary is to have a consistent naming convention of the files in order to
> pull down the correct files for the day being processed. I'll need to modify
> the current process to remove the logic that pulls the manifest to determine
> which files to download.

Thanks Matt for the clarification. This sounds like a sprint work which we can prioritize in our calls.
(Assignee)

Comment 6

a year ago
The bug is not dependent on the state of vertica/tableau, and the data has been backfilled appropriately.

2017-08-22 17:15:12  118578540 processed-2017-06-01.csv
2017-08-22 19:05:04  117934363 processed-2017-07-01.csv
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.