Closed
Bug 1386639
Opened 8 years ago
Closed 8 years ago
Backfill monthly search rollups for 201706 with patch from in-content searches
Categories
(Data Platform and Tools :: General, enhancement, P1)
Data Platform and Tools
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: amiyaguchi, Assigned: amiyaguchi)
Details
Searches for the month of June are inflated due to incontent searches. See related bug 1380002 for the daily backfill.
Assignee | ||
Updated•8 years ago
|
Assignee: nobody → amiyaguchi
Points: --- → 1
Priority: -- → P1
Assignee | ||
Comment 2•8 years ago
|
||
Data is backfilled for June and July, under s3://net-mozaws-prod-us-west-2-pipeline-analysis/spenrose/search/to_vertica/monthly/processed-2017-*-01.csv
The naming convention has changed slightly, but I also want to get rid of the manifest files.
Comment 3•8 years ago
|
||
Following up from Anthony's comment-
Matt does that mean we have data in vertica now to be able to fetch via Tableau?
Flags: needinfo?(mpressman)
Comment 4•8 years ago
|
||
Anthony and I have agreed to get rid of the manifest files. They were cumbersome and added complexity to the process. The only thing that's really necessary is to have a consistent naming convention of the files in order to pull down the correct files for the day being processed. I'll need to modify the current process to remove the logic that pulls the manifest to determine which files to download.
Flags: needinfo?(mpressman)
Comment 5•8 years ago
|
||
(In reply to Matt Pressman [:mpressman] from comment #4)
> Anthony and I have agreed to get rid of the manifest files. They were
> cumbersome and added complexity to the process. The only thing that's really
> necessary is to have a consistent naming convention of the files in order to
> pull down the correct files for the day being processed. I'll need to modify
> the current process to remove the logic that pulls the manifest to determine
> which files to download.
Thanks Matt for the clarification. This sounds like a sprint work which we can prioritize in our calls.
Assignee | ||
Comment 6•8 years ago
|
||
The bug is not dependent on the state of vertica/tableau, and the data has been backfilled appropriately.
2017-08-22 17:15:12 118578540 processed-2017-06-01.csv
2017-08-22 19:05:04 117934363 processed-2017-07-01.csv
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Updated•4 years ago
|
Component: Datasets: Search → Datasets: General
Updated•3 years ago
|
Component: Datasets: General → General
You need to log in
before you can comment on or make changes to this bug.
Description
•