No main_summary data since Aug 10

RESOLVED FIXED

Status

Data Platform and Tools
Datasets: Main Summary
P1
normal
RESOLVED FIXED
10 months ago
9 months ago

People

(Reporter: chutten, Assigned: robotblake)

Tracking

Details

(Whiteboard: [SvcOps])

(Reporter)

Description

10 months ago
11:47 <@bsmedberg> Is there a bug on file where main_summary only has data through 10-Aug?
11:49 <@bsmedberg> robotblake, ^^
11:50 <chutten> I was just about to ask about that, too
12:10 <@bsmedberg> chutten, can you make sure this gets filed?
12:11 <chutten> bsmedberg: I'll see what I can do
12:12 <chutten> Oh, file a bug? Yeah, I can do that
12:12 <chutten> Do you have a query I should ref?

My queries can't get through at the moment (Error running query: ec2-34-212-32-219.us-west-2.compute.amazonaws.com: java.net.SocketException: Connection reset) so I can't 100% confirm, but my queries were getting 0 data from the last two days. That's consistent with BDS' report.

Comment 1

10 months ago
Blake, any ideas?
Flags: needinfo?(bimsland)

Updated

10 months ago
Assignee: nobody → bimsland
Priority: -- → P1
Whiteboard: [SvcOps]
(Reporter)

Updated

10 months ago
Blocks: 1380298
Benjamin says this main_summary bug is preventing some of the Flash Click-to-Activate Statistics dashboard's graphs from updating.

https://sql.telemetry.mozilla.org/dashboard/flash-click-to-activate-statistics
(Reporter)

Comment 3

10 months ago
:robotblake did some magic and got Presto's main_summary back working again. Unfortunately the data only goes up to 20170815.
(Reporter)

Comment 4

10 months ago
For comparison, Athena's main_summary only goes up to 20160928
(Assignee)

Comment 5

10 months ago
I'm currently working on fixing the parquet2hive code to do partition diffs and apply them instead of relying on MSCK REPAIR TABLE as that has a tendency to fail (as it did in this case). For Athena, we're hitting a partition limit that will be resolved by moving the dataset to the Athena instance in the data IAM and that is next on my list.
Flags: needinfo?(bimsland)
(Reporter)

Comment 6

9 months ago
This particular failure has been resolved. It's indicative of a broader problem, though (https://github.com/mozilla/telemetry-airflow/pull/168) which has hopefully been taken care of.
Status: NEW → RESOLVED
Last Resolved: 9 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.