Airflow task bqetl_event_rollup.telemetry_derived__events_daily__v1 failing on 2022-12-14
Categories
(Data Platform and Tools :: General, defect)
Tracking
(Not tracked)
People
(Reporter: relud, Assigned: akomar)
Details
(Whiteboard: [airflow-triage])
Attachments
(1 external file)
The Airflow task bqetl_event_rollup.telemetry_derived__events_daily__v1 failing on 2022-12-14.
Error in query string: Error processing job 'moz-fx-data-shared-prod:bqjob_r77720627a06a6424_000001850f749c72_1': Cannot query rows larger than 100MB limit.
Comment 1•3 years ago
•
|
||
Also failed on 2022-12-17 with the same error.
Comment 2•3 years ago
|
||
Also failed on 2022-12-29 with the same error.
Comment 3•3 years ago
|
||
Also failed on 2022-12-31 with the same error.
| Assignee | ||
Comment 4•3 years ago
|
||
This table seems to be no longer used: https://sql.telemetry.mozilla.org/queries/89535/source, I'm wondering if we should retire it.
:frank - looks like you were working on this DAG in early 2021 before :wlach took ownership. What was the problem this was solving? Do we still need this?
Comment 5•3 years ago
|
||
It appears the specific run in question in this ticket is showing currently as successful in Airflow.
Comment 6•3 years ago
|
||
We should leave it kicking for a little while longer, until there's more decisions made around event analysis. My understanding is that this will be worked on in Q1 or Q2. However I don't have the bandwidth to tackle this bug right now. cc'ing Rob Miller.
Comment 7•3 years ago
|
||
Someday soon we'll have a data catalog and I'll be able to look things like this up (hooray!), but until then I have to come ask y'all. Arkadiusz's query shows that this data set isn't being used in STMO at all. Is it used anywhere else? Does this feed into the regex-based event funnel explore you built, Frank?
Comment 8•3 years ago
|
||
Does this feed into the regex-based event funnel explore you built, Frank?
Yes. Looker depends on it
| Assignee | ||
Comment 9•3 years ago
|
||
But if it's not coming up in monitoring.bigquery_usage then no one looks at it in Looker?
Comment 10•3 years ago
|
||
I chatted w some folks and I think we can turn this off. We're going to be working on some new event processing and user flow / funnel analysis stuff soon, and the ETL code and the event funnel explore will both be useful as references and starting points, but we don't need the DAG to keep running and new data to keep flowing for that case. So for now we're clear to turn off the DAG, just keep the table(s) and don't remove any of the bq-etl or lookml code or artifacts. Thanks!
| Assignee | ||
Comment 11•3 years ago
|
||
| Assignee | ||
Updated•3 years ago
|
Updated•3 years ago
|
Description
•