1854405 - Airflow task probe_scraper.lookml_generator failed for exec_date 2023-09-21

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect)

Description

[Mike Williams [:mwilliams]]

Airflow task probe_scraper.lookml_generator failed for exec_date 2023-09-21

Task link:
https://workflow.telemetry.mozilla.org/dags/probe_scraper/grid?root=&dag_run_id=scheduled__2023-09-20T00%3A00%3A00%2B00%3A00&task_id=lookml_generator

Log extract:

[2023-09-21, 05:34:48 UTC] {pod_manager.py:235} INFO - + git push -f --set-upstream origin main-working
[2023-09-21, 05:34:50 UTC] {pod_manager.py:235} INFO - To github.com:mozilla/looker-spoke-default.git
[2023-09-21, 05:34:50 UTC] {pod_manager.py:235} INFO -  + 2683736...b93e231 main-working -> main-working (forced update)
[2023-09-21, 05:34:50 UTC] {pod_manager.py:235} INFO - branch 'main-working' set up to track 'origin/main-working'.
[2023-09-21, 05:34:50 UTC] {pod_manager.py:235} INFO - + git diff --quiet main...main-working
[2023-09-21, 05:34:50 UTC] {pod_manager.py:235} INFO - + gh pr create --title 'Auto-push from LookML Generator' --base main --head main-working --body ''
[2023-09-21, 05:34:50 UTC] {pod_manager.py:235} INFO - a pull request for branch "main-working" into branch "main" already exists:
[2023-09-21, 05:34:50 UTC] {pod_manager.py:235} INFO - https://github.com/mozilla/looker-spoke-default/pull/624
[2023-09-21, 05:34:52 UTC] {pod_manager.py:288} INFO - Pod lookml-generator-1-fqgpw81e has phase Running
[2023-09-21, 05:34:54 UTC] {kubernetes_pod.py:691} INFO - Skipping deleting pod: lookml-generator-1-fqgpw81e
[2023-09-21, 05:34:54 UTC] {taskinstance.py:1824} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/google/cloud/operators/kubernetes_engine.py", line 532, in execute
    result = super().execute(context)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 516, in execute
    return self.execute_sync(context)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 545, in execute_sync
    self.cleanup(
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 671, in cleanup
    raise AirflowException(
airflow.exceptions.AirflowException: Pod lookml-generator-1-fqgpw81e returned a failure:

Comment 1

[Curtis Morales [:curtis]]

The pull requests needed to be merged; the task ran successfully once we did that.