Closed
Bug 1880272
Opened 2 years ago
Closed 2 years ago
Airflow task ltv_daily.ltv_daily failed for exec_date 2024-02-14
Categories
(Data Platform and Tools :: General, defect)
Data Platform and Tools
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: lvargas, Unassigned)
Details
(Whiteboard: [airflow-triage])
Airflow task ltv_daily.ltv_daily failed for exec_date 2024-02-14
Log extract:
[2024-02-14, 05:37:32 UTC] {taskinstance.py:1937} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 1518, in _run_raw_task
self._execute_task_with_callbacks(context, test_mode, session=session)
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 1684, in _execute_task_with_callbacks
self.task.post_execute(context=context, result=result) # type: ignore[union-attr]
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/subdag.py", line 190, in post_execute
raise AirflowException(f"Expected state: SUCCESS. Actual state: {dag_run.state}")
Comment 1•2 years ago
|
||
Error here was of model convergence:
24/02/14 13:14:20 WARN org.apache.spark.util.Utils: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.
fun: -150.67940576586616
hess_inv: array([[1.65548971, 1.7232383 , 0.18719648, 0.0533623 ],
[1.7232383 , 6.35185852, 0.25046752, 0.04126487],
[0.18719648, 0.25046752, 4.17361378, 1.80086176],
[0.0533623 , 0.04126487, 1.80086176, 3.73694248]])
jac: array([-1.14056470e-07, -3.88347797e-08, 5.38994670e-08, -4.93878932e-08])
message: 'Desired error not necessarily achieved due to precision loss.'
nfev: 86
nit: 20
njev: 74
status: 2
success: False
x: array([-1.46818669, -5.88405973, -2.54970807, -1.41937746])
Traceback (most recent call last):
File "/tmp/ltv-daily_83a4891d/ltv_daily.py", line 252, in <module>
main(**vars(args))
File "/tmp/ltv-daily_83a4891d/ltv_daily.py", line 138, in main
model_perf, model = train_metric(search_rfm_ds, metric, plot=False, penalty=0.8)
File "/tmp/ltv-daily_83a4891d/ltv_daily.py", line 30, in train_metric
bgf.fit(train[frequency], train[recency], train[T])
File "/opt/conda/default/lib/python3.6/site-packages/lifetimes/fitters/beta_geo_fitter.py", line 129, in fit
**kwargs
File "/opt/conda/default/lib/python3.6/site-packages/lifetimes/fitters/__init__.py", line 119, in _fit
"""
lifetimes.utils.ConvergenceError:
The model did not converge. Try adding a larger penalizer to see if that helps convergence.
| Reporter | ||
Comment 2•2 years ago
|
||
SubDAG ltv_daily.ltv_daily is showing No Status for all tasks, and overall status running on 2024-02-15.
| Reporter | ||
Comment 3•2 years ago
|
||
Depends on past = True. Further runs are blocked until the issue is resolved.
Comment 4•2 years ago
|
||
still blocked
Comment 5•2 years ago
|
||
Restarting fixed this.
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•