Closed Bug 1311853 Opened 8 years ago Closed 8 years ago

Progress bars on spark clusters busted

Categories

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P2)

Product:
Cloud Services Graveyard
Component:
Metrics: Pipeline
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: isegall, Assigned: frank)

References

Details

roughly 75% of the time I'm missing progress bars when using spark. With data sets blowing up, this is slowing down the debugging process substantially.
Flags: needinfo?(jezdez)
:Ilana Can you elaborate what "busted" looks like? I don't have the appropriate Jupyter notebooks to reproduce right away. Any details would help, like screenshots or steps to reproduce. Thanks!
Flags: needinfo?(jezdez)
Most of the time, they simply don't appear. Unfortunately, I haven't been able to discern a pattern for this. Most recently, it was with this notebook: https://gist.github.com/ilanasegall/b3ce1aa0d3cc8c117a35b4a4fb9d4681, but it seems to be a problem tied to the cluster, not the script.
bump to p3 as needed
Points: --- → 2
Priority: -- → P3
Assignee: nobody → fbertsch
Priority: P3 → P2
I noticed something similar today, but it was actually due to the kernel that wasn't responding. After killing the kernel and restarting it, I could see the progress bars.
Is this still an issue? I haven't been able to reproduce.
Flags: needinfo?(isegall)
Can confirm that at least the last few times this occurred was when the kernel wasn't responding. Is there a good fix for this case? I generally have to reconnect several times in order to mediate.
Flags: needinfo?(isegall)
You can restart the kernel from the Jupyter interface, from the "Kernel" menu. Doesn't it work for you?
Not always, no - I often require a few restarts. If this occurs again, I'll file a new bug for it.
Sorry about the delay. It makes sense that there is no response when the kernel isn't responding, since the extension can't respond either. I don't think there's anything we can really do about this case. A workaround can be to use the Spark UI [0]. If it becomes a serious issue we can look into fixing it, so please let us know. [0] https://wiki.mozilla.org/Telemetry/Custom_analysis_with_spark#Accessing_the_Spark_UI
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
Product: Cloud Services → Cloud Services Graveyard
You need to log in before you can comment on or make changes to this bug.