Spark console disappears when executing a second Python notebook

RESOLVED WONTFIX

Status

Cloud Services
Metrics: Pipeline
RESOLVED WONTFIX
2 years ago
2 years ago

People

(Reporter: Dexter, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

2 years ago
I've been experiencing a weird issue in the last couple of days.

- Run the notebook from [0] and wait for it to finish.
- Open the Spark shell.
- Run the notebook from [1]
- Try to refresh the Spark shell page.
- It will take some time to load and will eventually display an Hadoop status page.
- The SSH proxy command will display the following errors:

"$ ssh -i RSA_KEY -N -D 8157 hadoop@ec2-54-184-179-139.us-west-2.compute.amazonaws.com
channel 2: open failed: connect failed: Connection refused
channel 2: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 2: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 6: open failed: connect failed: Connection refused
channel 2: open failed: connect failed: Connection refused
channel 2: open failed: connect failed: Connection refused
channel 2: open failed: connect failed: Connection refused
channel 2: open failed: connect failed: Connection refused
channel 12: open failed: connect failed: Connection refused
channel 12: open failed: connect failed: Connection refused
channel 12: open failed: connect failed: Connection refused
channel 12: open failed: connect failed: Connection refused"

All computations will stop and the only way to regain control of the cluster is to kill it and create a new one.

[0] - https://gist.github.com/Dexterp37/52247591346eb3c26dac
[1] - https://gist.github.com/Dexterp37/3b7019849425ddae52f3
Running more than one notebook on the same cluster is not supported for now.
(Reporter)

Comment 2

2 years ago
I see, then I think we can close this one right now!
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.