Closed
Bug 1390332
Opened 8 years ago
Closed 7 years ago
Set spark.scheduler.mode to FAIR
Categories
(Data Platform and Tools Graveyard :: Telemetry Analysis Service (ATMO), enhancement, P4)
Data Platform and Tools Graveyard
Telemetry Analysis Service (ATMO)
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: amiyaguchi, Unassigned)
Details
Attachments
(1 file)
First in first out is a good policy for long running batch jobs, but isn't necessarily the best policy for analysis work. ATMO should set the scheduler mode to fair, which enables a round robin scheduling policy for jobs.[1] This would make working in Jupyter a bit nicer (Zeppelin already does this by default).
[1] https://spark.apache.org/docs/latest/job-scheduling.html#scheduling-within-an-application
|
Reporter | |
Updated•8 years ago
|
Assignee: nobody → amiyaguchi
Points: --- → 1
Priority: -- → P1
|
|
Reporter | |
Comment 1•8 years ago
|
||
|
|
Reporter | |
Updated•8 years ago
|
Points: 1 → 2
|
|
Reporter | |
Updated•8 years ago
|
Assignee: amiyaguchi → nobody
Priority: P1 → P4
|
|
Reporter | |
Comment 2•7 years ago
|
||
The responsibility of fair resource allocation probably lies with YARN. Enabling the fair scheduling mode in Spark didn't help with multitenancy in my small trial usage.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
|
||
Updated•5 years ago
|
Product: Data Platform and Tools → Data Platform and Tools Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•