Closed
Bug 1076769
Opened 10 years ago
Closed 10 years ago
When an job with an unparsed log is accessed via the job details panel, parse it immediately
Categories
(Tree Management :: Treeherder, defect, P1)
Tree Management
Treeherder
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: emorley, Assigned: mdoglio)
References
Details
Attachments
(2 files)
Broken out from bug 1074927.
One of the problems with the current treeherder log parsing implementation compared to TBPL, is that there is no way to force log parsing for a job of interest, or otherwise raise the priority of the existing queued task.
This would alleviate many of the use-case problems highlighted in bug 1074927's deps and dupes, since even if there is a backlog of queued log parser tasks, or if we didn't quite get the prioritisation algorithm right, the user can still access the log fairly rapidly.
ie:
1) User clicks on job in job details panel
2) The job artefacts are requested, and either:
a) The service recognises that the log isn't parsed and pokes the log parser. It then returns the artefact as before, since we can't block on it.
b) The UI makes another call to a log parser endpoint once the returned artefact shows an unparsed log.
3) In response, the service either:
a) Bumps the priority of the current queued parse job
b) Adds a new queued task at the front of the queue (and the log parser knows to bail out when it reaches the original tasks)
4) If the UI received an artefact indicating an unparsed log, it knows to re-request the log every N seconds (with appropriate spinner/message in the UI) and/or has a retry button in the failure summary panel (next to the message about log parsing not complete).
Assignee | ||
Updated•10 years ago
|
Assignee: nobody → mdoglio
Reporter | ||
Updated•10 years ago
|
Status: NEW → ASSIGNED
Assignee | ||
Comment 1•10 years ago
|
||
Attachment #8510309 -
Flags: review?(cdawson)
Assignee | ||
Comment 2•10 years ago
|
||
Attachment #8510310 -
Flags: review?(cdawson)
Comment 4•10 years ago
|
||
Commits pushed to master at https://github.com/mozilla/treeherder-service
https://github.com/mozilla/treeherder-service/commit/b03b3320994c0461892898cb1262c76cc1831eb7
Bug 1076769 - setup lock strategy for the log parser
During a log parser run, the same log cannot be parsed by another
worker. This is done using a memcached-based lock mechanism.
https://github.com/mozilla/treeherder-service/commit/b6898b6f87e7189232a9e386307e6dd33b466164
Bug 1076769 - support parsing a log on demand
https://github.com/mozilla/treeherder-service/commit/f4e979a2485dbb5cb0d2163d2f86c72d00faff77
Merge pull request #261 from mozilla/bug-1076769-parse-log-on-demand
Bug 1076769 parse log on demand
Comment 5•10 years ago
|
||
Commits pushed to master at https://github.com/mozilla/treeherder-ui
https://github.com/mozilla/treeherder-ui/commit/847dda8aeedfa128c8a859ddcb48d01e37cf83a8
Bug 1076769 - support parsing a log on demand
https://github.com/mozilla/treeherder-ui/commit/19a695f29de0c6eaee81b86d6323148c86a0dc8c
Merge pull request #245 from mozilla/bug-1076769-parsing-logs-on-demand
Bug 1076769 - support parsing a log on demand
Updated•10 years ago
|
Attachment #8510309 -
Flags: review?(cdawson) → review+
Updated•10 years ago
|
Attachment #8510310 -
Flags: review?(cdawson) → review+
Reporter | ||
Comment 6•10 years ago
|
||
Partially reverted in:
https://github.com/mozilla/treeherder-service/commit/0cdf0092a1e71ba49227e288747f2586f2dec04c
desc: "revert latest changes to the parse-log task to reduce the load on memcached"
Assignee | ||
Comment 7•10 years ago
|
||
The code reverted was an optimization to avoid 2 log parsers to run in parallel on the same log. Given the performance consequences of using memcached delete, it's not worth it. This is in on production now
Assignee | ||
Updated•10 years ago
|
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Comment 8•10 years ago
|
||
Commits pushed to master at https://github.com/mozilla/treeherder
https://github.com/mozilla/treeherder/commit/d548b279a85aa2c3bc43352f092686fba79e1905
Bug 1076769 - support parsing a log on demand
https://github.com/mozilla/treeherder/commit/19eec1b5604b2a989210c6a4dca14ab88cfc7c68
Merge pull request #245 from mozilla/bug-1076769-parsing-logs-on-demand
Bug 1076769 - support parsing a log on demand
You need to log in
before you can comment on or make changes to this bug.
Description
•