Closed Bug 1310157 Opened 8 years ago Closed 7 years ago

ATMO V2: Show cluster logs in detail page

Categories

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P2)

Product:
Cloud Services Graveyard
Component:
Metrics: Pipeline
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: jezdez, Assigned: jezdez)

References

Details

User Story

With --log-uri EMR updates logs on S3 every 5 minutes. You can find an example of the log layout dumped to S3 at [1]. There might be hundreds of files and some are more useful than others depending on what one is trying to debug (e.g. spark.log). 

It would be nice if the UI had a S3 explorer with an inline view which would allow the job owner to navigate and visualize the logs. Furthermore, as some logs tend to be generally more useful than others, it would be convenient if the UI came with a default set of bookmarks, like in the case of [1]:
- node/i-2a1580f7/applications/spark/spark.log.gz
- steps/s-2G6TKVVOX60B7/[stdout|stderr].gz

The log viewer should support both ad-hoc jobs (which don't have steps) and scheduled ones (which have steps).

[1] https://console.aws.amazon.com/s3/home?region=us-west-2#&bucket=telemetry-parquet&prefix=logs/mreid%40mozilla.com/Main%20Summary%20View/j-10MI4M52BB1MZ/

Attachments

(1 file)

[telemetry-analysis-service] mozilla:bug-1310157-fix-job-logs-urls > mozilla:master
61 bytes, text/x-github-pull-request
Details | Review 
Once bug 1307528 is storing log files for the clusters we should show the log files in the cluster detail page.

This is basically a continuation of bug 1309543
Points: --- → 3
Priority: -- → P2
Assignee: nobody → jezdez
Status: NEW → ASSIGNED
User Story: (updated)
User Story: (updated)
The above attachment is a fix for bug 1316035, I attached it here by mistake
Moved to https://github.com/mozilla/telemetry-analysis-service/issues/193
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → INVALID
Product: Cloud Services → Cloud Services Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: