Closed Bug 1477733 Opened 7 years ago Closed 6 years ago

All ATMO scheduled jobs are being duplicated

Categories

(Data Platform and Tools Graveyard :: Telemetry Analysis Service (ATMO), enhancement)

Product:
Data Platform and Tools Graveyard
Component:
Telemetry Analysis Service (ATMO)
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: bugzilla, Assigned: robotblake)

Details

Looking at the EMR cluster list, it looks like starting 7/19 ~23:00 UTC all ATMO scheduled jobs were launched twice. Seems like we're running two copies of the scheduler somewhere..
Found the two most recent RunJobFlow events on cloudtrail: event Ids c9eb4826-662c-497d-98ff-f8311e50c446 and eee4e8d8-8e7f-11e8-af2e-6f53817c4aed Looks like they're coming from 1. two different AWS IPs, 2. the "atmo" user and 3. both are tagged with "environment: Prod" I think that's about the extent of what I can do to investigate this.
Assignee: nobody → bimsland

:sunahsuh Do we know if this is still happening?

Flags: needinfo?(ssuh)

I haven't noticed it lately (iirc you tried restarting some of the services? and that seemed to do it) -- we can close the bug

Status: NEW → RESOLVED
Closed: 6 years ago
Flags: needinfo?(ssuh)
Resolution: --- → WORKSFORME
Product: Data Platform and Tools → Data Platform and Tools Graveyard
You need to log in before you can comment on or make changes to this bug.