Closed Bug 1602932 Opened 5 years ago Closed 5 years ago

Services still reporting to the old Papertrail account

Categories

(Taskcluster :: General, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: armenzg, Unassigned)

Details

Attachments

(1 file)

I deleted a bunch of the systems I knew about.

https://papertrailapp.com/systems/stateless_dns-1

I'd like to leave this one in place if it's OK with you. It's a service that's very difficult to redeploy but that we can't kill very soon, unfortunately. And it doesn't run in Heroku, so we can't use a Heroku add-on for it.

I also see taskcluster-tools -- I'm not sure how to turn that off, but we probably should. Hassan, can you check if there's a setting for that in Heroku? I don't have access anymore.

Flags: needinfo?(helfi92)

It's fine to leave what cannot be easily moved. I appreciate the house cleaning!

There doesn't seem to be a papertrail addon in the taskcluster-tools heroku app. Weird...

Flags: needinfo?(helfi92)

It's a log drain, which isn't visible in the UI.

dustin@lamport ~ $ heroku drains -a taskcluster-tools
=== Drains
syslog+tls://logs6.papertrailapp.com:26354 (d.975ea0b0-9cab-4855-94bd-329e3d9e8ea4)
dustin@lamport ~ $ heroku drains:remove -a taskcluster-tools d.975ea0b0-9cab-4855-94bd-329e3d9e8ea4
Successfully removed drain syslog+tls://logs6.papertrailapp.com:26354

Thanks for the clean up!

What do you know about i-0197706ac7ca23e2e..usw2.mozilla.com?
It is in the group called group-awaiting-deletion?

I peeked at that -- it seems to be a Windows system. I have no idea :)

(In reply to Dustin J. Mitchell [:dustin] (he/him) from comment #6)

I peeked at that -- it seems to be a Windows system. I have no idea :)

Possibly a generic-worker test instance? cc-ing pmoore just in case.

Armen: are there still TC services reporting here?

Flags: needinfo?(armenzg)
Attached image Non-Treeherder apps

There's still these. Not sure if some belong to releng.

Flags: needinfo?(armenzg)
Attachment #9142438 - Flags: data-review?(coop)

The ones that look like IPs don't have anything useful.

The last two have logs up to today.

I think relops shut down the tc-coalesce service /cc fubar

stateless-dns is still running, so we should figure out how to log that elsewhere (or not at all..)

Flags: needinfo?(klibby)

Any objections if I remove the IP-like services?

34.213.136.122 and 52.49.228.53 are also stateless DNS hosts running in different AWS regions, so let's not remove them quite yet until we figure out what to do with them per comment #11.

For the other two IPs, can you find out when they last reported, and maybe even discern what they're logging about from the output?

Flags: needinfo?(armenzg)
Comment on attachment 9142438 [details] Non-Treeherder apps Analysis is in comment #13.
Attachment #9142438 - Flags: data-review?(coop) → data-review+

(In reply to Chris Cooper [:coop] pronoun: he from comment #13)

For the other two IPs, can you find out when they last reported, and maybe even discern what they're logging about from the output?

I don't have admin access in this papertrail any more, but looking through the purge events menu, I can tell those two other IPs (185.202.1.161 and 5.188.86.98) are part of the Treeherder custom group.

(In reply to Armen [:armenzg] from comment #12)

Any objections if I remove the IP-like services?

OK, I've circled completely around back to "Yes, just remove all the IP-based services." They're not logging anything anyway. Sorry for the churn.

Flags: needinfo?(armenzg)

I've changed the Treeherder group not to accept messages from new senders (that will exclude some of those static IP systems; even though they were not emitting).

I can't remove those static IP systems.

fubar or coop: Could you please remove them? I see you both have full privileges
https://papertrailapp.com/systems/5442785752/edit
https://papertrailapp.com/systems/5442793302/edit
https://papertrailapp.com/systems/5184777802/edit
https://papertrailapp.com/systems/5187210812/edit

Flags: needinfo?(coop)

(In reply to Armen [:armenzg] from comment #17)

fubar or coop: Could you please remove them? I see you both have full privileges
https://papertrailapp.com/systems/5442785752/edit
https://papertrailapp.com/systems/5442793302/edit
https://papertrailapp.com/systems/5184777802/edit
https://papertrailapp.com/systems/5187210812/edit

Oh look , I do have admin privs. Their interface is just terrible. :/

I've removed those systems after verifying that the individual stateless DNS hosts are being aggregated under the single, remaining stateless_dns system. Just need :fubar to verify the status of the coalesce service.

Flags: needinfo?(coop)

Coalescing service decomm'ed in bug 1621404.

Flags: needinfo?(klibby)

Great!

fubar, coop: could you please remove it? https://papertrailapp.com/systems/releng-tc-coalesce/edit

That would only leave us with the Treeherder app and the stateless-dns services.

(In reply to Armen [:armenzg] from comment #20)

fubar, coop: could you please remove it? https://papertrailapp.com/systems/releng-tc-coalesce/edit

Done.

Thank you coop!
From my perspective this can be closed unless you want to track doing something with stateless-dns.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: