missing home directory on mpt jumphost

VERIFIED FIXED

Status

Infrastructure & Operations
Infrastructure: Other
VERIFIED FIXED
5 years ago
5 years ago

People

(Reporter: Robert Kaiser, Assigned: jabba)

Tracking

Details

(Reporter)

Description

5 years ago
Somewhere in the last days, suddenly the mpt jumphost changed and is missing a home directory for me.

In my worfkflow of getting stuff up for my custom reports on crashes, I rely on checking in some adjustments to me own repo and then run a nice nested ssh command like this:

> hg push && ssh-add && ssh vpn "ssh crashanalysis.dmz.phx1.mozilla.com 'cd /mnt/crashanalysis/rkaiser/crash-report-tools/ && hg pull -u && chmod -R a+r /mnt/crashanalysis/rkaiser/crash-report-tools/{overview*,dashboard}'"
[...hg spew]
Identity added: [...]
Could not chdir to home directory /home/rkaiser: No such file or directory
Could not create directory '/home/rkaiser/.ssh'.
Host key verification failed.

Also, when I logged into the mpt jumphost, I usually had a bash history available (so I don't need to remember the host names I tend to access) and didn't always need to confirm the ssh fingerprints of hosts I access (as long as this is always asked and then deleted, it's no security feature but only annoying anyhow for everyone except the three people in the world who actually remember and check those fingerprints manually).

Updated

5 years ago
Assignee: server-ops-infra → pradcliffe
:jabba found archived homedirs in /home/archive, from 3am May 10th:
-rw-r--r--    1 root root      2861 May 10 03:09 rkaiser.tar.gz
and is restoring those homedirs.

Theory is the homedir cleanup script running when the host could not connect to ldap since this is a legacy host which is managed directly from ldap and not via puppet.

My missing home directory is not in that set, however, so there may be other missing directories which would need to be manually recreated.

I've suggested the homedir cleanup script be tweaked to not let this happen again before the legacy hosts are retired.
Assignee: pradcliffe → jdow
(Assignee)

Comment 2

5 years ago
I've brought up the issue with :kang and we'll discuss if some better intelligence can be built into the script. This host is slated to be replaced soon with something with more stable user management vs. using LDAP directly, so it's mostly a temporary problem if it does re-occur.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
(Reporter)

Comment 3

5 years ago
verified that I get my home directory again. Thanks a lot!
Status: RESOLVED → VERIFIED
Component: Server Operations: Infrastructure → Infrastructure: Other
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.