1434170 - please setup new Buildduty team (6 ppl) with jumphost access and MFA

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Jordan Lund (:jlund)]

Hi,

We have a new team for Buildduty who now have LDAP access. Like Alin and Andrei, they need access to our hosts internally that Buildbot and Taskcluster use.


Names and LDAP email:

  - Cosmin Bontea <[e-mail]>
  - Bogdan Crisan <[e-mail]
  - Danut Labici <[e-mail]>
  - Radu Iman <[e-mail]>
  - Roland Mutter <[e-mail]>
  - Zsolt Fay - <[e-mail]>


They will need an MFA account invite for Duo Mobile. As well as some instructions on how to connect to the machines once they have VPN setup.

Comment 1

[:dhouse]

I added the six users to the Duo account.
```
Cosmin Bontea,[e-mail]
Bogdan Crisan,[e-mail]
Danut Labici,[e-mail]
Radu Iman,[e-mail]
Roland Mutter,[e-mail]
Zsolt Fay,[e-mail]
```

They will receive an email with a link for enrolling their MFA device and a link to the jumphost doc:
https://mana.mozilla.org/wiki/display/IT/Releng+JumpHosts

Comment 2

[:dhouse]

Attachment: add new buildduty users to jumphost users list

Comment 3

[:dhouse]

Comment on attachment 8946864 [details] [diff] [review]
add new buildduty users to jumphost users list

Applied and pushed to default.
https://hg.mozilla.org/build/puppet/rev/8947722bdea32b88ad18d64197e2cc461b445dac
Travis passed. Pushed to production:
remote:   https://hg.mozilla.org/build/puppet/rev/5496da42dd6bcab28a64bf7ebbf4d1842f62c6fe

Comment 4

[:dhouse]

Dragos, could you apply this patch on the jumphosts and check if the new softvision buildduty folks have access?

Comment 5

[Dragos Crisan [:dragrom]]

Fail to create user rmutter
 sudo puppet agent -t --noop
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed to realize virtual resources Users::Person[rmutter] on node rejh2.srv.releng.mdc1.mozilla.com
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run


The user rmutter is sick leave, and didn't make any changes on his LDAP account. To unlock the rest of the users, I'll remove this user on my puppet environment, and then I'll run puppet on jumhosts pinning to my puppet environment

Applied puppet on rejh2.srv.releng.mdc1.mozilla.com host (sudo puppet agent -t --environment=dcrisan --server=releng-puppet2.srv.releng.scl3.mozilla.com). The users were created, except rmutter.

Comment 6

[Dragos Crisan [:dragrom]]

User dlabici fail to login into rejh1 mdc1 jumphost. Here is what I found into the server logs:

Jan 31 02:01:40 rejh1.srv.releng.mdc1.mozilla.com sshd[2243]: Partial publickey for dlabici from 10.22.248.34 port 58136 ssh2: RSA 8d:bd:9c:f6:5b:c7:85:10:cc:24:df:7f:f4:ed:70:04
Jan 31 02:01:40 rejh1.srv.releng.mdc1.mozilla.com sshd[2247]: Aborted Duo login for 'dlabici' from 10.22.248.34: Access Denied. The username you have entered cannot authenticate with Duo Security. Please contact your system administrator.
Jan 31 02:01:40 rejh1.srv.releng.mdc1.mozilla.com sshd[2243]: error: PAM: Authentication failure for dlabici from 10.22.248.34
Jan 31 02:01:40 rejh1.srv.releng.mdc1.mozilla.com sshd[2243]: Failed keyboard-interactive/pam for dlabici from 10.22.248.34 port 58136 ssh2
Jan 31 02:01:40 rejh1.srv.releng.mdc1.mozilla.com sshd[2243]: Connection closed by 10.22.248.34 [preauth]

:dhouse Can you help me please with this?

Comment 7

[Ludovic Hirlimann [:Usul]]

(In reply to Dragos Crisan [:dragrom] from comment #6)

> :dhouse Can you help me please with this?

Duo is self-served, people need to go to login.mozilla.com and enroll in duo. Try that it might fix the issue.

Comment 8

[Bogdan Crisan [:bcrisan] (EEST - GMT + 3)]

We did enroll in duo and we can also can confirm that in other apps/services that have support for duo it's working.

Comment 9

[Danut Labici [:dlabici]]

To add more information:
The new MFA menu (and shows accurate information) is also available inside login.mozilla.com dashboard and the enrollment (both sides) was completed.
We now have a new option inside Duo named Mozilla Relops and the invitation link also confirms that the enrollment was completed.

Comment 10

[Kendall Libby [:fubar] (he/him)]

(In reply to Ludovic Hirlimann [:Usul] from comment #7)
> (In reply to Dragos Crisan [:dragrom] from comment #6)
> 
> > :dhouse Can you help me please with this?
> 
> Duo is self-served, people need to go to login.mozilla.com and enroll in
> duo. Try that it might fix the issue.

IT Duo is self-served, Relops Duo is decidedly not.

Comment 11

[:dhouse]

There were two problems, now confirmed resolved as dlabici was able to login:
1. I had incorrectly set the user real names as their usernames (bulk import sets username for primary auth).
2. I had not added the new users to a group (I added them to "releng").

We'll keep this bug open until we've confirmed all the users have activated their Duo accounts and accessed the jumphosts.

Comment 12

[:dhouse]

A second user account for "cbontea" was created yesterday by duo because the original enrollment was tied to the previous username.

We'll need to be aware of this for rmutter when he enrolls: I sent a new enrollment email, but if the first one is used the original user will be re-created and we will need to fix it (permanently delete user "rmutter", rename "roland mutter" to "rmutter" and add the "releng" group membership).

Comment 13

[Roland Mutter Michael (:rmutter)]

I've tested the access and it seems fine.