1061579 - GPO images to name existing file ffxbld_dsa as ffxbld_rsa (but content of file should remain the same)

Status: RESOLVED WONTFIX

(Infrastructure & Operations :: RelOps: General, task)

Description

[Pete Moore [:pmoore][:pete]]

This either needs to be a coordinated roll out at the same as the RelEng source code changes in parent bug 1061188, or alternatively; ffxbld_rsa could be provided as a copy of ffxbld_dsa (i.e. both ffxbld_rsa and ffxbld_dsa are included in GPO), and after bug 1061188 lands, ffxbld_dsa can be deleted in a future bug. Perhaps this is the sanest solution?

Explanation: due to a security incident, we needed to generate a new key. Policy states that this should be an rsa key, not a dsa key. We initially created a new key, but to minimise impact, left it named as ffxbld_dsa. We would like to switch the name over to ffxbld_rsa to avoid confusion, since it is an rsa key.

Please advise which of these two potential roll out procedures are preferred.

Comment 1

[Amy Rich [:arr] [:arich]]

We should definitely make a copy first, since that means we won't need a tree closure to switch form one to the other. We can go back later and delete the file named ffxbld_dsa after all the hosts have been updated and the releng code has been modified to use only ffxbld_rsa.

Comment 2

[Mark Cornmesser [:markco] OOO 2024/04/15]

The GPO that does the file copy now includes ffxbld_rsa. Once we are comfortable i will remove ffxbld_dsa.

If, for any reason, we wanted to remove it on a limited number of machines first, I can add file deletion to the existing GPO and limit it through item level targeting.

Comment 3

[Pete Moore [:pmoore][:pete]]

Many thanks Mark!

We'll have to update our tools to use ffxbld_rsa, and I can coordinate back in this bug when we are ready to get rid of ffxbld_dsa, when everything has been switched over.

Comment 4

[Pete Moore [:pmoore][:pete]]

(In reply to Mark Cornmesser [:markco] from comment #2)
> The GPO that does the file copy now includes ffxbld_rsa. Once we are
> comfortable i will remove ffxbld_dsa.
> 
> If, for any reason, we wanted to remove it on a limited number of machines
> first, I can add file deletion to the existing GPO and limit it through item
> level targeting.

Do we need to reimage all of our machines to pick up this change?

For example:
https://bugzilla.mozilla.org/show_bug.cgi?id=1082535#c62

Or what is the best way to make sure this is rolled out to all machines?

Thanks,
Pete

Comment 5

[Amy Rich [:arr] [:arich]]

GPO runs at boot time, so no reimaging required.

Comment 6

[Mark Cornmesser [:markco] OOO 2024/04/15]

Pmoore: I will take a look into what is happening with https://bugzilla.mozilla.org/show_bug.cgi?id=1082535#c62 tomorrow or Friday this week.

Comment 7

[Pete Moore [:pmoore][:pete]]

(In reply to Mark Cornmesser [:markco] from comment #6)
> Pmoore: I will take a look into what is happening with
> https://bugzilla.mozilla.org/show_bug.cgi?id=1082535#c62 tomorrow or Friday
> this week.

No worries Mark - Coop and Amy both pointed out that the GPO imaging occurs after a reboot, and no doubt this slave had not rebooted.

I'm going to upload the key to the windows slaves to catch the ones that haven't rebooted, and try again.

Thanks!
Pete

Comment 8

[Pete Moore [:pmoore][:pete]]

Whoops, I closed this rather prematurely.

I think we are now ready to drop the dsa key from GPO, and clean up the .ssh dir of the windows machines to not include the rsa key.

Once this is done, we can close the bug.

Comment 9

[Pete Moore [:pmoore][:pete]]

(In reply to Pete Moore [:pete][:pmoore] from comment #8)

> I think we are now ready to drop the dsa key from GPO, and clean up the .ssh
> dir of the windows machines to not include the rsa key.


I mean, "not include the ffxbld_dsa key" (not the rsa key).

Comment 10

[Pete Moore [:pmoore][:pete]]

Hi Mark,

Can you delete the ffxbld_dsa key now from the GPO images, and also from the slaves (if needed)?

Nothing should be relying on that key any more.

Many thanks!
Pete

Comment 11

[Mark Cornmesser [:markco] OOO 2024/04/15]

I have a GPO ready to roll for this. Do we want to roll it out to a smaller test pool first?

Comment 12

[Pete Moore [:pmoore][:pete]]

Hey Mark,

I've just found out that some new dependencies have recently been added on the old file (see https://bugzilla.mozilla.org/show_bug.cgi?id=1061188#c36), so we should probably hold off for now until I've fixed that up. When I've landed a fix for that and it has propagated, we should be good to go.

Then indeed we could roll out to a smaller test pool initially, if that is not too much work for you.

Thanks Mark!
Pete

Comment 13

[Mark Cornmesser [:markco] OOO 2024/04/15]

Sounds good.

Comment 14

[Mark Cornmesser [:markco] OOO 2024/04/15]

Just checking in. Any update on the dependencies ?

Comment 15

[Pete Moore [:pmoore][:pete]]

Dependencies landed I believe - can we try on a test pool?

Comment 16

[Pete Moore [:pmoore][:pete]]

Hey Mark,

Now we're setting up puppet for Windows, is this no longer needed? Should I mark as WONTFIX?

Thanks,
Pete

Comment 17

[Mark Cornmesser [:markco] OOO 2024/04/15]

WONTFIX is probably a good choice. This has been at the bottom of my list to address because of Puppet.