1555100 - Change firewall policy to allow "phoning-home" from macs in srv.releng

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: NetOps: DC ACL Request, task)

Description

[:dhouse]

Please allow the mac minis in srv.releng.[mdc1,mdc2].mozilla.com to connect to the apple servers. This is similar to the same allow for test.releng in bug 1520951

We need this for reaching the apply internet recovery for turning off SIP in MacOS Mojave

Comment 1

[Nadia]

Hi Dave,
I looked at the old bug you referenced and it looks like that issue was fixed by disableling the deny rule for all of releng in mdc1/2, it looks like that should take care of all of releng, can you verify that this is not working, and if its not can I get an IP of a system that is not working so I can see the attempt in the firewall.

Regards,
Nadia

Comment 2

[:dhouse]

(In reply to Nadia from comment #1)

Hi Dave,
I looked at the old bug you referenced and it looks like that issue was fixed by disableling the deny rule for all of releng in mdc1/2, it looks like that should take care of all of releng, can you verify that this is not working, and if its not can I get an IP of a system that is not working so I can see the attempt in the firewall.

Regards,
Nadia

Hi Nadia,

Thank you.

Here are the machines I get a failed connection to apple on:
mac-v3-signing1.srv.releng.mdc2.mozilla.com has address 10.51.48.234
mac-v3-signing2.srv.releng.mdc2.mozilla.com has address 10.51.48.235
mac-v3-signing3.srv.releng.mdc2.mozilla.com has address 10.51.48.236
mac-v3-signing4.srv.releng.mdc2.mozilla.com has address 10.51.48.237
mac-v3-signing5.srv.releng.mdc2.mozilla.com has address 10.51.48.238

(Jake/Dragos, Nadia may need some testing for this; the failure is QTS cannot reach apple's internet recovery on these 5 in srv.releng)

Comment 3

[Jake Watkins [:dividehex]]

Hi Nadia,

My working hypothesis here is that NTP traffic from the releng networks to apples NTP servers is being dropped. This is causing the macos internet recovery to fail to download via https because ssl is failing from the out-of-sync host clock.

I see this in panorama where ntp is being drop and the ssl traffic is logging as incomplete. Apples public internet block is 17.0.0.0/8. So I'd like to request a rule to explicitly allow NTP traffic to apples public block from ALL releng networks.

10.49.0.0/16 -> 17.0.0.0/8 MDC1 releng
10.51.0.0/16 -> 17.0.0.0/8 MDC2 releng

Comment 4

[Jake Watkins [:dividehex]]

Attachment: apple_ntp_blocked.png

Panorama screenshot attached as an example of ntp being dropped by the interzone-default rule.

Comment 5

[Nadia]

Attachment: Screen Shot 2019-06-03 at 11.32.44 AM.png

I went ahead and created an apple allow all rule for all of releng in MDC1/2 (see screenshot) Please test and let me know if this is working for you now.

Comment 6

[Nadia]

Hi can you confirm if this is working for you or not.

regards,
Nadia

Comment 7

[Jake Watkins [:dividehex]]

(In reply to Nadia from comment #6)

Hi can you confirm if this is working for you or not.

regards,
Nadia

I won't be able to confirm this until I visit MDC1 so let's call this fixed for now. I'll re-open if it isn't. Thanks for creating the allow rule!