586580 - Implement DNSSEC for mozilla.org

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Shyam Mani [:fox2mike]]

Q3 goal for IT, I'm running point on this and so filing a bug for tracking/information purposes.

Comment 1

[Shyam Mani [:fox2mike]]

Test setup is up and running on dm-dnssec01. 

[root@dm-dnssec01 config]# dig @localhost mozilla.org

; <<>> DiG 9.7.0-RedHat-9.7.0-1.rhel5 <<>> @localhost mozilla.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12455
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 3

;; QUESTION SECTION:
;mozilla.org.                   IN      A

;; ANSWER SECTION:
mozilla.org.            3600    IN      A       63.245.209.11

;; AUTHORITY SECTION:
mozilla.org.            3600    IN      NS      pm-ns02.mozilla.org.
mozilla.org.            3600    IN      NS      pm-ns01.mozilla.org.
mozilla.org.            3600    IN      NS      ns1.mozilla.org.

;; ADDITIONAL SECTION:
ns1.mozilla.org.        3600    IN      A       10.2.74.125
pm-ns01.mozilla.org.    3600    IN      A       10.2.74.125
pm-ns02.mozilla.org.    3600    IN      A       10.2.74.127

;; Query time: 2 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Aug 31 02:02:21 2010
;; MSG SIZE  rcvd: 155

[root@dm-dnssec01 config]# dig @localhost +dnssec mozilla.org

; <<>> DiG 9.7.0-RedHat-9.7.0-1.rhel5 <<>> @localhost +dnssec mozilla.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33324
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 4, ADDITIONAL: 7

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;mozilla.org.                   IN      A

;; ANSWER SECTION:
mozilla.org.            3600    IN      A       63.245.209.11
mozilla.org.            3600    IN      RRSIG   A 5 2 3600 20100930052242 20100831052242 61896 mozilla.org. XAGiQ+K44UhTeeXHFtQUIh1FKi2oU0Bgzrz/Yyn2GDJUpY8f3WmLZgnz nbaNfId//FXoMiCL9GvYkaHvcyIBzuNqDxaKNAynV2w4hXcy6gLYHiGg 1c9duJOuTZNtY7S1XLKmC5d7gKHNTYdIdVYk39ihhoMnzmMxsDujK9Ih vms=

;; AUTHORITY SECTION:
mozilla.org.            3600    IN      NS      ns1.mozilla.org.
mozilla.org.            3600    IN      NS      pm-ns02.mozilla.org.
mozilla.org.            3600    IN      NS      pm-ns01.mozilla.org.
mozilla.org.            3600    IN      RRSIG   NS 5 2 3600 20100930052242 20100831052242 61896 mozilla.org. vxjIx4cX+tCQj6asjhTL+s/xHBiiFdlTMns0nMrx2XG16LmnbwomkDfW rbiUwGfZaNDEkih2pWs/LCBRg+sYpJs7IytDVguhyMiW7COro85sfKL3 wz++LGtdHG9g2QzOocsE4kcMeSVnpUuKNEpU6l8MxHALANiA6mJEb6ok Bo8=

;; ADDITIONAL SECTION:
ns1.mozilla.org.        3600    IN      A       10.2.74.125
pm-ns01.mozilla.org.    3600    IN      A       10.2.74.125
pm-ns02.mozilla.org.    3600    IN      A       10.2.74.127
ns1.mozilla.org.        3600    IN      RRSIG   A 5 3 3600 20100930052242 20100831052242 61896 mozilla.org. gBT/oOoqCqz0hADA/zPwvXBJ1akJrzPTL6SKgprHBB+Upr9ZEQ0ghnsH mAF3k6/Sy5IKM1GxiGhvJafSFNXpqJ0OPuBFTZQewl1tKP+8DLaOVjZP NueTO1LmDbi8oPVqwnZxaEJfB7ZVD1wWGzbR5uQIBnscBuhf8cSxwvyb KhI=
pm-ns01.mozilla.org.    3600    IN      RRSIG   A 5 3 3600 20100930052242 20100831052242 61896 mozilla.org. aXLsmZ8VtBfW2BJHqdsd5v9A54iZTj1ZAyVKVAUkmFnV1UKtQ7vDF8l3 OJsXLoZpWRHrX5uLAVhoedEiPJ0SM3dakwlRFjqZ9RyIMhKXLH2owlLO s4C6t0cZPUywuDiV2+KO0UqbpStyV90MVmKHTUIxFwbIuDSIf+Od7ac/ fWg=
pm-ns02.mozilla.org.    3600    IN      RRSIG   A 5 3 3600 20100930052242 20100831052242 61896 mozilla.org. YGCJT/4Y4EOAlbbLoyFFODqeO63sg47RutAi0X1ZFXSLB55TW/GT4WYz FX0OxxV45B8QiTH4OtbHCKOkT52SwdjEwfjZeMwfj6EU9q1DuDHtWuHp ELDcJBxWfStJpq+4ouctJUTR8dpdlOerfinloxr53+1AKKB60aun0wz0 qmg=

;; Query time: 3 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Aug 31 02:02:27 2010
;; MSG SIZE  rcvd: 1021

Comment 2

[Shyam Mani [:fox2mike]]

Ready to go live, we'll be pushing this out coming Thursday.

Comment 3

[Shyam Mani [:fox2mike]]

Slave setup tested on dm-dnssec02. We now have our DS records up at the .org root.


trinity:~ shyam$ dig @4.2.2.2 +dnssec DS mozilla.org

; <<>> DiG 9.6.0-APPLE-P2 <<>> @4.2.2.2 +dnssec DS mozilla.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10863
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;mozilla.org.                   IN      DS

;; ANSWER SECTION:
mozilla.org.            43176   IN      DS      51618 7 1 B996ED047EF73B9CF9342491761EDCB63622A150
mozilla.org.            43176   IN      RRSIG   DS 7 2 86400 20100929214601 20100915204601 37812 org. Fym3vPXYfbivNYMTYUG1uUaHLe5nYXrcRSxbls2WwLJE4psuR8sKMVSV qmEsAMoybeO1b9uQvDgE78FL6qFYA6RE0RlehXS2tdQKqRfQoD2YJCoy TlDv+J8EcFp/VRtfyVYl+cL7XvLDVJg2S2bQUrnt5PN4aalxW3LqgxxI w0s=

;; Query time: 208 msec
;; SERVER: 4.2.2.2#53(4.2.2.2)
;; WHEN: Thu Sep 16 12:05:34 2010
;; MSG SIZE  rcvd: 239

Comment 4

[Shyam Mani [:fox2mike]]

Pre-empted this because I slightly failed and made sure DS records were out before the signed zones were out. Should have been done in reverse. This meant that anyone trying to reach mozilla.org with the EDNS bit set (for DNSSEC) or with validating resolvers were getting errors resolving mozilla.org for a few hours.

While our general users would have not had any issues, we still decided to go ahead and push it live as people were complaining on twitter and the IT blog.

This is now complete. mozilla.org is DNSSEC compliant, stuff is signed and can be verified from the root downwards. w00t.

Calling this fixed, reopen in case of user facing issues, if any.

http://dnsviz.net/d/mozilla.org/dnssec/
http://dnsviz.net/d/hg.mozilla.org/dnssec/

Comment 5

[Steve Snyder]

Looks like this became unfixed.  I'm getting a *lot* of these logged in my BIND v9.8.2 server:

22-Oct-2013 15:00:35.025   validating @0x7f272000ee60: mozilla.org DNSKEY: no DS record
22-Oct-2013 15:00:35.025 validating @0x7f2720ac0fb0: VERsIonCHecK-BG.aDdonS.MOZILLA.oRg A: no valid signature found
22-Oct-2013 15:00:35.049   validating @0x7f272000ee60: mozilla.org DNSKEY: no DS record
22-Oct-2013 15:00:35.049 validating @0x7f272011abf0: versioncheck.aDdonS.MOZILLA.oRg A: no valid signature found
22-Oct-2013 15:02:29.507 validating @0x7f272853e680: adDoNS.MoziLla.ORG A: no valid signature found

Comment 6

[Dave Miller [:justdave]]

This bug is too old to reopen, it was fixed when it was resolved, and the unfixed is on purpose because of a provider migration which required changing keys and we needed the keys to not conflict, so it was disabled for the duration of the migration process. See bug 924663 and bug 929516

Comment 7

[Steve Snyder]

(In reply to Dave Miller [:justdave] ([e-mail]) from comment #6)
> This bug is too old to reopen, it was fixed when it was resolved, and the
> unfixed is on purpose because of a provider migration which required
> changing keys and we needed the keys to not conflict, so it was disabled for
> the duration of the migration process. See bug 924663 and bug 929516

Given that I am apparently unauthorized to view either of the bugs you referenced, can you guesstimate when mozilla.org will again have valid keys?

Thanks.

Comment 8

[Dave Miller [:justdave]]

Provided the change advisory board approves at their meeting tomorrow, it'll be turned back on this coming weekend.  Sorry for linking closed bugs, apparently they have confidential info about the providers in them.