Closed Bug 1515788 Opened 5 years ago Closed 5 years ago

DigiCert: Underscores - CVS Pharmacy

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: jeremy.rowley, Assigned: brenda.bernal)

References

Details

(Whiteboard: [ca-compliance] [ov-misissuance]) 

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0

Steps to reproduce:

This is the first of the companies where I have information referenced by https://bugzilla.mozilla.org/show_bug.cgi?id=1515564 . We didn't receive permission to identify them by name, but we can say they are "Major Pharmacy Benefits Manager".

The follow up requested information is that they have a blackout period from October 15-February 1. These blackout periods represent annual enrollment in benefits and when companies subscriber to new or modified plans. Unsurprisingly, this is their busiest time of the year. 

The certificates are:
67445bd22255feaedeeb9791d88d60f8	https://crt.sh/?id=15806036
52e74dc14d0c6c76f68e62e575964aba	https://crt.sh/?id=15806037
06095344b72bbabd6cd10f5dbdc688e9	https://crt.sh/?id=15944127
7c0c3ddb652391a7268202bfd4a91e38	https://crt.sh/?id=16158275
192c2961a3f7b0450aaa8c6eaf125302	https://crt.sh/?id=18018590
2aa5f189cbe698a69c5e32504a8a6680	https://crt.sh/?id=18019190
109248039b4f476c7115d86531fe3e57	https://crt.sh/?id=18019888
0456341a964334f750f4e792a3fb8f77	https://crt.sh/?id=18021060
461bfe9cd2fac2cea23ab8fd9d096cd0	https://crt.sh/?id=18020962
01834d35d3d7a5a406ead082774d6e63	https://crt.sh/?id=18022527
505be82a14f8d400157872139643b2b0	https://crt.sh/?id=18021658
11ea6f4edfafc8c6248f8f487e109a5b	https://crt.sh/?id=18022848
65f2eeedfc9c7d0854c83e349152abfa	https://crt.sh/?id=18023281
10f082fe9bcf93ea0c0a4fb223ae1294	https://crt.sh/?id=18023904
3f9c4fdc0060720d9ee2042007f94c4a	https://crt.sh/?id=18024039
208e7aff0247c2887a6d4ef7d1b400b6	https://crt.sh/?id=18024708
35faff98f4864aeb256b23012bdfb770	https://crt.sh/?id=18024575
359e30c3cd5387dd9b7f90b51a8b9346	https://crt.sh/?id=18024892
59e9ee5086b2f07f3d1b3405b5a75c00	https://crt.sh/?id=18025126
18ab5f52f789b12b765fc9d9aa18e672	https://crt.sh/?id=18025117
56dcb64ea65b24d8c4f013fe4a98aa41	https://crt.sh/?id=18025340
528dd479144840d26c5c75a9f97881fc	https://crt.sh/?id=18025170
4d3ea144e9924dc06828ab23f11fef19	https://crt.sh/?id=18025443
078c09fe1b413359d94b2ab4d90edeb5	https://crt.sh/?id=18025544
789345552b410cc3db6ae77b1b1d9995	https://crt.sh/?id=18025397
34f43bbf33279aa5da82d5445f8c4c5f	https://crt.sh/?id=18025637
354378848975c82a4763983d6db673a7	https://crt.sh/?id=18025585
5ca6b9b0889c2e08a1a9dea5e9ae4a62	https://crt.sh/?id=18025807
38950e698899f45eaa2ff57f92b6edd9	https://crt.sh/?id=18025911
023cb77924e1c2cead3d3cf5d25aeed6	https://crt.sh/?id=18026104
c9528e411e373d48d373ac5b5434a7	https://crt.sh/?id=18025850
3eed5acf9c5103b57ef4d05bfc251a94	https://crt.sh/?id=18026034
3705e4e7ff18251a267aa32743e8ec24	https://crt.sh/?id=18026229
4e6d929297f80203c02bdace9a9b60eb	https://crt.sh/?id=18026355
5db909c5ff1c8193a1436839c427aa8c	https://crt.sh/?id=18026404
15370808309e7c45220c6d090d7265cb	https://crt.sh/?id=18026511
05a78f6a1d2ad92039cf3421f7021ba1	https://crt.sh/?id=18026588
415332ddb7633cfbd759df558773cf40	https://crt.sh/?id=18026471
48eb0d28b952ef01e01afa03b68bd9cc	https://crt.sh/?id=18026649
6b77bf5e07357ad2622ea4db7f7a6839	https://crt.sh/?id=18026700
3e27dd272f9c3002ba8ffeee6f0419ae	https://crt.sh/?id=18026820
62f829f2482d7eb302257f015c128427	https://crt.sh/?id=18027045
24d6476449e58913d99d409e71ffd3bd	https://crt.sh/?id=18027019
5f7792309472a8c0ca9afa954ec8c979	https://crt.sh/?id=18027037
7f3e176b7469e58039fe62b983a7e289	https://crt.sh/?id=18026987
451ebbb3e3f50c9dbb44ec4c0e43a457	https://crt.sh/?id=18027105
758db81306e480a7b941bff69154a024	https://crt.sh/?id=18027060
650579d4600b3d412e4a7ff4d9f9ebaf	https://crt.sh/?id=18027084
7f46303e73845014e03d8b0a32db1f15	https://crt.sh/?id=18027257
0cd76f731107709276eed07b39211aea	https://crt.sh/?id=18027380
2f8198481cdcd0858901a0887dc3a54a	https://crt.sh/?id=18027440
424e2db9852f42e3aa763c242d347e29	https://crt.sh/?id=18027493
608d8024cfa04819d536786f818f74c8	https://crt.sh/?id=18027449
47a0be5dded051a86cf435fcca27e181	https://crt.sh/?id=18027383
1e0ca540a08598f180e21143d8db790f	https://crt.sh/?id=18027471
68b47788707a26d77ab8628da448d453	https://crt.sh/?id=18027390
37611be416d736dcda41800d5e67558e	https://crt.sh/?id=18027569
4fe3ceaa2c9dc8632e4f6427d790910b	https://crt.sh/?id=18028274
7b800fb40f6ac9bb4c172e18b459d118	https://crt.sh/?id=18028275
47bb9032e90d25ee2afcfda16fea3bd4	https://crt.sh/?id=18028428
5c21ea9a8c08974d555655deb50b3e31	https://crt.sh/?id=18028538
4e5a7489a9d0a015994c14cfff5e645c	https://crt.sh/?id=18028437
1255a82ad9bbfda11b2cee1147d61e6a	https://crt.sh/?id=18028451
49b36013bd6c3d08577ae9814ac3beee	https://crt.sh/?id=18028458
13dd5d79c44cdb00b93df43dc65f86fc	https://crt.sh/?id=18028652
6a094d13a2e3440d64d6dfac638d57ac	https://crt.sh/?id=18028655
71c7885057e40f129811fb1fcefd2097	https://crt.sh/?id=18028662
74a7b4177ba242fd7338e9a4a1df5890	https://crt.sh/?id=18028665
5e25ed875293905babc83843802553af	https://crt.sh/?id=18028756
2a685431a4699784af78b363995212b5	https://crt.sh/?id=18028755
69aaa9cc2524d34bb52210005ae3e8a4	https://crt.sh/?id=18028771
2b7febfc0502efdc110559d5ef554ffc	https://crt.sh/?id=18028773
19bf71d78a2f97e68ae6191588868f9e	https://crt.sh/?id=18028796
7a71ceb77643f7e75ac89397be002b7b	https://crt.sh/?id=18028871
2136a13c3a31c48c72764ae22bf6dea4	https://crt.sh/?id=18028881
2a57660063f8110ae0dac5a7b78928f9	https://crt.sh/?id=18028993
5962a42b5b9770ae8b7625629075695b	https://crt.sh/?id=18029126
37c3592d9423087e87bb5c0a29c2c7b3	https://crt.sh/?id=18029218
532ee437dc15f2ac37c92504d9989098	https://crt.sh/?id=18029402
08ee00735878902bd4f77c21f727bd35	https://crt.sh/?id=18029515
06d22c048d6ea043c9b660fd73b3c7b5	https://crt.sh/?id=18030396
327a077bfeb9e215cc3acf42e4e291d4	https://crt.sh/?id=18987751
6f7580f1f3e2e58480aea57f42152eec	https://crt.sh/?id=19618950
3c1e25aeabd5ae68b929af5196beb923	https://crt.sh/?id=19618982
0f42e137fb0c4da96e7dc6c4d7150ad8	https://crt.sh/?id=19618981
696399909917b2b75672dec65c184f70	https://crt.sh/?id=19619016
39c29e22e167afa0417474f68298f52f	https://crt.sh/?id=20294817
06d01e058f5fd2d56fdc43ed320bdff1	https://crt.sh/?id=20585902
057f5449753f12145517f376036f4636	https://crt.sh/?id=21263756
760cb1857651511e948ba74187d26604	https://crt.sh/?id=21456231
44f55d976e480c88ed0cfd0e04f6dcea	https://crt.sh/?id=21456233
737a8c6c7b63a1c9554c5d6e2778fc29	https://crt.sh/?id=21456236
129daeb5cf820ac220395f3c52fd52f2	https://crt.sh/?id=21566758
4e914db5203f5071a4da72e5fa59b094	https://crt.sh/?id=21566772
6f892ae9617a5c79844025f1a8eb532d	https://crt.sh/?id=21566773
4d10228dd42b19aa8a11e9a9d61a5fb7	https://crt.sh/?id=21566777
3757fb14ddad6d25525c87c40d76015f	https://crt.sh/?id=21566778
2a29878d3e2ca68935da158c621cf5c0	https://crt.sh/?id=21566788
44f19a5475f3d549258ed47dba8142f2	https://crt.sh/?id=21566789
35d149f4861b478c321da4b3693e2e43	https://crt.sh/?id=21566807
715db6947d30c212df34513c148c4c4f	https://crt.sh/?id=21660987
4399147ee0ca3b135bf0f94b4ace3659	https://crt.sh/?id=23482299
3db2026ebe667acaa6b9194f966ec102	https://crt.sh/?id=23482276
1a3f0e2a0e3b5d234f119c9bfc1cfc50	https://crt.sh/?id=23482300
2dcc82fb72aa61186b91ba0825889c91	https://crt.sh/?id=23482291
336057c5dba7013dbf1672070d273a6a	https://crt.sh/?id=23482308
65338f499b41b56d3ecc6073c7253e3f	https://crt.sh/?id=23482273
208ae33d17719d2d1b5134eabe5ddc97	https://crt.sh/?id=23482340
18fb150aaa0753ff56224fcf73bfcb28	https://crt.sh/?id=23482332
07a8a915a6ae2606d1ece60c54a5f7f3	https://crt.sh/?id=24036891
1fbf0a7f4741b8f37207ba4437cb8c80	https://crt.sh/?id=24188280
50a067bf8aef8c3e36cc58c63286a0d5	https://crt.sh/?id=24982549
654844b4c0b4547c706308714b3ca381	https://crt.sh/?id=25074249
2f701715ba054bdeb91e6b80c46fb314	https://crt.sh/?id=31225124
316e8795cacb37c15e4c8e24dffe107c	https://crt.sh/?id=31225220
5ade3b6d766679e47806bcff00a07f45	https://crt.sh/?id=31306095
6f1b99ece5ef6dde2e364a906818adfd	https://crt.sh/?id=31306235
5320aea023233e0a1a4d7bd8157a1a88	https://crt.sh/?id=31306396
709313236e7cb5e42497021506cd9b65	https://crt.sh/?id=31306498
6dead098368b47908e121860d56ffacd	https://crt.sh/?id=31306511
7eec5c22379bdfb7b886f81ce6d56a07	https://crt.sh/?id=31306808
2508d668c975efd994a3c55569a75e37	https://crt.sh/?id=31307248
5b963a5dbeab8f1cb4e828a746a0c497	https://crt.sh/?id=31307367
541574545a4bfc46355e65b91d4235a7	https://crt.sh/?id=31307504
5dbedeec13f61a7f121a0ced00fa3983	https://crt.sh/?id=31307505
5ad5c7a93ad953378e8d78a3cce30080	https://crt.sh/?id=31307507
313170bda9d8fe2fdda27de2c7093183	https://crt.sh/?id=31307534
7b308b5e383c207366a2d53d11f2c1bf	https://crt.sh/?id=31307539
66db4bc46e927c46da2d8d8ff9fa19e5	https://crt.sh/?id=31307543
3b2a646f852e0d72d9199010bd4c2b23	https://crt.sh/?id=31307544
70272fa465e75a78a7f3621cd50fc654	https://crt.sh/?id=31307545
53b9737f3327c8c8558a445c5a360c91	https://crt.sh/?id=31307546
2c8bbae04bd04c9b9dce83d8edad4c5b	https://crt.sh/?id=31307552
25ac1c53f225b792999922b806efefcd	https://crt.sh/?id=31307553
662bc46e17bb68f8f027b98665968aff	https://crt.sh/?id=31307556
cab5a87c7880107a8266bc917a7d1c	https://crt.sh/?id=31307568
3c4bd3520206898b1563da5daa202d33	https://crt.sh/?id=31307567
17deaa2a59183831fe2e6e39bca30bd4	https://crt.sh/?id=31307587
29916fdac34027768efed01b02739da0	https://crt.sh/?id=31307586
34c9307ea175829b571094931a652cbc	https://crt.sh/?id=31307594
043a410a42c94bc7942bcc8a11cf65d7	https://crt.sh/?id=31307595
41010916c1ea4e5134530ea0d508e530	https://crt.sh/?id=31307608
50f42a1b1a9b5b409d332930f68c01cc	https://crt.sh/?id=31307612
492b85a1447f21f854dba859b5d9e81a	https://crt.sh/?id=31307613
0a7f2d8e480a5e93d8e2e359c5b0ef32	https://crt.sh/?id=31307614
5aa127f343105a3eff39def6db3b4a7d	https://crt.sh/?id=31307624
0eb21717a8a3a3d932b7e238e0246750	https://crt.sh/?id=31307632
25fa9d36a8d2235e59de794e68c1bf1f	https://crt.sh/?id=31307633
4ad9d938e61a8011c4633fab89bff0bf	https://crt.sh/?id=31307648
3ed7cc00cedc36e45760e748fe1f2ed3	https://crt.sh/?id=31307656
57bc987edb831fd35588bdc1ec9322a6	https://crt.sh/?id=31307657
401971c410b47542b46ed0f8d5161d50	https://crt.sh/?id=31307670
7a39efbc0ee3213edb561ff1947fe414	https://crt.sh/?id=31307705
7027617d41ee70f7638025238f501cf7	https://crt.sh/?id=31307759
31a38b009dff5eeaf2ad902307d3de71	https://crt.sh/?id=31307747
5192972cd1bfd7bb6e177a35c6615987	https://crt.sh/?id=31307749
7aabd23e80be9cf2cdbbf977a4129080	https://crt.sh/?id=31307765
4e273056bd077b6a94c0243bfe1604b8	https://crt.sh/?id=31307772
1c149b6faa304f639e54ac150a604484	https://crt.sh/?id=31307786
2b38cccdbbf2bbb3f87250a2be651159	https://crt.sh/?id=31307810
075fb745bf1942bb2566d8d30a233a51	https://crt.sh/?id=35419975
584a74e23c92cbf8c319f86a8ac8a92c	https://crt.sh/?id=35419905
0a043b332f0a197c39250f3dc13c9111	https://crt.sh/?id=35420082
2406f92bb820180d00f18f4d5d557305	https://crt.sh/?id=35419982
0c75895c20fb58d7b111114652300903	https://crt.sh/?id=35420048
3c736286ba00c7ed2f1894e05519d070	https://crt.sh/?id=35420075
5281acb3554268f76dd036d280762bf4	https://crt.sh/?id=35420137
0830cd3fc1d1eceb48fcbe90b7fea6bd	https://crt.sh/?id=35420065
2078e58004ec88dcec69bb665b47c0c4	https://crt.sh/?id=35420275
1ab475d89c0019e2af875d675a4b94ce	https://crt.sh/?id=35420265
7971fec52f58599a255f9405734ba537	https://crt.sh/?id=35420298
5364cc6084e5705517db081e7aa0042e	https://crt.sh/?id=41001122
1f74bf6a2dc175632382b612a9cef5ea	https://crt.sh/?id=41001120
3703726629184df0b7be062c5e21e1cd	https://crt.sh/?id=41001119
58f864b60e9f020a114e9ca3695c83fe	https://crt.sh/?id=41001118
641767db2e2cc630cc6b70db6e62447f	https://crt.sh/?id=41744438
443cb09d2eb85da60fb67b92a3666010	https://crt.sh/?id=46281655
2638b31d24c3344ff73d9086b3007ccd	https://crt.sh/?id=48158433
258a0f4da249511f7c99e39c0d7823e1	https://crt.sh/?id=48173804
14c467efa61467f73a53ed3794797269	https://crt.sh/?id=48315924
3a0799eb03b6cd56dcf60d5a46e504a1	https://crt.sh/?id=48440200
46411cd30d346cf55fc758b2c64f3679	https://crt.sh/?id=48634112
6bc46cc415d2070b1d40a39b08670f25	https://crt.sh/?id=50220239
03ec3e7858aca04b6b3c3c3d97900520	https://crt.sh/?id=50232662
7509661a48b93f04e593c8d30ee03bdc	https://crt.sh/?id=56035640
02bc7a682237a2e3a4026f1db4adb243	https://crt.sh/?id=56035639
4a1ab346e4f89b7818b5e60409fac89c	https://crt.sh/?id=56035705
0a95861c600ce536b7086639a01cda5b	https://crt.sh/?id=56035866
1e23b75463d5d5dd767b4fd46cfb2206	https://crt.sh/?id=56035879
3957cfb57b18eb8f64986a896cf12137	https://crt.sh/?id=56376628
3cb1abc5fc5d2f5cff49666598610b69	https://crt.sh/?id=56376635
3ce105ebdf4dbe1e91ab2ea5db1cc445	https://crt.sh/?id=56376520
0c322bb30450c82ca966873a11f7ea1a	https://crt.sh/?id=56376627
50f18cba53fe05733527247d2d2471e1	https://crt.sh/?id=56981693
6ac1593b0f3211fe39700d58b04e66e3	https://crt.sh/?id=58720767
4b0f7f17c4b73189b8828b86d4a746ac	https://crt.sh/?id=58734196
57971d58cc21a58e588d435eeba21ae2	https://crt.sh/?id=59125817
5a3e1228c378f41a0828f755d28f7cd7	https://crt.sh/?id=59255553
2ee34de4c3f7f8e2c46960cbb625c1cd	https://crt.sh/?id=59255782
40d8a34e136cc11e6928f39194916b2f	https://crt.sh/?id=59255796
03a30102b5a05993146ac9b0851129fb	https://crt.sh/?id=59255774
17b0c912db2f4cb4c28bf3df95b279a6	https://crt.sh/?id=59255785
16295f692611fe3c80a512674713c581	https://crt.sh/?id=59255786
265ea3e64a52ca56be6b3b03275c0636	https://crt.sh/?id=73048232
60f5ecbb401219193dc778572bc60b26	https://crt.sh/?id=76770006
127276265b24a2c85cacac66af3b1e9b	https://crt.sh/?id=76770016
4c079636cdac89d76ac98fff7a08f274	https://crt.sh/?id=76802781
0c9d4ee72e917ba3a6f4e1b7096c37e0	https://crt.sh/?id=88203012
2f570eb37e62275b5c69750a4050dc49	https://crt.sh/?id=98104332
7c494ff7277d776dabf68916820132a1	https://crt.sh/?id=108364192
3a076b403942ba6cce4df0d653add0da	https://crt.sh/?id=108364188
23ecb16d972a3e8a701ee7796563c0b0	https://crt.sh/?id=108364191
582a3d0dc4b86282464d69b3781cbba7	https://crt.sh/?id=108364190
20a44e6155e8bd4fe80ccdbdd08648c0	https://crt.sh/?id=108364193
52832f46ad3752dfb31b0bbea1791feb	https://crt.sh/?id=108364194
480dc3be159671517e0a73f9f07839fe	https://crt.sh/?id=118423082
3dc26e72014d72fc0341fe780b4b0855	https://crt.sh/?id=124407871
4bb55ea977613503bf73552cf8169061	https://crt.sh/?id=124407855
2870303b9b13b53d533f543adfcf7c89	https://crt.sh/?id=124407865
5dedcd12fa9c7fa6fd20cf13adaf7abb	https://crt.sh/?id=124407866
229c29ae54ebc61cbe2c026d22836d5f	https://crt.sh/?id=124720612
045b5e8ddc1741090c98c97036187bb7	https://crt.sh/?id=124713413
6465423564731be15f9eadabb1eb1702	https://crt.sh/?id=154896584
7c08d8f80d5010618151fc5f038c92a8	https://crt.sh/?id=154896586
4983e1c6122e4b0ce8dc3c1011d23a23	https://crt.sh/?id=158715746
3b7264ca13c11cb2d4a9225160040ab1	https://crt.sh/?id=159320641
1faf5870dcaad315efb2e0dfdbc1ead6	https://crt.sh/?id=167855126
29a5ec146c09dc4a7f42e707f9684b52	https://crt.sh/?id=167883854
0c71b3cff47b4a5cdfe021b42225f450	https://crt.sh/?id=167883855
4e1967a9b2adebf0549faeaa5938e054	https://crt.sh/?id=170429183
6791802e7680cbf3d180a9b1764f98a6	https://crt.sh/?id=172742708
33cc35c2f9fea3ad7439f283ee669573	https://crt.sh/?id=188559470
2ed73b584748cda244649180fb6d4f11	https://crt.sh/?id=188559447
7ced6c4f6dd1a1330a3643d33b671853	https://crt.sh/?id=188559453
4b1f9b27e77ed51d20461eec14d84cbd	https://crt.sh/?id=188559473
76254beb5bf1c0ffa58c4e61ca02660a	https://crt.sh/?id=192546540
400e3035ab3ff071bcf8c7fd5a407887	https://crt.sh/?id=200748333
1a90f10f1b5c583784e3ce95f9cd2236	https://crt.sh/?id=201316075
254dc96dd44d94136c3ba1a593397ad0	https://crt.sh/?id=205723804
2c967ae6765e7eebe896bfaaf00e82e9	https://crt.sh/?id=208943705
59b92d578038ea2dc8a6b05cb74dbd5d	https://crt.sh/?id=210092209
1c77295159063ca78ae4b73b094185c3	https://crt.sh/?id=210185237
1bf79e9c7c54951c11de1ce9af0a490c	https://crt.sh/?id=218375953
12c3fd38a2e667b8fbb39049b5207e7d	https://crt.sh/?id=218375970
6e12d284f343cdc89dfff68408ce7e14	https://crt.sh/?id=218375988
471ee0cd8fb1e493ebc416e2728cfb27	https://crt.sh/?id=218375942
1d969cac9738d57ad6e57aadadbc12f8	https://crt.sh/?id=218375944
64894fdeeb1ece7ce26dc84f958f7538	https://crt.sh/?id=218375946
747dba78367795be9aecd79b857f3744	https://crt.sh/?id=218375948
23c899c42b49ae18cc543b78a9e4d3ff	https://crt.sh/?id=218375949
6a76b57a5fe2dca798d4a9b35b0e8988	https://crt.sh/?id=218375950
7bcad887fa089f8a9c6ba2dbac6ef63f	https://crt.sh/?id=218375951
39823debffc92acb9efc8ef1471b32db	https://crt.sh/?id=218375969
734a10edaa9a58008477f3c73a5b63ed	https://crt.sh/?id=218375968
1f24659cc341f9c830613cdb38d91371	https://crt.sh/?id=218492381
35853d718aadf4feedfbb2d642dd59af	https://crt.sh/?id=218492535
70f83e8c31f9887f6e12b965682ce296	https://crt.sh/?id=218492815
57694dc34ad8bd4bc0485d9fbdae104e	https://crt.sh/?id=218492966
5fd705d3b1420d3e0b2a2dff99e6ef96	https://crt.sh/?id=218493013
3c5f2f57d43ddbc1516c19ec1362429d	https://crt.sh/?id=218492250
2679d9bfd80a8b37891ed1521592ee2f	https://crt.sh/?id=218492379
16921ff4305b95d11a5980d754fc4878	https://crt.sh/?id=218492537
482ddb654c974e8da8557ecf25e62070	https://crt.sh/?id=218492651
67fab84b2cb0069e88101a6cab4e1f98	https://crt.sh/?id=218492703
32c8848c845184e833481383e14b1708	https://crt.sh/?id=218492814
57b59052fa4bdcff71c43dbd73971afd	https://crt.sh/?id=218492831
1745c1882d0b0bb7b5ff946b4eb91c41	https://crt.sh/?id=218493014
61ce9d827950e1149a933b250f82c043	https://crt.sh/?id=218493023
3d87e72152722c1da23ff66bea707cda	https://crt.sh/?id=218928133
680f63d8b6ec593d02593c762687edc4	https://crt.sh/?id=220939016
5f039a008f89490123e855808ad8b151	https://crt.sh/?id=220939030
7b05b2ac8ef737758bf32074685b7020	https://crt.sh/?id=224763330
36cdb9c5c6b93f1e8320c27786b90f08	https://crt.sh/?id=224763374
5842da861c593795b738c30bd4b45b07	https://crt.sh/?id=224763186
62d124a1095360fb75e7a7e97d6b0dbb	https://crt.sh/?id=224763220
260087a2de99ababf366c84263ae572e	https://crt.sh/?id=224763221
0683992795efac4716b142822f14a859	https://crt.sh/?id=224763278
17a5532698727ed15c3b2a83b7517f63	https://crt.sh/?id=224763277
608d23d4983e76f49112244593f073dd	https://crt.sh/?id=224763287
04282c05691ecab84660902a2947f6af	https://crt.sh/?id=224763288
77c0256fe0cd5d977667868ed667a545	https://crt.sh/?id=224763332
6c4d3770e0d112f509c7a638f5b746e8	https://crt.sh/?id=224763334
430acc5e769afe076cc740f38b5f0757	https://crt.sh/?id=224763345
082ba63e14a1d548149bb726213c94ef	https://crt.sh/?id=224763370
4bc5f24f6adf048c754fc12f8c54920f	https://crt.sh/?id=224763371
17e7950c1abc30425dcd4ee0b8f29e43	https://crt.sh/?id=224763383
35003fa133e252ad8196c795efd28e44	https://crt.sh/?id=224763410
24ad633566b4af463447a0aa65c26abc	https://crt.sh/?id=224763411
61e39f4ce34d65ad7760dcb2192cebc8	https://crt.sh/?id=224763417
36e803210cbe79bb443c88e184d84a86	https://crt.sh/?id=232961893
2bc902616764f4846b675feab4263588	https://crt.sh/?id=232961603
5bb15d213c1910060b32277b7e5ad07a	https://crt.sh/?id=232961599
3b7cd9ea99c805912504749c7143b99c	https://crt.sh/?id=253368616
7e8546974b82685b39c62631d6237199	https://crt.sh/?id=259443531
0e6ef1621eab0f10c63ef2aaebab5d2d	https://crt.sh/?id=272911464
02ba28080f341cb53a7b368026f64641	https://crt.sh/?id=296922495
0d6177e212ceade6333a602f3d474253	https://crt.sh/?id=296922499
0a040a32a71c7ff64dde38e626d4f4d3	https://crt.sh/?id=296922510
05e69eebe7b36ba6ca12c6aba6131d88	https://crt.sh/?id=296922553
06fc5035a117a150b593389bde9baeb5	https://crt.sh/?id=296922578
0d8cc159f93e9b5c8368d22c4d71edc1	https://crt.sh/?id=296922580
02b4f3eeddb1e4e1cfff5372537d1870	https://crt.sh/?id=296922589
0ab674e553d44ef4250d1b56fca31373	https://crt.sh/?id=296948101
0399e96657049a930eaa6c714a5e1bdd	https://crt.sh/?id=296922608
0705fe7b1b2194fdb0b8b3dfa585734f	https://crt.sh/?id=296922610
013548f8e481507535f11988a58b3a60	https://crt.sh/?id=296922661
0b958ab3ec2018d144a0b0e215fc598a	https://crt.sh/?id=296922666
03ffe1e33f0990b6da339604e3daac5f	https://crt.sh/?id=296922687
0eaf0a5c2eef0ac0c1a4bfc271ddd596	https://crt.sh/?id=296922689
06e56573d2a3a3e9bd363f66ec0dce51	https://crt.sh/?id=296922691
0b35f4ba90c55425909a805f7192ec91	https://crt.sh/?id=296922695
05c33e5bcadc5db849e990b040a56c33	https://crt.sh/?id=296922697
06bdec0208f2facb9c610ea269d8fa42	https://crt.sh/?id=296922702
0d37a84d85d1c689ac47716cf8e03b2a	https://crt.sh/?id=296926712
0b458a46a5a4bbd95e01ff83bcfd642f	https://crt.sh/?id=296922391
0e1cf285916ea84195c0929c03571eb3	https://crt.sh/?id=296922590
04d03f8a4b8ef8461c501b099c276a5f	https://crt.sh/?id=296922700
0c7a378f07548d2d89ae6c3a29f71678	https://crt.sh/?id=296922321
0d28e3934648558319cbc59109948118	https://crt.sh/?id=296946993
0c023c44bcc3139c0c772531de4827c9	https://crt.sh/?id=296922366
08bcdce91ffa44c25899e011424f785e	https://crt.sh/?id=296922419
0fed701d9af3b6cb4f7fa1bfcbf5a915	https://crt.sh/?id=296922458
06c44292eb3d2b88b2c263964aef3629	https://crt.sh/?id=296922493
07ad70e166d6796781243188bb3e2027	https://crt.sh/?id=296922512
0a5a44b47b402efac8ed81bad92e8afd	https://crt.sh/?id=296922574
0be473b7c40bae0bfb1b1154f6a82d6f	https://crt.sh/?id=296922669
0c3e64c8af488f3dc1c32a7bb1f97607	https://crt.sh/?id=296922364
0c3a59be6409201adbf919479d1e931c	https://crt.sh/?id=296922365
0a42b4f147ae7291df61692278799308	https://crt.sh/?id=296922414
01fedfc6b3889aadce442db9125ed8e1	https://crt.sh/?id=296922551
09c9def4933ae6e1abb29f6c7b7aeb60	https://crt.sh/?id=306059030
0be630019c642b8b9e9fbbb2f3e44fdd	https://crt.sh/?id=311884656
09e82e26b9e3e1b18bb2951c016e793b	https://crt.sh/?id=311884679
0120f6a47387cbc91eb50817a2b985cf	https://crt.sh/?id=311891010
04396ee112fb3c6c35cfd47fe8d8155b	https://crt.sh/?id=313983791
0acb5eeb3c87c84f548beeb018f09467	https://crt.sh/?id=318645732
0759925e3dbd229f7555ff549ce7e8fb	https://crt.sh/?id=318696838
02a4563f97eedce9eb74dd3cd2c09cec	https://crt.sh/?id=323481848
075a779b38067c8cacebd1ef57871c01	https://crt.sh/?id=325895256
0c2f5fe88424cb44fe8513ea60796a77	https://crt.sh/?id=338299100
0a0f192696dda61a6f6c1958be8964a7	https://crt.sh/?id=355745155
07e4cc0fee62fb6839d58160d2ecbfa0	https://crt.sh/?id=356136781
0ef46d310ba28055cbe3654ec3e703e6	https://crt.sh/?id=356606368
05b769f47fb313283c34fe2afcf69be3	https://crt.sh/?id=368994445
03fcf5adebf6f816b0c7124dabd380e3	https://crt.sh/?id=438172292
0ac092a0d7c228c55d0c3d9da0be81c6	https://crt.sh/?id=480511602
0a9b2808c88b36ce15e70b39fc1c8b95	https://crt.sh/?id=493519562
0242480a94e6426a955e0ed7fcfdbc74	https://crt.sh/?id=493519670
056abffc522b25954404d1cfcd434a55	https://crt.sh/?id=493520462
0ad9cc3ed799fd00ed17f23ea20203af	https://crt.sh/?id=508222437
0ac25abea4253a9754bfab6fb941f728	https://crt.sh/?id=508222552
0bf9a37ff30c90a310cc59fa9461c140	https://crt.sh/?id=508221860
070256e32574a9a22d4121ad5b94218a	https://crt.sh/?id=509869387
08c97a407448bb480b8e9a431994393b	https://crt.sh/?id=509868916
0248a134a422c14766050b281df7a148	https://crt.sh/?id=509868921
06a6b3079b07a03a5fd5e3d02c438418	https://crt.sh/?id=511816320
0123865ab8fc8f6bad283d53d1395bf0	https://crt.sh/?id=511816328
076765fbec64d549f9395a8bbbb7be18	https://crt.sh/?id=511816322
05d10c5d560dda8e9b8af609c49cd375	https://crt.sh/?id=526224863
032df73e687e5f0755564eb142d1cc47	https://crt.sh/?id=606681579
0399644eee168145b1b942d912f891b8	https://crt.sh/?id=615930676
0ff900890d42a817985187e24c11e945	https://crt.sh/?id=615931252
0d72725ddb10fa4e1cd6a1ae2aad6b7c	https://crt.sh/?id=615931257
06413d80733298590d976786f462f540	https://crt.sh/?id=627188575
01b316b0eea87c561ee5b18c89ea8e0f	https://crt.sh/?id=635938547
06927c072fdfa4e7ceac78db14fff858	https://crt.sh/?id=667182614
0a231c25b1f2f13f9038e3caa42fdf66	https://crt.sh/?id=709938975
049f6ea294c2611c99a626f88ea0f748	https://crt.sh/?id=672703418
07ec282ab6036200698111da572690c7	https://crt.sh/?id=675070363
07f2f73e861d22a4db361444d8dccfce	https://crt.sh/?id=675070320
0db83a823b5d2c6b00fb29e93cfa75f1	https://crt.sh/?id=684581732
0887b6b1a28a7ccc055dd6b53bc09bd4	https://crt.sh/?id=684581651
064e8f1b3ff1e70290e5e37dbdd6b217	https://crt.sh/?id=684581806
0a29320a4c77327522aeb7e8b94194e7	https://crt.sh/?id=698005053
071272ad74789d6fc0ce8ab8fe4dc17f	https://crt.sh/?id=698171373
05234d7778ab62f8a2457a88781f2df5	https://crt.sh/?id=700451004
0321fcedf95a2e5c9287a6eccfd2d4a0	https://crt.sh/?id=719012435
03e27ae3449dc47049bdefe741748c7e	https://crt.sh/?id=722324245
08bfa3b069972af7769c1df7c470e9f0	https://crt.sh/?id=722326218
070e0c6551eda173434b31c656311776	https://crt.sh/?id=722323579
0e137ff794deaa106335da82f2b3a6e3	https://crt.sh/?id=722323863
04f65e9127f8e5890ba303741036873a	https://crt.sh/?id=722115663
0a5ce3d1c77c7c3c15f2b5965e602328	https://crt.sh/?id=722324654
015c6f3635ced5673a329490e4f179cf	https://crt.sh/?id=722326223
09d538ea8912474a86bef6452dbf3e46	https://crt.sh/?id=722326192
021ed11e330d22d8467e05c9e6470085	https://crt.sh/?id=722326706
0ef1cf22b26be22597daa2362de5f3ed	https://crt.sh/?id=722326195
057090f2190b6547651dd465e1bfa0cb	https://crt.sh/?id=724663338
019b9b7f84aa275e88ca626997791ed6	https://crt.sh/?id=724514264
09cee89425abdac209020b42fcb4f375	https://crt.sh/?id=735191141
0968f81dd996925023e207848718212c	https://crt.sh/?id=735191941
0f79cd1cba4d065b31b59fe3a5953d70	https://crt.sh/?id=735191992
05c64ca1094212e8036cbf7227216652	https://crt.sh/?id=736911672
0d3e7c10d6f6511ec4aae66dc859d4cd	https://crt.sh/?id=748219103
013441fe9fe1f217d1bbca4051b4a20a	https://crt.sh/?id=767703657
08c14e4350e79f31824806baa832878e	https://crt.sh/?id=767902631
05d3a78a4f0967d213312ad0bcf52c78	https://crt.sh/?id=767902702
081c0b3225c210a174af16c46cfec900	https://crt.sh/?id=767704309
0afbe8673337e99eb975a56b4eb1cc87	https://crt.sh/?id=767704365
0d55eab6c6296a684d3443d1a8b95a57	https://crt.sh/?id=767903024
0316c5b129d12170177a12111af98ab0	https://crt.sh/?id=767903178
08581317a3fedbdeb0786c29830857c5	https://crt.sh/?id=767705252
0d1f1a7aa1487688d5deab9ba6497c2b	https://crt.sh/?id=767903213
099a2cd3d7e9c964a7cd068657ab3eb7	https://crt.sh/?id=767903196
03f61e406bc3b08cac346cafacf4e50e	https://crt.sh/?id=767704414
09f07c05f35970277a78e32620f6844c	https://crt.sh/?id=767903260
0dc6ff761a83a3e758ef750da7beb340	https://crt.sh/?id=767697565
03ef7b40babf23a794c0f1798e2cab9d	https://crt.sh/?id=767902849
04c6179eb1bad969f3895a25fdf5643b	https://crt.sh/?id=767903052
0a44fdf715320175cbeb87b057cc7c54	https://crt.sh/?id=767903244
05a095e944200df84c61fe3cf915ba40	https://crt.sh/?id=767903571
0b1ae5e3725cb852f9114979ed959b8b	https://crt.sh/?id=767903254


They plan on replacing all certificates by April 30th. They can't be replaced before then because of the code freeze and the risk of an outage.  Because these are critical part of the enrollment systems, replacing the certs requires internal full stack validation and coordination with the external partners and customers. Coordinating a replacement on this takes weeks because of the application testing required.

The systems use publicly trusted certificates because they are accessed by corporations everywhere to enroll employees in benefits and systems. Browser use is essential during these periods.

The 30 day certificates do not help because the code freeze doesn't permit certificate changes. The main difficulty is the external buyoff by the corporate partners and the testing required. Changing certificates and domains together takes about the same effort.
Jeremy,

Thanks for filing this. Unfortunately, there's still missing details, although hints of it are in the messages you've shared on the list.

For example, you mentioned "The actual revocation occurs sometime between Jan 15 and April 30"

For this set of certificates, it's necessary that we have a clearer understanding about what the migration plan looks like. Similarly, in analyzing what you present as the risks, you say 30 day certificates don't help because the code freeze doesn't permit certificate changes - but it's unclear how long that freeze lasts, when progress will begin, and how progress will be measured.

It sounds like you're saying that changing certificates takes as much work as changing the entire domain name - is that correct?

Concretely:
1) When does the code freeze end?
2) When do certificates start to:
  * Be replaced (e.g. with 30 day certificates)
  * Be rendered obsolete (e.g. due to domain changes or switching to wildcard certificates)
3) What are the progress milestones to expect
  * For example, "We expect that the majority will be replaced from February to March by renaming the host."
  * For example, "For those that renaming the host is difficult, we expect to transition to 30 day certificates."
  * For example, "We're working with the customer to get more concrete plans. We think roughly X, but will have a closer idea of what certificates are most problematic by Y."
Flags: needinfo?(jeremy.rowley)
Discussion is at https://groups.google.com/d/msg/mozilla.dev.security.policy/0oy4uTEVnus/pnywuWbmBwAJ
Assignee: wthayer → jeremy.rowley
Summary: Underscore character - Pharmacy company → DigiCert: Underscore character - Pharmacy company
Whiteboard: [ca-compliance]
This is in response to Ryan's questions for clarification.

1) When does the code freeze end?
There are two freeze dates that are important.  1st, 10/15 through 2/1.  This is the Annual Enrollment Period for certain major health plans.  2nd,  12/1 through 2/1 this is the Welcome season for people to access their new or modified Pharmacy Benefits plans.  These are our absolute busiest volume times of the year, and are critical to our customers so they can successfully navigate they prescription benefits.

2) When do certificates start to:
* Be replaced (e.g. with 30 day certificates)
* Be rendered obsolete (e.g. due to domain changes or switching to wildcard certificates)

Certificates are starting to be replaced now, in the lower level environments (small subset of the total list).  Planning for the remaining is happening and will continue through January.  We will start swapping out the remaining certificates by February 15th and target completion of the certificate swaps by the end of March.

3) What are the progress milestones to expect

During our Code freezes, we can’t accept a risk to change out significant pieces of our technology. Replacing these certs requires not only internal full stack validation, but also coordination with the large population of external partners and customers.  That isn’t feasible to manage in the window that we have been given, especially since this is in the code freezes. Coordinating a change of this nature takes many weeks due to the number of the parties involved in the application testing.
(In reply to Brenda Bernal from comment #4)
> This is in response to Ryan's questions for clarification.
> 
> 1) When does the code freeze end?
> There are two freeze dates that are important.  1st, 10/15 through 2/1. 
> This is the Annual Enrollment Period for certain major health plans.  2nd, 
> 12/1 through 2/1 this is the Welcome season for people to access their new
> or modified Pharmacy Benefits plans.  These are our absolute busiest volume
> times of the year, and are critical to our customers so they can
> successfully navigate they prescription benefits.

Thanks for clarifying. To make sure I've got it summarized: There are two different freezes, both of which end on 2/1, correct?

> 
> 2) When do certificates start to:
> * Be replaced (e.g. with 30 day certificates)
> * Be rendered obsolete (e.g. due to domain changes or switching to wildcard
> certificates)
> 
> Certificates are starting to be replaced now, in the lower level
> environments (small subset of the total list).  Planning for the remaining
> is happening and will continue through January.  We will start swapping out
> the remaining certificates by February 15th and target completion of the
> certificate swaps by the end of March.

To clarify: The goal is to have existing certificates swapped with 30-day, underscore-carrying certificates by EOM March, is that correct?

> 3) What are the progress milestones to expect
> 
> During our Code freezes, we can’t accept a risk to change out significant
> pieces of our technology. Replacing these certs requires not only internal
> full stack validation, but also coordination with the large population of
> external partners and customers.  That isn’t feasible to manage in the
> window that we have been given, especially since this is in the code
> freezes. Coordinating a change of this nature takes many weeks due to the
> number of the parties involved in the application testing.

This didn't quite answer the question, but it looks like the previous answer provided many of these details, so I want to make sure I've got it summarized correctly:

- Now - 2019-01-30: Planning for migration for all existing certificates (to...?)
- 2019-02-01: Freeze is lifted
- 2019-02-15: Migration begins (to... 30 day certs? non-underscore hosts?)
- 2019-03-31: Migration complete
- 2019-04-01: No new underscore certificates are needed or will be issued

Is that right?

If so, I think the core questions are:
* Is this conflating changing the host name (which, understandably, can be more risky for an organization) with swapping the certificate to a 30 day certificate? That is, could the migration to shorter-lived certificates be completed independent of the name migration, and thus completed sooner? I'm wanting to confirm this was explored, and if it was ruled out, we've got it documented why it was ruled out.
* Could you explain a bit more about the two week gap in February between 02-01 and 02-15? It would seem like the migration could begin immediately, but perhaps I've misunderstood the timeline.
* Can you confirm that the expectation is that on-or-after 2019-04-01, no new underscore certificates are expected, as captured in SC12?
Flags: needinfo?(jeremy.rowley) → needinfo?(brenda.bernal)
Replying the follow-up questions from Ryan:

1) Correct, two different freezes, both of which end on 2/1.

2) The certificate swap will be for non-underscore certificates.

3) The timeline is correct with one clarification below:

- 2019-02-01: Freeze is lifted
- 2019-02-15: Migration begins to non-underscore
- 2019-03-31: Migration complete
- 2019-04-01: No new underscore certificates are needed or will be issued

Core questions responses:

* Is this conflating changing the host name (which, understandably, can be more
risky for an organization) with swapping the certificate to a 30 day
certificate? That is, could the migration to shorter-lived certificates be
completed independent of the name migration, and thus completed sooner? I'm
wanting to confirm this was explored, and if it was ruled out, we've got it
documented why it was ruled out. 

Based on our analysis, the effort requires the same due diligence and shakeout testing, so we are going to just move to the non-underscore hostnames and certs.

* Could you explain a bit more about the two week gap in February between 02-01
and 02-15? It would seem like the migration could begin immediately, but
perhaps I've misunderstood the timeline. 

The expectation is it will start between 2/1 and 2/15, the commitment is it will start by 2/15, based on schedule coordination across all the systems, partners, and customers.

* Can you confirm that the expectation is that on-or-after 2019-04-01, no new
underscore certificates are expected, as captured in SC12? 

Correct.
Flags: needinfo?(brenda.bernal)
(In reply to Brenda Bernal from comment #6)
> 2) The certificate swap will be for non-underscore certificates.
> <snip>
> Core questions responses:
> 
> > * Is this conflating changing the host name (which, understandably, can be
> > more risky for an organization) with swapping the certificate to a 30 day
> > certificate? That is, could the migration to shorter-lived certificates be
> > completed independent of the name migration, and thus completed sooner? I'm
> > wanting to confirm this was explored, and if it was ruled out, we've got it
> > documented why it was ruled out. 
> 
> Based on our analysis, the effort requires the same due diligence and
> shakeout testing, so we are going to just move to the non-underscore
> hostnames and certs.

Just to make sure it's captured, Wayne had posed a question on list specific to this that looked unanswered:

> I agree that more information is needed here. My hypothetical is that of a
> critical vulnerability in one of Organization One's systems being
> discovered on 16-Oct. Does Organization One hold off on patching until Feb?
> If not, what makes these certificates different? Why is so much
> coordination required if they are just used in browsers? Was a risk
> assessment performed to evaluate the possibility of replacing them during
> the freeze? Are routine changes permitted during the change? If so, why is
> a certificate replacement not a routine change?

I'm highlighting this, because it does sound like renaming the hosts (which wouldn't need to be completed until April 30) is being conflated with replacing the certificates (which SC12 sets at Jan 15). From the reply, it sounds like the view is that it is just as risky to replace certificates as it is to wholly rename hosts, which is both surprising and concerning, and I think something worth understanding more about.

From looking at these hosts in sampling (I have not plugged the whole set into a query/analytic engine), it seems they follow the pattern that was discussed leading up to the meeting in October; namely, that potentially renaming the hosts could be delayed much longer, through the use of wildcard certificates, which can be issued after April 1, still, and will then just depend on whether the client software support underscores for hostnames, which, presumably all platforms this is being used on does.

I think it's absolutely good to move away from such hostnames, to be clear, but in that risk assessment, it seems like that larger task/effort (of renaming hosts and updating references) is being conflated with what is meant to be 'simpler', namely, changing certificates.

I think Wayne can add any color to that question, but I did want to highlight it was still hanging out there on the list, and was specific to this ("Organization One").


On my side:

During the discussions of underscores and SC12, multiple requests were made to CAs to examine their systems and ensure any historic-but-still-valid underscore certificates (i.e. those issued before the CT transition date) were logged, to make sure that analysis properly considered and scoped the impact. From the remarks on the mailing list and this bug, it does appear that new certificates were recently logged, hence the delay for crt.sh links.

**1**) What steps has DigiCert taken to log its full corpus of underscore certificates (those requesting exception and those revoked)?
**2**) If no steps have been taken, when will they be taken?

I ask these questions, because it is clear that CVS Health (as Matt highlighted on the list, is clear from the O in the certificates) is and was DigiCert's largest customer of impact. This was discussed during and following the Shanghai F2F. This is captured https://cabforum.org/pipermail/servercert-wg/2018-October/000331.html , noting that outside of vIPtela (which used old Symantec roots), CVS was the largest/most impacted organization. 

Knowing the full corpus of certificates (not just CVS) that DigiCert has issued with underscores helps us have a concrete discussion about whether we're talking about a third of the misissued certs, or something less than that. The current request is, it seems, 412 certificates, which is a marked increase from even what was discussed. Understanding the cause of 139 certificates to be 'missing' from conversation - whether it was DigiCert that didn't log them or a flawed analysis on my part - helps further inform the risk.

**3**) Is it also fair to say that, looking at the corpus of certificates, CVS Health was a legacy customer of Symantec's that was acquired in the transition to DigiCert?

It looks like the transition of DigiCert issuing a larger number of underscore certificates begins around https://crt.sh/?id=272911464 , which aligns with the Symantec/DigiCert transition date. Based on the aforementioned threads, it seems this was a practice that was 10x more prevalent with Symantec than other CAs, which then caused an increase in DigiCert-related issuance following the integration and transition. I'm just wanting to confirm this analysis is correct, in the context of this request, since it seems to fit into the "How we got here" timeline.
Flags: needinfo?(brenda.bernal)
Sorry for the delay.

Re: Wayne's question:
The two are related because trying to get approvals to a replacement first, then a name change, then another certificate replacement is far longer and more difficult than changing everything at one. The slow part is not the updating, it's the testing by third parties and third party approvals using the services required to make the change. 

Re: Ryan's questions
We've identified all of the certificates but have logged all certificates that will be part of the incident report. The original bug shows all of the certificates where there will be an incident on Jan 15. We aren't planning on logging the rest because they will be revoked on Jan 15th.  The non-logged certificates are used only for server-to-server transactions and don't require trust in Google or Mozilla. 

The total corpus of certificates originally was about 2200. However, we've been reducing that number as we work with the customers to identify additional certificates that can be replaced prior to the date or where revocation won't impact use of the certificates. 

You are correct about legacy DigiCert vs. legacy Symantec. However, DigiCert did issue certs to underscore domains up until Oct 1. Verizon, for example, was legacy DigiCert. Now that I think about it, Verizon was probably the only legacy DigiCert customer on that list.
(In reply to Jeremy Rowley from comment #8)
> Sorry for the delay.
> 
> Re: Wayne's question:
> The two are related because trying to get approvals to a replacement first,
> then a name change, then another certificate replacement is far longer and
> more difficult than changing everything at one. The slow part is not the
> updating, it's the testing by third parties and third party approvals using
> the services required to make the change. 

I can't speak for Wayne, but it's not clear to me how this response answers those questions. There were specific examples given in the hypothetical, such as:

>> My hypothetical is that of a
>> critical vulnerability in one of Organization One's systems being
>> discovered on 16-Oct. Does Organization One hold off on patching until Feb?
>> If not, what makes these certificates different?

That's about replacing the certificates with shorter-lived certificates, not about renaming.

The problem is trying to put together a coherent picture of the problems, and the shorter answers aren't really helping put together a narrative that explains the timelines or the challenges. We're left trying to piece it together from the bits.

For example, from the timeline, we know that migration starts 02-15 and is expected to complete by 04-01 - meaning 6 weeks to both replace certificates and rename hosts. In that time, the expectation is that hosts will be renamed, certificates replaced, and acceptance testing performed. What's missing from that, which Wayne's questions got to, is "Are you saying there is no way to replace certificates from 10-15 to 02-01" - that is, replace the certificates, not rename the hosts.

Given that we know what the bound on that timeline is - that no work starts until 02-15 as planned - it's unclear why or how it's far longer and far more difficult to replace the certificates in that interim. It's also unclear where the acceptance testing comes in - presumably, these would be issued from the same hierarchy, and thus present no issues.

Now, I can try and guess reasons that aren't being stated. For example, it may be implied that "It will take longer than two weeks to replace the remaining certificates. Further, once those certificates have been replaced, they would need to be replaced again within 30 days of their issuance, thus likely in the early part of February. As a result, work to rename hosts will not be able to begin until replacing the certificates the second time has completed. Further, given the timeline, certificates will need to be replaced a third time in parallel during that migration, sometime in early-mid March, thus causing further risk to the timeline to rename".

Of course, that's not stated, and it's all based on trying to understand what the challenges are with replacing the certificates, which still has not really been responded to in substance since the discussion begin.


> Re: Ryan's questions
> We've identified all of the certificates but have logged all certificates
> that will be part of the incident report. The original bug shows all of the
> certificates where there will be an incident on Jan 15. We aren't planning
> on logging the rest because they will be revoked on Jan 15th.  The
> non-logged certificates are used only for server-to-server transactions and
> don't require trust in Google or Mozilla. 

I would encourage DigiCert to revisit this decision as quickly as possible. One of the key aspects of transparency is understanding the (incident) versus (total) certificates. Knowing this prior to SC12 may have helped DigiCert avoid an incident entirely - for example, if the problem was 2X worse than it was, or 4X, or however much, discussions about the feasability of migration and shorter-lived certificates could have gone differently. Post-SC12, knowing about whether DigiCert is planning to treat 98% of their certificates as part of an incident vs 2% of their certificates seems very valuable in measuring the impact.
>> My hypothetical is that of a
>> critical vulnerability in one of Organization One's systems being
>> discovered on 16-Oct. Does Organization One hold off on patching until Feb?
>> If not, what makes these certificates different?

Security vulnerabilities are patched based on their rating, the practical mitigation efforts that don't require a change to key systems, the system, and number. Whether the vulnerability waits until Feb or not depends specifically on the vulnerability. The difference between the two is there is no CVSS to point to, there's no way to identify what options there are other than replace the cert, and there's no process for exceptions where the CAB Forum decides something. 

>> For example, from the timeline, we know that migration starts 02-15 and is expected to complete by 04-01 - meaning 6 weeks
>> to both replace certificates and rename hosts. In that time, the expectation is that hosts will be renamed, certificates 
>> replaced, and acceptance testing performed. What's missing from that, which Wayne's questions got to, is "Are you saying 
>> there is no way to replace certificates from 10-15 to 02-01" - that is, replace the certificates, not rename the hosts.

I'll get back to you on this one. I'm not intending to have short answers only. This incident report is different than any other ones I've filed because the root cause is not directly with DigiCert. If we mis-issue a certificates, finding the issue, reporting what happened, and remediating the issue isn't difficult. Having third parties involved is increasing the complexity in reporting the information.
(In reply to Jeremy Rowley from comment #10)
> >> For example, from the timeline, we know that migration starts 02-15 and is expected to complete by 04-01 - meaning 6 weeks
> >> to both replace certificates and rename hosts. In that time, the expectation is that hosts will be renamed, certificates 
> >> replaced, and acceptance testing performed. What's missing from that, which Wayne's questions got to, is "Are you saying 
> >> there is no way to replace certificates from 10-15 to 02-01" - that is, replace the certificates, not rename the hosts.
> 
> I'll get back to you on this one. I'm not intending to have short answers
> only. This incident report is different than any other ones I've filed
> because the root cause is not directly with DigiCert. If we mis-issue a
> certificates, finding the issue, reporting what happened, and remediating
> the issue isn't difficult. Having third parties involved is increasing the
> complexity in reporting the information.

Is there an update here?
Flags: needinfo?(jeremy.rowley)
Summary: DigiCert: Underscore character - Pharmacy company → DigiCert: Underscores - CVS Pharmacy

Based on the conversation on the forum, the post from Wayne, and instruction from Google, our understanding is there is no exception or extension possible and the expectation is that all CAs will revoke the certificates on the date required by the BRs. We hope that the same rules/penalties/expectations will be applied to those CAs who fail to revoke on the required date. Thank you for the discussion. Although we were hoping for more compassionate results, we do appreciate the feedback and clarification on expectations.

Flags: needinfo?(jeremy.rowley)

Seems there was a mis-communication on the intent of the discussions. We will post an update answering Ryan's questions tomorrow. Please ignore my previous post. Apologies for the confusion.

In reply to Ryan's prompt on comment #11, here's the response back to the following:

For example, from the timeline, we know that migration starts 02-15 and is expected to complete by 04-01 - meaning 6 weeks
to both replace certificates and rename hosts. In that time, the expectation is that hosts will be renamed, certificates
replaced, and acceptance testing performed. What's missing from that, which Wayne's questions got to, is "Are you saying
there is no way to replace certificates from 10-15 to 02-01" - that is, replace the certificates, not rename the hosts.

Customer response:
"It’s just as much work AND risk to both change the name of and replace a cert as it is to just replace the cert, because in both cases we’re “touching” the application that relies on the cert – which means application testing in non-production before the cert replacement, and in production after the cert replacement. Any touch to core PBM application now, in the heart of welcome season, constitutes significant, inadvisable, risk."

The customer is mitigating the risk of loss of customer service and benefits disruption during the busiest time of the annual enrollment period.

Flags: needinfo?(brenda.bernal)

That risk assessment is surprising and somewhat against what would result from common industry practice. It is hoped that any remediation plan, if an incident occurs, will detail how such a situation will be mitigated in the future.

For example, past incidents have revealed a number of possible options:

  1. Migration to a private (non-BR audited) PKI
  2. The use and adoption of certificate automation, such that acceptance testing is not tied to individual certificates (a similar concern with pinning)
  3. The use of TLS intermediary devices (reverse proxies) that support more rapid upgrade and deployment, as explored through the SHA-1 and Symantec deprecations

While the BRs have long had an industry-standard, CA-agnostic revocation requirement, ultimately, the CA is responsible for making the decision to revoke or not revoke. It is hoped that a CA that makes a decision not to revoke will take concrete steps to prevent a reoccurrence in the future and identify concrete steps that they will take to ensure that.

Jeremy: Can you confirm whether an incident occurred and ensure that all of the details of affected certs are accurate?

Flags: needinfo?(jeremy.rowley)

Hi Ryan, I will be responding to provide updates on the underscore incidents. I can confirm that an incident has occurred and the details provided are accurate to the best of our knowledge. Our planned extension to revoke the remaining certificates (listed above) is 31-May-2019. We will provide periodic updates as progress is made.

Brenda: I believe that date is different than the past discussions, and so want to understand how this target moved.

Comment #0 stated:

They plan on replacing all certificates by April 30th

Comment #4 stated:

target completion of the certificate swaps by the end of March

Comment #6 stated:

2019-04-01: No new underscore certificates are needed or will be issued

So now I'm trying to understand Comment #17:

Our planned extension to revoke the remaining certificates (listed above) is 31-May-2019

Flags: needinfo?(jeremy.rowley) → needinfo?(brenda.bernal)

Hi Ryan, I will say the 31-May-2019 is an error on my part. I meant to align it to the March 31st date. With that said, I'd like to report that the customer has made significant progress and Digicert plans to revoke their remaining underscores by Friday, February 8, 2019.

Flags: needinfo?(brenda.bernal)
Assignee: jeremy.rowley → brenda.bernal
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
QA Contact: kwilson → wthayer
Whiteboard: [ca-compliance] → [ca-compliance] Next Update - 8-February 2019

The remaining underscore certificates listed above in Jeremy's initial report were all revoked as of today, February 8, 2019.

Thanks for the update, Brenda. I'm glad to hear this was resolved more timely than the originally proposed March 31.

I spot-checked a dozen, and they all show revoked, so I'm going to close this issue as Resolved, tagging Wayne in case he has any questions.

Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Flags: needinfo?(wthayer)
Resolution: --- → FIXED
Flags: needinfo?(wthayer)
Product: NSS → CA Program
Whiteboard: [ca-compliance] Next Update - 8-February 2019 → [ca-compliance] [ov-misissuance]
You need to log in before you can comment on or make changes to this bug.