GlobalSign: Incorrect whois information for TLD
Categories
(CA Program :: CA Certificate Compliance, task)
Tracking
(Not tracked)
People
(Reporter: christophe.bonjean, Assigned: christophe.bonjean)
Details
(Whiteboard: [ca-compliance] [uncategorized])
GlobalSign has been informed of incorrect whois information for a TLD within our automated whois validation processes, that may have potentially lead to mis-issuance.
So far we see no signs of mis-issuance, however, we are further investigating and raising this ticket as a precaution out of transparency. We will provide an update by the end of the week.
This may be what the bug is referring to: https://labs.watchtowr.com/we-spent-20-to-achieve-rce-and-accidentally-became-the-admins-of-mobi/
Have you taken any action so far?
Given the scope of the bug referenced in Comment 1 I would gather a larger discussion is required on handling this verification method going forward. The researchers noted more CAs are affected, so I'm wary of a single CA creating an incident as if this were a sole failure on their behalf.
Would a thread on MDSP would more relevant, and how should we handle every CA with an affected infrastructure going forward?
|
||
Comment 3•10 days ago
|
||
(In reply to amir from comment #1)
This may be what the bug is referring to: https://labs.watchtowr.com/we-spent-20-to-achieve-rce-and-accidentally-became-the-admins-of-mobi/
Have you taken any action so far?
As soon as we were rendered aware of the issue and identified root cause we updated our WHOIS references for .mobi to the IANA referenced .mobi WHOIS server. This action was completed on September 10 2024 at 2.48 UTC.
(In reply to Arvid Vermote from comment #3)
(In reply to amir from comment #1)
This may be what the bug is referring to: https://labs.watchtowr.com/we-spent-20-to-achieve-rce-and-accidentally-became-the-admins-of-mobi/
Have you taken any action so far?
As soon as we were rendered aware of the issue and identified root cause we updated our WHOIS references for .mobi to the IANA referenced .mobi WHOIS server. This action was completed on September 10 2024 at 2.48 UTC.
Thank you, and thank you for opening this bug as soon as you were made aware of the issue that's happening. I agree with Comment 2 that this is probably something that should be discussed in MDSP in addition to the bugs posted here.
As soon as we were rendered aware of the issue and identified root cause we updated our WHOIS references for .mobi to the IANA referenced .mobi WHOIS server.
Did you also audit your WHOIS reference list for other TLDs whose WHOIS references might also be out of date? If so, what did you find?
|
|
||
Comment 6•10 days ago
|
||
(In reply to mpalmer from comment #5)
As soon as we were rendered aware of the issue and identified root cause we updated our WHOIS references for .mobi to the IANA referenced .mobi WHOIS server.
Did you also audit your WHOIS reference list for other TLDs whose WHOIS references might also be out of date? If so, what did you find?
We reviewed our full WHOIS TLD configuration and noted other discrepancies, which we have rectified as well. We will share the further details in our incident report once ready.
|
||
Updated•9 days ago
|
|
Assignee | |
Comment 7•8 days ago
|
||
We are on track to release the full incident report on Tuesday September 17, 2024.
|
|
||
Comment 8•4 days ago
|
||
Summary
GlobalSign received an email from technology website Ars Technica explaining that a security researcher had successfully registered the domain of a TLD's (.mobi) expired WHOIS server, and observed GlobalSign systems still queried it for WHOIS-based domain approvals.
Impact
Within our platform, we support WHOIS-based domain validation both as an automated process and a manual process as a fallback mechanism in case of issues with the automated flow. Only the automated process was impacted.
We identified 264 certificates with domains across 14 TLDs for which the domain validation could not be relied upon. These were covered in 3 batches:
- Batch 1: Certificates for which domain validation was initiated using WHOIS validation for affected TLDs.
- Batch 2: Certificates for which domain validation was re-used from previously issued certificates, for which the WHOIS option was initiated for affected TLDs.
- Batch 3: Certificates issued using our MSSL service, for which domain validation was initiated using the WHOIS option for affected TLDs.
Example:
- We receive Certificate Request A for affected TLD. Customer initiated WHOIS validation for Certificate A. Certificate A was identified in Batch 1.
- We receive Certificate Request B for the same customer as Certificate A, for the same domain, within the permitted timeline for re-use. Certificate Request B is processed using the validation previously performed for Certificate A. Certificate B is identified in Batch 2.
- Customer requests validation for domain in their MSSL service. Customer initiated WHOIS validation for this domain. After activation of the domain, the customer places Certificate Request C for the activated domain. The resulting Certificate C is identified in Batch 3.
We stopped issuance for *.mobi domains on 10/09/2024 at 00:23. We audited and updated our complete WHOIS configuration in two iterations, which was completed by 11/09/2024 02:12. After further investigation and internal discussion, we also disabled the automated WHOIS process at on 11/09/2024 at 15:57.
Timeline
| DD/MM/YYYY - Times in UTC | Description |
|---|---|
| 09/09/2024 - 19:25 | Email from technology website Ars Technica received by our public relations team requesting comments on an upcoming publication by security researchers. The publication related to the hijacking of an old .mobi WHOIS servers' domain and the possibility of using that domain to impersonate WHOIS towards GlobalSign and as consequence obtain a TLS certificate for a .mobi domain they do not own. |
| 09/09/2024 - 19:51 | Public relations team notices the email from Ars Technica and forwards to CISO, product management, legal and marketing. |
| 09/09/2024 - 20:42 | Internal major incident process started |
| 09/09/2024 - 21:43 | CISO reached, who reaches out to key compliance members outside of their office hours / during their night time |
| 09/09/2024 - 22:20 | Initial investigations identified a potential avenue of this being a flaw in the manual WHOIS validation process, as a precaution all issuance based on manual WHOIS validation was stopped |
| 09/09/2024 - 23:20 | Identified another potential avenue where this vulnerability might exist, the automated issuance process in our retail certificate ordering platform. Development team is contacted in order to assist further investigation. |
| 10/09/2024 - 00:20 | Confirmed the issue exists through retail platform orders where WHOIS was selected as automated validation method. |
| 10/09/2024 - 00:23 | All issuance for .mobi blocked in the affected platform |
| 10/09/2024 - 02:48 | Updated our WHOIS references for .mobi to the IANA referenced .mobi WHOIS server. |
| 10/09/2024 - 03:28 | Audit of the further WHOIS configuration to check for any other TLD that have a different WHOIS server configured compared to the current WHOIS servers listed by IANA and other sources (e.g. Linux WHOIS tool) |
| 10/09/2024 - 07:20 | First cycle of WHOIS configuration completed, updates to other incorrect WHOIS server references have been completed |
| 10/09/2024 - 12:57 | Finalized audit of all issued certificates on .mobi, concluded there have been no miss-issuances for this TLD |
| 10/09/2024 - 15:08 | Bugzilla created |
| 11/09/2024 - 02:12 | Second cycle of WHOIS configuration completed, for servers that were not clearly incorrect and required case-by-case analysis. |
| 11/09/2024 - 12:41 | Identified potentially affected domains which relied on WHOIS validation for a TLD that had a different WHOIS server compared to the current WHOIS servers listed by IANA and other sources (e.g. Linux WHOIS tool) |
| 11/09/2024 - 14:48 | Informed vetting management to be on standby for possible incoming revocations |
| 11/09/2024 - 15:57 | All automated WHOIS validation functionality disabled and any associated issuance stopped. |
| 12/09/2024 - 05:58 | Confirmed batch #1 certificates (for which domain validation was initiated using WHOIS) that require within 24-hour revocation under TLS Baseline Requirements 4.9.1.1 (CRLReason #4, superseded) and set for revocation by 13/09/2024 05:58 |
| 12/09/2024 - 08:21 | Received additional report on pending orders where email challenge was sent to WHOIS contact, which showed none found for affected TLDs |
| 12/09/2024 - 17:54 | Confirmed batch #2 certificates (for which domain validation was re-used from previously issued certificates) and set for revocation by 13/09/2024 17:54 |
| 12/09/2024 - 18:03 | Confirmed batch #3 certificates (issued using our MSSL service, for which domain validation was initiated using the WHOIS option for affected domains) and set for revocation by 13/09/2024 18:03 |
| 13/09/2024 - 05:08 | Certificate batch #1 revocation completed |
| 13/09/2024 - 15:54 | Certificate batch #2 revocation completed |
| 13/09/2024 - 17:00 | Certificate batch #3 revocation completed |
| 13/09/2024 - 17:00 | All issued certificates based on WHOIS validation performed by servers differing from IANA and other sources revoked. |
Root Cause Analysis
At the time of the incident, our retail ordering platform supported automated lookup of WHOIS contact information for data validation. The application configuration for which WHOIS servers to query had not been actively maintained for all TLD's, allowing non-current WHOIS servers to be queried for WHOIS requests during automated lookups.
We did not have a process in place to reflect each IANA WHOIS update in our own WHOIS configuration. Rather than having a rolling configuration, one of the primary historical drivers to have a self-maintained WHOIS configuration was to immediately support WHOIS validation for new TLD's launched by our sister company GMO Registry for combined launch of the TLD, ordering domains and supporting certificates.
Even now that we have rendered our WHOIS configuration up-to-date with IANA entries it appears, after further investigation and the discussion on MDSP, that this information is non-authorative and cannot be relied upon since IANA has no mechanism to adequately enforce and ensure the TLD-operators' self-reported WHOIS server entries are up-to-date, functional and trustworthy at all time. As a result we disabled automated WHOIS-lookup and have endorsed CA/Browser Forum SC-080 V1: "Sunsetting use of WHOIS to identify Domain Contacts" and will be further supporting the CA/Browser Forum's efforts into further deprecating this method.
Lessons learned
What went well?
- The compliance team was able to quickly identify the issue and block *.mobi issuance.
- It was possible to reach and summon the necessary stakeholders outside of office hours / during their night time.
What didn’t go well?
- The initial email did not contain a lot of details, as a result it took some time on our side to replicate the suggested vulnerability and hence respond and pinpoint root cause.
- The queries for reporting on the domains and certificates impacted by this specific validation method were not readily available. After identifying the affected domains, development of ad-hoc queries was required which delayed the identification of the corresponding certificates during various phases of the lifecycle (i.e. already issued, domain validation completed but not issued, vetting completed but not issued).
- Disabling WHOIS validation was not a previously encountered and tested scenario.
Where we got lucky
- Only a limited number of certificates were issued through WHOIS for the impacted TLDs.
- Ars Technica reported the issue to us - the security researchers themselves did not. If Ars Technica would not have informed us, there could have been a time window between the disclosure by the security researchers and us being able to respond / block issuance in which a malicious party could have actually exploited the vulnerability to obtain rogue certificates.
Action Items
| Action Item | Kind | Due Date | Status |
|---|---|---|---|
| Configuration updates: for .mobi WHOIS server. | Prevent | 10/09/2024 - 02:48 | Done |
| First cycle of configuration updates for other TLDs. | Prevent | 10/09/2024 - 07:20 | Done |
| Second cycle of configuration updates for other TLDs. | Prevent | 11/09/2024 - 02:12 | Done |
| Disable automated WHOIS validation. | Prevent | 11/09/2024 - 15:57 | Done |
| Extend platform testing scenarios with scenarios for disabling specific validation methods. | Mitigate | 29/10/2024 | Pending |
| Review and prepare reporting queries based on common compliance issues in order to more efficiently respond to similar future incidents. | Mitigate | 17/11/2024 | Pending |
Appendix - Details of affected certificates
Batch #1: Certificates for which domain validation was initiated using WHOIS validation for affected TLDs.
Batch #2: Certificates for which domain validation was re-used from previously issued certificates, for which the WHOIS option was initiated for affected TLDs.
Batch #3 Certificates issued using our MSSL service, for which domain validation was initiated using the WHOIS option for affected TLDs.
|
|
||
Updated•3 days ago
|
Description
•