Open Bug 1586795 Opened 6 months ago Updated 5 months ago

NetLock: Issuance of intermediates after 2019-01-01 that do not comply with Mozilla Policy

Categories

(NSS :: CA Certificate Compliance, task)

task
Not set

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: ryan.sleevi, Assigned: varga.viktor)

Details

(Whiteboard: [ca-compliance])

During a spot-check of Mozilla Policy Compliance, I discovered that NetLock issued intermediates that do not conform with Mozilla Policy 2.6.1

Mozilla Policy 2.6.1 requires (formatted for readability), in Section 5.3:

Intermediate certificates created after January 1, 2019, with the exception of cross-certificates that share a private key with a corresponding root certificate:

  • MUST contain an EKU extension; and,
  • MUST NOT include the anyExtendedKeyUsage KeyPurposeId; and,
  • MUST NOT include both the id-kp-serverAuth and id-kp-emailProtection KeyPurposeIds in the same certificate.

The following intermediates, issued after 2019-01-01, lack an EKU extension:

Please provide an incident report, as detailed at https://wiki.mozilla.org/CA/Responding_To_An_Incident

Flags: needinfo?(varga.viktor)
Status: NEW → ASSIGNED
  1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

As part of the ongoing audit we are reviewing the complete certificate issuance and Policy compliance.
On 2019-10-04, we finished this verification and identified the non-compliant and mentioned section.
I was preparing the report, but Mr. Ryan reported it on 2019-10-07 before we did.

  1. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

i. On the 2019-10-04 we identified and analyzed the bug, and the following tickets were introduced to our JIRA:

  1. We decided to replace those certificates and migrate the end entity certificates to the new CAs. We didn't revoke immediately the subCAs, because the end-entity should migrated before as we do not want to disrupt the usage of those.
  2. The modification of the x509lint to disable the possibility of this error by the code.
    The following modification will suits this:
    SetCertInfo(CERT_INFO_NO_EKU);
    if (type == SubscriberCertificate)
    { Set--Error(WARN_NO_EKU); }
    to
    SetCertInfo(CERT_INFO_NO_EKU);
    if (type == SubscriberCertificate)||(type == IntermediateCA)
    { Set-Error(ERR_NO_EKU); }

ii. On 2019-10-07 we got the report from Ryan.

  1. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

We are not issuing certificates with this problem anymore.

  1. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

The report already includes all the three affected certificates.
Number of certs: 3
First and last date: 2019-04-02; 2019-05-27

  1. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

The report already includes all the three affected certificates.

  1. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The following causes leaded to this errors:

i. There is some contradiction between the practice and the BRG and EVGL.
According to BRG 1.6.6 section 7.1.6.3 and EVGL 1.7.0 section 9.3.4 for a Subordinate CA that is an affiliate of the Issuing CA MAY include the AnyPolicy identifier in the CertificatePolicies.
Following these policies, we created the following 2 SubCA certificates back in September of 2016 :
https://crt.sh/?caid=33991
https://crt.sh/?caid=34165
After we finished the EV inclusion at the Microsoft we did a lot of inspection, to find out, why the end entity certificates aren't green in IE or EDGE.
In the end we found out, that the general implementation and the policy of those sections are not the same, the subordinate CA that is an affiliate of the Issuing CA SHALL have an AnyPolicy at minimum.

ii. After we finally found out the cause of the "greenlessness" we immediately replaced it. Before the issuance of these SubCA certificates we checked everything that is technically possible for a precheck before the actual issuance, but there was no tool to detect this type of problem.
The third certificate was issued with the same settings.

  1. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

The following development steps will exclude the possibility of future errors, and to resolve the actual problem.

i. Replacement of these intermediate certificates and migration of the end entity certificates to this new CA,
the target deadline for the new certs is 10-15 except the last one:
https://crt.sh/?q=f90aca63cb8bb44f8d91b864474bb42aae177e73fdfe6f4acbd12a013a415c15
This is a signer CA and for these there are no recommendations for the subCA EKUs and it's need to be tested for the usable EKU combinations before the replacement, which needs more time.

ii. The modification of the x509lint to disable the possibility of this by code. the target deadline for this 10-15.
The following modification will suits this:
SetCertInfo(CERT_INFO_NO_EKU);
if (type == SubscriberCertificate)
{ Set--Error(WARN_NO_EKU); }
to
SetCertInfo(CERT_INFO_NO_EKU);
if (type == SubscriberCertificate)||(type == IntermediateCA)
{ Set-Error(ERR_NO_EKU); }

Flags: needinfo?(varga.viktor)

Viktor: Mozilla specifically notified CAs of this change and Netlock responded that they understood and would comply [1]. Please explain why Netlock failed to meet the commitment that was made, and how this will be prevented in the future.

[1] https://ccadb-public.secure.force.com/mozillacommunications/CACommResponsesOnlyReport?CommunicationId=a051J00003rMGLL&QuestionId=Q00072,Q00073

Flags: needinfo?(varga.viktor)

If I understand correctly:

  • The first two certificates were misissued because NetLock originally issued certificates in violation of Microsoft policy, revoked them, but re-used the (old) profiles to generate new versions, and did not review that the new profiles conformed to the new requirements.
  • There's no explanation as to why the third certificate happened.

The fix mentioned in Comment #1 would still not comply with Mozilla Policy, so that's equally concerning.

(In reply to Ryan Sleevi from comment #3)

If I understand correctly:

  • The first two certificates were misissued because NetLock originally issued certificates in violation of Microsoft policy, revoked them, but re-used the (old) profiles to generate new versions, and did not review that the new profiles conformed to the new requirements.

No. The first two certificates are not violating the MS policy. The original certificates were compliant with all published policies at issuance time, but because they are not going to green, we need to replace it. After we identified what are the causes of the problem we replaced those certificates. Unfortunately we missed this requirement at the review before the issuance.

It was a replacement of the earlier near to expiration CA certificate:
https://crt.sh/?id=12721773
It was issued with the same settings as the two before, because we missed this requirement.

The fix mentioned in Comment #1 would still not comply with Mozilla Policy, so that's equally concerning.

Can you tell me, why the code not fits for this? We are blocking issuance, if x509lint gives an error.

(In reply to Wayne Thayer [:wayne] from comment #2)

Viktor: Mozilla specifically notified CAs of this change and Netlock responded that they understood and would comply [1]. Please explain why Netlock failed to meet the commitment that was made, and how this will be prevented in the future.

[1] https://ccadb-public.secure.force.com/mozillacommunications/CACommResponsesOnlyReport?CommunicationId=a051J00003rMGLL&QuestionId=Q00072,Q00073

Human error. To prevent human errors like this, we reviewed our processes, and added another contact person to CCADB (which was not possible at the time of commitment).
(look at https://bugzilla.mozilla.org/show_bug.cgi?id=1572992 )

Flags: needinfo?(varga.viktor)

Dear Ryan,
Can you give me clarification about your opinion?
Yours, Viktor

The fix mentioned in Comment #1 would still not comply with Mozilla Policy, so that's equally concerning.

Can you tell me, why the code not fits for this? We are blocking issuance, if x509lint gives an error.

Flags: needinfo?(ryan.sleevi)

Thanks for the NI Viktor!

The issue is the proposed change doesn't encompass all of the Mozilla Requirements within Policy 2.6.1. It is not simply sufficient to ensure the EKU extension is present, but that the EKU extension is consistent with policy. Comment #0 captures those, while the proposed change in Comment #1 does not fully account for those.

Flags: needinfo?(ryan.sleevi)

Thank you Ryan!
At first I thought you found some bug in the code.
You have right, for blocking the full combination the following code also needed:

// checks.c
// insert after line 1391
if (type == IntermediateCA) {
SetError(ERROR_ANY_EKU_IN_INTERMEDIATE_MOZILLA);

}
//insert after line 1425
if (GetBit(cert_info, CERT_INFO_SERV_AUTH) && GetBit(cert_info, CERT_INFO_EMAIL) )
{
if (type == IntermediateCA) { SetError(ERROR_INVALID_EKU_COMBO_IN_INTERMEDIATE_MOZILLA); }
}

Maybe both the type checking is unnecessary, because

  1. on 2020-04-01 AnyEKU will be not usable anymore in end entity certs.
  2. if the serverAtuth, emailProtection combo invalid on the level of intermediate, it needs to be invalid on end entity too.

Can you do a quick check on this code too?
Thanks, Viktor

Flags: needinfo?(ryan.sleevi)

The first check - anyEKU in intermediate - is good to have regardless. The second check - two EKUs in intermediates - is good, but as you note, it'd also be a problem for leaf certs. So you could simply ignore the type of cert, and check it as an invalid combo.

Note that the second check will result in false-positives for certificates that are cross-signed (which are permitted to have EKUs, under certain constraints, per Comment #0).

You can see discussion about this in the following check - https://github.com/zmap/zlint/pull/323 - which proposes to add a variety of Mozilla policy checks to zlint.

Flags: needinfo?(ryan.sleevi)

The modified checking code will be in PROD in 2019-11-18.
We are not planning cross-certificates, so its good for us.
We will mention this, when commiting back the changes to x509lint.

You need to log in before you can comment on or make changes to this bug.