Open Bug 1965264 Opened 1 year ago Updated 6 months ago

IZENPE: Postpone removal of Izenpe Root CA TLS trust bit

Categories

(CA Program :: CA Certificate Root Program, task)

Product:
CA Program
Component:
CA Certificate Root Program
Type:
task

Tracking

(Not tracked)

Status:
ASSIGNED

People

(Reporter: d-fernandez, Assigned: bwilson)

Details

(Whiteboard: [ca-extension-requested])

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36

Steps to reproduce:

Izenpe's Root Certificate with Serial Number 2530CC8E98321502BAD96F9B1FBA1B099E2D299E0F4548BB914F363BC0D4531F (https://ccadb.my.site.com/s/detail/001o000000HshFEAAZ) was issued on 2007/12/13.
In Mozilla´s Transition Schedule, CAs issued on 2006-2007 will be distrusted on 2026/04/15th/(CA/Root CA Lifecycles - MozillaWiki).

We would like to ask browsers to put off the removal of the trust bit for a year, this is 2027/04/15, while we finish deploying a new solution, according to the following information:

  • This CA was issued just only 18 days before 2008, in which case, it would had the distrust date proposed.
  • On July 2023, Izenpe signed a contract with ENTRUST to provide a new PKI infrastructure and generate a new CA ROOT for TLS certificates.
  • On September 2024, Izenpe issued the new CA Root.
  • On October 2024, the new CA Root was audited.
  • On November 2024, Izenpe notified Entrust that their solution did not comply with ACME. Entrust pledged to apply a solution.
  • On February 2025, Entrust notifies their solution will not comply with ACME.
  • On March 2025, Izenpe signed a contract with Keyfactor(EJBCA).
  • On May 2025, Izenpe will finish EJBCA configuration in our testing environment.
  • On June 2025, Izenpe plans to finish configuration in the production environment and will request for a new root inclusion.
Assignee: nobody → bwilson
Status: UNCONFIRMED → ASSIGNED
Type: enhancement → task
Ever confirmed: true

Thank you for your update. We understand that Izenpe is working toward deployment of a new solution and plans to request inclusion of a new root. However, before we can consider an extension to the removal of the "websites" trust bit until 2027-04-15, we require additional detailed information to evaluate your request.

Please provide a comprehensive timeline and supporting details that address the following:

Deployment Timeline (with Milestones)

Please include specific estimated dates for:

Completion of final EJBCA testing (has this been completed?)

Completion of production environment configuration

Key generation and key ceremony (auditor-witnessed)

Internal and external audit activities and completion

Root inclusion request submission to the CCADB/Mozilla

Estimated approval and trust enablement timeline (based on feedback from root stores)

Planned first issuance of TLS certificates under the new hierarchy

Migration milestones (e.g., expected % of subscriber base migrated by each quarter)

Root Inclusion Process Timeline

When do you expect to submit the root and intermediate certificates to the CCADB?

What audit reports will accompany your request (key generation and subsequent period-of-time audit reports)?

What timeline are you anticipating for approval and usage?
(Assume that all required information is submitted and reviewed on a timely basis)

Subscriber Transition Plan

How many active TLS certificates currently rely on the affected root?

What are your plans to reduce issuance of new certificates from the existing root?

How will you transition subscribers to the new hierarchy?
    (Note that you must support ACME or other automation for issuance)

What tools, communications, and dates will be involved in subscriber migration?

Contingency Plans

What happens if the new root is not accepted or is delayed?

What risk mitigations or alternative solutions are you preparing?

Justification for One-Year Extension

A full-year extension is substantial. Please:

Provide clear reasoning why 6 months is insufficient.

Identify which project components require the full year duration

Explain any flexibility or earlier milestones (or hurdles to overcome) that might enable earlier removal.

Additional Information

To support transparency and measurable progress, we will also need monthly progress updates.

Please update this Bug with the information requested above so that we can fully evaluate the feasibility and necessity of extending the trust bit removal deadline.

Thank you.

Flags: needinfo?(d-fernandez)

Deployment Timeline (with Milestones)

Completion of final EJBCA testing (has this been completed?)

We have finished a preliminary environment configuration setup in order to test the integration of different items such as pkimetal, CT logs, HSM and so on.
Our plan now is to develop a full testing environment by 2025/06/20 which will also be used in the testing environments of our clients.

Completion of production environment configuration

By 27th of June we plan to have the initial setup of the production environment.

Key generation and key ceremony (auditor-witnessed)

Key generation took place on 24th September 2024 and was auditor-witnessed.

Internal and external audit activities and completion

The related audit activities took place the last week of October 2024 and were included in the annual audit report of December 15th 2024.

Root inclusion request submission to the CCADB/Mozilla

We plan to request the Root inclusion by the 2nd fortnight of July.

Estimated approval and trust enablement timeline (based on feedback from root stores)

According to other cases and the time it takes to enable the trust bit in browsers we estimate a 12 months period. Assuming we will submit the request inclusion on July 2025, this will take us to July 2026.

Planned first issuance of TLS certificates under the new hierarchy

As soon as we can test the trust bit is enabled in all the major browsers, according to this, it could be August/September 2026.

Migration milestones (e.g., expected % of subscriber base migrated by each quarter)

2024-09-24 - New Root CA Ceremony.
2025-06-xx - New PKI Software in the production environment.
2025-07-xx - CCADB request submission.
2026-03-15 - TLS certificates duration reduced to 200 days.
2026-04-15 - Official Distrust of Izenpe current Root TLS Certificate.
2026-09-xx - First issuances with the new Root.
2027-04-15 - Proposed date for one year extension.

Root Inclusion Process Timeline

When do you expect to submit the root and intermediate certificates to the CCADB?

We plan to request the Root inclusion by the 2nd fortnight of July.

What audit reports will accompany your request (key generation and subsequent period-of-time audit reports)?
The current audit report covers the new CA.
***What timeline are you anticipating for approval and usage?

(Assume that all required information is submitted and reviewed on a timely basis)***
We are planning to be able to issue the first certificates on August/September 2026.

Subscriber Transition Plan

How many active TLS certificates currently rely on the affected root?

So far, we have slightly more than one thousand TLS certificates, coveríng around two thousand FQDN.

What are your plans to reduce issuance of new certificates from the existing root?

As soon as the new root is included, we will start issuing all the certificates from the new  Root.

How will you transition subscribers to the new hierarchy?
(Note that you must support ACME or other automation for issuance)

ACME has been the main reason why we have to deal with this delay. Entrust software for issuing TLS certificates was supposed to be
fully compliant with ACME and we discovered it was not and they would not be able to develop a new version in less than one year if they decided to do. We have had to move to EJBCA to be comly with this.
Subscribers have been informed about the current problem and the timelines and milestones. In case we had TLS certificates which their expiration date goes over the distrust date, we will help them to renew them under the new hierarchy. If the date proposed for extension is accepted, no actions will be needed.

What tools, communications, and dates will be involved in subscriber migration?

Most clients are public administrations of the Basque Country. Quarterly meetings are held with them and this issue has been added to agenda.

Contingency Plans

What happens if the new root is not accepted or is delayed?

We will provide the subscribers a full detailed communication to allow them substitute those certificates in timely manner.

What risk mitigations or alternative solutions are you preparing?
Communication plans with th highest detail possible are being prepared to inform subscriptors and most of them are aware of it.
Justification for One-Year Extension

Provide clear reasoning why 6 months is insufficient.

Izenpe TLS certificates are widely used by the whole Basque public Administration, which means Departments such as Justice, Health, Homeland and so on may cause them certain disruption.
With a 6 months extension (October 15th 2026), a migration of TLS certificates will have to be done as certificates issued after this September 15th (our certificates last 13 months), will expire after said extension. We are planning to be able to issue  new certificates by September 2026 but in case there is any problem, we could go beyond that date and would not be reasonable to ask for new delay.
Another small detail is that if the current Root had been issued just a few days later, it wouldn't be necessary to ask for this time extension as the trust bit would have been removed on April 15th, 2027.

Identify which project components require the full year duration

It is not a matter of projects which may need a full year but when we will be ready to issue TLS certificates under the new CA Root. We could shorten the 1 year extension by some months if we issue certificates on September 2026.

Explain any flexibility or earlier milestones (or hurdles to overcome) that might enable earlier removal.

Once the new Root is enabled and updated in browsers of our clients (Corporate clients with controled updates) we could handle the update of the "old" certificates within a month.
Flags: needinfo?(d-fernandez)

Thanks for this updated information. Beginning 2025-07-15, please post monthly updates to this bug, covering:

  • Any changes to your timeline or milestones
  • CCADB submission status
  • Audit progress or delays
  • Subscriber communication or migration progress
  • ACME deployment and validation status
Whiteboard: [ca-extension-requested] Next update 2025-07-15

#1 Monthly update regarding postposal of Izenpe Root CA TLS trust bit removal.

  • The following milestones, will suffer a delay:

    • 2025-06-xx - New PKI Software in the production environment.
      The new estimated date will be 2025-07-31.

    • 2025-07-xx - CCADB request submission.
      The new estimated date will be 2025-08-14.

  • Although not all the items needed to submit an inclusion request to CCADB can be filled, we will have most of them prepared for the estimated date.

  • Audit process was performed last October, no changes at this point will be necessary until next October.

  • ACME deployment will be tested as soon as the production environment is fully operational which will happen in the following days.

#2 Monthly update regarding postposal of Izenpe Root CA TLS trust bit removal.

  • The following milestones, are suffering a delay:
    • 2025-06-xx - New PKI Software in the production environment.
      The new estimated date will be 2025-09-30.
    • 2025-07-xx - CCADB request submission.
      The new estimated date will be 2025-12-15.
  • Regarding CCADB submission status, we will postpone it until December. The main reason is that the software is not fully compliance with the latest requirements by Chrome Root Program Policy which states that ACME solutions must support Renewal Information. By november, a new version of the software will be released that will fix this.
  • On the other hand, foreseeing that all the milestones will not be reached in time and to ensure a continuity in our issuing process, Izenpe has disclosed a public tender to provide TLS certificates to Izenpe.
  • Once the public tender is awarded, these will be the following milestones:
    • 2025/10/03. Communication to subscribers about the new situation and the next steps to give to renew all the certificates which expiring date goes beyond 2026/04/15.
    • 2025/12/15. Finishing integration with the new provider's API.
    • 2026/01/15. Begin issuing certificates with the new provider's hierarchy.

#3 Monthly update regarding postposal of Izenpe Root CA TLS trust bit removal.

  • This week the public tender will be awarded. Once it is done we will proceed with the previously mentioned steps in the #2 monthly update.
  • At the same time we still work with the last previously scheduled dates.

Could you please provide an update? We understand that a public tender has been awarded. Thanks!

Flags: needinfo?(d-fernandez)
Whiteboard: [ca-extension-requested] Next update 2025-07-15 → [ca-extension-requested]

While the circumstances that led us here were avoidable, in an effort to minimize breakage and potential disruption to Spanish government services and relying parties due to the planned removal of CN=Izenpe.com,O=IZENPE S.A.,C=ES from the Chrome Root Store, Chrome will instead phase-out this root using the SCTNotAfter feature.

The SCTNotAfter constraint will be set to April 15, 2026, the root’s original scheduled term-limit removal date. This means all existing unrevoked certificates issued and logged to CT prior to that date will continue to be trusted until their natural expiration.

It is our opinion that rather than continuing to issue Subscriber certificates whose validity extended past the term-limit removal date (i.e., 4/15/2026), Izenpe should have begun to gradually reduce certificate validity to avoid breakage at the time of removal, which was first communicated in 2023.

We encourage Izenpe to prioritize the transition of Subscribers to the solution presented in Comment 5.

Izenpe urgently needs to provide an update on its migration status, including any changes to the previously stated timelines and milestones. Additionally, if warranted, Izenpe should respond to Chrome’s recent decision to apply an SCTNotAfter date of 2026-04-15 to the Root CA where CN=Izenpe.com, especially given that Mozilla will likely to apply a similar distrustAfter date of 2026-04-15 for this root.

Hi,
first of all, thanks for this approach. This extra time will ease our transiction to the new Certificate provider. Right now, our efforts are focused on the integration between our TLS certificates web application and the new TLS provider's API to be able to reach the following milestones.
Following the previous list:

  • Regarding the new Izenpe PKI infrastructure and the new Root submission to CCADB:

    • 2025-12-xx The last version of the PKI software will be released and will comply with the last ACME requirements.
    • 2026-04-xx Once migration to the new TLS certificate provider API ends, we will resume the actions for inclusion of the new Izenpe Root.

  • Regarding the new Certificate Provider and integration development:

    • 2025-07-24 Izenpe discloses a public tender to provide TLS certificates to Izenpe.
    • 2025-09-10 Izenpe's new TLS certificate provider is awarded.
    • 2026-01-15 Start issuing DV certificates with the new TLS provider and stop issuing DV certificates under Izenpe's one.
    • 2026-02-02 tart issuing OV certificates with the new TLS provider and stop issuing OV certificates under Izenpe's one.
    • 2026-02-16 Start issuing EV certificates with the new TLS provider and stop issuing EV certificates under Izenpe's one.

  • Regarding comunication to subscribers.

    • 2025-11-19 A massive meeting will be held with subscribers where the situation will be explained (new web, new hierarchy, validation methods...) and suggestions will be welcome.
    • 2026-01-15 New certificates under Izenpe PKI will have a 200 days duration to prioritize issuance under the new CA provider.
Flags: needinfo?(d-fernandez)

Hi,
as we have planned, from now on, any TLS certificate issued under Izenpe's CA will last 195 days. We expect to start issuing all new certificates through our current reseller.
At the same time, we still continue our efforts to include our new root once we migrate the current EJBCA version to the last one.
Regards,

Mozilla intends to proceed with a distrustAfter date of 2026-04-15 to the root certificate CN=Izenpe.com,O=IZENPE S.A.,C=ES, aligned with Chrome’s SCTNotAfter constraint and previous communications. Izenpe should already be migrating its subscribers and performing its transition activities accordingly to ensure that no disruption occurs as a result of the distrustAfter date. We've opened a bug for this - Bug #2017322. We'll be closing this bug on or before 4/15/2026.

You need to log in before you can comment on or make changes to this bug.