Open Bug 1565270 Opened 1 year ago Updated 14 days ago

Telia: Qualified BR Audit Statement

Categories

(NSS :: CA Certificate Compliance, task)

task
Not set
normal

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: kwilson, Assigned: pekka.lahtiharju)

Details

(Whiteboard: [ca-compliance] - Next Update - 1-November 2020)

Telia received a Qualified BR audit statement:
https://support.trust.telia.com/download/CA/Telia-WebTrust-For-CA-SSL-Baseline-With-Network-Security-Report-2019.pdf

The purpose of this bug is to track closure of all of the qualifications, and for the CA to provide incident reports for them.

For each qualification that was already reported in Bugzilla, it is fine to just reference that Bugzilla Bug number, and finish resolving in that bug.

Status: NEW → ASSIGNED

Telia's Incident report of BR findings. Note! Telia has provided the most important information related to all of the qualifications in its public posting since June 2019 here: https://support.trust.telia.com/download/CA/Telia-Public-Response-To-Audit-2019.pdf. Most issues are old ones so that report and fix was done a year ago. Below is the full incident report for all listed DEVIATIONs like instructed by Mozilla:

DEVIATION 1: The Key Usage extension in the root CA certificates of TeliaSonera Root CA v1 and Sonera Class 2 CA is not marked critical and TeliaSonera Root CA v1 certificate’s subject information does not include subject:countryName.

This qualification was old one and discussed already a year ago here: https://bugzilla.mozilla.org/show_bug.cgi?id=1475115. However, in that discussion there wasn't requirement to add an incident report so it is below.

1.1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.
Telia became aware of the problem when Telia applied EV permission for its roots in March 2017. Mozilla commented that Telia Roots have these errors (check https://bugzilla.mozilla.org/show_bug.cgi?id=1322668). At that point it wasn't a problem. Then in June 2018 Telia got BR audit reports and this issue was mentioned as qualification. After that Telia started a process to update its roots. Issue was again in this BR audit report received in June 2019.

1.2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
In April 2001 Telia created its first root CA and the other was created In October 2007. At that time there weren’t any clear requirements for CA certificates. The first CA Browser Forum BR document appeared in 2011 (effective 2012).
In June 2018 Auditor report listed this root CA issue as qualification
In November 2018 Telia created new root CA that is fully compliant. Auditors witnessed the creation.
In January 2019 PIT Audit report proved that new root CA is compliant.
June 2019: Telia wanted to wait for the next full audit results to June 2019 so that Mozilla and others could see that Telia has improved all its processes so that browser vendors could approve the upcoming root application without concerns.
Next step in 3Q2019 is to apply the new root to be trusted in all browsers.

1.3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.
All new CA certificates have been compliant and only CA root certificates created before year 2012 had these problems. Telia needs Mozilla to continue trusting the old roots until the new root is widely accepted.

1.4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.
1.5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.
https://crt.sh/?id=3819
https://crt.sh/?id=989582

1.6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
At the time in 2001 and 2007 when these were created there weren’t any clear requirements for CA certificates. The first CA Browser Forum BR document appeared in 2011 (effective 2012). Telia didn't understand that BR root rules may affect also older root certificates because BR Overview chapter states: "these Requirements apply only to relevant events that occur on or after the Effective Date." None of the BR Effective dates are dated before 2012. Trust anchors in browsers have special treatment which make possible to use exceptional trust rules (like accept otherwise unacceptable SHA1).

1.7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
Telia has always followed latest BR rules. New fully compliant Telia root has been created but it is not yet included in browsers. The new root applications will be provided to browser vendors in Q3/2019. Telia can't stop using its older roots before new root is widely accepted by all browsers.

DEVIATION 2: The subscriber certificates issued by Telia Domain Validation SSL CA v1 contained wrong policy identifier until 25 June 2018.
Because this problem overlapped two audit periods the same qualification is mentioned in 2018 and 2019 audit reports. This incident report was provided a year ago in thread https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/34gZxUX3EBc. Fix was verified by PIT audit Oct 2018.

DEVIATION 3: Many subscriber certificates with email address as an optional subject attribute were not adequately verified
Because this problem overlapped two audit periods the same qualification is mentioned in 2018 and 2019 audit reports. This incident report was provided a year ago in thread https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/5K7LZfIveNQ. Fix was verified by PIT audit Oct 2018.

DEVIATION 4: 13 SSL certificates with inappropriate L value were issued to three different organizations by TeliaSonera Server CA v2. According to the CPS, locality name should be a city name, but in these cases locality name consisted of a country name or organization’s postal office name.
This was already reported and discussed here: https://bugzilla.mozilla.org/show_bug.cgi?id=1551372. There one additional Telia improvement is planned to be done in 3Q2019.

DEVIATION 5: Three weekly security configuration reviews were not properly implemented before Oct 2018. In addition one new weekly review failed for several months.
Because this problem overlapped two audit periods the same qualification is mentioned in 2018 and 2019 audit reports. Mostly this was already reported and discussed here: https://bugzilla.mozilla.org/show_bug.cgi?id=1475115. Fix was verified by PIT audit Oct 2018.

The failed weekly review was a new discovery and its incident report is below.

5.1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.
Telia became aware of the problem when Auditors reported this related to 2019 BR audit.

5.2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
November 2018: Secondary Firewall weekly review was implemented, but its change reports were mostly lost until June 2019
May 2019: Auditors informed that this particular weekly review doesn't work properly. All other reviews were functional.
June 2019: Secondary Firewall weekly review was fixed

5.3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.
This was only a minor reviewing problem and it didn't affect issuing systems or cause threat to CA. There were multiple compensative controls at all times.

5.4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.
None.

5.5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.
None.

5.6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
Telia has mostly automated change review of relevant security configurations. However, there weren't previously checks to verify that all this automation is functional. A reviewing system that caused very few messages (like secondary firewall) was potentially risky to be undetected if failing. Unfortunately failure happened to our secondary firewall monitor so that its failure wasn't detected.

5.7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.
The planned change in 3Q/2019 is to monitor with a new monthly human review the functionality of all implemented reviewing systems.

Thank you for the incident report. It appears that one remediation action remains to be completed:

planned change in 3Q/2019 is to monitor with a new monthly human review the functionality of all implemented reviewing systems.

Please update this bug when that change has been put into place.

Whiteboard: [ca-compliance] Qualified BR Audit → [ca-compliance] - Next Update - 1-October 2019

We have started in August 2019 a new process to manually verify monthly that our automatic reviewing systems are functional. This will finalize actions related to DEVIATION 5 (weekly reviews).

For DEVIATION 4 (Inappropriate L value) we've been investigating new automation for L checking but that solution is delayed. The original idea was to use API against Finnish company register (ytj.fi) but in evaluation it was old fashioned and we didn't like that it would solve the problem only partly (only Finnish L values). That's why we decided to find a more generic API and we've found that in September. We get test account for commercial generic API next week. If tests are OK we are able to automate all O/L/C verifications related to all our target countries (currently Finland and Sweden). The first values will be auto-verified at the end of 2019. In 1Q2020 this should be in full usage. I will update the status later when the full automation is implemented. Until that we continue using the manual pre-approval of all L values. It has been very reliable process after it was implemented and no bad L values have been accepted since.

Pekka: thank you for the update. Please provide regular updates on the progress toward automated O/L/C verifications.

Whiteboard: [ca-compliance] - Next Update - 1-October 2019 → [ca-compliance]

Pekka: It's been four months without regular updates, as mentioned in Comment #5. Do you have updates regarding the EOY2019 and the 1Q2020 progress?

Flags: needinfo?(pekka.lahtiharju)

The new O/L solution is still in development but almost ready. Testing is in progress. Pilot will begin probably in March 2020 and hopefully new Customers will go to the new system before summer. We'll migrate old customers to new system one by one. Full migration will go to 3Q2020. There are lot of changes in this release so there are some risks involved and we'll do testing accordingly.

The new solution will automatically fetch business ID, O, street, postalcode, L and C values when Customer is trying to create OV/EV level certificate and previous verified data is old or missing in our CA database. We may also fetch company phone number from there to be used if Customer representative is new to us and we want to verify that he/she is representing the company. We'll leave the old manual method (manual pre-validation) for the time being as a secondary method because automation can't solve all the cases (e.g. scandinavic letter conversion used or side office address used instead of primary address). All manually inserted values are reviewed monthly by another person.

Flags: needinfo?(pekka.lahtiharju)

Pekka: Thanks for the update. It's unclear what's happened in the intervening four months since Comment #5, because it seems like progress hasn't really been made. Full migration was originally planned 1Q2020 and is now suggested for 3Q2020, with possibility of further slippage.

I'm not fully sure I understand the 'new solution'. Could you indicate where this information is being fetched from? e.g. a national registry of some sorts, using an automated API?

Regarding the manual review, it seems quite problematic to do post-hoc review in 2020. It seems like manual exceptions should require multi-person sign-off before issuing, rather than after-the-fact review. Could you explain some of the rationale behind allowing potentially invalid certificates to be issued and undetected for up to a month? What factors made that the ideal solution?

Flags: needinfo?(pekka.lahtiharju)

There are multiple reasons why the original schedule has delayed.

  1. We decided to go to new registryAPI instead of the originally planned national Finnish registry API. This enables us to validate almost all data automatically instead of just Finnish data.
  2. We decided to combine this data automation solution only to our new database solution. Originally the plan was to integrate the automation to our old database solution. This prevents us developing the same solution twice but will cost couple of extra months to the schedule because there are lot of other features in the new solution also. All new features must be ready before we give the new GUI to Customers.
  3. We lost our key developer in October. New ones have now replaced him.

We'll fetch the data from Bisnode (https://bisnode.com). They collect B2B data from 21 European countries including all 7 Telia home countries. In each country they use national registers as data source. They provide REST API to fetch all this data. They are also a partner with Dun&bradstreet. When Customer is trying to use some Company name and L value we verify that those can be found using the REST API (or its recent stored result or manual data if such was necessary) and all data is owned by the company (Business ID) that the data and person represents.

The current manual method that we have used has been very reliable because verification officer has to store the actual text from the registry that includes the company data. Since we started using Registry data blocks as evidence there haven't been any problems. EV process includes multi-person verification but we haven't seen any requirements for such in OV process. That is one of the main reasons EV is using "Extended validation" and OV only "Standard validation". Multi-person processing would mean extra manual work and delays to OV processing time and that's what we'd like to avoid.

Flags: needinfo?(pekka.lahtiharju)

Thanks. It sounds like you're using Bisnode as a Reliable Data Source, Pursuant to 3.2.2.1, 3.2.2.2, and 3.2.2.7 of the BRs. I think it's questionable whether Dun & Bradstreet represents a Reliable Data Source, so hopefully you have carefully documented the steps used to evaluate Bisnode against 3.2.2.7

Note that the EV Guidelines do not make use of the Reliable Data Source of 3.2.2.7, which is a lower standard than required of the Agency of Registration/Incorporation or QGIS. The recommendation to use multi-party review was specifically for places where human entry was involved, since the underlying issues with entering that information have lead to misissuance.

I'm concerned that you've known there were delays since October, but did not update this issue with an updated timeline. These are the sorts of the delays the CA should be communicating as soon as they know about them, since the ability to deliver remediations is tied to the continued trust in that organization.

I'm setting N-I for Wayne if he has further questions. It does appear the remediations are not only not complete, but not yet widely deployed, and will not be until Q3 2020.

Flags: needinfo?(wthayer)

No further questions, but as commented in bug 1551372, Mozilla expects regular updates from Telia on the progress of migration to the new solution.

Flags: needinfo?(wthayer) → needinfo?(pekka.lahtiharju)

Here is the status of Telia's fully automated subject solution (O,L,C). As part of this Telia will stop using OU because it can't be properly automated.

This is a big project. Telia has divided the new solution to several tracks. Status of each is today as follow:

  1. Commercial agreement: Telia is now very close to sign the agreement to get the official access via API to registry data. Scheduled to be ready next week.
  2. Legal compatibility: Telia is gathering evidence of data and national sources so that new API can pass requirements in BR 3.2.2.7. Finnish and Swedish data is already evaluated to be good in random cases. Full data evaluation (check track 6 also) and data quality for other 4 Telia countries and proper documentation will follow. Scheduled to be ready in February.
  3. Ordering process: The first application to start using this API is Ordering process. Its beta version is now ready for testing but the access to data using our test license has expired. Track 1 has to proceed first. Scheduled to be ready in March.
  4. Approval process. If some data can't be found automatically there is a manual approval process. This development is done together with track 3. Approval process is still partly on development. Scheduled to be ready in March.
  5. Portal process: In the second step also Telia's SSL self-service portal start using the same API. Specification for that should be ready next week. Then its development will start. The biggest schedule risks are related to track 7 features of this. Launch may go to Q3 even though Subject automation parts are ready in Q2.
  6. Migration process: Telia plan is to revalidate currently stored Company information using the API and compare the two data sources (old and new). This will tell a lot about data quality and let us migrate data to new portal. In a way this is part of track 2. Database model is ready. Script for data comparison and migration will be done in February. Tracks 3&4 start using this data in late March.
  7. Other features: Telia can't put the new solution to production before independent process is fully developed and tested. This testing has been going on for several months but there are still areas that are not yet ready especially related to self-service portal. This work will now continue on daily basis. Focus is on features related to application that is launched.
Flags: needinfo?(pekka.lahtiharju)

Thanks for the clear and detailed plans. This is a great example of the communication we're looking for to better understand the work involved and what the risk factors are that may cause delays.

Based on this, I've set an update for the end of February to check in.

If I'm understanding correctly, Track 1 and 2 should be complete, and Tracks 3 & 4 should be nearing completion or at least have meaningful progress. This will help inform if there's risks to delay for Tracks 6, 5, 7 (seemingly in that order?). Since a lot of this is gated on the commercial agreement and compliance evaluation, it's expected that Telia will communicate back with Mozilla if/as they become aware of issues or possible delays.

Flags: needinfo?(pekka.lahtiharju)
Whiteboard: [ca-compliance] → [ca-compliance] - Next Update - 29-February 2020

Status at the end of Feb 2020 and question to Mozilla in track 2. There is 2 week delay because of problems with street data verification (check below)

  1. Commercial agreement: This has delayed because the agreement is now tied to bigger contract between Telia and Bisnode related to other non-related data queries (related to consumer data needed by other Telia services). We have been able to continue using test account so that this project hasn't suffered yet. I expect this to proceed within week or two so that Telia CA will get a separated agreement (we wish that). Nordic winter holidays have also delayed negotiations.

  2. Legal compatibility: We have delays on this track because Bisnode data structure is more complex than estimated related to street&postalcode. Also special characters in company names like '&' have caused problems to our scripts. The hardest thing is to get accurate street address because basic query result only includes postal address, City (locality) and Name (O) value but not visiting address. QUESTION: Is it mandatory to get & store also accurate visiting street address in this automation when street is not used at all in the process? What is the purpose of sentence "the CA SHALL verify the identity and address of the organization and that the address is the Applicant’s address of existence or operation" in BR 3.2.2.1? Or is it enough that we validate that O & L values are correct and that we verify that company status in the register is "active". All other data verification/migration except street&postalcode has been solved. Still only partial evidence exists mainly because problems to fetch real physical street address. There is still one non-tested option to solve this street issue. API has a secondary object group under the company object that probably includes visiting street information because manual web interface can show it. This makes queries more complex but is tested next week.

  3. Ordering process: The first application to start using this API is Ordering process. We have been developing and testing this as planned. still estimated to be ready in March providing that problem related to street information can be solved.

  4. Approval process. If some data can't be found automatically there is a manual approval process. This development is done together with previous track. Approval process is still partly on development. Scheduled to be ready in late March.

  5. Portal process: In the second step also Telia's SSL self-service portal start using the same API. The last specifications have been finished. This is mostly waiting the previous tracks to be finished.

  6. Migration process: Specification now exists how everything is migrated. Migration of data used by tracks 2-4 is waiting how to solve street issue (check track 2). Customer ID, O and address (street, postalcode, city, country) migration is needed.

  7. Other features: Telia can't put the new solution to production before independent process is fully developed and tested. This testing has been going on for several months but there are still areas that are not yet ready especially related to self-service portal. This work will now continue on daily basis. Focus is on features related to application that is launched.

Flags: needinfo?(pekka.lahtiharju)

(In reply to pekka.lahtiharju from comment #14)

  1. Legal compatibility: We have delays on this track because Bisnode data structure is more complex than estimated related to street&postalcode. Also special characters in company names like '&' have caused problems to our scripts. The hardest thing is to get accurate street address because basic query result only includes postal address, City (locality) and Name (O) value but not visiting address. QUESTION: Is it mandatory to get & store also accurate visiting street address in this automation when street is not used at all in the process? What is the purpose of sentence "the CA SHALL verify the identity and address of the organization and that the address is the Applicant’s address of existence or operation" in BR 3.2.2.1? Or is it enough that we validate that O & L values are correct and that we verify that company status in the register is "active".

My interpretation of BR 3.2.2.1 is that the street address must be verified - O & L are not sufficient. I can only speculate that the purpose of this requirement is to ensure that the exact organization has been identified and operational existence for that exact organization has been verified, even if the street address is not included in the certificate.

Whiteboard: [ca-compliance] - Next Update - 29-February 2020 → [ca-compliance]

We succeeded to solve issues related to automatic street data verification and verification of O/L data compared to our old data. We have compared now about 10 % of all data but I think it is good base for reliable conclusion. Result is that this API is reliable source for the planned O/L/ADDR automation. Below is why we think like this:

REST API to Business data
Included countries In Bisnode are: Austria, Belgium, Czech, Republic, Denmark, Estonia, Finland, France, Germany, Greece, Italy, Latvia, Lithuania, Netherlands, Norway, Poland, Russia, Slovakia, Spain, Sweden, Switzerland, UK. At first Telia is utilizing only data from Telia home countries: Denmark, Estonia, Finland, Lithuania, Norway, Sweden. Only those are now verified.

Based on Bisnode data description they handle vast amounts of data every day. It synchronizes information from hundreds of suppliers. Altogether, they have access to 2,000 official data sources that are provided to API users. All information is handled and analyzed in accordance with a number of decision rules. For Bisnode, it is both essential and obvious to safeguard customers' confidentiality and handle sensitive information in a secure manner. Bisnode uses local authority registries. Details are in separate document “BBC Suite – Data Sources created by Bisnode 2019-03-22". That document was carefully verified by Telia and stored as evidence of data quality.

Telia has verified that currently used manually fetched company data matches almost perfectly with the data from Bisnode API. Separate Telia internal Data migration description explains the details and all differences. This comparison covered so far about 10 % of all company data stored to Telia CA database. This data is originated from Finland or Sweden. The rest of the data 90 % is verified little later. Main sources for differences in this comparison were:
• Minor allowed variations to actual name (mainly scandinavic letters or company suffix difference like "AB")
• Company had changed name/address since it was stored manually
• Old stored name was DBA name instead of official company name fetched from API
• Old address was side office instead of main office that was fetched from API

Bisnode Data from Telia countries is trusted based on data source description from Bisnode and because all so far verified data has been correct. Summary is below (minor data sources are complementing the data derived from the main data source explained below):

Finland: Yritys- ja yhteisötietojärjestelmä (YTJ) is used as source. YTJ covers approx. 2 000 000 business Ids (100% of all businesses in Finland - both active and dissolved businesses) and is updated daily from the source to Bisnode Finland. Source website: www.ytj.fi/en/index.html. Coverage: 100 %.
Sweden: Every new company is registered at the source and Bolagsverket sends all the company information to the Swedish National Tax Board (Skatteverket) for completion and updates Bisnode Sweden daily. Source website: https://bolagsverket.se/.
Estonia: Company data is updated daily and is based on official registry for all companies in Estonia – Estonian Business Register. Source website: https://ariregister.rik.ee. Coverage: 100 %.
Denmark: CVR is the Danish Government main business information register and contains information about all companies in Denmark. Note! For small companies it is not mandatory to register in Denmark (turnover of less than DKK 50,000 on a yearly basis). The CRV data is updated/sent to Bisnode Denmark daily. Source website: www.cvr.dk. Coverage: 100 %.
Norway: The Central Coordinating Register for Legal Entities is one of several public registers that is part of the Brønnøysund Register Centre and covers the registration of all companies and other entities in Norway. All entities have a unique organization no. that also is the VAT no. This source covers 100% of all registered entities in Norway and updates are sent to Bisnode Norway daily. Bisnode uses Posten as source to enhance and verify the address data (street addresses). Source website: www.posten.no. Coverage: 100 %.
Lithuania: There is one official registry for all companies in Lithuania: the Lithuanian Business Register. Data supplier: Professional Partner OÜ (via Bisnode Estonia). Source website: www.registrucentras.lt. Coverage: 100 %.

Next Telia will concentrate to verify the rest of the existing company data and to do the phase 3 in above plan. There is about 2 week delay in implementation mainly because of troubles with street data verification.

Thank you for the update Pekka. Please provide another update by the end of March.

Whiteboard: [ca-compliance] → [ca-compliance] - Next Update - 01-April 2020

Status per track at the end of March 2020:

  1. Commercial agreement: Ready
  2. Legal compatibility: Ready
  3. Ordering process: Ongoing, two week additional delay. Exception processes related to automation are more complex than estimated. New estimate to be ready is at the end of April 2020.
  4. Approval process: Like track 3. 3&4 are done together.
  5. Portal process: Ongoing. Target date May 2020.
  6. Migration: Target 2Q-3Q/2020 like originally planned.
  7. Other features: Ongoing.

Can you provide more detail about #3? Also, when were you aware of these delays? In general, updating sooner than later is preferred, so that the community can be sure of the progress, and not just missed milestones.

Flags: needinfo?(pekka.lahtiharju)
Whiteboard: [ca-compliance] - Next Update - 01-April 2020 → [ca-compliance]

Delay of automation on ordering process is related to address data quality of used API data. In API data the address data is structured to one main company object and under that are all separate side office details. Now we have found (based on testing) that in many cases the main object includes only postal address or even box office address when BR requires real physical address. In addition the side office address won't include information which of those is the main office; all are equal. We'd like to use the main office address. We didn't expect this kind of behavior and we are still trying to solve this. E.g. we are creating automatic country based rules which postal codes in each country mean that address is not real physical address. We also try to communicate with the data vendor. The problem came to our knowledge about a week ago when we started extensive testing with real company data. The first tests earlier didn't reveal this because only few data objects have the problem. We want to provide a good customer experience for all from the beginning even if it would delay the project.

Also our new ordering system is using new authentication solution for verifiers. That has delayed because the team responsible for its installation has been extra busy doing critical digitalization support tasks to Finnish government and enterprises related to Corona issue. Corona issue has escalated in Finland within last two weeks. I can't estimate how this team's workload will proceed in April. This installation has been promised to us but we understand if some other tasks get now higher priority. Our test system is already using the new authentication solution and only production installation is delayed.

I'd like to emphasize that this project has already now brought value to Mozilla community because used O&address data objects have been re-verified. We re-documented why each individual value was approved even though it was not exactly the same than retrieved from API (minor variance, scandinavic letters, business name, side office address,...). Our internal rules related to all exceptions were enhanced. And all changed company names or addresses (localities) have removed from approved status even if they were still valid according to BR 825 day validity rule.

Flags: needinfo?(pekka.lahtiharju)

Now it looks like the L automation in Telia ordering system will delay to May. The next phase related to self-service tool will be postponed to June. There are still too many bugs that probably prevent us putting the new software to production in April. It is better for all to use old functional system until we can be sure that there are not essential bugs in the new code.

To compensate the additional delay in L automation Telia has implemented a new second person L verification that is done on daily basis.

Status: same than in comment 21. New automatic L model should be in ordering system in May. Self-service will start in June and bigger migration of it will happen in Q3.

Do you have a more concrete date about when in May? and June?

Flags: needinfo?(pekka.lahtiharju)

This project has still problems: a) We haven't get the new authentication system installed to production because of other projects. Finally last Friday it was installed but its API isn't fully working yet. We continue installation today. b) The user interface still isn't perfect. We have found it is very hard to give clear errors in case of partially bad user input or partially good old data in our internal database. The new algorithm to handle errors is in tests this week.

=>There is a risk that we will postpone ordering system launch to June. If we have to postpone, it will affect also the next phase (self-service) to be moved over summer vacations to August. But we still try to get ordering system ready this week. This is our number one project.

Flags: needinfo?(pekka.lahtiharju)

We will be expecting a status update by next Monday.

Whiteboard: [ca-compliance] → [ca-compliance] - Next Update - 8-June 2020

Unfortunately we couldn't yet solve the problems in the new authentication system. Its vendors work now every day to fix it. Without it we can't put the new L automation to production. There is an unknown problem in its SAML API. After it starts working we'll need about a week to finalize other settings. We hope authentication is solved in the next week so that we could start using automation in June.

Because the ordering phase (#3&4 above) is delayed we have to delay also portal phase (#5 above) to August. Note! We have change freeze period in July because it is summer vacation period in Nordic countries. Only minor updates in July are possible.

The compensative actions mentioned in comment 21 are in use until new automation is ready (a new second person semi-automated L verification that is done on daily basis).

Finally the authentication problem was solved last week. Reason was a bug in the external authentication platform. Because of that our L project got about 8 weeks extra delay. Problem was solved too late to put the new L system to production before Telia's general summer freeze (from midJune to midAugust). We'll start the production in late August. Migration phase will be delayed to start in October so that project should be completed in November 2020. Until that our above mentioned improvements to our current system will take care of good L quality.

We just got the new Webtrust/BR audit results and there weren't any problems related to L any more. Only issue in our audit results was the old problem that our older root CA certificates from year 2002&2007 weren't fully compatible to BR root CA requirements that were published in 2012. Our audit results and our public response to the problem will be published on our web pages within few days on https://cps.trust.telia.com/

Perhaps this bz related to our previous audit could be closed and we may complete the planned L automation without this bz being open to the end? I suppose that L automation isn't a mandatory requirement but an alternate way to further improve L quality.

I think there have been some ongoing problems, and so, if we were to close bug #1551372, I still want tracking and follow up in this bug. I want to see what the status of your efforts are to improve the L system, which according to Comment 27, will not be completed until November 2020. We will need to set a new "Next Update" for this bug. Should that be August 30, September 15, or October 15? Also, meanwhile, aren't the new 2020 audit reports due?

Flags: needinfo?(pekka.lahtiharju)
Whiteboard: [ca-compliance] - Next Update - 8-June 2020 → [ca-compliance] - Next Update - 15-July 2020

Due to holiday season, I will comment this behalf of Pekka Lahtiharju. We would like to propose next update for this bug on September 15. Latest audit result could be found https://bugzilla.mozilla.org/show_bug.cgi?id=1649683

Whiteboard: [ca-compliance] - Next Update - 15-July 2020 → [ca-compliance] - Next Update - 15-September 2020
QA Contact: wthayer → bwilson

Telia has now created the root inclusion application (Mozilla bz and CCADB request) to add new Telia Root CA v2 to Mozilla root store to fix the last remaining audit issue (problem was that Telia's old root certificates don't completely follow BR rules that were published after root certificate creation). Check: https://bugzilla.mozilla.org/show_bug.cgi?id=1664161

The older audit report from 2019 had also the issue that few L values were invalid in audit period 2018-2019. For that Telia has already implemented several improvements discussed earlier in this bz and in the related other bz (and verified by PIT audit). A long-term plan was to improve Telia's L process so that most values are automatically verified against national business registers. That project is still ongoing and its status is this: The L automation project has now proceeded well and there are no known problems. Extensive testing has been done. Production deployment of the first phase (Order system) will be done at the end of September 2020. From that point all incoming SSL certificate orders to Telia get automatically verified data to attributes: street,postalcode,L,O,C. Data is fetched based on given organization name or business ID number. Customers will still have the option to put the values manually but in that case (if the values are new) there is manual verification by two Telia Employees (one at order approval and the other next business day to do verification). Automatically fetched values are cached for 60 days. Manually approved values are valid for 820 days.

Implementation of the second phase is to extend the same automation for Telia's current certificate portal customers. Now they use only manually pre-validated data (street,postalcode,L,O,C) but the new automation enables them to easily use also company information of their affiliates or other authorized companies. Then only data authorization is manually verified by CA. Actual data is automatically verified. The schedule for the second phase is now 4Q2020. Then the automation project will end.

Flags: needinfo?(pekka.lahtiharju)
Whiteboard: [ca-compliance] - Next Update - 15-September 2020 → [ca-compliance] - Next Update - 1-October 2020

Telia CA automation related to attributes: street,postalcode,L,O,C has been put to production in Telia SSL ordering system. This step is completed.

The last step of the project is to do the same in Telia SSL portal system. There are still some open tasks related to other features of the portal (e.g Search and Batch jobs). Schedule is to finalize the portal system in October-November 2020 and then do customer migrations gradually in November - January 2021.

Whiteboard: [ca-compliance] - Next Update - 1-October 2020 → [ca-compliance] - Next Update - 1-November 2020
You need to log in before you can comment on or make changes to this bug.