1715929 - Sectigo: Incorrect EV businessCategory

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Tim Callan]

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

On our SSL abuse line we received a report of two EV certificates with erroneous businessCategory information. We received the report June 3 at 11:17 pm.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

All times Eastern Daylight Time

June 3, 11:17 pm
Inbound report received.

June 4, 1:59 am
Investigation concluded and misissuance confirmed.

June 8, 4:56 pm
Both certificates revoked.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

These appear to be isolated incidents of human error. We have searched our corpus of certificates and can find no other instances of this problem.

See part 7 for how we are addressing this problem going forward.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

The two certificates were issued June 12, 2019 and June 26, 2019.

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

https://crt.sh/?id=1570797855
https://crt.sh/?id=1614408593

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

We have to attribute these to human error. At issuance time (and still today) we had no check to compare businessCategory against other potential indicators like serialNumber.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

Our Guard Rails project includes a roadmap item to check for logical conflicts in the serialNumber and businessCategory fields and disallow issuance until the conflict is resolved. Please see bug #1715024 comment #0 for more information on Guard Rails.

Comment 1

[Ryan Sleevi]

(In reply to Tim Callan from comment #0)

We have to attribute these to human error. At issuance time (and still today) we had no check to compare businessCategory against other potential indicators like serialNumber.

Can you help me understand how this response is different than the problematic response called out in https://wiki.mozilla.org/CA/Responding_To_An_Incident

For example, it’s not sufficient to say that “human error” of “lack of training” was a root cause for the incident, nor that “training has been improved” as a solution.

Our Guard Rails project includes a roadmap item to check for logical conflicts in the serialNumber and businessCategory fields and disallow issuance until the conflict is resolved. Please see bug #1715024 comment #0 for more information on Guard Rails.

So both this and Bug 1715024, Comment #0 seem to be a bit piecemeal in disclosing what Guard Rails entails. That is, there's a high level abstract concept, but very little details, or at least, only specific details to specific incidents.

Can you provide a more detailed description of all the checks you are adding as part of this project?

Comment 2

[Tim Callan]

(In reply to Ryan Sleevi from comment #1)

Can you help me understand how this response is different than the problematic response called out in https://wiki.mozilla.org/CA/Responding_To_An_Incident

For example, it’s not sufficient to say that “human error” of “lack of training” was a root cause for the incident, nor that “training has been improved” as a solution.

Let's look at the full passage from the Mozilla Wiki:

[I]t’s not sufficient to say that “human error” of “lack of training” was a root cause for the incident, nor that “training has been improved” as a solution. While a lack of training may have contributed to the issue, it’s also possible that error-prone tools or practices were required, and making those tools less reliant on training is the correct solution.

That's the model we followed here. We said it was human error. We stated why our systems made that error possible. We stated the programmatic solution we plan to put in place so that human error is no longer possible in this instance.

Comment 3

[Tim Callan]

(In reply to Ryan Sleevi from comment #1)

Can you provide a more detailed description of all the checks you are adding as part of this [Guard Rails] project?

These aren't fully defined by any stretch of the imagination. These checks are very modular, so we can be implementing some even as others are undefined or even unimagined. We also can be, and are, agile in this development, so priorities likewise can be fluid. Some of the checks on the roadmap are listed below, but I want to be clear that we still view the exercise of examining fields and how to build programmatic checks to be largely in front of us.

Here are the ground rules we laid out for potential checks to consider:

• For every field we should specify the rules for contents to the fullest degree that we are able
• These rules should leave as little wiggle room as possible without limiting the ability to populate the field with valid and appropriate data
• Rules can be proscriptive (e.g. “no numerals in this field”) or prescriptive (e.g. “this field must contain at least one numeral”)
• Logic can be as complex as we want, so long as we can articulate it clearly and be confident it is right
• In the event that alternatives exist, we can choose to programmatically eliminate some data options so long as other equally good options will always be available. For example, some states in some countries have more than one short name. We could choose to eliminate all but one of these short names per state if that served a purpose for us.
• Preferably rules are hard blocks (if it doesn’t meet these conditions, it doesn’t issue) but we can imagine rules that are almost always right but not necessarily always. For these we could implement some kind of escalation/override functionality instead. (An example using deprecated fields [but still illustrative]: If the Street address contains the correct postal code, it is most likely wrong. But it could be, by sheer coincidence, that that particular address is the same as the local postal code. In that case we would be able to confirm that in the documentation and issue the cert.)
• It’s perfectly permissible to implement more than one rule for the same field, or to construct more than one idea into a single, complex rule. It’s also permissible for us to think of these rules individually without having the whole thing worked out, and it’s even permissible to implement rules and then later update them or add new ones to make the field’s contents more restrictive.
• All else being equal, prioritize in these ways:
  • Prioritize fields that appear in many certs over fields that appear in few (e.g. EV-only fields are lower priority than fields that appear in all OV certs)
  • Prioritize fields that we deem to be higher risk of missuance over those that appear to be lower risk
  • Prioritize easy wins over difficult hairy projects

Some examples of candidate checks include:

• Require at least one letter in localityName
• For all fields except postalCode, check for strings exactly matching the postal code format in this country
• Where possible, ensure registration numbers match formatting expectations
• Limit allowed character set for all fields to that country's languages
• For government-issued domains, ensure the business category is Government entity

There is much more to investigate. This project is in early days. As we respond to bug reports and our own investigation of our corpus of certificates, these will drive candidate checks. We also can learn from other CAs' misissuance reports.

Comment 4

[Ryan Sleevi]

(In reply to Tim Callan from comment #3)

We also can learn from other CAs' misissuance reports.

Indeed, it's precisely this that I'm trying to make sure other CAs can learn from. There's obvious benefit to what you're doing, and ostensibly, what CAs were expected to already have been doing. I appreciate that you've provided a few examples, but I think as this progresses, it benefits from greater transparency here. This provides assurances to the community (in terms of seeing how Sectigo looks at things systemically) and provides opportunities for the ecosystem to improve. In particular, the goal is to ensure these things can be reliably implemented.

For example, "Where possible, ensure registration numbers match formatting expectations" is, well, a bit devoid of useful meaning. You can see on other CA incidents that CAs have been asked to disclose the "Where isn't it possible", as well as disclose "How do you determine formatting expectations".

The goal here is understanding the process.

This also then speaks to Comment #2. I acknowledge that you

stated the programmatic solution we plan to put in place so that human error is no longer possible in this instance.

But that doesn't really speak to the how or why these issues happened, which is what the question is asking. In particular here, the questions that arise are "Are these things that Sectigo should already have been doing?" (e.g. were they reasonable expectations of the community, were they disclosed in other CA incidents, are they necessary to meet the compliance obligations of the BRs) and trying to understand "Why is Sectigo only doing these now, as part of incident response?"

These are the systemic issues that the human error is just a manifestation of. Or, put differently, human error happened, but the goal is to understand why it was possible for human error to result in a situation like that. I understand you're looking to prevent this particular incident going forward, but that's also why I asked about what the guard rails project entails: trying to understand what systemic solutions are looking at, and how systemically things are addressed.

Put differently: the concern that Comment #1 was trying to express was one of transparency and thoroughness. Comment #0 feels a little perfunctory, but lacked a detailed introspection on the incident (after all, it's just two certs), or failed to look how to generalize that in a level of detail that would benefit other CAs.

Comment #3 is useful for capturing the principles you're using to evaluate these, and that is the sort of useful detail that should be included going forward, because it helps better understand how the CA approaches compliance. Where it's perhaps less detailed is concrete results and concrete timelines, the latter of which is explicitly required as part of the incident report, and fails to provide detail at a level that other CAs could implement (at least, with respect to some of these).

This isn't to rake you over the coals, but rather, to highlight the expectations: that compliance is less a competitive zero-sum game of winners and losers, but about how we all learn from incidents and share knowledge and good practices on how to prevent them, for all CAs.

Comment 5

[Tim Callan]

(In reply to Ryan Sleevi from comment #4)

I think as this progresses, it benefits from greater transparency here. This provides assurances to the community (in terms of seeing how Sectigo looks at things systemically) and provides opportunities for the ecosystem to improve.

We can and will commit to sharing details of the specific “guard rails” as they are implemented, with their logic spelled out in plain English. The reason we can’t offer that now is because this logic needs to be vetted, combined, refined, and occasionally rejected.

Let’s take an example. The fifth bullet in comment #3 advances the idea of requiring that the business category be listed as Government entity when a government-issued domain is in the cert. Seems like a no-brainer, right? To test this idea we started looking at our own issued certificates with government domains. We easily found multiple examples of companies with .gov in their domains what were legitimate private entities.

• https://crt.sh/?q=1677671559. This certificate issued for, among others, e-lindsey.gov.uk and sholland.gov.uk belongs to a British Local Authority Trading Company (LATCo). This company is the result of a partnership formed by multiple District Councils to create a private entity to run certain services. It should be, and is, categorized Private Organization.

• https://crt.sh/?q=3065386745. This certificate for ciac.gov.co is issued to a business joint-stock company in Colombia which should be, and is, categorized Private Organization. It is an aerospace manufacturer and maintenance services provider with ties to the Ministry of National Defense.

• https://crt.sh/?q=2115374342. This Turkish certificate for tarsim.gov.tr belongs to a business joint-stock company that provides insurance services including premium collection, damage assessment, compensation payment, finding reinsurance collateral and statistical studies, advertising, and fund management. Its seven member Board consists entirely of representatives from government ministries. It should be, and is, categorized Private Organization.

That is enough for us to conclude that the idea as originally conceived is not to be. It at least will need refinement or reinvention if we can salvage it at all. We just received these research results on Friday, so I don’t yet know what the next step will be for this particular idea.

I hope this is illustrative of the exploratory nature of this process. When a rule is specified, it goes into the roadmap. We mostly prioritize the controls that we judge to eliminate the most misissuance risk. We make this call based on a mix of how many certs this control will impact, how risky the field content is that is being controlled, and whether or not the control would have prevented past instances of known misissuance.

"Why is Sectigo only doing these now, as part of incident response?"

We are not. The Guard Rails initiative and a number of other big projects set to drive down misissuance rates (the biggest of which we list in bug #1714628 comment #4) all were in the works before this current flurry of bugs.

It happens that each reported incident exposes a weakness in our systems and processes, and if we can find a way to address that weakness, we will be better off. As I stated above, if a proposed control would have prevented a known issue, that raises its priority. After all, this is not hypothetical. Misissuance occurred.

So it shouldn’t be surprising that we look at these incidents with an eye to how to put a programmatic, preventative measure in place. And it shouldn’t be surprising that we discuss those intentions in our public response to the incident.

(after all, it's just two certs),

I have no recollection of being dismissive of any incident because the number of affected certificates was low. In bug #1714193 we removed a partner’s RA status because of two certs.

Now, we may alter our priority decisions based on how much misissuance actually occurs. If one control would have eliminated two misissued certs in the past two years and another would have eliminated 200, you can bet I’ll be asking for latter before the former. But this is not to belittle the importance of any issuance problem; rather it’s a healthy product of prioritizing the releases that will reduce risk the most.

Where it's [comment #3] perhaps less detailed is concrete results and concrete timelines, the latter of which is explicitly required as part of the incident report, and fails to provide detail at a level that other CAs could implement (at least, with respect to some of these).

Per the above, I don’t see a way to provide detail that is not presently known or a timeline for features we’ll approach on a highly agile basis. Instead let me reiterate my proposal from the top of this comment. We will publish real-English descriptions of all our Guard Rails programmatic controls as they go into place. We can post them as comments right here in this bug. I know that decision will likely keep this bug open for the remainder of the year or beyond, but it will give us a single repository for this information. That’s better than having them scattered around in whatever incident report happens to be open at the time.

Comment 6

[Tim Callan]

In the process of investigating the .gov programmatic check detailed in comment #5, we discovered two certificates with .gov in their domain names and incorrect businessCategory of Private Organization. The earlier was issued April 30, 2019 and the latter May 14, 2020. We revoked both certificates today, June 23, at 4:13 pm EDT.

https://crt.sh/?id=2808666638
https://crt.sh/?id=1432566423

Comment 7

[Tim Callan]

(In reply to Tim Callan from comment #5)

We will publish real-English descriptions of all our Guard Rails programmatic controls as they go into place. We can post them as comments right here in this bug.

We have no new Guard Rails deployments to report since the last comment. For the moment we have put our engineering focus into our big DCV overhaul as covered in bug 1718771.

Comment 8

[Ryan Sleevi]

Thanks for the update, Tim.

I think you've made it clear Sectigo has a lot going on, and certainly, I think you're right for focusing on the recently discovered issues around DCV, because those pose the most immediate risk to users.

In terms of understanding this issue, it seems in root causes, we're at:

• Mistakes were made by humans
• Mistakes were possible because a lack of automated checks/controls
• A lack of automated checks/controls is an acknowledged systemic issue being resolved

In terms of resolving this:

• Comment #0 stated:

  Our Guard Rails project includes a roadmap item to check for logical conflicts in the serialNumber and businessCategory fields and disallow issuance until the conflict is resolved. Please see bug #1715024 comment #0 for more information on Guard Rails.

  but that doesn't give a timeframe or deliverable for that.

• Comment #7 acknowledges:

  We have no new Guard Rails deployments to report since the last comment.

So my questions are:

• What's the timeframe for the roadmap item for the Guard Rails project to check this conflict?
• If the timeline is unclear (because of the DCV issues being worked on), what's the timeframe to the timeframe?

I realize that we might have to take this week-by-week, but that's certainly not ideal, because we want CAs to make progress on all issues and incidents. I'm aware that Sectigo has some very severe incidents in play, and I'm not wanting to detract from getting those resolved, but there are definitely some obvious negative incentives if CAs get more time to resolve incidents the more incidents they have :)

Comment 9

[Nikola Maksimovic]

(In reply to Ryan Sleevi from comment #8)
Hi this is Nikola Maksimovic and I am VP Engineering at Sectigo.

A unit test of the API seems like it would miss this - because the companyNumber would be provided - so perhaps I'm misunderstanding how your tests are actually structured.

To address this need, we have created a suite of integration tests containing multiple steps which gave us the opportunity to localize the problem to an interaction between modules, rather than a module itself.

We used this test structure:

Precondition: Fixture created (input data prepared, both Company Registration Number and Date of Incorporation or Registration are provided)
Step 1: Place an order
Assert: Was the order placed?

Step 2: Validate the order
Assert: Was the order validated?

Step 3: Issue and collect Certificate
Assert: Was Certificate collected?

Step 4: Parse Certificate
Assert: Was Certificate Subject Serial Number equal to companyNumber?

With minor modification of the fixture, we check another scenario when Date of Incorporation or Registration is provided only:

Precondition: Fixture created (input data prepared, Date of Incorporation or Registration is provided, Company Registration Number is omitted)
Step 1: Place an order
Assert: Was the order placed?

Step 2: Validate the order
Assert: Was the order validated?

Step 3: Issue and collect Certificate
Assert: Was Certificate collected?

Step 4: Parse Certificate
Assert: Was Certificate Subject Serial Number equal to dateOfIncorporation?

We also check the negative scenario where neither the comanyNumber nor dateOfIncorporation is included by validation:

Precondition: Fixture created (input data prepared, Date of Incorporation or Registration is omitted, Company Registration Number is omitted)
Step 1: Place an order
Assert: Was the order placed?

Step 2: Validate the order
Assert: Was the order validated?

Step 3: Issue and collect Certificate
Assert: Was Certificate collected?

In this case the test would fail at Step 3 as certificate issuance could not occur.

It's unclear to me which of these steps should have failed. It sounds like the issue was in your database storage of validated fields, which if I understand from your description, happens before the "order creation" of the test.

As it can be seen from the test flow, the fourth step should have failed where certificate content is checked. You are right, the initial problem was introduced by a coding bug which resulted in the failure to obtain the Registration Number from the appropriate database. At the same time, this test framework is functioning separately from the software mentioned and we create and treat the test input data independently from the storage caused the initial bug.

Figuring out where in the pipeline inputs are validated is important, and for me, the biggest ambiguity is trying to figure out where 1 is describing an "internal API" or an "external API", and where the data validation is performed.

As this test is meant to replicate the full user experience, step 1 is describing any of our available interfaces such as public APIs, UI, or ACME. We place the order with a set of input parameters taken from the test fixture and validate the success of the step (implementation details depend on the interface we used). It is worth mentioning that any step relies on the state stored by the previous step (if applicable) and the fixture created as a precondition to the test. The flow mentioned allows us to validate data and check the population of the certificate profile in the final fourth step, making sure content is set with the same values as the order had been placed with.

Comment 10

[Ryan Sleevi]

Thanks Nikola. I believe your Comment #9 was meant for Bug 1712120 :)

Comment 11

[Tim Callan]

(In reply to Ryan Sleevi from comment #8)

• What's the timeframe for the roadmap item for the Guard Rails project to check this conflict?

This morning we slotted this feature for our July 18 release window.

Comment 12

[Tim Callan]

I’d like to follow up with some detail on the disadvantages of public commitments to specific Guard Rails features and dates too far into the future. This is an important point as it’s integral to how we’re managing this project to optimize results. The reality is that this project is a bunch of mini-projects, all of which are agile and all of which to some degree are exploratory.

Comment 5 gives an example of how the exploratory nature of these bugs makes it difficult to offer a hard timeline. In that case the basic idea had to be rewrought. The current candidate, in plain English, looks like this:

• If any SAN in an EV certificate contains a .gov domain, check to see if the businessCategory is Government Entity.
• If yes, the certificate passes this check.
• If no, hold issuance pending review and approval by the validation escalation team.

But I have another example for you, which is my favorite poster child, State/Locality exclusivity.

The basic idea here is that we would rather include stateOrProvinceName than localityName in a certificate since the former comes from a locked-down list and the latter does not. The opportunity for error is simply much, much lower in the stateOrProvinceName field. Today it turns out that we very frequently include BOTH fields, which is the opposite of what we’re going for.

The plain English logic goes like this:

• If this country has State values available in our lookup table, stateOrProvinceName must be present.
• If stateOrProvinceName is present, localityName must be absent.
• If stateOrProvinceName is absent, localityName must be present.

Pretty simple, right? Should be cake. Turns out not so much.

As we’ve been specifying this feature, the edge cases keep coming up. What happens for a customer where we can reuse organization details? What if these details include locality but not state, do we allow it or send it back for reauthentication? What happens if more than one validation for this organization has occurred in the allowed timeframe and they have different sets of information included? What is our mechanism for checking for that? Does any of these decisions change if it’s a renewal as opposed to a new certificate order? How about a replacement?

Or what happens when the QIIS or QGIS contains locality information but not state information in a country that allows this field? How do we redefine our process to account for this scenario and how do we update our software systems to support this process change?
And who knows what we’ll come up with tomorrow? We are working through these questions and I’m sure at the end we’ll be able to implement this “simple” feature to the benefit of issuance quality, but the path to getting there is more time consuming than it seemed at first glance. At this point we don’t even have an estimated release date. Too much is unclear.

Now, as we realized this feature would not be a quick win in July, it left room to squeeze in something else if that something turned out to be genuinely simple. By pure coincidence we settled on enforced matching of businessCategory and serialNumber for government entities, which started this whole bug. The plain-English logic is:

• If serialNumber equals Government Entity then businessCategory must equal Government Entity. Otherwise block issuance.

We have examined it and sized it and are confident this one is genuinely simple. We have a ticket open and plan on deployment July 18.
All this goes back to why I am cautious about our hard commitments for these programmatic checks. The trend is clear. The vision is unambiguous. But the specifics are highly fluid. Should new bugs come in, they might reveal Guard Rail needs that trump what we already are planning. As we investigate the specifics of each programmatic check, our sizing estimate may change. Once we’re able to return to proactive research of our own corpus of active certificates for misissuance, we surely will identify Guard Rails checks worth putting in place. And as we continue to follow the industry and other CAs’ activity, we may adjust our priorities.

I hope this explanation makes it clear this fluidity isn’t due to the negative incentives you mention in comment 8. Rather, it’s the healthy by-product of our attempt to maximize how much risk reduction we can fit in and how soon. It won’t be in the interest of certificate quality if we go too far in forcing rigid schedules on these programmatic checks.

Comment 13

[Tim Callan]

(In reply to Tim Callan from comment #12)

The basic idea here is that we would rather include stateOrProvinceName than localityName in a certificate since the former comes from a locked-down list and the latter does not. The opportunity for error is simply much, much lower in the stateOrProvinceName field. Today it turns out that we very frequently include BOTH fields, which is the opposite of what we’re going for.

The plain English logic goes like this:

• If this country has State values available in our lookup table, stateOrProvinceName must be present.
• If stateOrProvinceName is present, localityName must be absent.
• If stateOrProvinceName is absent, localityName must be present.

Our code for state/locality exclusivity is checked in and in QA right now. We will announce when it is deployed.

It’s worth noting that we expect state/locality exclusivity to have the pragmatic effect of entirely removing the localityName field from our public SSL certificates.

We now have a table of discrete values available to the stateOrProvinceName field. Every country in the table contains states, meaning there will always be a predictable value available for this field. Because the BRs require at least one of stateOrProvinceName and localityName, so long as we always include stateOrProvinceName, we never need to include localityName. We plan to require stateOrProvinceName to be present for all new certificates except replacements, which means we will never require localityName for new, full-term certificates.

For the replacement certificates exception, the point behind a replacement certificate is immediate issuance with all the same organization details and the same expiry date. Because these certificates may have been issued without a stateOrProvinceName field initially, forcing them to have this field now can hold up replacement while we undergo new validation and cause production outages. For those certificates we can allow the localityName field to remain and the risk of a mississued certificate will be no greater than if no replacement had occurred. As the existing certificate base expires out, this risk will approach and then reach zero as the number of existing certificates with localityName diminishes.

Removing localityName entirely in combination with our stateOrProvinceName lookup table would have eliminated bug 1714193, bug 1715024, bug 1710243, and bug 1548713.

Comment 14

[Tim Callan]

In last weekend's window we deployed enforced matching of serialNumber and businessCategory for government entities.

Comment 15

[Tim Callan]

In addition to the government businessCategory-registrationNumber match we just deployed (see comment 14) our upcoming release targets are:

August 1: Forced escalation for postalCode match to other fields

• Compare the organization’s postalCode to the contents of all other variable fields
• In the event of a match, hold issuance for review by escalation team

August 1: Match businessCategory to organizationName

• Parse organizationName value for known markers of organization type such as Inc. or LLC
• Compare to expected businessCategory value
• Block issuance in case of mismatch

August 15: stateOrProvinceName-localityName exclusivity

• We have described this feature in bug 1715929 comment 12.

As features are delivered or added to the list or expected dates change, we will provide updates here.

Comment 16

[Tim Callan]

Attachment: business_type_by_country_with_org_name_20210725.xlsx

Comment 17

[Tim Callan]

We managed to fit both forced escalation for postalCode match and match businessCategory to organizationName into this last weekend's deployment, so they are both live. The matching list for the second is a first-release list that we will add to over time. I have added it in the file https://bugzilla.mozilla.org/attachment.cgi?id=9233126&action=edit. This should be pretty easy to read. If the organizationName field, ends in one of these extensions, the businessCategory must match as indicated in the spreadsheet. Otherwise block issuance.

stateOrProvinceName-localityName exclusivity remains scheduled for August 15.

Comment 18

[Tim Callan]

(In reply to Tim Callan from comment #17)
Okay, I messed that up. It's attachment 9233126 [details].

Comment 19

[Tim Callan]

We expect to be posting new Guard Rails updates for quite a while, including:

• Descriptions of new programmatic checks
• Updates on deployment of checks
• Lists of upcoming checks with target dates

Exactly how long it will make sense to issue these regular reports is unclear, but certainly we have months of reporting ahead of us. While we originally simply started appending this information to the bug in which the main dialog about the Guard Rails project had occurred, now that we’re into the regular cadence of updates it is strange to be providing this information in a bug called “Incorrect EV businessCategory” that initially was about an entirely different matter.

Therefore we suggest a new bug to track Guard Rails projects, similarly to the way we created new bug 1718579 to carve out Manual DCV from other issues we were addressing in bug 1712188.

We will give the community a few days to consider and provide feedback on this idea. In the absence of any feedback to the contrary, we will create a new bug next week to track our Guard Rails initiative. We will immediately mark it closed as the purpose will be informational rather than to track an ongoing issue.

Comment 20

[Mathew Hodson]

(In reply to Tim Callan from comment #19)

We will give the community a few days to consider and provide feedback on this idea. In the absence of any feedback to the contrary, we will create a new bug next week to track our Guard Rails initiative. We will immediately mark it closed as the purpose will be informational rather than to track an ongoing issue.

If the proposed informational updates aren't associated with any incident, then maybe it would be better to post them on the [e-mail] mailing list instead.

If they are related to localityName (bug 1720744)

Comment 21

[Mathew Hodson]

(In reply to Tim Callan from comment #19)

We will give the community a few days to consider and provide feedback on this idea. In the absence of any feedback to the contrary, we will create a new bug next week to track our Guard Rails initiative. We will immediately mark it closed as the purpose will be informational rather than to track an ongoing issue.

If the proposed informational updates aren't associated with any incident, then maybe it would be better to post them on the [e-mail] mailing list instead.

If the updates are related to localityName (bug 1720744), registration numbers (bug 1721271), stateOrProvince (bug 1715024), or another incident, then they should be in the appropriate bug.

Comment 22

[Tim Callan]

(In reply to Mathew Hodson from comment #21)

If the updates are related to localityName (bug 1720744), registration numbers (bug 1721271), stateOrProvince (bug 1715024), or another incident, then they should be in the appropriate bug.

Of course we will report on updates relevant to any bug in that bug. The point is to have a single ongoing thread that reports on these programmatic checks we have been calling Guard Rails. Some of the checks already deployed and scheduled for future deployment are independent of any misissuance event, and therefore they don't have bugs associated with them. If we don't have a proper place to report these checks, they simply will go unreported.

Comment 23

[Tim Callan]

We deployed stateOrProvinceName-localityName exclusivity in this last weekend's deployment window.

Comment 24

[Tim Callan]

Per comment 19, we have created bug 1724476 to contain information for Guard Rails moving forward. We intend to include all future Guard Rails updates and discussion in that bug.

Comment 25

[Tim Callan]

The originating incident for this bug is resolved, and this bug evolved into the repository for discussion of our Guard Rails initiative. Now that we have created bug 1724476 to hold that information, we believe this bug can be closed.

Comment 26

[Ben Wilson]

I'll plan closing this incident on or about next Wed. 18-Aug-2021.