(In reply to Ryan Sleevi from comment #13)
.. as I shared in the related bugs, I want to encourage Sectigo to be clearer and more detailed in its reports, to help build confidence that it's not just weekly updates, but that meaningful progress is being made. Obviously, sharing who the candidate is would definitely be on the /oversharing/ side, but as it relates to some of the technical controls and investigation on the dependent bugs, there's a lot of opportunity to actually dive in and explain.
I appreciate the drivers for your comments there and we are making a sincere effort to usefully increase the level of detail in communication.
While we all work in PKI and are subject matter experts in our respective fields, it might help to adopt a position of "assume they don't know what we mean", and try to add detail about the design and/or implementation. Alternatively, you might think of it like describing an algorithm: how could another CA implement a similar process, and get similar results, to Sectigo, which will hopefully be the "right" results.
That's a useful pointer. The JoI policy checker, for example, we can share an outline of the psuedo-code for that. I'll ask for that to be prepared.
As it currently exists the source code itself is not going to be useful to anyone else, but the psuedo-code can be useful.
Having that alongside the ruleset would allow other CAs to implement the same checks. Or of course it could be incorporated into one of the existing lint-checkers.
For example, for this issue, sharing how you structure your meeting, what are the challenges you face, and how you're overcoming these challenges. Comment #9 said y'all were underresourced for the experts, but presumably, Comments #10 and Comments #11 are the meetings that involve the folks with the skills, making the explicit time to discuss and address the issues. Comment #9 provided assurances that significant resources were expended despite the lack of non-communication, but we're not really getting a picture of what that was then, or what that is now. Several bugs, two months later, don't even feel like there's a good preventative mitigation in place.
This meeting is structured around incident reports. All of the incident reports we're currently examining are the public Mozilla ones.
It really helps compliance and development to talk through the tasks and the responses so far and to get direction from a wider team of clueful people in the company and it also helps to avoid unnecessary rat-holing and headlight-staring.
For the three open incident bugs (so excluding this one) there is preventative mitigation in place.
The mitigation can always be improved.
We have our adversarial second review process which is proving far more effective than before. By 'second review' there I'm really referring to the 'Final Cross-Correlation and Due Diligence' step outlined in section 11.3 of the EV guidelines. We now separate that second review team in the same way that you'd separate an IT security team from developers.
We also have improved automated checks in place to identify and reject the most egregious examples of incorrect address data, such as what we have come to call 'meta-data' (although I think that term is probably misused here) by which we mean data that consists of dots, dashes, spaces, and other punctuation and especially sequences or repetitions of those characters instead of city or state names or other address fields.
Those same automated checks also aim to identify frequently used words and phrases that identify that these fields do not contain real data, such as '(none)', 'NA', 'Not Applicable', and we use that same mechanism to catch the common default subject phrases ('Some-State', 'Default City', etc).
Although those checks may seem unnecessary since we have the required initial verification checks and second review process in place, but those characters and phrases are exactly the ones that become so familiar to our human validators that they can become practically invisible to them.
The further automated checks of subject address fields that we are still working on are to further improve the accuracy and further reduce the scope for error by human validators. These have yielded some improvements but remain a work in progress, and something on which we are expending R&D effort. I can feel you willing me to go into more detail about the R&D avenues being explored but in this I'm going to fail you, for today at least. I think there are limits on how much of this R&D work is reasonably responsive to the incidents we're dealing with, although we probably aren't quite there yet and I absolutely realize we can't close out at least Bug 1575022 until the identification and remediation of prior problematic issuance is completed.
The more we automate the subject detail checks, the less we rely on our human validators and that is a win both because it serves to reduce ongoing costs and because it decreases error rates.
Helping understand where the challenges are helps browsers build confidence that things are going as expected. It also provides opportunities to help, to the best of their ability, to find solutions, or to allow the community to do more research into options.
Basically, what's missing here is the narrative I'd previously discussed. Comment #12 feels a bit like "Last week, we met in Rivendell. There was some debate about next steps" which... like, is both accurate and completely overlooks the critical discussions ;)
I've never been a Tolkien reader. I had friends at school who read him at 9 and 10, but I was more into running around with a stick. My running days are now behind me (unless chased by something large) and I left my stick somewhere.
But I take your point.