Adding Alice on CC as I believe she was the original author of this test, at least I reviewed my recent changes on it.
(In reply to Jonathan Kew (:jfkthame) from comment #2)
It's unclear to me whether this test is entirely valid/correct, for a couple of reasons:
(1) The spec text quoted in comment 1 is one of the points listed under "User agents can choose their own heuristics for when to match :focus-visible; however, the following (non-normative) suggestions can be used as a starting point". As such, it is explicitly not a normative requirement but just a suggestion.
All the :focus-visible spec is full of suggestions, and the tests check those suggestions. It's true that user agents can do whatever they want regarding them.
Anyway regarding WPT tests, as the spec text says "should" it looks like having regular tests is the way to go (https://web-platform-tests.org/writing-tests/file-names.html#test-features):
.optional : Indicates that a test makes assertions about optional behavior in a specification, typically marked by the RFC 2119 “MAY” or “OPTIONAL” keywords. This flag should not be used for “SHOULD”; such requirements can be tested with regular tests, like “MUST”.
Thought maybe we have to add "should" meta flag (https://web-platform-tests.org/writing-tests/css-metadata.html#requirement-flags).
(2) It's not clear to me whether pressing the Shift key should be considered as "interact[ing] with the page". A clearer "interaction" would be, for example, using an arrow key to move the caret within an editable element, after having placed the caret (and focused the element) via a mouse action. But merely pressing or releasing a modifier key such as Shift does not (usually) amount to an "interaction" with the page at all.
An editable element will always match :focus-visible when it's focused, so it's not an option. Maybe we could use other keys like ENTER or SPACE, or a regular letter on this test, instead of a modifier key like SHIFT, to make things clearer. WDYT?