1847292 - always report crashes with wpt

Status: ASSIGNED

(Testing :: web-platform-tests, task)

Description

[Joel Maher ( :jmaher ) (UTC -8) (PTO back normal Nov 17)]

we had a perma crash on wdspec tests for 1+ week on the tree, but the jobs were green because in our structuredlogger we send allow_crash=True. Flipping this should work and things should be green now.

A risk is an intermittent crash might get marked as intermittent.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8) (PTO back normal Nov 17)]

Attachment: Bug 1847292 - Allow crashes to be reported in wpt by default. r=jgraham!

Comment 2

[Joel Maher ( :jmaher ) (UTC -8) (PTO back normal Nov 17)]

a new direction here is to allow wpt to accept crashes of certain signatures, from matrix chat:

In mozlog we'd need an extra field on the crash action to say if the crash was allowed. Then we'd need a way to pass in a set of allowed signatures to mozcrash which would cause the crash to be marked as expected, plus a way to set it as expected even without allowed signatures (because the test itself was expected to crash).

Then instead of mozharness counting crashes it could just check for any crashes without expected=True and that would always fail the job.
The metadata update would work just like for lsan_allowed; we'd collect any signatures for non-allowed crashes and add them to the allow list for the current scope (i.e. test group). I guess we'd also need to add the scope to the crash log data; that seems unfortunate but I don't currently have a better idea.

this is half baked, but a direction to go in order to make this a bit more robust.