This can use the original HTML report as a template, but has at least one additional requirement; results that are better than expected should not be flagged up.
Created attachment 8447998 [details] report.html So I made a start at an improved report. Attached is a very basic sample. Feedback welcome before I spend too much time on this.
Much better, thanks! I like the practice of prefacing unexpected failures with UNEXPECTED_. I know it's redundant since we include the expected outcome, but I worry about partners (particularly those not super fluent in English) not understanding the difference between: expected: FAIL, result: FAIL and expected: PASS, result: FAIL Using "UNEXPECTED_" gives them something more explicit to look for.
Well this won't ever say expected: FAIL result: FAIL, or even expected: FAIL result: PASS. So I will certainly experiment with adding that prefix, but I sort of suspect it will just make the page more noisy.
(In reply to James Graham [:jgraham] from comment #3) > Well this won't ever say expected: FAIL result: FAIL, or even expected: FAIL > result: PASS. So I will certainly experiment with adding that prefix, but I > sort of suspect it will just make the page more noisy. Oh we don't have expected failures or unexpected successes? In that case, it's probably OK to leave it as-is.
Right, the idea was to not report expected failures or unexpected successes in order to concentrate on the issues most likely to be real cert-affecting problems.
In that case, it looks great! Thanks.
Created attachment 8450450 [details] [review] Updated report format
Attachment #8450450 - Flags: review?(jgriffin)
Btw, the look of the new report is a big improvement, IMO.
Comment on attachment 8450450 [details] [review] Updated report format I see ato has reviewed this in critic, so I'll give my r+ for the general look and format of the report.
Attachment #8450450 - Flags: review?(jgriffin) → review+
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.