Closed Bug 1592608 Opened 4 months ago Closed 4 months ago

ReftestManifest does not capture enough manifest information

Categories

(Testing :: Reftest, enhancement)

Version 3
enhancement
Not set

Tracking

(firefox72 fixed)

RESOLVED FIXED
mozilla72
Tracking Status
firefox72 --- fixed

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(1 file)

The python ReftestManifest is used by the TestResolver to read reftest manifests and is the source of reftest manifest data for tools like 'mach test-info' and test-verify.

ReftestManifest provides access to all of the files referenced from reftest manifests, but it is missing important data:

  • it does not capture any of the manifest annotations (eg. "skip-if(skiaContent)")
  • it does not capture the relationship between a test file and a reference file
Blocks: 1528471, 1584310

Add test annotations to the ReftestManifest and TestResolver. For example, a
test description from the TestResolver might now contain 'skip-if': 'skiaContent';
similar to the content provided for manifestparser tests; this will allow
'mach test-info report' to filter tests based on reftest manifest test
annotations.
Also add the referenced-test field which identifies the test file associated
with test entries for reference files; this will allow test-verify to
run the correct reftest when only the reference file has been modified.

Blocks: 1592754
No longer blocks: 1528471
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/082bcb14a8c5
Add more fields to ReftestManifest test descriptions; r=ahal,froydnj
Status: NEW → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla72
You need to log in before you can comment on or make changes to this bug.