787267 - Add capability to mark tests as failing in import-tests.py

Status: RESOLVED FIXED

(Core :: Layout, defect)

Description

[Scott Johnson (:jwir3)]

Our import-tests.py script imports tests from the css-wg's hg repository. Although all of the tests are imported for a given module once that module is added to the suite, not all of the tests will pass, thus turning tbpl perma-orange if they are imported into the tree.

We should add the capability for the import-tests.py script to do the following:
  - Be able to automatically mark reftests as failing based on a reftest name.

This will be controlled via a text file that will sit next to import-tests.py. This text file will essentially house a list of reftest names that will be marked in the reftest.list file as "failing".

Comment 1

[Scott Johnson (:jwir3)]

Attachment: b787267

Comment 2

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 657320 [details] [diff] [review]
b787267

>+import sys

>+    dirname = os.path.dirname(sys.argv[0])

Use __file__ or os.path.realpath(__file__) instead of sys.argv[0].

>+    failList = [x.lstrip().rstrip() for x in failListFile if bool(x.lstrip().rstrip()) and not x.lstrip().startswith("#")]

May as well assign directly to gFailList.

Also, can you avoid calling lstrip() three times by nesting list comprehensions, e.g. something like:

gFailList = [x for x in [x.lstrip().rstrip() for x in failListFile] if bool(x) and not x.startswith("#")]

(or maybe a better way)?


r=dbaron with that or similar

Comment 3

[Scott Johnson (:jwir3)]

Fixed issues found in review and landed on inbound:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b8e8c00e00ff

Comment 4

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/b8e8c00e00ff