Closed Bug 1182817 Opened 9 years ago Closed 9 years ago

In some cases, running tests on Try with the --tag option blows up with python error

Categories

(Testing :: Mochitest, defect)

defect
Not set
normal

Tracking

(firefox42 fixed)

RESOLVED FIXED
mozilla42
Tracking Status
firefox42 --- fixed

People

(Reporter: kats, Assigned: ahal)

References

Details

Attachments

(1 file)

See [1]. My goal was to run just the tests in gfx/layers/apz/test/mochitest.ini via "--tag apz" with extra logging enabled so that I could investigate an intermittent failure that I can't reproduce locally. Some of the tests ran ok, in chunks 2 and 5. But the one test I cared about (test_basic_pan.html) didn't get run, and instead chunk 8 has some python error:

21:47:20     INFO -  Traceback (most recent call last):
21:47:20     INFO -    File "runtestsb2g.py", line 481, in run_remote_mochitests
21:47:20     INFO -      retVal = mochitest.run_tests(options)
21:47:20     INFO -    File "runtestsb2g.py", line 127, in run_tests
21:47:20     INFO -      self.logPreamble(self.getActiveTests(options))
21:47:20     INFO -    File "/builds/slave/test/build/tests/mochitest/runtests.py", line 1961, in getActiveTests
21:47:20     INFO -      exists=False, disabled=disabled, filters=filters, **info)
21:47:20     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/manifestparser/manifestparser.py", line 748, in active_tests
21:47:20     INFO -      tests = fn(tests, values)
21:47:20     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/manifestparser/filters.py", line 192, in __call__
21:47:20     INFO -      end = tests.index(chunk_tests[end])
21:47:20     INFO -  IndexError: list index out of range
21:47:20     INFO -  Automation Error: Exception caught while running tests

Same thing applies to the debug suite, except chunks 17, 18, and 19 all had the error.

I'm trying again with the tests rearranged [2] in mochitest.ini on the assumption that this is a bug in the python code that barfs on the last test when spread too thinly over many chunks.

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=141a9039e029
[2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=ba7429d94f13
(Treeherder doesn't schedule the jobs :-) )
Summary: In some cases, running tests with --tag on TreeHerder blows up with python error → In some cases, running tests on Try with the --tag option blows up with python error
Thanks, I'll take a look. Looks like there are a total of 3 unskipped tests with the 'apz' tag being chunked, so all the "No tests found" messages are expected. The bug here is that the last one fails.
This happens anytime there are at least twice as many chunks as there are tests. It makes the 'end' index == len(tests):
https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/manifestparser/manifestparser/filters.py#178

Patch coming shortly.
Bug 1182817 - [manifestparser] Fix exception in chunk_by_slice when there are two times more chunks than tests, r=chmanchester
Attachment #8632947 - Flags: review?(cmanchester)
Comment on attachment 8632947 [details]
MozReview Request: Bug 1182817 - [manifestparser] Fix exception in chunk_by_slice when there are two times more chunks than tests, r=chmanchester

https://reviewboard.mozilla.org/r/13155/#review11755

Ship It!
Attachment #8632947 - Flags: review?(cmanchester) → review+
Assignee: nobody → ahalberstadt
Status: NEW → ASSIGNED
https://hg.mozilla.org/mozilla-central/rev/43eea082d662
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla42
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: