973601 - xpcshell chunk options broken

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect)

Description

[Geoff Brown [:gbrown]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=34696796&tree=Ash&full=1#error0

Android 2.3 seems to be the first platform to try to run xpcshell tests in more than 1 chunk...specifying --this-chunk/--total-chunks fails:

11:07:07     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py --deviceIP=127.0.0.1 --devicePort=20701 --xre-path=/builds/slave/test/build/hostutils/xre --testing-modules-dir=/builds/slave/test/build/tests/modules --apk=/builds/slave/test/build/fennec-30.0a1.en-US.android-arm.apk --no-logfiles --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/ash-android/1392401572/fennec-30.0a1.en-US.android-arm.crashreporter-symbols.zip --total-chunks 3 --this-chunk 1 --manifest tests/xpcshell_android.ini
...
11:19:07     INFO -  pushing /builds/slave/test/build/tests/xpcshell/tests
11:19:07     INFO -  Traceback (most recent call last):
11:19:07     INFO -    File "/builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py", line 610, in <module>
11:19:07     INFO -      main()
11:19:07     INFO -    File "/builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py", line 605, in main
11:19:07     INFO -      **options.__dict__):
11:19:07     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 1337, in runTests
11:19:07     INFO -      self.buildTestList()
11:19:07     INFO -    File "/builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py", line 455, in buildTestList
11:19:07     INFO -      xpcshell.XPCShellTests.buildTestList(self)
11:19:07     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 806, in buildTestList
11:19:07     INFO -      self.chunkTests()
11:19:07     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 815, in chunkTests
11:19:07     INFO -      totalTests += len(self.alltests[dir])
11:19:07     INFO -  TypeError: list indices must be integers, not dict

Comment 1

[Geoff Brown [:gbrown]]

Attachment: allow test chunks in xpcshell tests

self.alltests is a simple array of test objects; I suppose it was a 2 dimensional array at one time? Anyway, I don't think we need to divide the tests by directory - I'm happy to simply divide alltests into N slices of approximately equal number of tests.

Sample run works the way I expect it:

xpcshell-1 https://tbpl.mozilla.org/php/getParsedLog.php?id=34817992&tree=Ash&full=1
chrome/test/unit/test_abi.js ... netwerk/test/httpserver/test/test_start_stop.js

xpcshell-2 https://tbpl.mozilla.org/php/getParsedLog.php?id=34818824&tree=Ash&full=1
rdf/tests/unit/test_rdfredirect.js ... intl/uconv/tests/unit/test_bug317216.js

xpcshell-3 https://tbpl.mozilla.org/php/getParsedLog.php?id=34817954&tree=Ash&full=1
intl/uconv/tests/unit/test_bug321379.j ... security/manager/ssl/tests/unit/test_intermediate_basic_usage_constraints.js

Comment 2

[(not currently active) Ted Mielczarek]

Comment on attachment 8377903 [details] [diff] [review]
allow test chunks in xpcshell tests

Review of attachment 8377903 [details] [diff] [review]:
-----------------------------------------------------------------

Looks like we broke this way back in bug 616999, FWIW:
http://hg.mozilla.org/mozilla-central/rev/40195c0187d3#l7.65

We're probably okay without the chunk-by-dir semantics for now, but we might want to reconsider that in the future.

Comment 3

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/0f57e0bf64b8

Comment 4

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/0f57e0bf64b8

Comment 6

[Andrew Halberstadt [:ahal]]

Xpcshell on b2g28 broke because they didn't have this patch. Landed there:
https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/2a9736d5d4f3

Comment 7

[Ryan VanderMeulen [:RyanVM]]

Thanks!