611007 - update maemkit to optionally run a complete reftest.list file instead of chunking by default

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: optionally don't split the reftest.list (1.0)

currently maemkit chunks all the reftest include files to run them 1 by 1.  In some cases it will also split a large list of tests in a single .list file into multiple .list files.

I have a patch that adds a option to maemkit.cfg where we can run the reftest.list as it was originally intended.  This is good for tests like crashtests and jsreftest where (on the n900) we can run the whole test suite end to end with no OOM or crashing of the browser.

In addition I removed an extra directory that was being inserted in the list of tests to run which is causing an error for crashtests.

Comment 1

[Aki Sasaki (not active)]

Is this included in the tarball in bug 608784 ?

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

yes, that is why the crashtests are green now!

Comment 3

[Aki Sasaki (not active)]

K, should we dup this?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

well the patch is unique, I just wanted to minimize the staging runs.  The over test run results look great for this patch!

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

odd, why do we show green and 0 failures on tinderbox MobileTest when the log shows 2 failures:
http://tinderbox.mozilla.org/showlog.cgi?log=MobileTest/1289571754.1289573261.25089.gz#err0

If we have some magic going on to ignore X failures, it would be nice to remove it.

Comment 6

[Aki Sasaki (not active)]

(In reply to comment #5)
> odd, why do we show green and 0 failures on tinderbox MobileTest when the log
> shows 2 failures:
> http://tinderbox.mozilla.org/showlog.cgi?log=MobileTest/1289571754.1289573261.25089.gz#err0
> 
> If we have some magic going on to ignore X failures, it would be nice to remove
> it.

Yes, as noted on the front of the mobile tinderbox page:

http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mobile

Unit tests:
# passed / # failed / # known / [mobileFailCount]
mobileFailCount is subtracted from #failed.
mobileFailCount is added to #known.

This has been true since day 1.

We can get rid of it; new bug?

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

pushed:
http://hg.mozilla.org/qa/maemkit/pushloghtml?changeset=ba8623e3b181