873149 - Report the slowest test in mochitest

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Martijn Wargers (dead)]

Attachment: path

Reftests report the slowest test file, it might be handy if mochitest would do the same? Especially since we have a timeout file where a test file can be timed out after 330 seconds.
It might be good to find out if certain files would be very close to that value on certain platforms (like b2g mochitest for instance).

Comment 1

[(not currently active) Ted Mielczarek]

Comment on attachment 750551 [details] [diff] [review]
path

Review of attachment 750551 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good!

Comment 2

[Martijn Wargers (dead)]

Attachment: patch for checkin

Comment 3

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b06b9c8b19a3

Comment 4

[Phil Ringnalda (:philor)]

Backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/b7dbf40666b1 - must have messed up the horrifying regexes in mozharness that look for pass/fail/todo for Android mochitests, because it turned every single Android mochitest orange.

Comment 5

[Martijn Wargers (dead)]

Wow, I really thought this would not give any trouble.

Comment 6

[Martijn Wargers (dead)]

Where can I find those horrifying regexes in mozharness?

Comment 7

[Phil Ringnalda (:philor)]

My mistake, Fennec isn't using mozharness yet, so it would be the horrifying source regexes, not the copies in mozharness. Also probably not what I thought it was, but instead http://mxr.mozilla.org/build/source/buildbotcustom/steps/unittest.py#267, which believes that a passing test will match "Failed: 0" with one space, not your "Failed:  0" with two spaces.

Comment 8

[Martijn Wargers (dead)]

Attachment: unittest.diff

So something like this would fix it, right? I have no idea how to test this.

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 780684 [details] [diff] [review]
unittest.diff

Review of attachment 780684 [details] [diff] [review]:
-----------------------------------------------------------------

Looks like it will work. Feel free to land on the default branch at your convenience. Someone else will put it into production during a reconfig.

Comment 10

[Martijn Wargers (dead)]

Attachment: unittest.diff (checked in)

Ok, so unittest.diff can be checked in ( attachment 780684 [details] [diff] [review] ), after that patch has been put into production, the first patch can be checked in.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/build/buildbotcustom/rev/687de43bbf8d

Comment 12

[Rail Aliiev [:rail]]

In production

Comment 13

[Martijn Wargers (dead)]

Ok, that means that the first patch (attachment 762398 [details] [diff] [review]) can be checked in safely now and this would then be fixed.

Comment 14

[Matthew N. [:MattN]]

https://hg.mozilla.org/integration/fx-team/rev/5d49f49bebb9

Comment 15

[Martijn Wargers (dead)]

First patch was checked in some time ago: http://hg.mozilla.org/mozilla-central/rev/5d49f49bebb9
So this bug is now fixed.