Open Bug 670229 Opened 13 years ago Updated 2 years ago

[Tracking bug] Determine which mochitests are 'slow'

Categories

(Testing :: General, defect)

defect

Tracking

(Not tracked)

People

(Reporter: jgriffin, Unassigned)

References

(Depends on 2 open bugs)

Details

(Whiteboard: [buildfaster:p2])

Determine which mochitest-plain tests are slow, so they can be targeted for further investigation.
For mozilla-central on 2011-07-07, the timings for all tests (in ms) are here:

http://people.mozilla.org/~jgriffin/mochitest.json

The tests that take greater than 10s are extracted here:

http://people.mozilla.org/~jgriffin/mochitest-more-than-10s.txt

I'll setup a system to collect this data on an ongoing (weekly?) basis.
Assignee: nobody → jgriffin
One of the criteria for considering a test as "slow" is if the debug duration of the test is much larger than the opt duration of the test.

Crunching the data above, I come up with these average ratios:

{
  "win32": {
    "debug": 3079053,
    "opt": 849342,
    "ratio": 3.6252216421653469
  },
  "macosx": {
    "debug": 5510793,
    "opt": 95161,
    "ratio": 57.91020481079434
  },
  "macosx64": {
    "debug": 5065500,
    "opt": 145407,
    "ratio": 34.836699746229549
  },
  "linux64": {
    "debug": 4934970,
    "opt": 776520,
    "ratio": 6.3552387575336118
  },
  "linux": {
    "opt": 798284,
    "debug": 5980156,
    "ratio": 7.4912637607668451
  }
}

Each ratio is the number of ms for all the debug tests / ms for all opt tests.  The mac numbers are so high because apparently I didn't capture all the opt mac tests during the period in which this data was saved.
Depends on: 673277
Depends on: 673285
Depends on: 673300
Depends on: 673305
Depends on: 673310
Depends on: 673458
Depends on: 673459
Depends on: 673539
Depends on: 673614
Depends on: 673634
Whiteboard: [buildfaster:p1] → [buildfaster:p2]
Assignee: jgriffin → nobody
Product: mozilla.org → Release Engineering
Found in triage. 

from irc w/jgriffin, this is a tracking bug for work by ATeam to find+fix slow running mochitests (in dependent bugs), so morphing summary and move to Testing:General
Component: Other → General
Product: Release Engineering → Testing
Summary: Determine which mochitests are 'slow' → [Tracking bug] Determine which mochitests are 'slow'
Version: other → unspecified
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.