Open Bug 962156 Opened 8 years ago Updated 8 years ago

Investigate why mochitest-bc runs slowly on all platforms, especially debug

Categories

(Core :: General, defect)

defect
Not set
normal

Tracking

()

People

(Reporter: RyanVM, Unassigned)

Details

We have seen many intermittent failures in the mochitest-bc suite that appear to be related to intermittent spikes in test run time.

For example, browser_webconsole_console_logging_api.js on Linux32 debug typically runs in ~4s, but intermittent would spike to 9+s, leading to bugs like bug 946066. I've seen other instances of tests hitting the 30s timeout that normally run in a fraction of that time.

Overall, debug mochitest-bc has a runtime of over 100 minutes on all platforms now. Linux32/64 and OSX 10.6 are generally 140-150min (and we've hit the 9000s timeout even then on occasion).

Since mochitest-bc chunking seems to be constantly held up by various factors, it would be good if we could investigate what is going on here and whether there are things we can be doing better now until that work is finished.
I think Taras was looking at profiling mochitest runs at the end of last year.  I don't know if he got any useful results out of that or not.
Would it be possible to populate a database with test files and runtimes, for every machine and build, every run? We need data to learn about what's going slower and slower.
I have a new round of intermittent test failures in the webconsole. Again it seems that the #1 culprit is slower and slower test runners...
(In reply to Mihai Sucan [:msucan] from comment #3)
> I have a new round of intermittent test failures in the webconsole. Again it
> seems that the #1 culprit is slower and slower test runners...

For avoidance of doubt: when you say "test runners", you mean the machines that the tests run on, correct?
Flags: needinfo?(mihai.sucan)
(In reply to Nathan Froyd (:froydnj) from comment #4)
> (In reply to Mihai Sucan [:msucan] from comment #3)
> > I have a new round of intermittent test failures in the webconsole. Again it
> > seems that the #1 culprit is slower and slower test runners...
> 
> For avoidance of doubt: when you say "test runners", you mean the machines
> that the tests run on, correct?

Yes, that is correct. However, it might as well be slower and slower builds. We have no measurements to tell us why tests run slower.
Flags: needinfo?(mihai.sucan)
You need to log in before you can comment on or make changes to this bug.