Created attachment 620173 [details] [diff] [review] configurable number of iterations; misc. fixes This patch contains everything you reviewed in the github peptest repo plus one more changeset (https://github.com/mozilla/peptest/commit/2f04593e61b7784ff0e1d68b4075869070c4ae68). This last changeset does a few things: - changes from iterating over each test to iterating over the whole testsuite. I found the numbers to be much more consistent this way. - a few fixes to tests, notably closing context menus that were previously left open (which interfered with other tests), and removing the add-on-bar toggling, since it didn't seem to do anything (I guess no appropriate add-ons installed). - catch exceptions in performAction() so that we clean up properly. - remove the failure threshold for open bookmarks. Since this is now a performance framework and not pass/fail, there's no point in hiding unresponsiveness below a certain level. I did *not* add a --iterations option to the makefile. Devs can rerun it via 'make peptest' as much as they want. Mozharness will handle the number of iterations for the automated runs. Please push this to either m-i or m-c after reviewing. :) I will then ask aki to do the appropriate mozharness-config modifications.
https://hg.mozilla.org/mozilla-central/rev/a2647e37b065 (Normally left open until the bug merges to mozilla-central :-))