Closed Bug 698561 Opened 11 years ago Closed 11 years ago

run_tests.py should err out early if firefox is already running

Categories

(Testing :: Talos, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: k0scist, Unassigned)

Details

(Whiteboard: [mozbase][good-first-bug])

Attachments

(1 file)

If you do `run_test.py something.yml` and firefox is running, it runs
through the whole suite:

(talos)│python run_tests.py 20111031_1309_config.yml 
RETURN:s: qm-pxp01
RETURN:id:20111031031100
RETURN:<a href =
"http://hg.mozilla.org/mozilla-central/rev/04b4ea333800">rev:04b4ea333800</a>
qm-pxp01: 
                Started Mon, 31 Oct 2011 13:12:59
Running test ts: 
             Started Mon, 31 Oct 2011 13:12:59
Failed ts: 
           Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: ts
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test ts: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test ts_paint: 
             Started Mon, 31 Oct 2011 13:12:59
Failed ts_paint: 
                 Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: ts_paint
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test ts_paint: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test ts_cold: 
             Started Mon, 31 Oct 2011 13:12:59
Failed ts_cold: 
                Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: ts_cold
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test ts_cold: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test ts_cold_generated_max: 
             Started Mon, 31 Oct 2011 13:12:59
Failed ts_cold_generated_max: 
                              Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: ts_cold_generated_max
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test ts_cold_generated_max: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test ts_cold_generated_min: 
             Started Mon, 31 Oct 2011 13:12:59
Failed ts_cold_generated_min: 
                              Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: ts_cold_generated_min
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test ts_cold_generated_min: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test ts_cold_generated_med: 
             Started Mon, 31 Oct 2011 13:12:59
Failed ts_cold_generated_med: 
                              Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: ts_cold_generated_med
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test ts_cold_generated_med: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test ts_places_generated_max: 
             Started Mon, 31 Oct 2011 13:12:59
Failed ts_places_generated_max: 
                                Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: ts_places_generated_max
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test ts_places_generated_max: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test ts_places_generated_min: 
             Started Mon, 31 Oct 2011 13:12:59
Failed ts_places_generated_min: 
                                Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: ts_places_generated_min
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test ts_places_generated_min: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test ts_places_generated_med: 
             Started Mon, 31 Oct 2011 13:12:59
Failed ts_places_generated_med: 
                                Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: ts_places_generated_med
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test ts_places_generated_med: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test tp: 
             Started Mon, 31 Oct 2011 13:12:59
Failed tp: 
           Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: tp
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tp: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test tp4: 
             Started Mon, 31 Oct 2011 13:12:59
Failed tp4: 
            Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: tp4
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tp4: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test tp5: 
             Started Mon, 31 Oct 2011 13:12:59
Failed tp5: 
            Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: tp5
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tp5: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test tp_js: 
             Started Mon, 31 Oct 2011 13:12:59
Failed tp_js: 
              Stopped Mon, 31 Oct 2011 13:12:59
FAIL: Busted: tp_js
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tp_js: 
               Stopped Mon, 31 Oct 2011 13:12:59
Running test tdhtml: 
             Started Mon, 31 Oct 2011 13:12:59
Failed tdhtml: 
               Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: tdhtml
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tdhtml: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test tgfx: 
             Started Mon, 31 Oct 2011 13:13:00
Failed tgfx: 
             Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: tgfx
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tgfx: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test tsvg: 
             Started Mon, 31 Oct 2011 13:13:00
Failed tsvg: 
             Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: tsvg
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tsvg: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test tsvg_opacity: 
             Started Mon, 31 Oct 2011 13:13:00
Failed tsvg_opacity: 
                     Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: tsvg_opacity
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tsvg_opacity: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test v8: 
             Started Mon, 31 Oct 2011 13:13:00
Failed v8: 
           Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: v8
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test v8: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test twinopen: 
             Started Mon, 31 Oct 2011 13:13:00
Failed twinopen: 
                 Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: twinopen
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test twinopen: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test tpaint: 
             Started Mon, 31 Oct 2011 13:13:00
Failed tpaint: 
               Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: tpaint
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tpaint: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test tsspider: 
             Started Mon, 31 Oct 2011 13:13:00
Failed tsspider: 
                 Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: tsspider
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tsspider: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test tscroll: 
             Started Mon, 31 Oct 2011 13:13:00
Failed tscroll: 
                Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: tscroll
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test tscroll: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test dromaeo_css: 
             Started Mon, 31 Oct 2011 13:13:00
Failed dromaeo_css: 
                    Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: dromaeo_css
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test dromaeo_css: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test dromaeo_dom: 
             Started Mon, 31 Oct 2011 13:13:00
Failed dromaeo_dom: 
                    Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: dromaeo_dom
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test dromaeo_dom: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test dromaeo_jslib: 
             Started Mon, 31 Oct 2011 13:13:00
Failed dromaeo_jslib: 
                      Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: dromaeo_jslib
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test dromaeo_jslib: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test dromaeo_sunspider: 
             Started Mon, 31 Oct 2011 13:13:00
Failed dromaeo_sunspider: 
                          Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: dromaeo_sunspider
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test dromaeo_sunspider: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test dromaeo_v8: 
             Started Mon, 31 Oct 2011 13:13:00
Failed dromaeo_v8: 
                   Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: dromaeo_v8
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test dromaeo_v8: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test dromaeo_basics: 
             Started Mon, 31 Oct 2011 13:13:00
Failed dromaeo_basics: 
                       Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: dromaeo_basics
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test dromaeo_basics: 
               Stopped Mon, 31 Oct 2011 13:13:00
Running test a11y: 
             Started Mon, 31 Oct 2011 13:13:00
Failed a11y: 
             Stopped Mon, 31 Oct 2011 13:13:00
FAIL: Busted: a11y
FAIL: Found processes still running: firefox, plugin-container. Please
close them before running talos.
Completed test a11y: 
               Stopped Mon, 31 Oct 2011 13:13:00
RETURN: cycle time: 00:00:01<br>
qm-pxp01: 
                Stopped Mon, 31 Oct 2011 13:13:00

In this case, it should err out before running any tests.
Whiteboard: [mozbase]
Whiteboard: [mozbase] → [mozbase][good-first-bug]
I think we should have more error types instead of a generic talosError like:
- talosPlatformError
- talosUncleanEnvError
- talosScriptError

etc. where all are subclasses of the generic talosError class. This way we can check the exception type and if it is a common error which will affect all the tests such as talosPlatformError or talosUncleanEnvError we can break the whole loop and quit.
After discussing this on irc, I think we agreed that it's best to always just error out as soon as an exception is thrown. We want to fail as close to the error as possible, to make it obvious what went wrong.
I solved the problem with raising the caught talosError instance though not sure this is the best way to do it.
Attachment #572032 - Flags: review?(jhammel)
Comment on attachment 572032 [details] [diff] [review]
Patch for a simple solution

tested and looks good to me! thanks!
Attachment #572032 - Flags: review?(jhammel) → review+
pushed: http://hg.mozilla.org/build/talos/rev/817fdb60730b

Thanks, BYK!
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.