398463 - talos runs tests in a different order than listed in the config file

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

Talos seems to run tests in a random order. I would expected that they be run in the same order listed in the config file. Not a big deal, but unintuitive.

Comment 1

[alice nodelman [:alice] [:anode]]

This has to do with how the yaml file is formatted - will get fixed up as I'm working on the structure of how talos gets configured.

Comment 2

[alice nodelman [:alice] [:anode]]

Attachment: [Checked in]run the tests in the order that they appear in the config file

It's a minor nuisance, but I think that it is worth the effort - especially if we go to a system where we report test results after each test has run, we'll want to put them in an order of speed so that developers can get quick results for ts/twinopen/etc and then run tp.

Comment 3

[[e-mail] (:bhearsum)]

Comment on attachment 342672 [details] [diff] [review]
[Checked in]run the tests in the order that they appear in the config file

YAML format is weird...but this looks fine.

Comment 4

[alice nodelman [:alice] [:anode]]

Comment on attachment 342672 [details] [diff] [review]
[Checked in]run the tests in the order that they appear in the config file

Checking in sample.config;
/cvsroot/mozilla/testing/performance/talos/sample.config,v  <--  sample.config
new revision: 1.19; previous revision: 1.18
done
Checking in run_tests.py;
/cvsroot/mozilla/testing/performance/talos/run_tests.py,v  <--  run_tests.py
new revision: 1.31; previous revision: 1.30
done
Checking in PerfConfigurator.py;
/cvsroot/mozilla/testing/performance/talos/PerfConfigurator.py,v  <--  PerfConfigurator.py
new revision: 1.6; previous revision: 1.5
done

Comment 5

[alice nodelman [:alice] [:anode]]

Reported numbers look good - the patch shouldn't have affected anything but it is always good to be careful.

Comment 6

[Chris Cooper [:coop] (he/him)]

Moving closed Future bugs into Release Engineering in preparation for removing the Future component.