786308 - using a sample.config file doesn't yield the expected results in the output .yml file

Status: RESOLVED WONTFIX

(Testing :: Talos, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

I am trying to get the new updated xperf tests online, and I ran into a problem modifying the xperf.config file and using that to seed the tests.  Here is an my xperf.config file:

jmaher@jmaher-MacBookPro:~$ cat xperf.config 
# Extra preferences to set in the test
extraPrefs:
  extensions.autoDisableScopes:
  extensions.enabledScopes:
  talos.logfile: browser_output.txt

tests:
- name: ts_paint
  cycles: 1
  xperf_providers : ['PROC_THREAD', 'LOADER', 'HARD_FAULTS', 'FILENAME', 'FILE_IO', 'FILE_IO_INIT']
  xperf_user_providers: ['Mozilla Generic Provider', 'Microsoft-Windows-TCPIP']
  xperf_stackwalk : ['FileRead', 'FileWrite', 'FileFlush']
  w7_counters: []
  win_counters: []
  linux_counters: []
  remote_counters: []
  mac_counters: []
- name: tp5n
  cycles: 1
  tppagecycles: 1
  tpcycles: 1
  xperf_providers : ['PROC_THREAD', 'LOADER', 'HARD_FAULTS', 'FILENAME', 'FILE_IO', 'FILE_IO_INIT']
  xperf_user_providers: ['Mozilla Generic Provider', 'Microsoft-Windows-TCPIP']
  xperf_stackwalk : ['FileRead', 'FileWrite', 'FileFlush']
  w7_counters: []
  win_counters: []
  linux_counters: []
  remote_counters: []
  mac_counters: []jmaher@jmaher-MacBookPro:~$


When I run a perfconfig command line:
python PerfConfigurator.py -v -e `which firefox` -a tp5n --sampleConfig xperf.config --develop --output x.yml --xperf_path=c:/perftools/xperf.exe


I get both ts_paint and tp5n in the resulting x.yml file (attached)

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: resulting output x.yml file

Comment 2

[Jeff Hammel]

So the problem is here: http://hg.mozilla.org/build/talos/file/6d79047595a4/talos/PerfConfigurator.py#l387 . We just extend the tests that are in the file.  I'll have to think about a more elegant way of doing this.  Maybe the thing to do is to add a test_overides section?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

the only reason for needing an xperf.config is to specify custom counters and reduce the cycles we run.  While [most of] this is configurable via the commandline, making ugly buildbot configs is just pushing the problem elsewhere.  If we could resolve the xperf_counters outside of buildbot, we could adjust buildbot configs to resolve the cycles.  Then we wouldn't need xperf.config.

The question then gets pushed out to is it easier to understand a buildbot config and log file journaling the entire operation or is it easier to follow along with a .config file to overload the default values in test.py ?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

ok, I am going to close this as won't fix since we are using test_overrides successfully.