Closed Bug 1192965 Opened 5 years ago Closed 5 years ago
./mach talos-test should allow changing the number of tests run
When running ./mach talos-test chromez-e10s for example, the resize test runs multiple times. While this is fine as a default, sometimes I just want to run a single iteration of the test and there doesn't appear to be a way to do that. A flag for talos-test or env variable to control this behaviour would be nice.
there are a few use cases: * you want what is by default * you want a few data points * you want a single data point I think we should figure out a way to add a flag to mach to describe scenarios, and if we pass in something like 'chromez-e10s', it would use production since that is a suite, but if you pass in 'tresize' e10s, it would assume local development and run a single data point. I am not sure how mach handles commands.
(In reply to Joel Maher (:jmaher) from comment #1) > I think we should figure out a way to add a flag to mach to describe > scenarios, and if we pass in something like 'chromez-e10s', it would use > production since that is a suite, but if you pass in 'tresize' e10s, it > would assume local development and run a single data point. This would be nice for sure. Just having some way to control it at all is a "good enough" start for me though :)
Kartikaya, maybe you could pass the talos options ('--cycles', '--tpcycles' and '--tppagecycles') using 'talos_extra_options' from the make_config function in mach_commands.py. I suspect this way you won't need to make any changes into talos.py.
Thanks for the tip! I had tried doing that earlier but I was putting talos_extra_options in make_args and that didn't work. Putting it in make_config works though.
Thanks Kartikaya for reporting this and the work here! This is a great (almost required!) feature for running Talos using mach. Hmm, not sure why but I can't apply the patch on m-c because of conflicts. Could you look into that please ? hg patch https://bug1192965.bmoattachments.org/attachment.cgi?id=8646300 applying https://bug1192965.bmoattachments.org/attachment.cgi?id=8646300 patching file testing/talos/mach_commands.py Hunk #1 FAILED at 18 Hunk #2 FAILED at 76 Hunk #3 FAILED at 116 3 out of 3 hunks FAILED -- saving rejects to file testing/talos/mach_commands.py.rej abort: patch failed to apply Otherwise this looks good to me and we should land that I think - assuming it works (don't need to push try, this is not used by harness, just a local test). :jmaher, thoughts ?
The patch applies on top of the patch in bug 1192964, but unfortunately that one has try failures. I can rebase it so this one applies first but ideally I'd like to land them both anyway.
this is sort of an ugly hack as we are setting cycles, tpcycles, tppagecycles all the same. If we have iterations >1, it will be confusing. one thought is we could fix tcanvasmark to either use tppagecycles or cycles and then we would have everything using tpcycles=1, this could eliminate that variable. How to differentiate between cycles/tppagecycles? We could split it on tests which are pageloader vs startup. I think for now ignoring tpcycles would be ok, and we can figure out how to simplify our usage internally while making cycles more clear inside of talos.
ok, now we can just use the standard talos flags (from bug 1206415) so '--cycles', '--tpcycles' and '--tppagecycles' are available. Closing the bug, please reopen it if that does not work or if I'm missing something.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.