./mach talos-test --suite tp5o should provide a way to run a subset of pages

NEW
Unassigned

Status

defect
3 years ago
3 years ago

People

(Reporter: bholley, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [talos_wishlist])

Reporter

Description

3 years ago
Per IRC discussion today, the only way to avoid running the entire pageset is first to do:

./mach talos-test --suite tp5o

Then wait until stuff downloads. Then kill the process. Then modify testing/talos/talos/tests/tp5n/tp5o.manifest.develop . Then run:

./mach talos-test --suite tp5o --no-download

There should be an easier workflow for this.
Reporter

Comment 1

3 years ago
Looks like I also need to give the same treatment to tp5o.manifest, otherwise it gets copied over and clobbers tp5o.manifest.develop.
I am thinking here we need a few things:
1) not use mozharness to setup talos, but follow the path of other tests
2) provide a |mach talos-test bootstrap| method that will download tp5 pageset once for you
3) as part of #2, we wouldn't download tp5n.zip by default.

the last part would be modifying the list of tests, there is confusion around tp5o.manifest vs tp5o.manifest.develop - maybe we dynamically generate the .develop all the time (could be related to bug 699128).
Whiteboard: [talos_wishlist]
You need to log in before you can comment on or make changes to this bug.