1286809 - Provide mechanism to run a subset of tasks in nss-try

Status: RESOLVED FIXED

(NSS :: Build, defect)

Description

[Pete Moore [:pmoore][:pete]]

At the moment all try pushes run all tasks, so it might be useful to have a simple mechanism to run a subset of the tasks only.

One option might be if it is possible to work out from which files changed, which tasks need to run, and reduce it that way. If that isn't so easy, another option might be introducing something like try-syntax that is used in gecko (the commit message specifying which tasks to run).

I'm not sure what the additional cost is of running everything vs the cost of implementing such a feature plus the cost of tasks not getting executed and later causing problems. It may be that the engineering cost is more than the savings, I'm not sure - but certainly worth looking into.

Comment 1

[Tim Taubert [:ttaubert] (inactive)]

Attachment: 0001-Bug-1286809-Provide-mechanism-to-run-a-subset-of-tas.patch

This patch introduces a try syntax that is intentionally very similar to what we already have, even if that's not optimal. I think there's benefit in keeping it similar though.

It uses -b do to specify build types, if none are given or we see -b with garbage then we'll fall back to "do".

Platforms are specified via -p linux,linux64,linux64-asan,win64 - or "all" / "none". If the value is empty or garbage we'll default to run all platforms. "linux64-asan" includes LSan as long as they're still separate. When a try syntax is given we'll never run memleak.sh tests, they will disappear with the ASan/LSan merge from Taskcluster. "none" only makes sense here if you want to run only tools, like "try: -p none -t all".

Tests are specified via -u crmf,chains,cipher,db,ec,fips,gtest,lowhash,merge,sdr,smime,tools,ssl - or "all" / "none". If -u is not specified or only garbage we'll run no tests.

Tools are specified via -t clang-format,scan-build - or "all" / "none". If -t is not specified or only garbage we'll run no tools.

I wasn't sure what to do about the "Various Builds" that check build flags and compilers. Currently they're included for each platform if chosen.

To be on the safe side, here's a try build with Docker image v0.0.18:

https://treeherder.mozilla.org/#/jobs?repo=nss-try&revision=b166deac21ce84f36838b361513eb8b1a9d12218

Here a few try runs that test parts of the try syntax:

https://treeherder.mozilla.org/#/jobs?repo=nss-try&revision=054b2c09b6569838e0e2de2a16a207deebcdb87d
https://treeherder.mozilla.org/#/jobs?repo=nss-try&revision=856edc73f86fb1606e9630f1eaa1b3df5aee9d36
https://treeherder.mozilla.org/#/jobs?repo=nss-try&revision=f5eae020cff888bf62d6fab30a801c77584d56fa
https://treeherder.mozilla.org/#/jobs?repo=nss-try&revision=1784167876fdcb76e14720b996208b86c1935403

Oh, btw, if one wants you can of course use --build do --platform linux64 --unittests all --tools none :)

Comment 2

[Franziskus Kiefer [:franziskus]]

Comment on attachment 8773733 [details] [diff] [review]
0001-Bug-1286809-Provide-mechanism-to-run-a-subset-of-tas.patch

Review of attachment 8773733 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm. Maybe we want to have the definition of available unittests, platform ect. in separate files so they're easier to update. But I think we can do that in a follow up if we think that makes life easier.

Comment 3

[Tim Taubert [:ttaubert] (inactive)]

(In reply to Franziskus Kiefer [:fkiefer or :franziskus] from comment #2)
> lgtm. Maybe we want to have the definition of available unittests, platform
> ect. in separate files so they're easier to update. But I think we can do
> that in a follow up if we think that makes life easier.

Yeah, that's not a bad idea. I can put it in a separate json file maybe.

Comment 4

[Tim Taubert [:ttaubert] (inactive)]

https://hg.mozilla.org/projects/nss/rev/e8327de7628e