1854095 - convert nss CI configuration to taskgraph

Status: RESOLVED FIXED

(NSS :: Build, task, P3)

Description

[Julien Cristau [:jcristau]]

nss uses bespoke code to schedule its CI tasks. Over the last few years most mozilla projects running on taskcluster have migrated over to the common taskgraph library. We want to convert nss over so we don't have to maintain that custom code going forward, so nss can benefit from taskgraph's features, and eventually so we can automate more tasks like releases and maybe imports into m-c.

Comment 1

[Benjamin Beurdouche [:beurdouche]]

Assigning priority/severity to make it disappear from the triage lists : )

Comment 2

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add initial taskgraph configuration and "base" docker image. r?#releng

• upgrade the docker image from ubuntu bionic to focal, required by
  run-task for env -S support
• add run-task
• move home dir and work dir to /builds/worker for consistency with
  taskgraph conventions

Comment 3

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - switch .taskcluster.yml to taskgraph. r?#releng

Comment 4

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "builds" docker image. r?#releng

• use the "base" image as parent to reduce duplication
• use bionic package sources for old toolchains

Comment 5

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "acvp" docker image. r?#releng

Comment 6

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "clang-format" docker image. r?#releng

Comment 7

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "gcc-4.4" docker image. r?#releng

Use the base image as parent, with ubuntu trusty package sources for the
old toolchain.

Comment 8

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "fuzz" docker image. r?#releng

Delete default "ubuntu" user to keep run-task happy.

Comment 9

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "tools" task kind. r?#releng

Update scripts to:

• stop cloning the source, taken care of by run-task
• look for repos in ${VCS_PATH}, copying to the work dir in some cases
  to avoid polluting the cache

Comment 10

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "build" task kind. r?#releng

This requires changes to the build scripts to:

1. let run-task check out the repos, and look for them in $VCS_PATH
2. copy the source to the work dir, since the scripts aren't set up to
   build outside of the source tree, and we don't want to be polluting the
   cache

I've attempted to preserve the existing build variants, skipping linux
arm64 for now since they haven't worked for a couple of years.

Comment 11

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "certs" task kind. r?#releng

Comment 12

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "test" task kind. r?#releng

Comment 13

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - add "fuzz" task kind. r?#releng

Comment 14

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - parse try syntax on pushes to nss-try. r?#releng

Comment 15

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - delete old docker image definitions and task scheduling code. r?#releng

Comment 21

[Julien Cristau [:jcristau]]

Attachment: Bug 1854095 - apply nspr patch in acvp script.

Comment 22

[Julien Cristau [:jcristau]]

add initial taskgraph configuration and "base" docker image. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/6658376362d4153d85a9cdc4beb673bc2410ca2c
switch .taskcluster.yml to taskgraph. r=releng-reviewers,hneiva,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/dd4d478ebe7ca88f84afc5fa42e206d8c875927f
add "builds" docker image. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/681a18adee68f4cfac56a5875ae09b562b401b72
add "acvp" docker image. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/33a2bb9335f850386e72355235b04c341f0b758d
add "clang-format" docker image. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/43db686fd5b0316ace97bf91b96d68c115ca48ab
add "gcc-4.4" docker image. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/4d99064dac02f2e3d1296ecfb2b71ce180b5c1f1
add "fuzz" docker image. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/5bd54ebf9c28aff6fdaceec9b0348c1058d08c6e
add "tools" task kind. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/657beea6b5d3252f32e70bf48a2b3b631bc3775a
add "build" task kind. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/eb6acca04de6ee7336ee5bf5aeef739ac7ba656a
add "certs" task kind. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/28e075fdcec842ebe127021cf0c03e30165cda4b
add "test" task kind. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/496931b177897649ff33a109c63fbcd13303a838
add "fuzz" task kind. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/c921ebbd6f85aa459e32f5e78cf4b1458a1c526b
parse try syntax on pushes to nss-try. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/9db7042caf17f52c3104c20c3c82a25fd5cd6a33
apply nspr patch in acvp script. r=bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/0af6526bb36bdacb1e668921d81247bcec696367
delete old docker image definitions and task scheduling code. r=releng-reviewers,bhearsum,bbeurdouche
https://hg.mozilla.org/projects/nss/rev/627896a4dda9e31e4acb132f8b840c7d49113ea4