Closed Bug 1245969 Opened 8 years ago Closed 1 year ago

Automated testing for `mach bootstrap`

Categories

(Firefox Build System :: Bootstrap Configuration, defect)

Product:
Firefox Build System
Component:
Bootstrap Configuration
Type:
defect

Tracking

(firefox110 fixed)

Status:
RESOLVED FIXED
Milestone:
110 Branch
Tracking Flags:
Tracking Status
firefox110 --- fixed

People

(Reporter: gps, Assigned: glandium)

References

Details

Attachments

(8 files, 1 obsolete file)

Bug 1245969 - TaskCluster tasks for performing system bootstrap;
58 bytes, text/x-review-board-request
dustin
: review-
Details
Bug 1245969 - Use working yasm rpm URLs
58 bytes, text/x-review-board-request
Details
Bug 1245969 - Support fetching bootstrap files from an arbitrary revision
58 bytes, text/x-review-board-request
Details
Bug 1245969 - TaskCluster tasks for `mach bootstrap`; r?glandium
46 bytes, text/x-phabricator-request
Details | Review
Bug 1245969 - Add tasks that perform a bootstrap on various bare Linux environments.
48 bytes, text/x-phabricator-request
Details | Review
Backed out 1 changesets (Bug 1245969)
48 bytes, text/x-phabricator-request
Details | Review
Bug 1245969 - Create level 1 pool for image testing r=#releng-reviewers
48 bytes, text/x-phabricator-request
Details | Review
Backed out 1 changesets (Bug 1245969)
48 bytes, text/x-phabricator-request
Details | Review 
We currently don't have automated test coverage that `mach bootstrap` works. This bug is about creating that automation.
Depends on: 1246025
Depends on: 1246033
Depends on: 1246040
(THIS PATCH IS FEEDBACK QUALITY AND ISN'T READY FOR FULL REVIEW)

We've had `mach bootstrap` and the one-line wget + `python bootstrap`
bootstrap tool for years. Unfortunately, it's never been formally
tested. TaskCluster enables us to finally do that.

This commit introduces support for testing the system bootstrap tool
on various Linux distros.

The bootstrap tasks are implemented as generic non-build/test tasks.
They are part of the "bootstrap" tag, so you can include "-j bootstrap"
in your try syntax to schedule them.

TODO

* Arch does not work (interactive prompt is hanging execution)
* Gentoo does not work
* Figure out how to schedule every ~24 hours
* Support Fennec bootstrap
* Tidy up TreeHerder names/groups?
* Work with sheriffs on TreeHerder visibility policy

Review commit: https://reviewboard.mozilla.org/r/35345/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/35345/
Attachment #8720527 - Flags: review?(dustin)
Comment on attachment 8720527 [details]
Bug 1245969 - TaskCluster tasks for performing system bootstrap;

https://reviewboard.mozilla.org/r/35345/#review32141

It would be better to define docker images for each of these tasks, with the system setup complete, and then just run the VCS clone (please use tc-vcs to take advantage of caching) and `mach configure` in the task itself.  In terms of in-tree complexity, it's about the same, but has the advantage of not pulling down loads of software for every run of the task.  Plus it nicely separates system configuration from the task itself.

Note, too, that TaskCluster jobs run as root (there's a bug to make that configurable..), so you'll probably want to create a user in your docker image and then switch to that user before cloning and running mach.

You can schedule these on a periodic basis with the taskcluster hooks service.  We're also looking at something like .cron.yml in the root of the gecko tree to allow more ad-hoc scheduling.
Attachment #8720527 - Flags: review?(dustin)
Depends on: 1251352
Comment on attachment 8720527 [details]
Bug 1245969 - TaskCluster tasks for performing system bootstrap;

https://reviewboard.mozilla.org/r/35345/#review94352

::: taskcluster/ci/bootstrap/kind.yml:74
(Diff revision 2)
> +            docker-image: "debian:jessie"
> +        run:
> +            system-setup: "apt-get update && apt-get -y install python2.7 wget"
> +            python: python2.7
> +
> +    bootstrap/fedora22:

Fedora 22 entered end-of-life in July. I'd suggest Feodra 23 and 24. https://lists.fedoraproject.org/archives/list/announce@lists.fedoraproject.org/thread/GCMM2KQQRY5PKFY42S3K772BDRENXZ7U/

Fedora 25 comes out next week, which will obsolete 23 as well, but there's no tag for it on dockerhub yet.

::: taskcluster/ci/bootstrap/kind.yml:110
(Diff revision 2)
> +                && echo ">=x11-libs/cairo-1.14.2 X" >> /etc/portage/package.use/firefox
> +                && emerge -UD world
> +            python: python2.7
> +
> +    bootstrap/ubuntu1404:
> +        description: "Ubuntu 16.04 Bootstrap Support"

Target should be `ubuntu1604` to match the image?

::: taskcluster/taskgraph/transforms/job/bootstrap.py:87
(Diff revision 2)
> +
> +    script_args = [
> +        run['system-setup'].strip(),
> +        'wget %s' % bsurl,
> +        bs,
> +        # Many distros run an ancient Mercurial that doesn't do cloen bundles.

s/cloen/clone/
Comment on attachment 8720527 [details]
Bug 1245969 - TaskCluster tasks for performing system bootstrap;

https://reviewboard.mozilla.org/r/35345/#review94684

With this in place, can we get rid of the "artifact build" builds?  I think those are doing a similar thing, right (verifying that the pieces necessary for an artifact build are in place)?

I like this!

::: taskcluster/ci/bootstrap/kind.yml:24
(Diff revision 2)
> +    run:
> +        using: bootstrap

Minor: I kind of prefer to have this co-located with the other contents of the `run` key. Otherwise the reader is left without much context for the other `run` keys.

::: taskcluster/taskgraph/transforms/job/bootstrap.py:39
(Diff revision 2)
> +@transforms.add
> +def clone_applications(config, jobs):

Transform should be in `taskcluster/taskgraph/transforms/bootstrap.py`.
Attachment #8720527 - Flags: review?(dustin) → review-
dustin: it's been a year since we visited this problem. How would you recommend we solve it today?

Recap: We don't want these tasks to run on every push. But because results are definitely not deterministic, we want them to run periodically. I was thinking we could use SCHEDULES to ensure we run them when python/bootstrap changes. And maybe supplement that with a cron scheduling? I just don't know the state of periodic scheduling on mozilla-central these days. I think it was in flux last I talked to bstack?
Flags: needinfo?(dustin)
Periodic scheduling is pretty solid -- it's used for nightlies and a number of other things.  So yeah, I think that approach makes a lot of sense.
Flags: needinfo?(dustin)
Product: Core → Firefox Build System
Component: General → Bootstrap Configuration
bstack: the desire for having CI coverage of `mach bootstrap` has come up again recently. These tasks are special in that they ideally start from a vanilla-as-possible OS install. I think it would be ideal if we could define them as Linux tasks that ran in a VM that was imaged from a fresh distro install. Do generic-worker or taskcluster-worker yet support running Linux tasks as root in a VM - not Docker - environment? And do you think we should attempt to shoehorn these tasks into docker-worker or wait for something better?
Flags: needinfo?(bstack)
Depends on: 1469441
Afaik, taskcluster-worker has experimental qemu support but that's as far as we are w.r.t. using a vm instead of docker. :jonasfj, what is the status of qemu support?

Could another possibility to have a 99% fresh ami but with generic worker and livelog binaries dropped in? Maybe even scriptworker would work in that case. I'm guessing that if docker is weird enough compared to a normal distribution install then maybe the ami are too?

I think I'm missing up above where the desire that these run in a VM rather than docker comes from? Are the docker images just plain weird compared to the actual disributions?
Flags: needinfo?(bstack) → needinfo?(jopsen)
Distro Docker images are stripped down installs (they are optimized for size). And we occasionally run into issues where certain packages don't install correctly inside Docker or that Docker does weird things to the environment. I'd feel better if we were able to tell a task to execute in a VM "image" (which could be versioned in source control), as I think it would be more representative of end-user systems.

Now, how we'd go about producing and consuming said image, I'm not sure. But if we could tell a task to start with a VM or filesystem snapshot and execute a process therein in a "real" OS environment (not Docker), then we'd be in a better position.
Depends on: 1383929
We've had `mach bootstrap` and the one-line wget + `python bootstrap`
bootstrap tool for years. Unfortunately, it's never been formally
tested. TaskCluster enables us to finally do that.

This commit introduces support for testing the system bootstrap tool
on various Linux distros.

The bootstrap tasks are implemented as their own kind because they
are somewhat special.

Each OS is defined once along with its base Docker images, a command
to pre-bootstrap the Docker environment (typically to install Python
and wget), and the name of the Python binary to use.

A transform "splits" each job into 4 separate jobs, each one testing a
different bootstrap application: firefox and fennec with normal and
artifact variations.

The emitted task is a giant pile of shell commands that essentially:

1) Perform (hopefully minimal) pre-bootstrap setup
2) Download bootstrap.py from the active revision
3) Invoke the bootstrapper
4) Clone the repo
5) Run `mach build`

Several tasks are broken:

* Arch is completely broken due to attempting to run a command as root
* CentOS7 is completely broken attempting a group install of a missing
  group.
* Debian, Ubuntu, and Fedora aren't installing Clang (likely due to the
  bootstrapper not installing it in non-interactive mode).
* Probably a handful of other failures.

The tasks are all tier 2. And they aren't enabled by default or picked
up by any dependencies. The only way to trigger them is via
`mach try fuzzy --full`.

Considering I've been sitting on this patch for ~2 years and that others
have wanted to test bootstrap in CI, I think getting a somewhat working
version of the CI tasks landed is a good first step and unblocks a lot
of incremental improvements to testing bootstrap. While there are
many shortcomings of this code, I think we should land now and iterate
later.
Flags: needinfo?(jopsen)

The bug assignee didn't login in Bugzilla in the last 7 months, so the assignee is being reset.

Assignee: gps → nobody
Status: ASSIGNED → NEW
Severity: normal → S3
Duplicate of this bug: 1745012
Assignee: nobody → mh+mozilla
Depends on: 1802271
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/03596bbe6b15
Add tasks that perform a bootstrap on various bare Linux environments. r=taskgraph-reviewers,firefox-build-system-reviewers,jlorenzo,ahochheiden

Backed out for causing bootstrap bustages.

Push with failures

Failure log

Backout link

[taskcluster 2022-11-25 02:47:59.507Z] Task ID: Wpd92JnGSse7_JlWFfNREg
[taskcluster 2022-11-25 02:47:59.507Z] Worker ID: 8584270821800645696
[taskcluster 2022-11-25 02:47:59.507Z] Worker Group: us-central1
[taskcluster 2022-11-25 02:47:59.507Z] Worker Node Type: projects/970387039909/machineTypes/n2-custom-16-73728
[taskcluster 2022-11-25 02:47:59.507Z] Worker Pool: gecko-3/b-linux-gcp
[taskcluster 2022-11-25 02:47:59.507Z] Worker Version: 38.0.4
[taskcluster 2022-11-25 02:47:59.507Z] Public IP: 34.121.40.147
[taskcluster 2022-11-25 02:47:59.507Z] Hostname: gecko-3-b-linux-gcp-lpesyk6ztaq-q5h9qhvjga
7d81aaeee7fa - Started downloading
7d81aaeee7fa - Downloaded in 0 seconds
0d1706afb11e - Started downloading
0d1706afb11e - Downloaded in 1.902 seconds
Digest: sha256:353e92f6b8bcd32ae55780feda22ae68cce47c2f41f0c41c2ee596a2e969c1f4
Status: Downloaded newer image for archlinux:latest
[taskcluster 2022-11-25 02:48:06.001Z] === Task Starting ===
+ pacman -Sy --noconfirm mercurial python-pip
error: failed to initialize alpm library:
(root: /, dbpath: /var/lib/pacman/)
could not find or read directory
[taskcluster 2022-11-25 02:48:07.318Z] Unsuccessful task run with exit code: 255 completed in 7.812 seconds
Flags: needinfo?(mh+mozilla)

So, archlinux failed with:

+ pacman -Sy --noconfirm mercurial python-pip
error: failed to initialize alpm library:
(root: /, dbpath: /var/lib/pacman/)
could not find or read directory

debian12 failed with:

W: GPG error: http://deb.debian.org/debian bookworm InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian bookworm InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://deb.debian.org/debian-security bookworm-security InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian-security bookworm-security InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://deb.debian.org/debian bookworm-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian bookworm-updates InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code

ubuntu* failed with:

W: http://security.ubuntu.com/ubuntu/dists/jammy-security/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg are ignored as the file is not readable by user '_apt' executing apt-key.
W: http://security.ubuntu.com/ubuntu/dists/jammy-security/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/ubuntu-keyring-2018-archive.gpg are ignored as the file is not readable by user '_apt' executing apt-key.
W: GPG error: http://security.ubuntu.com/ubuntu jammy-security InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 871920D1991BC93C
E: The repository 'http://security.ubuntu.com/ubuntu jammy-security InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: http://archive.ubuntu.com/ubuntu/dists/jammy/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg are ignored as the file is not readable by user '_apt' executing apt-key.
W: http://archive.ubuntu.com/ubuntu/dists/jammy/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/ubuntu-keyring-2018-archive.gpg are ignored as the file is not readable by user '_apt' executing apt-key.
W: GPG error: http://archive.ubuntu.com/ubuntu jammy InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 871920D1991BC93C
E: The repository 'http://archive.ubuntu.com/ubuntu jammy InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: http://archive.ubuntu.com/ubuntu/dists/jammy-updates/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg are ignored as the file is not readable by user '_apt' executing apt-key.
W: http://archive.ubuntu.com/ubuntu/dists/jammy-updates/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/ubuntu-keyring-2018-archive.gpg are ignored as the file is not readable by user '_apt' executing apt-key.
W: GPG error: http://archive.ubuntu.com/ubuntu jammy-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 871920D1991BC93C
E: The repository 'http://archive.ubuntu.com/ubuntu jammy-updates InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: http://archive.ubuntu.com/ubuntu/dists/jammy-backports/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg are ignored as the file is not readable by user '_apt' executing apt-key.
W: http://archive.ubuntu.com/ubuntu/dists/jammy-backports/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/ubuntu-keyring-2018-archive.gpg are ignored as the file is not readable by user '_apt' executing apt-key.
W: GPG error: http://archive.ubuntu.com/ubuntu jammy-backports InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 871920D1991BC93C
E: The repository 'http://archive.ubuntu.com/ubuntu jammy-backports InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code

opensuse-tumbleweed failed with:

+ zypper install -y mercurial python3-pip
terminate called after throwing an instance of 'std::system_error'
  what():  Operation not permitted

gentoo failed with:

Using keys from /usr/share/openpgp-keys/gentoo-release.asc
!!! Manifest verification impossible due to keyring problem:
OpenPGP key import failed:
gpg: keybox '/tmp/gemato.1ms72rou/pubring.kbx' created
gpg: key DB6B8C1F96D8BF6D: 2 signatures not checked due to missing keys
gpg: /tmp/gemato.1ms72rou/trustdb.gpg: trustdb created
gpg: key DB6B8C1F96D8BF6D: public key "Gentoo ebuild repository signing key (Automated Signing Key) <infrastructure@gentoo.org>" imported
gpg: can't connect to the agent: End of file
gpg: key 9E6438C817072058: 3 signatures not checked due to missing keys
gpg: key 9E6438C817072058: public key "Gentoo Linux Release Engineering (Gentoo Linux Release Signing Key) <releng@gentoo.org>" imported
gpg: can't connect to the agent: IPC connect call failed
gpg: key BB572E0E2D182910: 1 signature not checked due to a missing key
gpg: key BB572E0E2D182910: public key "Gentoo Linux Release Engineering (Automated Weekly Release Key) <releng@gentoo.org>" imported
gpg: can't connect to the agent: End of file
gpg: key A13D0EF1914E7A72: 1 signature not checked due to a missing key
gpg: key A13D0EF1914E7A72: public key "Gentoo repository mirrors (automated git signing key) <repomirrorci@gentoo.org>" imported
gpg: can't connect to the agent: IPC connect call failed
gpg: Total number processed: 4
gpg:               imported: 4
gpg: no need for a trustdb check with 'direct' trust model


 * IMPORTANT: 8 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.


Action: sync for repo: gentoo, returned code = 1

fedora* failed with:

+ dnf install -y mercurial python3-pip
Fedora 36 - x86_64                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'fedora':
  - Curl error (6): Couldn't resolve host name for https://mirrors.fedoraproject.org/metalink?repo=fedora-36&arch=x86_64 [getaddrinfo() thread failed to start]
Error: Failed to download metadata for repo 'fedora': Cannot prepare internal mirrorlist: Curl error (6): Couldn't resolve host name for https://mirrors.fedoraproject.org/metalink?repo=fedora-36&arch=x86_64 [getaddrinfo() thread failed to start]

These tasks work just fine on try. In fact, if you take any of them on firefox-ci-tc, edit it to replace gecko-3/b-linux-gcp with gecko-1/b-linux-gcp, without changing anything else, they all work. Whatever is happening that causes these failures (and weirdly enough, no failure on the debian10, debian11, opensuse-leap, rockylinux8 or voidlinux tasks) comes from whatever difference there is at the docker/host level between gecko-3/b-linux-gcp and gecko-1/b-linux-gcp.

Flags: needinfo?(mh+mozilla)

Michelle or David, do you know if there are differences between the l1 and l3 images that could explain comment 42?

Flags: needinfo?(mgoossens)
Flags: needinfo?(dhouse)

The images are built using https://github.com/taskcluster/monopacker
To explain how it works in a comment is a bit much, but I checked, and other than some secrets, there should not be any differences between the two.

However, I think when we run tasks on a level 3 repo, the tasks are provisioned in a different GCP project that may have restrictions on outbound network connectivity, this is just a wild guess (and I can't check this manually as I don't have the access yet so I am going to defer to dhouse for when he returns next week).

Flags: needinfo?(mgoossens)
Flags: needinfo?(dhouse)

(See above comment)

Flags: needinfo?(dhouse)

In theory, the images should be identical other than the CoT private key.

Is it possible that some tool or library got implicitly updated (due to lack of pinning) when we did https://bugzilla.mozilla.org/show_bug.cgi?id=1768865 ? If it works on L1 but not L3, maybe we just need to generate a new L3 image as well to trigger the same update there.

Perhaps a policy of always updating them both together would be good for reproducibility going forward.

I'll generate some new images.

Flags: needinfo?(dhouse)

(In reply to Michelle Goossens [:masterwayz] from comment #47)

I'll generate some new images.

Did this happen?

Flags: needinfo?(mgoossens)

The image I had made I cannot use so I'll build a new one without those changes.
It's building now.

Flags: needinfo?(mgoossens)

Leaving NI in place so I don't have to spent two hours looking through my email

Flags: needinfo?(mgoossens)
See Also: → 1803015

Backed out changeset 7a1d3c870d87 (bug 1245969)

Backed out one changeset for: image is busted.

Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/ubuntu/docker-worker/node_modules/fs-ext/build/Release/fs-ext.node)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at Object.Module._extensions..node (node:internal/modules/cjs/loader:1239:18)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at Module.load (node:internal/modules/cjs/loader:1033:32)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at Function.Module._load (node:internal/modules/cjs/loader:868:12)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at Module.require (node:internal/modules/cjs/loader:1057:19)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at require (node:internal/modules/cjs/helpers:103:18)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at Object.<anonymous> (/home/ubuntu/docker-worker/node_modules/fs-ext/fs-ext.js:22:15)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at Module._compile (node:internal/modules/cjs/loader:1155:14)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1209:10)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at Module.load (node:internal/modules/cjs/loader:1033:32)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker     at Function.Module._load (node:internal/modules/cjs/loader:868:12)
Nov 30 13:52:35Z gecko-3-decision-gcp-bemiofzkt9uykqadbjtdtq docker-worker 2022/11/30 13:51:24 exit status 1

https://hg.mozilla.org/ci/ci-configuration/rev/8cfd8bde6eb7

Still working on the image! Will let you know once it is fixed.

Pushed by mgoossens@mozilla.com:
https://hg.mozilla.org/ci/ci-configuration/rev/b0beb00b2141
Create level 1 pool for image testing r=releng-reviewers,gbrown
Flags: needinfo?(mgoossens)

Setting this so the bug doesn't close with my regular bugherder run.

Keywords: leave-open
Pushed by mgoossens@mozilla.com:
https://hg.mozilla.org/ci/ci-configuration/rev/ae27e7b9ac33
Use new level 3 GCP builder image with updates r=releng-reviewers,gbrown

Backed out changeset ae27e7b9ac33 (bug 1245969)

Do you have an update?

Flags: needinfo?(mgoossens)

Yesterday I landed a shiny new level 3 image attached to a different bug, forgot to say it here, sorry.
We have a new image in place right now should you want to try it again.

Flags: needinfo?(mgoossens)
See Also: → 1805731

I retriggered one of the jobs that failed on the first landing, and it worked. Yoohoo \o/

Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/675d7821269c
Add tasks that perform a bootstrap on various bare Linux environments. r=taskgraph-reviewers,firefox-build-system-reviewers,jlorenzo,ahochheiden
Regressions: 1806198

Glad that it's fixed now!

Attachment #9305700 - Attachment is obsolete: true
Status: NEW → RESOLVED
Closed: 1 year ago
Keywords: leave-open
Resolution: --- → FIXED
status-firefox110: --- → fixed
Target Milestone: --- → 110 Branch
Depends on: 1805731
See Also: 1805731 →
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: