Closed Bug 1426324 Opened 2 years ago Closed 2 years ago

Toolchain indexes should vary depending on the docker-image they were built on

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla59

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(1 file)

Example of problem this has just caused me:
- Made changes to build-gcc script *and* switched the gcc build task to a docker image based on Debian.
- This resulted in a toolchain slighly different from what would have been produced on Centos (for instance, built on Debian, we have a gcc/lib32 path, but that path doesn't exist when built on Centos, it's gcc/lib instead)
- I did a subsequent push with the same build-gcc script changes, but *without* the switch to a docker image based on Debian.
- The build tasks still got the GCC toolchain with the gcc/lib32 path.

The problem is that by the time we compute a hash for the index during transformations, we don't actually have access to the docker image to possibly add its hash to the equation.

Dustin, do you have thoughts?
Flags: needinfo?(dustin)
Maybe just pick new names for these new docker images?
Flags: needinfo?(dustin)
(In reply to Dustin J. Mitchell [:dustin] from comment #1)
> Maybe just pick new names for these new docker images?

And add the docker image name to the hash? That seems footgun-y but less so than the current status quo, so I can get behind this.
Yeah, I'm not sure this case is worth a lot of heroics..
Assignee: nobody → mh+mozilla
Comment on attachment 8938257 [details]
Bug 1426324 - Make toolchain cache indexes vary depending on the docker image name.

https://reviewboard.mozilla.org/r/209018/#review214888

::: taskcluster/taskgraph/transforms/job/toolchain.py:97
(Diff revision 1)
> +    # the index path as well. Ideally, the content of the docker image itself
> +    # should have an influence, but at the moment, we can't get that
> +    # information here. So use the docker image name as a proxy. Not a lot of
> +    # changes to docker images actually have an impact on the resulting
> +    # toolchain artifact, so we'll just rely on such important changes to be
> +    # accompanied with a docker image name change.

I like this reasoning :)
Attachment #8938257 - Flags: review?(dustin) → review+
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/f41ca59052be
Make toolchain cache indexes vary depending on the docker image name. r=dustin
https://hg.mozilla.org/mozilla-central/rev/f41ca59052be
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
Had to back this out temporarily to work around bug 1421100 not being deterministic.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Backout by mh@glandium.org:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c1e20de19b6f
Backout changeset f41ca59052be in an attempt to unbust win32 builds. a=bustage on a CLOSED TREE
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/mozilla-inbound/rev/841178c05781
Make toolchain cache indexes vary depending on the docker image name. r=dustin
https://hg.mozilla.org/mozilla-central/rev/841178c05781
Status: REOPENED → RESOLVED
Closed: 2 years ago2 years ago
Resolution: --- → FIXED
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.