Closed Bug 1527394 Opened 2 years ago Closed 2 years ago

Shrink and squash Docker images

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set
normal

Tracking

(firefox67 fixed)

RESOLVED FIXED
mozilla67
Tracking Status
firefox67 --- fixed

People

(Reporter: catlee, Assigned: catlee)

References

Details

Attachments

(2 files)

Our desktop1604-test image is about 1.7GB compressed, is composed of many individual docker layers, and contains many packages that aren't required by the tests.

We should use something like https://github.com/goldmann/docker-squash to squash the images after purging unneeded packages/files from the image.

Assignee: nobody → catlee

Please land this on Saturday night. ni Cristi.

From irc: catlee> it will probably need the trees closed after landing to give time for the new docker images and toolchains to be built

Flags: needinfo?(cbrindusan)

thank you!

also the downstream package/toolchain builds tend to fail connecting to snapshots.debian.org. please try and rerun those via the treeherder custom actions instead of retriggering

also, please only land D19541: Squash docker images for now. if that works, we can land the next piece, but that's far more likely to break tests :)

Flags: needinfo?(cbrindusan) → needinfo?(nbeleuzu)
Pushed by nbeleuzu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/07c7cd57cc90
Squash docker images before exporting/compressing them r=tomprince
Flags: needinfo?(nbeleuzu)
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/bec37c9a80b9
Remove unneeded packages/files in the test image r=jmaher
Status: REOPENED → RESOLVED
Closed: 2 years ago2 years ago
Resolution: --- → FIXED

Some numbers to follow up here.

I took a look at some graphs before/after the two changes above landed and measured the time from the task starting to the '=== Task Starting ===' line. This time represents downloading the docker image, loading it into the local docker daemon, and starting up a fresh container.

Before (3088 tasks inspected):
mean time: 219s
95th percentile: 619s

After (2586 tasks inspected):
mean time: 52s
95th percentile: 232s

This is an average improvement of 167s, or 76%

If we look at tasks where the worker has to download the image (i.e. it's not cached locally):

Before (1997 tasks inspected):
mean time: 338s
95th percentile: 753s

After (799 tasks inspected):
mean time: 164s
95th percentile: 323s

This is an average improvement of 164s, or 51%.

You need to log in before you can comment on or make changes to this bug.