Shrink and squash Docker images

RESOLVED FIXED in Firefox 67

Status

task
RESOLVED FIXED
3 months ago
3 months ago

People

(Reporter: catlee, Assigned: catlee)

Tracking

unspecified
mozilla67

Firefox Tracking Flags

(firefox67 fixed)

Details

Attachments

(2 attachments)

Assignee

Description

3 months ago

Our desktop1604-test image is about 1.7GB compressed, is composed of many individual docker layers, and contains many packages that aren't required by the tests.

We should use something like https://github.com/goldmann/docker-squash to squash the images after purging unneeded packages/files from the image.

Assignee

Updated

3 months ago
Assignee: nobody → catlee

Please land this on Saturday night. ni Cristi.

From irc: catlee> it will probably need the trees closed after landing to give time for the new docker images and toolchains to be built

Flags: needinfo?(cbrindusan)
Assignee

Comment 4

3 months ago

thank you!

also the downstream package/toolchain builds tend to fail connecting to snapshots.debian.org. please try and rerun those via the treeherder custom actions instead of retriggering

Assignee

Comment 5

3 months ago

also, please only land D19541: Squash docker images for now. if that works, we can land the next piece, but that's far more likely to break tests :)

Flags: needinfo?(cbrindusan) → needinfo?(nbeleuzu)

Comment 6

3 months ago
Pushed by nbeleuzu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/07c7cd57cc90
Squash docker images before exporting/compressing them r=tomprince
Flags: needinfo?(nbeleuzu)

Comment 7

3 months ago
bugherder
Status: NEW → RESOLVED
Last Resolved: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
Assignee

Updated

3 months ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 8

3 months ago
Pushed by catlee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/bec37c9a80b9
Remove unneeded packages/files in the test image r=jmaher

Comment 9

3 months ago
bugherder
Status: REOPENED → RESOLVED
Last Resolved: 3 months ago3 months ago
Resolution: --- → FIXED
Assignee

Comment 10

3 months ago

Some numbers to follow up here.

I took a look at some graphs before/after the two changes above landed and measured the time from the task starting to the '=== Task Starting ===' line. This time represents downloading the docker image, loading it into the local docker daemon, and starting up a fresh container.

Before (3088 tasks inspected):
mean time: 219s
95th percentile: 619s

After (2586 tasks inspected):
mean time: 52s
95th percentile: 232s

This is an average improvement of 167s, or 76%

If we look at tasks where the worker has to download the image (i.e. it's not cached locally):

Before (1997 tasks inspected):
mean time: 338s
95th percentile: 753s

After (799 tasks inspected):
mean time: 164s
95th percentile: 323s

This is an average improvement of 164s, or 51%.

You need to log in before you can comment on or make changes to this bug.