1391806 - Make valgrind task use its own docker image

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task, P1)

Description

[Wander Lairson Costa]

bug 1382280 tripled the size of desktop-build image because of installation of debug symbols. It is only used for valgrind, so let's move valgrind task to use its own image.

Comment 1

[Wander Lairson Costa]

Attachment: Bug 1391806: Move valgrind task to its own docker image.

bug 1382280 tripled the size of desktop-build image because of
installation of debug symbols. It is only used for valgrind, so let's
move valgrind task to use its own image.

Review commit: https://reviewboard.mozilla.org/r/170502/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/170502/

Comment 2

[(not currently active) Ted Mielczarek]

Comment on attachment 8899427 [details]
Bug 1391806: Move valgrind task to its own docker image.

https://reviewboard.mozilla.org/r/170502/#review175978

This is OK to stop the current bleeding, but we're still going to have the issue where the valgrind-build image is huge. Can you file a followup on making install-debug-symbols.sh install a smaller set of debug symbols? It should be possible to pare it down quite a bit.

::: taskcluster/docker/valgrind-build/Dockerfile:2
(Diff revision 1)
>  # TODO remove VOLUME below when the base image is updated next.
>  FROM          taskcluster/centos6-build-upd:0.1.7.20170801103900

Is it possible to make this inherit from desktop-build so this isn't all copied, or does that not work with the current taskcluster docker image building setup? I worry that this is not going to be maintainable in the long-term.

If not, can you file a followup bug so we can figure out how to make this more maintainable?

Comment 3

[Wander Lairson Costa]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> Comment on attachment 8899427 [details]
> Bug 1391806: Move valgrind task to its own docker image.
> 
> https://reviewboard.mozilla.org/r/170502/#review175978
> 
> This is OK to stop the current bleeding, but we're still going to have the
> issue where the valgrind-build image is huge. Can you file a followup on
> making install-debug-symbols.sh install a smaller set of debug symbols? It
> should be possible to pare it down quite a bit.
> 
> ::: taskcluster/docker/valgrind-build/Dockerfile:2
> (Diff revision 1)
> >  # TODO remove VOLUME below when the base image is updated next.
> >  FROM          taskcluster/centos6-build-upd:0.1.7.20170801103900
> 
> Is it possible to make this inherit from desktop-build so this isn't all
> copied, or does that not work with the current taskcluster docker image
> building setup? I worry that this is not going to be maintainable in the
> long-term.
> 
> If not, can you file a followup bug so we can figure out how to make this
> more maintainable?

Unfortunately it is not possible, as there is no way to inherit from docker-image built images. I think :dustin has a bug/patch/idea for that.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/38df377bad4e
Move valgrind task to its own docker image. r=ted

Comment 5

[Gregory Szorc [:gps]]

I think we should not inherit from in-tree Docker images. Instead, images should be defined/built by composing reusable parts. Inheritance is just fragile. It also adds many more layers to the image, which slows down I/O on images given our current mechanism for exporting/importing images.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

I have vague thoughts on the issue ("we should do that" and "some way to import an image from another task and edit it into the FROM line in a Dockerfile")

Comment 7

[(not currently active) Ted Mielczarek]

(In reply to Gregory Szorc [:gps] from comment #5)
> I think we should not inherit from in-tree Docker images. Instead, images
> should be defined/built by composing reusable parts. Inheritance is just
> fragile. It also adds many more layers to the image, which slows down I/O on
> images given our current mechanism for exporting/importing images.

I'm fine with any way it can be accomplished, I would just like the valgrind-build Dockerfile to be like two lines: "reuse this stuff from desktop-build (however), add this extra script" instead of copying the whole desktop-build Dockerfile.

Comment 8

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/38df377bad4e