1295380 - Really run tests with run-task

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Gregory Szorc [:gps]]

+++ This bug was initially created as a clone of Bug #1294234 +++

We tried to switch tests to use run-task in bug 1294234 but I had to back out that commit from the series because of intermittent permissions errors. I think the conversion to run-task somehow resulted in some tasks running as worker:worker instead of root:root.

Let's figure out what went wrong and finish up that conversion.

Comment 1

[Gregory Szorc [:gps]]

Attachment: Bug 1295380 - Teach run-task to chown paths;

It is common for tasks to chown paths before permissions are dropped
from root:root. This commonly occurs when Docker volumes/caches are
involved since they are default owned by root:root and not writable
by any other user.

Since this will likely be a common request, add the functionality to
run-task.

Review commit: https://reviewboard.mozilla.org/r/71998/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/71998/

Comment 2

[Gregory Szorc [:gps]]

Attachment: Bug 1295380 - Use run-task for test tasks;

Before, test.sh (duplicated between the desktop-test and
desktop1604-test images) was dropping permissions, creating a workspace,
and executing test-linux.sh. This is functionality now provided by
run-task.

So, convert the test tasks to use run-task.

It's worth noting that creating the workspace directory is no longer
necessary because it is defined as a VOLUME in the Dockerfile or a
cache in some task configurations. However, it is default owned by
root:root, so we do need to instruct run-task to chown it.

Since the test.sh files are no longer used after this change, they have
been deleted. The desktop-test image no longer has any files in the
bin/ directory, so the Dockerfile entry to copy those files has been
removed.

Review commit: https://reviewboard.mozilla.org/r/72000/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/72000/

Comment 3

[Gregory Szorc [:gps]]

Comment on attachment 8781605 [details]
Bug 1295380 - Use run-task for test tasks;

https://reviewboard.mozilla.org/r/72000/#review69488

This commit was pretty much already reviewed in bug 1294234. The main difference is the introduction of --chown to ensure permissions on the workspace cache/volume are happy.

Comment 4

[Gregory Szorc [:gps]]

Comment on attachment 8781604 [details]
Bug 1295380 - Teach run-task to chown paths;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/71998/diff/1-2/

Comment 5

[Gregory Szorc [:gps]]

Comment on attachment 8781605 [details]
Bug 1295380 - Use run-task for test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72000/diff/1-2/

Comment 6

[Gregory Szorc [:gps]]

Comment on attachment 8781604 [details]
Bug 1295380 - Teach run-task to chown paths;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/71998/diff/2-3/

Comment 7

[Gregory Szorc [:gps]]

Comment on attachment 8781605 [details]
Bug 1295380 - Use run-task for test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72000/diff/2-3/

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8781604 [details]
Bug 1295380 - Teach run-task to chown paths;

https://reviewboard.mozilla.org/r/71998/#review70300

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8781605 [details]
Bug 1295380 - Use run-task for test tasks;

https://reviewboard.mozilla.org/r/72000/#review70302

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/213a4986ccfd
Teach run-task to chown paths; r=dustin
https://hg.mozilla.org/integration/autoland/rev/905fb4a53386
Use run-task for test tasks; r=dustin

Comment 11

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Backed out in https://hg.mozilla.org/integration/autoland/rev/f90d73ed0269 because Treeherder is going to require some changes before it can read the changes to log formats.

Comment 12

[Gregory Szorc [:gps]]

Bug 1297184 is deployed to treeherder stage. Once that makes it to production, we should be clear to reland this.

Comment 13

[Gregory Szorc [:gps]]

Comment on attachment 8781604 [details]
Bug 1295380 - Teach run-task to chown paths;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/71998/diff/3-4/

Comment 14

[Gregory Szorc [:gps]]

Comment on attachment 8781605 [details]
Bug 1295380 - Use run-task for test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72000/diff/3-4/

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/805d466424e4
Teach run-task to chown paths; r=dustin
https://hg.mozilla.org/integration/mozilla-inbound/rev/4fc36d02980f
Use run-task for test tasks; r=dustin

Comment 16

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/805d466424e4
https://hg.mozilla.org/mozilla-central/rev/4fc36d02980f