Add a Taskcluster job to build with the tup backend

NEW
Assigned to

Status

()

Core
Build Config
a year ago
4 months ago

People

(Reporter: mshal, Assigned: chmanchester)

Tracking

(Depends on: 1 bug, Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

a year ago
There should be a separate Taskcluster job that builds the tree with the tup backend. We'll need to have tup in tooltool for this.
(Assignee)

Updated

5 months ago
Assignee: nobody → cmanchester
(Assignee)

Comment 1

5 months ago
I got pretty far with this, but the builds hit an issue to do with Docker: "fuse: failed to open /dev/fuse: Operation not permitted".

From searching around about this, the workaround I can find suggests adding "--cap-add SYS_ADMIN --device /dev/fuse" to the `docker run` command line, which sounds like it will significantly compromise the container's isolation. I'm not sure if this is an option for TC or if there is a better workaround available.
(Assignee)

Comment 2

5 months ago
Ok, I found the section in the docker docs: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities, which says we need "--cap-add SYS_ADMIN" for this.

To add this would require patching docker-worker I expect, as in bug 1221661, but more to the point we were ok with adding that feature because it was only for testers, this could be a significant hurdle when we get to the point of evaluating a Tup build's suitability for shipping.

Comment 3

5 months ago
CAP_SYS_ADMIN is basically root and would almost completely undermine the process sandbox used by docker-worker. I reckon you'll have a hard time convincing people to give Docker containers that capability. See http://man7.org/linux/man-pages/man7/capabilities.7.html for more.

I suspect there's a way to make fuse work without CAP_SYS_ADMIN. CAP_SYS_ADMIN is probably the most convenient. Although it likely requires changes to docker-worker to expose fuse to containers.

Comment 4

5 months ago
I bet the first problem here is that cgroups isn't allowing access to the /dev/fuse device. You can fix that as a one-off. Why it needs CAP_SYS_ADMIN, I dunno. Extended attributes?
mount()

Comment 6

5 months ago
mount() itself /might/ be safe to expose to the container because presumably the container was started with its own mount namespace. But, yeah, if you can't call mount() without CAP_SYS_ADMIN, then you are in a predicament. We may have to solve this by having the container started with a fuse filesystem pre-mounted. That requires docker-worker cooperation. That also means moving the tup fuse bits into docker-worker. That'll be a rabbit hole.
(Reporter)

Comment 7

5 months ago
I'll also be looking at using an LD_PRELOAD shim again in tup instead of fuse, assuming that will work inside docker. If it does, that would get around us having to fiddle with capabilities I believe. (tup used to use LD_PRELOAD, but I switched it to fuse a while back for various reasons, like staticically-linked binary support).

Comment 8

5 months ago
LD_PRELOAD will work inside Docker as long as the library being loaded doesn't require system calls that the process doesn't have privileges to call :)
(Reporter)

Updated

4 months ago
Depends on: 1387098
You need to log in before you can comment on or make changes to this bug.