1214809 - Figure out how to run pulseAudio in a docker container

Status: RESOLVED FIXED

(Taskcluster Graveyard :: Docker Images, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

I had no end of trouble in bug 1213325 trying to get pulseaudio to work, on the assumption it had been done before in the tester image.  I eventually figured out that's not the case, and nobody at Mozilla has successfully run pulseaudio in a docker container.

We'll probably need to sort that out to get tests running that rely on it.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

http://stackoverflow.com/questions/28985714/run-apps-using-audio-in-a-docker-container

Comment 2

[(not currently active) Ted Mielczarek]

I've just recently done some unrelated-but-relevant hacking, and found that running PulseAudio inside Docker worked fine, as long as you don't actually care about getting the audio *out* of the container.

Comment 3

[(not currently active) Ted Mielczarek]

You do have to manually start the PulseAudio daemon to make things work, but you can just run `pulseaudio --fail --daemonize --start`. I also ran `pactl load-module module-null-sink` to get a default audio output.

Comment 4

[(not currently active) Ted Mielczarek]

From bug 1213325, it looks like you're trying to get PulseAudio to have access to actual sound devices. I don't think that's something we need to do.

We probably do need to give the test containers access to /dev/videoN so they can run loopback video tests, since unfortunately that requires access to a real device.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

Starting pulseaudio with no /dev/snd/* defined failed -- the daemon exited.  Was there some config that you applied to make it work?  Also, I was unable to replicate the container-killing behavior at an interactive shell -- it only occurred when running a shell script that invoked pulseaudio (although it did so both on my desktop and on TC EC2 instances).

The video's a separate issue (I hope!)

Comment 6

[(not currently active) Ted Mielczarek]

I didn't do anything particularly fancy, I built the container using this Dockerfile:
https://github.com/luser/sip-transcribe/blob/master/Dockerfile

Running it with no special options I'm able to launch pulseaudio as per comment 3 and things work. I can run `paplay /path/to.wav` and it runs. (As you can see from the rest of that repository, I've also used the null sink as a loopback device to capture audio being output from another app.)

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Indeed, with no sound devices mounted:

root@taskcluster-worker:~# chown -R worker:worker /home/worker/
root@taskcluster-worker:~# su - worker
worker@taskcluster-worker:~$ pulseaudio --fail --daemonize --start
worker@taskcluster-worker:~$ pactl load-module module-null-sink
17
worker@taskcluster-worker:~$ paplay a2002011001-e02-ulaw.wav

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

https://tools.taskcluster.net/task-inspector/#QsO_p2HZSC6rSS3m74ywPg/
  "command": [
    "bash",
    "-c",
    "chown -R worker:worker /home/worker && su - worker bash -c 'pulseaudio --fail --daemonize --start && sleep 2 && echo success'"
  ],
=> success

----

https://tools.taskcluster.net/task-inspector/#F2dTVWJvTKqghx4RufaQiw/0
test-linux.sh:
https://hg.mozilla.org/users/dmitchell_mozilla.com/mozilla-central/file/d648607bb9fd/testing/taskcluster/scripts/tester/test-linux.sh
#! /bin/bash -xe

set -x -e

echo "running as" $(id)

    pulseaudio --fail --daemonize --start
    pactl load-module module-null-sink
    echo "let's see.."

    sleep 10
    echo "good!"

=> failure

----

Running the latter from my own desktop (Docker-1.8.2)

+ : GECKO_HEAD_REPOSITORY https://hg.mozilla.org/users/dmitchell_mozilla.com/mozilla-central
+ : GECKO_HEAD_REV d648607bb9fd
+ : WORKSPACE /home/worker/workspace
++ id -u
+ '[' 0 = 0 ']'
+ chown -R worker:worker /home/worker
+ exec su worker /home/worker/bin/test.sh -- --mochitest-suite plain-chunked --total-chunk=5 --this-chunk=1 --installer-url=https://queue.taskcluster.net/v1/task/EAOdruDTTq6OPTPkNB8tUw/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/EAOdruDTTq6OPTPkNB8tUw/artifacts/public/build/test_packages.json --no-read-buildbot-config --download-symbols ondemand
+ : GECKO_HEAD_REPOSITORY https://hg.mozilla.org/users/dmitchell_mozilla.com/mozilla-central
+ : GECKO_HEAD_REV d648607bb9fd
+ : WORKSPACE /home/worker/workspace
++ id -u
+ '[' 1000 = 0 ']'
+ '[' -d /home/worker/workspace ']'
+ mkdir -p /home/worker/workspace
+ cd /home/worker/workspace
+ script=testing/taskcluster/scripts/tester/test-linux.sh
+ url=https://hg.mozilla.org/users/dmitchell_mozilla.com/mozilla-central/raw-file/d648607bb9fd/testing/taskcluster/scripts/tester/test-linux.sh
+ curl --fail -o ./test-linux.sh --retry 10 https://hg.mozilla.org/users/dmitchell_mozilla.com/mozilla-central/raw-file/d648607bb9fd/testing/taskcluster/scripts/tester/test-linux.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   189  100   189    0     0    198      0 --:--:-- --:--:-- --:--:--   322
+ chmod +x ./test-linux.sh
+ exec ./test-linux.sh --mochitest-suite plain-chunked --total-chunk=5 --this-chunk=1 --installer-url=https://queue.taskcluster.net/v1/task/EAOdruDTTq6OPTPkNB8tUw/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/EAOdruDTTq6OPTPkNB8tUw/artifacts/public/build/test_packages.json --no-read-buildbot-config --download-symbols ondemand
+ set -x -e
++ id
+ echo 'running as' 'uid=1000(worker)' 'gid=1000(worker)' 'groups=1000(worker)'
running as uid=1000(worker) gid=1000(worker) groups=1000(worker)
+ pulseaudio --fail --daemonize --start
+ pactl load-module module-null-sink

=> failure

----

Reducing test.sh to the following, and running it in an image with /home/worker owned by worker:

#! /bin/bash -vex
set -x -e
if [ $(id -u) = 0 ]; then
    chown -R worker:worker /home/worker
    # drop privileges by re-running this script
    exec su worker /home/worker/bin/test.sh -- "${@}"
fi
pulseaudio --fail --daemonize --start || echo failed
pactl load-module module-null-sink || echo failed
echo "let's see.."
sleep 10
echo "good!"

docker run => failure
docker run -u worker => success

----

docker run:
+ id -a
uid=1000(worker) gid=1000(worker) groups=1000(worker)
+ id -r -g
1000
+ id -r -u
1000
+ id -r -G
1000

docker run -u worker:
+ id -a
uid=1000(worker) gid=1000(worker) groups=1000(worker)
+ id -r -g
1000
+ id -r -u
1000
+ id -r -G
1000

(no difference)

----

replacing 'su' with a python script:
#! /bin/bash -vex
set -x -e
if [ $(id -u) = 0 ]; then
    # drop privileges by re-running this script
    exec python <<'EOF'
import os
os.setgid(1000)
os.setuid(1000)
os.execv('/bin/bash', ['bash', '/home/worker/bin/test.sh'])
EOF
fi
id -a
id -r -g
id -r -u
id -r -G
pulseaudio --fail --daemonize --start || echo failed
pactl load-module module-null-sink || echo failed
echo "let's see.."
sleep 10
echo "good!"

=> success

I ran an strace of the 'su' process to see what silliness it's doing; trace is attached.

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: strace of su

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

The most relevant bits of that strace:

getuid()                                = 0
getuid()                                = 0
getuid()                                = 0
getuid()                                = 0
setgid(1000)                            = 0
setgroups(1, [1000])                    = 0
getuid()                                = 0
getuid()                                = 0
getresuid([0], [0], [0])                = 0
getresgid([1000], [1000], [1000])       = 0
setuid(1000)                            = 0

and even adjusting the python script to make those calls:

import os
print(os.getuid())
os.setgid(1000)
os.setgroups([1000])
os.setuid(1000)
print(os.getresuid())
print(os.getresgid())
os.execv('/bin/bash', ['bash', '/home/worker/bin/test.sh'])

still succeeds.

Comment 11

[Dustin J. Mitchell [:dustin] (he/him)]

Even simpler, on the advice of
  http://unix.stackexchange.com/questions/132663/how-do-i-drop-root-privileges-in-shell-scripts

#! /bin/bash -vex
set -x -e
if [ $(id -u) = 0 ]; then
    # drop privileges by re-running this script
    exec sudo -u worker bash $0
fi
pulseaudio --fail --daemonize --start || echo failed
pactl load-module module-null-sink || echo failed
echo "let's see.."
sleep 10
echo "good!"

=> success

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

Great Success!!
  https://tools.taskcluster.net/task-inspector/#PW-BDQ_XQVydS6k_lHTpJA/1
(the failures are because I was using one of ahal's task descriptions -- unrelated to this bug)

Comment 13

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: MozReview Request: Bug 1214809: use 'sudo' instead of 'su' to drop root privs; r?ted

Bug 1214809: use 'sudo' instead of 'su' to drop root privs; r?ted

Comment 14

[(not currently active) Ted Mielczarek]

Comment on attachment 8676524 [details]
MozReview Request: Bug 1214809: use 'sudo' instead of 'su' to drop root privs; r?ted

https://reviewboard.mozilla.org/r/22741/#review20239

Weird.

Comment 15

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d43d4d4e26fb

Comment 16

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/d43d4d4e26fb