1245931 - Use ip address to set DOCKER_HOSTNAME if hostname is 'localunixsocket'

Status: RESOLVED FIXED

(MozReview Graveyard :: General, defect)

Description

[Dan Minor [:dminor]]

At the moment, if the docker hostname is 'localunixsocket' we set it to 'localhost'. This breaks the DOCKER_HOSTNAME foo added in bug 1244588.

We should use the ip address of the docker network interface instead.

Comment 1

[Dan Minor [:dminor]]

Attachment: MozReview Request: testing: use ip address to set docker hostname if it is localunixsocket (bug 1245931) r=gps

If we get a hostname of "localunixsocket", we iterate over the docker
networks looking for the bridge interface so we can use the ip address to
set docker_hostname.

This only works if docker is listening on a tcp socket rather than a unix
domain socket. This was already the recommended configuration, I've updated
the documentation now that it is a requirement.

We might be able to do something similar for boot2docker, but I'm not able
to test that locally.

Review commit: https://reviewboard.mozilla.org/r/33803/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/33803/

Comment 2

[Dan Minor [:dminor]]

Comment on attachment 8716343 [details]
MozReview Request: testing: use ip address to set docker hostname if it is localunixsocket (bug 1245931) r=gps

I think this needs some more testing locally before it's ready for review.

Comment 3

[Gregory Szorc [:gps]]

https://reviewboard.mozilla.org/r/33803/#review30459

Not setting ship it per comment that more testing is needed.

::: docs/hacking-mozreview.rst:48
(Diff revision 1)
> +On Linux, it is necessary to configure Docker to listen on a tcp socket
> +rather than the default configuration, which uses an unix domain socket,
> +in order to get test results which match the expected results. This can be
> +done by editing the configuration file (located at ``/etc/default/docker``
> +on Debian and Ubuntu), to set:
> +
> +  DOCKER_OPTS="-H=tcp://127.0.0.1:4243".

Others have encountered errors when running Docker in the default mode of using a unix domain socket. If we could detect this and raise and raise an error, it would help with the new contributor experience.

::: testing/vcttesting/docker.py:161
(Diff revision 1)
>          # TODO look at network info for Docker and extract IP address instead.

I guess this TODO can go away.

::: testing/vcttesting/docker.py:168
(Diff revision 1)
> +            networks = self.client.networks()
> +            for network in networks:
> +                if network['Name'] == 'bridge':
> +                    ipam = network['IPAM']
> +                    addr = ipam['Config'][0]['Gateway']
> +                    self.docker_hostname = addr
> +                    break

Oh, cool. When I wrote this TODO, I don't think client.networks() existed!

FWIW, my Linux machine has a "bridge" interface as well. However, I don't have a "Gateway" entry:

[
    {
        "Name": "bridge",
        "Id": "602cef57da747ea18b0fee6de678d28c2aa1d8ebd5da146da8811013b9e95859",
        "Scope": "local",
        "Driver": "bridge",
        "IPAM": {
            "Driver": "default",
            "Config": [
                {
                    "Subnet": "172.17.0.0/16"
                }
            ]
        },
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        }
        
    }
]

You can reference `ifconfig docker0` to get the IP address, however:

docker0   Link encap:Ethernet  HWaddr 02:42:dd:c8:47:5d
          inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:ddff:fec8:475d/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:186790 errors:0 dropped:0 overruns:0 frame:0
          TX packets:423666 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:10209692 (10.2 MB)  TX bytes:637096707 (637.0 MB)

Comment 4

[Gregory Szorc [:gps]]

https://reviewboard.mozilla.org/r/33803/#review30459

> Oh, cool. When I wrote this TODO, I don't think client.networks() existed!
> 
> FWIW, my Linux machine has a "bridge" interface as well. However, I don't have a "Gateway" entry:
> 
> [
>     {
>         "Name": "bridge",
>         "Id": "602cef57da747ea18b0fee6de678d28c2aa1d8ebd5da146da8811013b9e95859",
>         "Scope": "local",
>         "Driver": "bridge",
>         "IPAM": {
>             "Driver": "default",
>             "Config": [
>                 {
>                     "Subnet": "172.17.0.0/16"
>                 }
>             ]
>         },
>         "Containers": {},
>         "Options": {
>             "com.docker.network.bridge.default_bridge": "true",
>             "com.docker.network.bridge.enable_icc": "true",
>             "com.docker.network.bridge.enable_ip_masquerade": "true",
>             "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
>             "com.docker.network.bridge.name": "docker0",
>             "com.docker.network.driver.mtu": "1500"
>         }
>         
>     }
> ]
> 
> You can reference `ifconfig docker0` to get the IP address, however:
> 
> docker0   Link encap:Ethernet  HWaddr 02:42:dd:c8:47:5d
>           inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
>           inet6 addr: fe80::42:ddff:fec8:475d/64 Scope:Link
>           UP BROADCAST MULTICAST  MTU:1500  Metric:1
>           RX packets:186790 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:423666 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:10209692 (10.2 MB)  TX bytes:637096707 (637.0 MB)

Actually, I don't think the Docker daemon on my Linux machine was running with -H tcp://. After configuring that, "Gateway" does show up in my config. Perhaps we should disable Docker if "Gateway" isn't set.

Comment 5

[Dan Minor [:dminor]]

That makes sense, I'll add a check for "Gateway" and die nicely if it is not present.

I'll also run with this locally for a little bit longer before I request review again, in case I notice any problems.

Comment 6

[Dan Minor [:dminor]]

Comment on attachment 8716343 [details]
MozReview Request: testing: use ip address to set docker hostname if it is localunixsocket (bug 1245931) r=gps

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/33803/diff/1-2/

Comment 7

[Gregory Szorc [:gps]]

Comment on attachment 8716343 [details]
MozReview Request: testing: use ip address to set docker hostname if it is localunixsocket (bug 1245931) r=gps

https://reviewboard.mozilla.org/r/33803/#review30921

::: testing/vcttesting/docker.py:173
(Diff revision 2)
> +                        print('ERROR: Unable to determine ip address of '
> +                              'the docker gateway. Please ensure docker '
> +                              'is listening on a tcp socket by setting '
> +                              '-H tcp://127.0.0.1:4243 in your docker '
> +                              'configuration file.')
> +                        raise

I know the logic is weird, but the way this class works with regards to failures connecting is `self.client` is set to None and no exception is raised.

This is treated as a recoverable error. Docker is simply marked as unavailable.

I think the print() here is acceptable. Although `warnings.warn()` is preferred. The `raise` should be replaced with `self.client = None` + `break`.

Comment 8

[Dan Minor [:dminor]]

Comment on attachment 8716343 [details]
MozReview Request: testing: use ip address to set docker hostname if it is localunixsocket (bug 1245931) r=gps

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/33803/diff/2-3/

Comment 9

[Gregory Szorc [:gps]]

https://reviewboard.mozilla.org/r/33803/#review30937

Comment 10

[Dan Minor [:dminor]]

https://hg.mozilla.org/hgcustom/version-control-tools/rev/95c4b41a5ba3