How to set DOCKER_HOST on 4.13.0 without /var/run/docker.sock

[joshua-at-aws]

Expected behavior

• DOCKER_HOST working in Docker Desktop 4.13.0

Actual behavior

• Docker Desktop for Mac removed /var/run/docker.sock per 4.13.0 Release Notes: https://docs.docker.com/desktop/release-notes/#docker-desktop-4130 "By default Docker will not create the /var/run/docker.sock symlink on the host and use the docker-desktop CLI context instead."

• I use Docker Desktop along with other tools that require DOCKER_HOST. The docs say to set this to the /var/run/docker.sock https://docs.docker.com/desktop/faqs/general/#how-do-i-connect-to-the-remote-docker-engine-api

Is there a different setting for DOCKER_HOST that will work in 4.13.0, or a preference to create the socket?
I poked around a bit but did not find anything in the UI. Thanks in advance!

Information

• macOS Version: 12.6 Monterey
• Intel chip or Apple chip: Intel
• Docker Desktop Version: 4.13.0

Steps to reproduce the behavior

1. Install 4.13.0 on Mac
2. DOCKER_HOST not working

[nicks]

Thanks for reporting!

We're trying to make Docker Desktop more userspace-friendly (e.g., fewer root privileges). As part of that, we're migrating the sockets to user-owned directories rather than root-owned directories (like /var/run).

Everything should work correctly if you unset DOCKER_HOST and let the default socket-picking behavior kick in?

There's supposed to be some shim code that fixes this automatically, let me check in with some people and get back to you.

[joshua-at-aws]

Hi, thanks for the reply!

If I do unset DOCKER_HOST then the other tools, e.g. Maven or AWS SAM CLI do not work Error: Running AWS SAM projects locally requires Docker.

Could the new user-owned directory for the socket be added to the docs?

[nicks]

Can you post a diagnostic id and the output of docker context list?

I think it's currently at ~/.docker/run/docker.sock, though I've been told that may change again. we're trying to get away from people hard-coding it.

[joshua-at-aws]

• Diagnostics ID: 8E63E195-B162-440D-826A-09E31E28D3A6/20221021210358

docker context list 
NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT                                KUBERNETES ENDPOINT   ORCHESTRATOR
default *           moby                Current DOCKER_HOST based configuration   unix:///Users/joshuf/.docker/run/docker.sock                         swarm
desktop-linux       moby                                                          unix:///Users/joshuf/.docker/run/docker.sock     

Setting DOCKER_HOST to the default socket works! I see docker context list has a JSON output format, which I can use in a script if that's the new recommended way, for example with a Python one-liner:

docker context list --format json | python3 -c "import sys, json; j=json.load(sys.stdin); [print(x['DockerEndpoint']) if x['Current'] else None for x in j]"

unix:///Users/joshuf/.docker/run/docker.sock

[nicks]

Our current plan is to roll this change back for now and figure out a more orderly upgrade path with docs for client library authors.

[hopeseekr]

Here is the workaround for OSX:

sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock

[hopeseekr]

I'd like to add that many thirdparty utilities, particularly IntelliJ and PhpStorm, expect there to be a /var/run/docker.sock. This change broke a wide swath of our toolchain.

[johndutchover]

Here is the workaround for OSX:

sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock

@hopeseekr I needed to force -f
sudo ln -sf "$HOME/.docker/run/docker.sock" /var/run/docker.sock

[tbeauvais-imagine]

This wasted 4 hours of my life thinking it was a configuration problem that I had.

As no valid configuration was found, execution cannot continue
Root cause NoSuchFileException (/var/run/docker.sock)

I appreciate the intent of this change, yet the intent should have been captured in the release notes. Also, with such changes, you should have a workaround available BEFORE release to ensure people can continue as necessary.

Still though, updated the DOCKER_HOST doesn't work since a lot of 3rd parties rely on the location of the sock descriptor since it has been there for a decade. To change it all of a sudden is really a really bold and strange call.

Hopefully, people find this (though I didn't) and don't waste hours like I did.

[johndutchover]

This wasted 4 hours of my life thinking it was a configuration problem that I had.

As no valid configuration was found, execution cannot continue Root cause NoSuchFileException (/var/run/docker.sock)

I appreciate the intent of this change, yet the intent should have been captured in the release notes. Also, with such changes, you should have a workaround available BEFORE release to ensure people can continue as necessary.

Still though, updated the DOCKER_HOST doesn't work since a lot of 3rd parties rely on the location of the sock descriptor since it has been there for a decade. To change it all of a sudden is really a really bold and strange call.

Hopefully, people find this (though I didn't) and don't waste hours like I did.

updated post

Desktop 4.13.0 (89412) no longer hangs with "workaround", however I am not able to connect to Docker daemon . See steps taken below. Yes, previous release 4.12 does work on this setup.

My env:

• macOS Version: 13.0 Ventura (previously didn't work in 12.6 Monterey)
• Intel chip or Apple chip: Apple M1 Pro
• Docker Desktop Version: 4.13.0,89412

Steps taken (zsh)

• sudo ln -sf "$HOME/.docker/run/docker.sock" /var/run/docker.sock
• add export DOCKER_HOST=unix://"$HOME/.docker/run/docker.sock" (note only two forward slashes after colon) to .zprofile
• restart

Output after restart and login:
zsh ➜ docker version
Cannot connect to the Docker daemon at unix:///Users/username/.docker/run/docker.sock. Is the docker daemon running?
Client:
Cloud integration: v1.0.29
Version: 20.10.20
API version: 1.41
Go version: go1.18.7
Git commit: 9fdeb9c
Built: Tue Oct 18 18:20:35 2022
OS/Arch: darwin/arm64
Context: default
Experimental: true

unclear why "true" since set to false in $HOME/.docker/daemon.json

~
zsh ❯ docker context list
NAME TYPE DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * moby Current DOCKER_HOST based configuration unix:///Users/username/.docker/run/docker.sock swarm
desktop-linux moby unix:///Users/username/.docker/run/docker.sock

[johndutchover]

I found an issue when tailing file DockerAppStderr.txt in ~/Library/Group\ Containers/group.com.docker/DockerAppStderr.txt after launching 4.13.0 (89412):

time="2022-10-28T12:59:28Z" level="info" msg="GoBackend: notifying bugsnag: setting overrides for proxies: computing HTTPIP of 192.168.65.0/28: prefix of 28 does not accommodate a host numbered 128

I interpreted this to mean that the default Docker subnet (/28) subnet is too small.

Steps taken (in addition to #6529 (comment))

1. Launch Docker Desktop 4.12.0 (85629)
2. Navigate: Preferences -> Resources -> Network and changed subnet to /24

3. Updated in-app to 4.13.0 (89412)

And it's working!