Automated Nginx Reverse Proxy for Docker

Tue, 25 Mar 2014 00:00:00 UTC

A reverse proxy server is a server that typically sits in front of other web servers in order to provide additional functionality that the web servers may not provide themselves.

For example, a reverse proxy can provide SSL termination, load balancing, request routing, caching, compression or even A/B testing.

When running web services in docker containers, it can be useful to run a reverse proxy in front of the containers to simplify depoyment.

Why Use A Reverse Proxy With Docker

Docker containers are assigned random IPs and ports which makes addressing them much more complicated from a client perspsective. By default, the IPs and ports are private to the host and cannot be accessed externally unless they are bound to the host.

Binding the container to the hosts port can prevent multiple containers from running on the same host. For example, only one container can bind to port 80 at a time. This also complicates rolling out new versions of the container without downtime since the old container must be stopped before the new one is started.

A reverse proxy can help with these issues as well as improve availabilty by facilitating zero-downtime deployments.

Generating Reverse Proxy Configs

Setting up a reverse proxy configuration can be complicated when containers are started and stopped. Typically the configuration needs to be updated manually which is error prone and time consuming.

Fortunately, Docker provides a remote API to inspect containers and access their IP, Ports and other configuration meta-data. In addition, it also provides a real-time events API that can be used for notifications when containers are started and stopped. These APIs can be used to generate a reverse proxy config automatically.

docker-gen is a small utility that uses these APIs and exposes container meta-data to templates. Templates are rendered and an optional notification command can be run to restart the service.

Using docker-gen, we can generate Nginx config files automatically and reload nginx when they change. The same approach can also be used for docker log management.

Nginx Reverse Proxy for Docker

This example nginx template can be used to generate a reverse proxy configuration for docker containers using virtual hosts for routing. The template is implemented using the golang text/template package. It uses a custom groupBy template function to group the running containers by their VIRTUAL_HOST environment variable. This simplifies iterating over the containers to generate a load-balanced backend and also enables zero-downtime deployments.

{{ range $host, $containers := groupBy $ "Env.VIRTUAL_HOST" }}
upstream {{ $host }} {

{{ range $index, $value := $containers }}
    {{ with $address := index $value.Addresses 0 }}
    server {{ $address.IP }}:{{ $address.Port }};
    {{ end }}
{{ end }}

}

server {
    #ssl_certificate /etc/nginx/certs/demo.pem;
    #ssl_certificate_key /etc/nginx/certs/demo.key;

    gzip_types text/plain text/css application/json application/x-javascript
               text/xml application/xml application/xml+rss text/javascript;

    server_name {{ $host }};

    location / {
        proxy_pass http://{{ $host }};
        include /etc/nginx/proxy_params;
    }
}
{{ end }}

The template can be run with docker-gen using:

docker-gen -only-exposed -watch -notify "/etc/init.d/nginx reload" templates/nginx.tmpl /etc/nginx/sites-enabled/default

-only-exposed - Only use containers that have exposed ports.
-watch - After starting up, watch for docker container events and regenerate the template.
-notify "/etc/init.d/nginx reload" - Reload the nginx config after the template is generated.
templates/nginx.tmpl - The nginx template.
/etc/nginx/sites-enabled/default - Destination file.

This is the rendered template with two containers configured with VIRTUAL_HOST=demo1.localhost and one with VIRTUAL_HOST=demo2.localhost

upstream demo1.localhost {
    server 172.17.0.4:5000;
    server 172.17.0.3:5000;
}

server {
    #ssl_certificate /etc/nginx/certs/demo.pem;
    #ssl_certificate_key /etc/nginx/certs/demo.key;

    gzip_types text/plain text/css application/json application/x-javascript
               text/xml application/xml application/xml+rss text/javascript;

    server_name demo1.localhost;

    location / {
        proxy_pass http://demo.localhost;
        include /etc/nginx/proxy_params;
    }
}

upstream demo2.localhost {
    server 172.17.0.5:5000;
}

server {
    #ssl_certificate /etc/nginx/certs/demo.pem;
    #ssl_certificate_key /etc/nginx/certs/demo.key;

    gzip_types text/plain text/css application/json application/x-javascript
               text/xml application/xml application/xml+rss text/javascript;

    server_name demo2.localhost;

    location / {
        proxy_pass http://demo2.localhost;
        include /etc/nginx/proxy_params;
    }
}

Try It Out

I created a trusted build with this setup to make it easier to try it out:

Run nginx-proxy container:

$ docker run -d -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock -t jwilder/nginx-proxy

Start your containers with a VIRTUAL_HOST environment variables:

$ docker run -e VIRTUAL_HOST=foo.bar.com -t ...

If you need HTTPS, would like to run docker-gen in a separate container from nginx, Websocket support or other features, take a look at the github project for more information.

Conclusion

Generating nginx reverse proxy configs for docker containers can be automated using the Docker APIs and some basic templating. This can simplify deployments as well as improve availability.

While this works well for containers running on a single host, generating configs for remote hosts requires service discovery. Take a look at docker service discovery for a solution to that problem.

Update: There’s a few other posts with similar ideas and variations that are worth checking out: