What Is Docker Volume?

Docker Volumes are persistent data stores for containers, created and managed by Docker, They provide a reliable and efficient way to ensure data persistence when working with containers. Docker volumes are file systems that are mounted on Docker containers to preserve the data generated by the container.

Why Do We Need Docker Volumes?

By design, Docker containers are ephemeral, meaning they are temporary and stateless. Any changes you make inside a running container, such as adding files, installing software, or modifying data, will be lost once the container is removed.

In a typical workflow, if you want to apply changes to a container, you:

• Update the Docker image,
• Remove the old container, and
• Launch a new one from the updated image.

While this approach works well for stateless applications, it becomes problematic for containers that need to retain state such as databases (e.g, MySQL, PostgreSQL) or services that generate or store user data.

Docker volumes solve this problem by allowing you to store data outside the container's filesystem, in a persistent and reusable location managed by Docker. Even if the container is stopped, removed, or rebuilt, the data in the volume remains intact.

This ensures:

• Data durability across container restarts or replacements.
• Separation of application logic (in containers) from persistent data (in volumes).
• A scalable and production-ready way to manage storage for stateful applications.

Independent Lifecycle of Docker Volumes

Unlike a container, which has a specific lifecycle that ends when it’s stopped or removed, a volume's lifecycle is independent of any particular container. This means that the data stored in a volume persists beyond the container that uses it, making it ideal for retaining important information like databases, configuration files, or logs.

• Persistence Beyond Containers: When a container is destroyed, its writable layer is also deleted, but any data stored in a volume remains intact. This ensures that your data is not lost even if the container is removed.
• Shared Access: A volume can be mounted into multiple containers at the same time, enabling easy data sharing across different applications or services.
• Volume Cleanup: Even if no container is currently using a volume, it remains available to Docker and won’t be automatically removed. If you need to free up space, you can remove unused volumes using the docker volume prune command.

This behavior allows you to manage and protect your data independently from the container lifecycle, making volumes an essential tool for ensuring data persistence and flexibility.

Bind Mounts Vs. Docker Volumes:

Bind Mounts

Bind mounts link a specific directory or file from the host system directly into the container. This means that any changes made inside the container or on the host will be instantly reflected on both sides.

• Best for development: Bind mounts are great for development workflows where live code changes need to be reflected immediately. For instance, if you're actively coding, you can bind mount your local source code into a container to test changes without needing to rebuild the image.
• Flexibility but risk: Since bind mounts are tied to the host’s file structure, they are often more flexible but also more prone to issues. Changes made outside the container (by the host) can sometimes interfere with the container's functioning, making them less reliable for production workloads.
• Example use case: Sharing configuration files or logging directories between the host and container.

Docker Volumes

Volumes, on the other hand, are managed by Docker and are independent of the host filesystem. Docker handles their creation and management, providing a more stable and isolated solution for persistent data storage.

• Best for production: Volumes are a better choice when you need to ensure data persists beyond container lifecycles. They’re perfect for databases, application states, or any data that needs to remain consistent even if the container is stopped or removed.
• Automatic management: Docker automatically handles volume creation, backups, and removal, reducing the risk of accidental data loss. Volumes are also less susceptible to errors caused by host-specific filesystem issues.
• Performance & scalability: Volumes often offer better performance than bind mounts, especially when using volume drivers that allow for advanced features like network storage or remote data locations.
• Isolation & security: Since volumes are managed by Docker and exist outside the host’s file structure, they offer better isolation and security compared to bind mounts.

Key Differences

• Use Case: Bind mounts are perfect for temporary or development storage, while volumes are designed for persistent, production-level storage.
• Management: Volumes are easier to manage since Docker takes care of their lifecycle. Bind mounts require manual handling of host directories.
• Performance: Volumes generally offer better I/O performance, particularly in production environments.

Types Of Mounts in Docker:

The data appears the same from within the container in all mount modes. In the filesystem of the container, it is shown as a directory or a single file.

• Volumes: Docker manages volumes kept in a section of the host filesystem (/var/lib/docker/volumes on Linux). This portion of the filesystem shouldn't be altered by non-Docker processes. In Docker, volumes are the most effective way to store data. Using the docker volume create command, we may directly create a volume, or Docker can do it for us when it creates a container or service.
• Named Pipes: To facilitate communication between a container and the Docker host, a named pipe mount can be employed. Using a named pipe to connect to the Docker Engine API while running a third-party program inside a container is the typical use case.
• Bind Mounts: On the host system, bind mounts can be kept anywhere. These might be crucial system folders or files. They are always modifiable by non-Docker processes running on a Docker host or in a Docker container. Comparatively speaking, bind mounts are less useful than volumes.
• Tmpfs Mounts: These mounts are never written to the host system's filesystem; instead, they are kept solely in the memory of the host system. Neither on the Docker host nor in a container is it stored on a disc. Sensitive or non-persistent state data can be stored on the tmpfs mount for the duration of the container.

Named Vs. Anonymous Volumes:

Docker supports two types of volumes: named and anonymous. While both serve the same purpose—persisting data outside of a container’s lifecycle—they behave differently when it comes to management and reuse.

Named Volumes

• Created with a specific name you assign.
• Easy to reference and reuse across multiple containers.
• Ideal for persistent data that needs to be accessed or managed long-term (e.g., db-data, app-config).

Anonymous Volumes

• Automatically assigned a random name (e.g, 13f6a8c5b6b7e0e22349c1...) by Docker.
• Useful for temporary storage when you don’t need to manage the volume manually.
• Persist even after the container is removed, unless the container was run with the --rm flag, in which case the volume is deleted along with the container.

Note: Each container using an anonymous volume gets its own unique volume, so they are not shared automatically between containers. To share an anonymous volume, you must explicitly use its generated volume ID when running other containers.

Docker Volume Plugins

Docker Engine volume plugins link Engine installations with external storage systems such as Amazon EBS, allowing data volumes to survive beyond the lifespan of a single Docker host. For further details, please refer to the plugin documentation.

Command-Line Changes

Use the --volume and --volume-driver options on the docker container run command to grant a container access to a volume. The host's volume name and path are accepted by the --volume (or -v) flag, whereas the driver type is accepted by the --volume-driver flag.

 $ docker volume create --driver=flocker volumename

 $ docker container run -it --volume volumename:/data busybox sh

Volume Plugin Protocol

If a plugin registers itself as a VolumeDriver when activated, it must provide the Docker Daemon with writeable paths on the host filesystem. The Docker daemon provides these paths to containers to consume. The Docker daemon makes the volumes available by bind-mounting the provided paths into the containers.

{
    "Name": "volume_name",
    "Opts": {}
}

Using Docker Volumes

Manually Creating and Linking Volumes with Proper Naming And Labeling Conventions

• Make sure you use appropriate name and labelling practices when establishing Docker volumes.

docker volume create \
--label description="my_vol" \
--label version="1.0.1" \
my_vol

Using Volumes in Dockerfiles with Controlling Permissions For Volumes

• In order to preserve data security and integrity, make sure the appropriate permissions are specified for Docker volumes.

FROM baseimage
RUN mkdir /app/data
RUN chown -R 1000:1000 /app/data
RUN chmod 647 /app/data
VOLUME /app/data

Mounting Volumes as Read-Only

Mounting volumes as read-only in Docker allows for the protection of sensitive or critical data from unintended modifications. By setting the volume option to read-only, you ensure that any changes made within the container are not persisted to the underlying volume, preserving data integrity and security.

docker run -d \
  -v /path/on/host:/path/in/container:ro \
  --name my_container \
  my_image

• -v /path/on/host:/path/in/container:ro mounts the directory /path/on/host on the host machine to /path/in/container in the container as read-only (ro).
• --name my_container assigns the name my_container to the Docker container.
• my_image is the name of the Docker image used to create the container.

Tracking And Controlling Volume Consumption

• To maximise resource consumption, track and adjust Docker volume usage on a regular basis.

$ docker system df -v

Populating Volume Content

When mounting volumes to container paths with existing data, Docker ensures data integrity by copying the existing container data into the new volume. Consequently, neighboring mount points and other containers using the volume will also access the populated content, preventing inadvertent data loss.

Reusing Volumes When Containers Start

Instead of manually specifying each volume with the -v flag, you can use --volumes-from to inherit volumes from an existing container when starting a new container:

# Create the first container
$ docker run -d --name test -v my_vol:/data image:latest

# Create the second container
$ docker run -d --name backup --volumes-from test image:latest

This command automatically mounts all volumes from the "test" container into the "backup" container, simplifying the setup process. It's handy for tasks like backing up data from one container to another.

Interacting With Docker Volumes

Each volume's name and the storage driver that supports it will be shown. Use docker volume inspect to obtain more in-depth details about a particular volume instead:

Inspecting Volumes

To inspect volumes in Docker, you can use the docker volume inspect command followed by the name or ID of the volume you want to inspect. For example:

docker volume inspect my_vol

Removing Volumes

To remove volumes in Docker, you can use the docker volume rm command followed by the name or ID of the volume you want to remove. For example:

docker volume rm my_vol

Pruning Volumes

To prune volumes in Docker, you can use the docker volume prune command. This command removes all volumes not used by at least one container. Here's how you can use it:

docker volume prune

Starting a Container with a Volume

On Using -v Option

• We may start a container with a bind mount using the -v option:

$ docker run -v $(pwd):/var/opt/project bash:latest \
  bash -c "ls /var/opt/project"

• This shows nothing from the mount position. However, if we write to the volume within a single execution of the container:

$ docker run -v data-volume:/var/opt/project bash:latest \
  bash -c "echo Baeldung > /var/opt/project/Baeldung.txt"

Using The –mount Option

• To indicate the volume we want to mount, we might find it easier to use the more obvious –mount option:

$ docker run --mount \
  'type=volume,src=data-volume,\
  dst=/var/opt/project,volume-driver=local,\
  readonly' \ 
  bash -c "ls /var/opt/project"

On Using Shared Volumes

Assume that we used the data-volume mount in a container to run our echo script. Afterwards, we could make a list of every container we've used:

$ docker ps -a
CONTAINER ID   IMAGE             COMMAND                  CREATED              STATUS                          PORTS      NAMES
5774502f857   bash              "docker-entrypoint.s…"   8 minutes ago        Exited (0) 8 minutes ago                   exciting_payne

How to use Docker Volumes?

The following command launches a fresh Ubuntu 22.04 container and connects your terminal to it (-it), enabling you to execute example commands in the ensuing stages. Within the container, a volume named demo_volume is mounted to /data. Use the following command right now:

$ docker run -it -v demo_volume:/data ubuntu:22.06

• Give a list of everything in the /data directory of your container:

$ ls /data

• Include a test file with any random content:

$ echo "foobar" > /data/foo
$ cat /data/foo
foobar

• Launch a fresh container with the same volume attached now:

$ docker run -it -v demo_volume:/app alpine:latest

• Add the --mount option to the docker run command in order to mount a data volume to a container.
• It stores the data created inside the virtual environment by adding volume to the designated container.
• To launch a container and mount a data drive to it, use the following syntax:

$ docker run --mount source=[volume_name],destination=[path_in_container] [docker_image]  

Troubleshooting Common Docker Volume Issues

Permission Denied When Mounting Volumes

• If you are mounting a local disc or host directory with the -v option while operating a Docker container, as follows:

docker run -it --rm \
    -p 8888:8888 \
    -v <my-vol>:<container-dir> \
    quay.io/jupyter/minimal-notebook:latest

Incompatible CPU detected

A processor (CPU) that supports virtualization—more especially, the Apple Hypervisor framework—is necessary for Docker Desktop to function. Only Mac computers with CPUs that support the Hypervisor framework may use Docker Desktop.

$ sysctl kern.hv_support

Path Conversion On Windows

When using Linux, mounting a route to another path is handled by the system. For instance, when executing the subsequent command on a Linux system:

 $ docker run --rm -ti -v /home/user/work:/work alpine

Permissions Errors On Data Directories For Shared Volumes

Docker Desktop defaults the read, write, and execute permissions for both users and groups on shared volumes to 0777 when sharing files from Windows.On shared discs, the default permissions are not customisable. You must either utilise non-host-mounted volumes or figure out a means to get the programmes to operate with the default file permissions if you are working with applications that need permissions different from the shared volume defaults during container runtime.