a common task in docker is moving the location of volumes and containers to a different drive; since, in my experience these can be many gigabites and need to be backed up frequently. however, there doesn't seem to be an 'official' way of doing this in the documentation. this writeup discusses some of the options. some people have managed it by adding a data-root to /etc/docker/daemon.json, but this didn't work for me (and many others). my solution involves changing the data root in the docker service directive.
Stop the docker service:
sudo systemctl stop docker
Move the data to your new location/mount point. You can do this with an NFS/iSCSI mount, I did it by passing through a virtual disk:
sudo rsync -axPS /var/lib/docker/ /new/docker/mount/
This doesn't delete the files in the old mount location. You can do this once you've made sure the transfer has worked. Then edit the docker service file:
sudo vi /lib/systemd/system/docker.service
Find the line starting with ExecStart=/usr/bin/dockerd and add a --data-root directive; e.g:
ExecStart=/usr/bin/dockerd --data-root /new/docker/mount/ -H fd:// --containerd=/run/containerd/containerd.sock
then restart the docker service:
sudo systemctl daemon-reload
sudo systemctl start docker
You can check it's worked with docker info | grep "Root Dir". At this point it should work, but for me I got an error that existing containers couldn't start because the docker root doesn't exist. apparently individual containers need to be told that the location has changed. you can do this by editing the config.v2.json of each container, according to this, but for me this would take a very long time so I just made a symlink of my new location to /var/lib/docker/ an alternative (better) solution would be to simply mount the new drive at /var/lib/docker/ in /etc/fstab/
My docker volumes are mounted using a virtual disk passed through from my hypervisor, truenas core; at /dev/vda. This is done since I assume it has less overhead than using NFS or iSCSI, and is possible since it is a VM.
for more information about my software stack, check out this blog post.