Welcome back! This is part 2 of a series of articles on Docker. In Part One we looked at setting up a private registry in order to store our own images without using Docker's public Hub. While the Hub does provide for a single private image, running your own registry is usually more convenient for managing multiple containers, and it also allowed us to get our feet wet.
In this article, we look at the Flocker project. My primary role in companies for the last 15 years has been Oracle databases (although not the only role!). When using Docker and exploring its functionality, one thing always niggles at me a bit. Containers are great for rapid deployment, modularity and scalability. If you start a container you can stop and start it as many times as you want and your data stays put. However, deploy that container to a NEW host and ... oh, it's started from scratch all over again without the data from the other node. You can use volumes, but this has problems of its own. If I expose the host filesystem to store my data from the container, what if I want to move the container? First I have to migrate it, then migrate my data. Containers are great for rapidly deploying but what about rapidly moving? If I have a cluster of 20 nodes meant for Docker containers, and I want to shut down node 1 for maintenance, I want to quickly move all the containers there to node 2 or node 3. What's a sysadmin or DBA to do? Follow below for an answer!
Let's get set up first, and then we can see the magic happen.
Flocker has two kinds of hosts. Any host with the CLI can manage containers. Flocker nodes have special configuration that allow them to do their magic. Using the flocker CLI we manage Docker containers that are deployed to flocker nodes. For this example I have three virtual machines. flocker.local is a regular Fedora 21 install with the flocker CLI installed. flocker1.local and flocker2.local have the flocker-node software installed. Note that currently flocker-node only runs on Fedora 20. You can also use Vagrant to spin up virtual machines with VirtualBox, use Amazon Web Services, DigitalOcean or rackspace. See the complete install guide if you want to try an alternate platform. I'm going to stick with F20 for the nodes.
First, I created my two virtual machines flocker1 and flocker2. Once built using Fedora 20, we install prerequisites:
Next we install flocker-node dependencies including ZFS and the flocker-node package itself:
Note: This is straight out of the documentation, which can be found here.
Note that docker is a dependency and will automatically be installed as well. Let's start it and make sure it's enabled. We'll also modify some firewall settings to enable some magic later on:
Next, Flocker uses ZFS in order to manage filesystems and move them from host to host. While you can use zfs to initialize a disk device for testing, you can also use a file as a backing store for a ZFS pool. Flocker requires a pool named "flocker". Run the following on flocker1 and flocker2 (or whatever you are calling them) to create a 10GB pool:
And last step, we need to set up SSH equivalency between the host you'll be running the CLI commands (in this example flocker.local) and the flocker-nodes (flocker1 and flocker2). There are several ways to do this, here's one. Use ssh-keygen -t dsa to generate DSA keys on flocker.local (the cli node). Press enter at all prompts:
This will create several files in ~/.ssh. We need the id_dsa.pub file. Copy the contents of this file to the flocker nodes (flocker1 and flocker2) into the file ~/.ssh/authorized_keys. Make sure the authorized keys file is mode 600 for security sake. You can do this by issuing the command chown 600 authorized_keys
,Whew! That takes care of the nodes. Now let's finish up by installing the CLI. Again, following the Flocker install document (but trimming some comments for brevity.) First we need to install some dependencies:
Once these are installed, run these commands to create a "flocker-tutorial" directory that will contain the CLI. Save the following script as install.sh:
Execute the script by typing sh install.sh. Once done you will now have a flocker-tutorial directory. Underneath that is a bin directory that contains the flocker-deploy command, which we need for the demo. To make life easier let's put the bin directory on your path. Assuming flocker-tutorial is in your home directory, this command will add it to the path:
To test this, type flocker-deploy which will show you a usage screen:
I can hear you now. "We've installed a lot of stuff and I don't see any magic!" You're right. The magic starts now. Flocker works by defining an application, and then deploying that application. To make for a quick example that doesn't require a lot of setup time, let's deploy a redis server, and move it around. In order to save on bandwith and time I'm going to pull the official Redis images into my local registry, and use that copy to deploy it to the flocker1 or flocker2 nodes. First, pull the image:
Now we tag the image, and push it into the local repository:
Ok, nothing too new there. Now let's define the redis application for flocker. Flocker uses YAML (Yet Another Markup Language), a reasonably easy document format that humans can read. Take a look at this application configuration and store it in a file called "redis-application.yml":
This is all fairly straightforward, and mirrors many docker commands. docker run -p 6379:6379 -v /tmp:/data flocker.local/redis would get you something similar, except that the volume is not mounted in /tmp, but in the ZFS pool we created earlier. More on that in a second.
Now let's look at a deployment file. This is very similar, and also somewhat shorter. Lets call this file deploy-node1.yml:
This file states the following:
Taking these two files together we can describe the redis application, and what node we want it run on. To put it all together, we just deploy this application using these files:
Kind of anticlimatic after all that work. Let's take a look at flocker1 and see if anything is running:
Yes! Let's put some test data in the redis database. If you are following along, you can install the redis package with yum install redis. Then run redis-cli like so:
Ok, so the redis server in the container that Flocker named "flocker--redis-example" is working. So far all of this is sort of expected. You create a container, you can read and write to it. But what happens if we deploy the application to node 2? If you did it with Docker by just doing a docker run -d ... you would get a new blank installation of Redis. Let's see what happens. First, create a deployment file deploy-node2.yml:
What's the difference? It's exactly the same, except the line for node1 is now empty, and node2 contains the redis-example application. Let's see what happens when we deploy:
Again, not very exciting. Let's connect to the redis datastore and see what's there.
Ok, the cool I promised has finally arrived. The data that was in the /data volume has moved with the container! Up until now databases have not been particularly suitable for Docker containers because of this. Now we can have our container AND storage! Even cooler is the way the data is moved. Imagine you have a 500GB dataset in a database and you need to move it from node1 to node2. Instead of shutting down node1, copying the data, and then starting node2, which might take a long time, Flocker does it smartly. The data is first cloned to the new node. Then the application is stopped on node1, a quick "catch up" on the filesystem is done to bring them into sync, and then the application is started on node 2. ZFS allows this catch up by maintaining pointers in the filesystem. It knows how much data was copied to the other system, and it knows what data is new. So the bulk of the data can be copied while the system continues functioning. Once the big copy is complete, then the app shuts down, only the data that changed since the clone is copied and the app started again. Clever!
Take a look at the redis command I used above to verify the data moved to node 2. Think about it and see if you can guess what's wrong with it. I'll wait here.
Give up? If you look at the last example above, after migrating the redis database to node 2 I used redis-cli to connect to .... NODE 1. So wait, did I cheat or something? The application's really still running on node 1 isn't it!
No. Flocker1 has no docker containers configured or running. What about node 2?
There it is, running on node 2. So what's going on? Well, Flocker adds a bit of magic to the networking. You can connect to ANY node in the flocker cluster and it will route the ports for your application to the right node. So applications can be configured to point to one node in a host, and you can move the container back and forth and the application will still work. I love it when people make my life easier.
Flocker enables a feature this DBA has been looking for since he first heard about Docker: persistent data stores that can migrate from node to node with a Docker container. Imagine an Oracle database running in a container, and being able to deploy new databases literally with the push of a button? The next article in this series will explore exactly that. Enjoy migrating your containers with filesystems, and catch Part 3 in the coming weeks.