As anyone with a web browser and a technical inclination knows, Docker has become a very popular and much discussed project. It provides functionality that at first glance might look like virtualization such as provided by VMWare, KVM, Xen and friends, but instead of virtualizing an entire physical server, it instead runs processes in an isolated container on the operating system. If you're familiar with chroot, or BSD Jails, Docker takes this concept, pumps it full of steroids and new ideas and the result is a project that is hard for IT professionals to resist.
This article is intended to be the first in a series that explores the new landscape that Docker created. Since Docker is so popular, we will not cover the very basics of Docker and containers. Docker already has an interactive tutorial that lets you get hands on with the tool immediately. They also provide a user guide which covers the basics of dockering. I strongly encourage you to investigate both of them.
Once you have your head around the basic concepts of Docker: images, Dockerfiles, volumes, etc. then you're ready to come back and explore the various tools and extensions that we will be investigating.
Enough talk. Let's play!
Note:A lot of the output listings are word wrapped due to formatting considerations. If you want to see the output with original line breaks hover over the code, and a little popup will appear. Click the first icon to see a text view
In this article I am going to focus on setting up a private Docker Registry that is suitable for development purposes. There are many guides on setting up full SSL protected registries with Nginx in front for security. I just want a small private registry that is safe behind a firewall and not exposed to the public. In fact my registry is running on VMware Workstation, and isn't even available to other machines except other VMs on the same network.
To those who have used the tool for awhile, you may think this is getting the cart before the horse. However, I want to start this way for several reasons:
First off, what exactly is a registry? If you look at the documentation linked above it's explained in detail, but the short summary is that it stores images. Images are the basis of a container. For example if I want to set up a container to run a Redis server, it needs an operating system to be installed. I could create a container and copy a distribution into it using debootstrap or febootstrap for Fedora, or another tool for your favorite distribution. However, that would most likely be a waste of time. The public Docker registry (called the Hub) contains images for many common applications, as well as plain installations of operating systems. There are "certified" or official images from Centos, Ubuntu and other Linux distros, as well as certified images for Redis servers, MySql, Nxinx and on and on. Certified images are guaranteed to be created by the company/organization that creates the product. So the certified Redis image comes from the Redis people, the certified Centos images comes from Centos, etc. Other images are created by the general public.
The idea is that you take the image closest to where you want to be and then build on it. For example the Redis image was most likely started by using the Ubuntu image, then installing Redis and configuring the container so it was easy to use with Docker. Instead of installing from scratch the Redis folks only had to build on the last piece of installing Redis. All of the images that you can build on are stored in a Registry. Docker would like you to put your image in the public registry (the Hub) so that other people can benefit from your image. However, there are times when you don't want your image public. Perhaps you're working on a new piece of software and are using Docker to enable rapid deployment and testing. You probably don't want your software publicly available for anyone to download. Perhaps you have personal information in configuration files in the images, and you'd rather not expose that to the public. Hence the need for private registries. As stated earlier you can set up a registry that is publicly accessible but still requires credentials to log in. That's great for companies with many developers who need to share. What I want is the single developer case. I just want a registry I can have access to to push my images into without worrying that I've scrubbed them of personal information or other private stuff. I don't want to have to login all the time since I'm the only user. So let's get a very basic private repository going.
Believe it or not the easiest way to get a registry going is to... deploy a Docker container with a registry in it. There's something very amusing about deploying a container that holds images that allow you to create other containers. Assuming you have docker installed already, simply execute the container:
As seen above all of the image layers that make up the registry image are downloaded and then applied to make the container. The -d switch runs the process as a daemon, otherwise you'd be attached to the registry console and pressing CTRL-C would abort it. The -p 5000:5000 means "take port 5000 in the container and make it port 5000 on the host machine." Now we have a registry accessible on port 5000. Let's test it out. In order to push an image into your registry you need to tag it with a unique name that includes the registry server's address. To verify it's running, do a docker ps:
Yes, it's been executing for 33 minutes, running the "docker-registry" command. Port 5000 on the local host is mapped to port 5000 inside the container. And because I forgot to name my container, a default name of "adoring_perlman" was assigned. I can use the name to refer to the container instead of using the ID. So it's running, now we need something to store inside it. For example lets just download the certified Centos image and put it into our own repository. This time instead of running the image, we'll just pull it down:
Yes, downloading images again. Fortunately once you have these images they are cached and you don't have to download them again every time you create a container. Once downloaded, let's check that we have in our local inventory:
Now that we have a copy of the centos image, let's create a (minimally) useful container and set it up to be stored in our registry.
Let's parse that command line a bit. The run command means execute a container. We're naming our container so that we can refer to it again later. If you don't name it you get a random name, as our registry did above. The -d means daemonize so we have it running in the background. Then we specify the image we want to base our container on (centos) and give it a command to run. In this case a very simple while loop in bash. Now whenever we want to know the current time, we can ask our container. Since it is running in the background, we need to get the logs from standard output:
Ok, not the most useful thing. There's a reason it's named silly-centos. Let's stop that container, and set it up to be saved to our local registry.
This stops the container (which is not strictly necessary), and then creates an image out of it. This image can now be cloned and copied and used to create new containers that you can run as-is or modify. Note the -a (author) and -m (message) parameters so an audit trail of what changes have been made to an image can be seen. Try docker history to see the list. After that you identify a container and specify what repository it should call home. For personal repositories the format is hostname:port/image-name. If the port is 80 or 443 it doesn't have to be specified.
Let's talk about naming for a minute. One thing that might seem confusing is how to name your images. By default docker pulls images from the Docker Hub. Images can be named like we've already seen as centos or redis. If it's a single word then it's a certified repository. Other images in the public registry are prefixed by a username and a slash, for example fred/special-image would be an image owned by user fred named special-image. So far all of these images are stored on the docker hub, so there's no need to specify a server name, docker already knows where the hub is. However, if you want to use a private registry, you need to store the name of the server as part of the tag. In the example above I am creating an image with docker commit. After the -a and -m options for metadata the next two parameters are "silly-centos" which means copy the container named silly-centos and make an image out of it called flocker.local:5000/silly-centos. That entire thing is the name of the image. As you can see the name is a server name, a port and then the image name that it should be called on that server. By naming the image this way you've already told docker what registry it should be stored in. A simple docker push then pushes the image where it's supposed to go. Often on my dev boxes I will run the registry on port 80 instead of 5000. Then I don't even need to publish the port number, and instead can call my image flocker.local/image-name.
Ok, now that we've talked about naming, let's take a look at our images on the local system:
Based on the repository name we can see I want to store my image on host "flocker.local" (which is the name of the VM I'm working in - why Flocker? See the article next week!) and the image name will stay silly-centos. So far this image still only lives on my personal machine. It's time to save it into the repository. Use the docker push command to push it into its new home.
Did you hear that? It was the sound of Docker weeping at your lack of respect for SSL encryption. Docker really wants you to use SSL, but I'm a very lazy developer and I don't want to. Fortunately, the fix is fairly simple. You need to tell your local docker process that it's ok to talk to insecure registries - which is what the error message says. Unfortuanately every Linux distribution handles services a little differently. Essentially you need to tell the service process that starts docker instead of just calling "docker" to call "docker --insecure-registry flocker.local:5000" (replacing the hostname with your own). Then when you push docker will ignore your obvious insecurities. I'm using Fedora to test with, so there is a file called /etc/sysconfig/docker which looks like this:
Note on line 4 above, command line options to the docker executable can be appended. Fedora passes in --selinux-enabled by default. We need to add the --insecure-registry parameter as well. Modify it to look like the below (replacing with your hostname).
Finally, a restart of your local docker process is required to pick up the new option. In Fedora that's systemctl restart docker. In Ubuntu it's service docker restart and so on. Now let's try to push again.
Success! Now our registry has a copy of the image.
Just for fun let's create a new container from our image just to close the loop on this project. We will modify the date command to report the number of seconds since 1970 instead of a more typical date/time.
Is our very-silly-centos image working?
And what's the output?
Success again. We can use our silly-centos image as the basis for new images.
This first article was written just to get us used to working in the Docker world, and along the way we learned how to create a local registry for ourselves. This will be very useful in the next several lessons as we fling Docker images back and forth while exploring other projects. Next time we'll be reviewing the capabilities of Flocker. This project extends the capabilities of containers in exciting ways. After that I hope to check out the features that CoreOS brings to the table. From these modest beginnings we will explore some great things! See you next time.
Labels: Docker