Showing posts with label dockerfiles. Show all posts
Showing posts with label dockerfiles. Show all posts

Wednesday, January 13, 2016

Docker: Up and Running on OS X (deprecated)

Update 13-July-2016: This is an old article.  Use Docker Machine going forward.

Introduction

 The good news is, docker is getting easier.

This time last year, if you wanted to install docker on your Macbook, there were a few hoops to jump through. Now it's easier than ever. Much of the s implication has come through the use of Docker Machine.

The Docker toolbox installer works well. Download, point and click, and set up. The initial launch performs a pre-create check, and then automatically creates a boot-2-docker machine on your computer.


Post-Installation

In Spotlight search, start typing "docker ... "
Launch the terminal.

If this is the first time running a docker terminal, it may take a few minutes to set up.

Successful output will look something like this:
Running pre-create checks...
Creating machine...
(default) Copying /Users/craigtrim/.docker/machine/cache/boot2docker.iso to /Users/craigtrim/.docker/machine/machines/default/boot2docker.iso...
(default) Creating VirtualBox VM...
(default) Creating SSH key...
(default) Starting VM...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect Docker to this machine, run: /usr/local/bin/docker-machine env default

                        ##         .
                  ## ## ##        ==
               ## ## ## ## ##    ===
           /"""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ /  ===- ~~~
           \______ o           __/
             \    \         __/
              \____\_______/


docker is configured to use the default machine with IP 192.168.99.100
For help getting started, check out the docs at https://docs.docker.com

~/workspaces $


Once you have the terminal accessible, the environment variables need to be established.

Run this command:
~/workspaces$ docker-machine env default
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="/Users/craigtrim/.docker/machine/machines/default"
export DOCKER_MACHINE_NAME="default"
# Run this command to configure your shell: 
# eval $(docker-machine env default)


and then follow the instructions to run:
$ eval $(docker-machine env default)
on the terminal.

This will set environment variables that the Docker client will read which specify the TLS settings. Note that you will need to do that every time you open a new tab or restart your machine.


Testing the Installation


From there, I can run this tutorial, and get:
~/workspaces$ docker build -t mybase .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM ubuntu
latest: Pulling from library/ubuntu

fcee8bcfe180: Pull complete 
4cdc0cbc1936: Pull complete 
d9e545b90db8: Pull complete 
c4bea91afef3: Pull complete 
Digest: sha256:b53bb7b0d18842214ac7472c2a8801e8682c247d30f1ba4bab0083a2e2e091ea
Status: Downloaded newer image for ubuntu:latest
 ---> c4bea91afef3
Step 2 : RUN touch myfile.txt
 ---> Running in 745e3d4c316b
 ---> f5644fc04a54
Removing intermediate container 745e3d4c316b
Successfully built f5644fc04a54



Troubleshooting

  1. Error checking TLS connection: machine does not exist
    1. Occurs if the environment variables were not set correctly use docker machine
    2. Last login: Wed Jan 13 12:11:02 on ttys005
      ~/workspaces$ docker ps
      Cannot connect to the Docker daemon. Is the docker daemon running on this host?
      ~/workspaces$ docker-machine env default
      export DOCKER_TLS_VERIFY="1"
      export DOCKER_HOST="tcp://192.168.99.100:2376"
      export DOCKER_CERT_PATH="/Users/craigtrim/.docker/machine/machines/default"
      export DOCKER_MACHINE_NAME="default"
      # Run this command to configure your shell: 
      # eval $(docker-machine env default)
      ~/workspaces$ eval $(docker-machine env default)
      ~/workspaces$ docker ps
      CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
      ~/workspaces$ 
      
  2. Error checking TLS connection: Error checking and/or regenerating the certs
    1. This appears to crop up from time to time on the Mac. 
    2. I took three actions to mitigate this - the first was to regenerate the certificates, but this did not appear to work:
                              ##         .
                        ## ## ##        ==
                     ## ## ## ## ##    ===
                 /"""""""""""""""""\___/ ===
            ~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ /  ===- ~~~
                 \______ o           __/
                   \    \         __/
                    \____\_______/
      
      
      docker is configured to use the default machine with IP 192.168.99.100
      For help getting started, check out the docs at https://docs.docker.com
      
      ]~$ docker-machine env default
      Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.99.100:2376": dial tcp 192.168.99.100:2376: i/o timeout
      You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
      Be advised that this will trigger a Docker daemon restart which will stop running containers.
      
      ~$ docker-machine regenerate-certs default
      Regenerate TLS machine certs?  Warning: this is irreversible. (y/n): y
      Regenerating TLS certificates
      Detecting the provisioner...
      Copying certs to the local machine directory...
      Copying certs to the remote machine...
      Setting Docker configuration on the remote daemon...
      
      This machine has been allocated an IP address, but Docker Machine could not
      reach it successfully.
      
      SSH for the machine should still work, but connecting to exposed ports, such as
      the Docker daemon port (usually <ip>:2376), may not work properly.
      
      You may need to add the route manually, or use another related workaround.
      
      This could be due to a VPN, proxy, or host file configuration issue.
      
      You also might want to clear any VirtualBox host only interfaces you are not using.
      ~$ 
      
    3. The second action was to open up VirtualBox and removing the host interfaces, in the event there was a conflict:

      I removed both of the Host-only networks listed above.
    4. The third and final action to restart my laptop, and try again with my Cisco VPN client disabled.  
      1. I've had trouble in the past with Cisco VPN and Docker on the Mac.  The VPN client does not allow split-tunneling, and this can cause issue with containers.  
      2. Having said that, I am using Docker and the Cisco VPN together now without any apparent issues.  



References

  1. https://github.com/docker/toolbox/issues/346
    1.  Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates #346

Thursday, March 26, 2015

Creating a Dockerfile

Introduction

You can launch a container from a docker image, modify the container, and commit the changes back to the image.  The advantage to this approach is it's likely easier for a beginner - once inside docker you have access to the terminal session and can install and configure software in a manner nearly resembling a full Linux environment.

The alternative is to modify your Dockerfile to contain the commands for installing and configuring file.  This appproach is preferred, because the Dockerfile can be easily checked into a SCM and built on other machines.


Creating a Dockerfile (Basics)


I'm going to create a directory in my home folder called "mydocker" and start editing a file within that directory called "Dockerfile".

My first Dockerfile is simple:
FROM ubuntu

RUN touch myfile.txt

and I can build it using this command:
sudo docker build -t basehadoop .
"basehadoop" is the name I give my image.
"." tells docker to look inside my current directory ("mydocker") to find the Dockerfile.


I can launch a container from the image that I've built using this command:
sudo docker run -i -t basehadoop /bin/bash

and once inside I notice the file I've created exists:
root@74e10ef8405d:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  myfile.txt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var



Setting Environment Variables


Use this pattern in your Dockerfile:
ENV key value

An environment variable like this
export JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre

becomes
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle/jre

Likewise, the PATH can be modified, using this format:
ENV PATH /usr/lib/jvm/java-8-oracle/jre/bin:$PATH

When I re-build my image with this change, I get this:
craig@dockerhost:~/mydocker$ sudo docker build -t basehadoop .
Sending build context to Docker daemon 3.072 kB
Sending build context to Docker daemon 
Step 0 : FROM ubuntu
 ---> b39b81afc8ca
Step 1 : RUN touch myfile.txt
 ---> Using cache
 ---> 0e83d857ff15
Step 2 : ENV JAVA_HOME /usr/lib/jvm/java-8-oracle/jre
 ---> Running in 1204292998bf
 ---> 90808840643c
Removing intermediate container 8cf05deb31d5
Successfully built 849723f3f1d2

When I view the json file under the graphs folder, I see this JSON:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
{
    "Size": 0,
    "architecture": "amd64",
    "config": {
        "AttachStderr": false,
        "AttachStdin": false,
        "AttachStdout": false,
        "Cmd": [
            "/bin/bash"
        ],
        "CpuShares": 0,
        "Cpuset": "",
        "Domainname": "",
        "Entrypoint": null,
        "Env": [
            "PATH=/usr/lib/jvm/java-8-oracle/jre/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre"
        ],
        "ExposedPorts": null,
        "Hostname": "2338b11efbe1",
        "Image": "90dd9b6eab1d605a10e151660f2a841c4bc64978fb85d3424f97bdb40c94be28",
        "Memory": 0,
        "MemorySwap": 0,
        "NetworkDisabled": false,
        "OnBuild": [],
        "OpenStdin": false,
        "PortSpecs": null,
        "StdinOnce": false,
        "Tty": false,
        "User": "",
        "Volumes": null,
        "WorkingDir": ""
    },
    "container": "5c5de1bd02358774ee5766979a95e59638884ae59d2af77523ced52805df944d",
    "container_config": {
        "AttachStderr": false,
        "AttachStdin": false,
        "AttachStdout": false,
        "Cmd": [
            "/bin/sh",
            "-c",
            "#(nop) ENV STO=/media/data2/STO/els"
        ],
        "CpuShares": 0,
        "Cpuset": "",
        "Domainname": "",
        "Entrypoint": null,
        "Env": [
            "PATH=/usr/lib/jvm/java-8-oracle/jre/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre",
            "ECLIPSE_HOME=/opt/eclipse/luna",
            "MAVEN_HOME=/usr/lib/apache/maven/3.2.5",
            "HADOOP_HOME=/usr/lib/apache/hadoop/2.6.0",
            "HADOOP_DATA_DIR=/home/craig/HADOOP_DATA_DIR",
            "HADOOP_CONF_DIR=/usr/lib/apache/hadoop/2.6.0/conf",
            "HADOOP_LOG_DIR=/usr/lib/apache/hadoop/2.6.0/logs",
            "WS=~/workspaces",
            "WSCOM=/home/craig/workspaces/swtk/common/projects/swtk-common",
            "WSSAN=/home/craig/workspaces/swtk/sandbox/projects/swtk-sandbox",
            "NYT=/media/data/NYT/02_Unzipped/02_TXT",
            "STO=/media/data2/STO/els"
        ],
        "ExposedPorts": null,
        "Hostname": "2338b11efbe1",
        "Image": "90dd9b6eab1d605a10e151660f2a841c4bc64978fb85d3424f97bdb40c94be28",
        "Memory": 0,
        "MemorySwap": 0,
        "NetworkDisabled": false,
        "OnBuild": [],
        "OpenStdin": false,
        "PortSpecs": null,
        "StdinOnce": false,
        "Tty": false,
        "User": "",
        "Volumes": null,
        "WorkingDir": ""
    },
    "created": "2015-03-26T19:04:55.686342953Z",
    "docker_version": "1.2.0",
    "id": "73e5400f4bb13d4e12b0c457d8fb77603691e983254ba1a55b593140e9c096a6",
    "os": "linux",
    "parent": "90dd9b6eab1d605a10e151660f2a841c4bc64978fb85d3424f97bdb40c94be28"
}
Note lines 16-17 that contain the modified PATH and environment variables

When I launch the container from this modified image, I get this:
craig@dockerhost:~/mydocker$ sudo docker run -i -t basehadoop /bin/bash
root@0dab6fe52a8e:/# echo $JAVA_HOME
/usr/lib/jvm/java-8-oracle/jre
root@0dab6fe52a8e:/# echo $PATH
/usr/lib/jvm/java-8-oracle/jre/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin



References

  1. [StackOverflow] How to update the PATH
  2. Reference Dockerfiles:
    1. [Github] Dockerfile for Java installation
    2. [Github] sequenceiq/hadoop
  3. [Blog] Removing untagged images from docker:
    1. Remove all stopped containers:
      sudo docker rm $(sudo docker ps -a -q)
      
    2. Remove all untagged images
      sudo docker rmi $(sudo docker images | grep "^<none>" | awk "{print $3}")