Docker Containers

From kipiki
Jump to: navigation, search

Docker Containers

Docker

Summary

Docker is a container system, it allows you to download, or make, packaged services that will run on many platforms, such as x86_64, arm, arm64, and IBM Power/Z.

Advantages and Disadvantages

This is by far not a complete list.

Building things in this manner has several advantages
Ease of deployment
Isolation of services allows multiple services to be run using the same ports but being mapped elsewhere, and better security if one gets compromised
Better Security, due to isolation of services and configurability of access
Update of dependencies is easier/testable, helps prevent breakages (such as relying on a particular version of python, and the system gets updated, breaking your program)
There are also disadvantages
Unless you build or review the docker image, you to not know exactly what is in it, whereas you usually trust the package creators (such as Apache Foundation) or package maintainers (such as Debian)
Security is good, but nothing is full-proof, this also applies to VMs
Knowledge of the product being deployed, unless you are building yourself, is greatly diminished

Primary Setup

Setting up Docker is generally different in different operating systems. Some have direct downloads from docker's site, some you can get from the repository for the OS. There are multiple versions, but we are concentrating on Docker CE (Community Edition) not Docker EE (Enterprise Edition). Here you can see the variety of the supported systems for the different products: https://docs.docker.com/v17.12/install/

Linux
Debian: https://docs.docker.com/v17.12/install/linux/docker-ce/debian/
Centos: https://docs.docker.com/v17.12/install/linux/docker-ce/centos/
Fedora: https://docs.docker.com/v17.12/install/linux/docker-ce/fedora/
Ubuntu: https://docs.docker.com/v17.12/install/linux/docker-ce/ubuntu/
Mac
https://docs.docker.com/docker-for-mac/install/
Windows
https://docs.docker.com/docker-for-windows/install/
As well as unsupported models such as (and many more unlisted)
Gentoo: https://wiki.gentoo.org/wiki/Docker
Arch: https://wiki.archlinux.org/index.php/Docker

Aliases

alias dc='/usr/local/bin/docker-compose -f /root/docker-compose.yaml'
alias ds_deploy='docker stack deploy -c docker-swarm.yaml swarm'
alias ds_ps='docker service ps swarm' 
alias ds_ls='docker service ls'
alias ds_rm='docker stack rm swarm'

Docker CE

Docker Community Edition is a great container system that allows users to build out environments with ease

## some helpful commands:
# cleanup system
docker system prune

# prune images
docker image prune

# check docker status
docker ps

# stop all containers, one container, or kill one container (not preferred)
docker stop $(docker ps -a -q)
docker stop <container name>
docker kill <container name>

# pull new images
docker pull <image name>

# Use a dockerfile to build a container
docker build -t <container name>

# Run things inside container, such as a shell
docker run <container name> -it bash

# Check the logs of the container if something is amiss
docker logs --follow <container name>

# Remove a container
docker rm <container name>

# Remove an image
docker rmi <image name>

Here are a few container examples
# smokeping - https://hub.docker.com/r/linuxserver/smokeping/
docker stop smokeping; docker rm smokeping; docker pull linuxserver/smokeping
docker create --name smokeping -p 4080:80 -e PUID=1002 -e PGID=1002 -e TZ="America/New_York" -v /home/public/smokeping/data:/data -v /home/public/smokeping/config:/config linuxserver/smokeping
docker start smokeping

# archiveteam - script to spawn 9 warriors - https://github.com/ArchiveTeam/warrior-dockerfile
docker pull archiveteam/warrior-dockerfile
for i in {01..09}; do
  docker stop archiveteam_$i; docker rm archiveteam_$i
  docker run -d --name archiveteam_$i --env DOWNLOADER="<your id here>" --env SELECTED_PROJECT="auto" --env CONCURRENT_ITEMS="6" -p 80$i:8001 --restart=unless-stopped -e PUID=1002 -e PGID=1002 archiveteam/warrior-dockerfile
done

# syncthing - https://hub.docker.com/r/linuxserver/syncthing/
docker stop syncthing; docker rm syncthing; docker pull linuxserver/syncthing
docker create --name=syncthing -e PUID=1002 -e PGID=1002 -e TZ="America/New_York" -e UMASK_SET=022 -p 8384:8384 -p 22000:22000 -p 21027:21027 -v /home/public/syncthing/config:/config -v /home/public/syncthing/data:/data1 --restart unless-stopped linuxserver/syncthing
docker start syncthing

# game/windows update caching - https://github.com/steamcache/
docker stop cache_all; docker rm cache_all; docker pull steamcache/generic:latest
docker create --name cache_all --restart unless-stopped -p 10.0.0.7:80:80 -v /export/steamcache/allcache/logs:/data/logs -v /export/steamcache/allcache/cache:/data/cache -e CACHE_MEM_SIZE=4000m -e CACHE_DISK_SIZE=2000g steamcache/generic:latest
docker start cache_all
docker stop sniproxy; docker rm sniproxy; docker pull steamcache/sniproxy:latest
docker create --name sniproxy --restart unless-stopped -p 10.0.0.7:443:443 steamcache/sniproxy:latest
docker start sniproxy

Docker Compose

Docker compose allows you to build docker containers using better tooling, and config files using yaml.

You need to install docker compose as an additional package after installing docker, and use special commands to control it

Some Commands for Docker Compose

## the main call to docker compose
docker-compose -f /path/docker-compose.yaml

## I usually make an alias that calls that in my .bashrc
alias dc='/full/path/to/docker-compose -f /path/to/docker-compose.yaml'

## then I can do fancy things using the nice short command like:
## Check status of docker images
dc ps

## pull new images down
dc pull

## run the containers as daemons, this will also automatically re-create the containers if they need an update, or just leaves them running if they dont
dc up -d

## and your other normal start/stop/kill/whatever, you can also just work on a single container that is defined like using docker:
dc stop <container name>

Example compose file

############# docker-compose.yaml #################

version: "3"
services: 

## Smokeping:
  smokeping:
    image: linuxserver/smokeping
    hostname: smokeping
    container_name: smokeping
    restart: unless-stopped
    ports: 
      - "4080:80"
    environment:
      - PUID=1002
      - PGID=1002
      - TZ="America/New_York"
    volumes:
      - /home/public/smokeping/config:/config
      - /home/public/smokeping/data:/data

## Caching Server (must be paired with SNI Proxy below) ## note-- need to work on getting away from root on this one and sni but they need 80/443 access
  cache_all:
    image: steamcache/generic
    hostname: cache_all
    container_name: cache_all
    restart: unless-stopped
    ports: 
      - "10.0.0.7:80:80"
    environment:
      - CACHE_MEM_SIZE=4000m
      - CACHE_DISK_SIZE=2000g
    volumes:
      - /export/steamcache/allcache/logs:/data/logs
      - /export/steamcache/allcache/cache:/data/cache

## SNI Proxy
  sniproxy:
    image: steamcache/sniproxy
    hostname: sniproxy
    container_name: sniproxy
    restart: unless-stopped
    ports: 
      - "10.0.0.7:443:443"

## Motioneye:
  motioneye:
    image: jshridha/motioneye
    hostname: motioneye
    container_name: motioneye
    restart: unless-stopped
    ports: 
      - "8765:8765"
    environment:
      - PUID=1002
      - PGID=1002
      - TZ="America/New_York"
    volumes:
      - /home/public/motioneye/config:/config
      - /export/iria_public/security_cameras:/home/nobody/media

Docker Swarm

Docker swarm allows you to deploy containers en-masse as well as across clusters and even define multiple-region setups.

This type of setup is very scaleable, and allows you to increase or decrease swarm managers, workers and containers.

Their documentation is pretty good at explaining how it works: https://docs.docker.com/v17.09/engine/swarm/how-swarm-mode-works/nodes/

Swarm commands

Remember aliases make your life easier, these are just the commands but you can make your own aliases, just add them your .bashrc to the user that runs the commands. Docker swarm (ver 3.0 and up) can use compose files

############################# node management ###################################
## create the swarm (first node)
docker swarm init

## secondary nodes that want to join the swarm will need a token, this will give you the manager token:
docker swarm join-token manager

## secondary nodes that want to join as workers would be given this key:
docker swarm join-token worker

## to get the secondary nodes to join, you need the token and the host's ip address, here's an example from my example swarm for the presentation at jaxlug:
docker swarm join --token SWMTKN-1-03x1i0zcemn8arp1lu4paxwf49qp3pynsq039reedm9bmss7fz-9gonen9npvp5i4crjg68yhc4v 10.10.10.8:2377

## show a listing of the nodes in the swarm
docker node ls

## check out a single node
docker node inspect <node hostname> --pretty

## drain a node so you can take it down for maintenance
docker node update --availability drain <node hostname>

## activate a node so you it starts taking work again
docker node update --availability active <node hostname>

## add or remove nodes from management
docker node promote <node hostname> <node2 hostname>
docker node demote <node hostname>

## have a node leave the swarm
docker swarm leave

##################### Networks ##########################
## look at the networks that are currently defined:
docker network ls

## check out the network settings of a specific network
## by default ingress is a special overlay network that helps load balancing between nodes
docker network inspect ingress

## overlay networks manage communication between the nodes participating in the swarm
## docker_gwbridge is a bridge that connects the overlay networks to the physical network
docker network inspect docker_gwbridge

##################### Creating swarm containers ###################
## deploy a swarm named 'swarm' using a compose file:
docker stack deploy -c docker-swarm.yaml swarm

## show the services currently running out of the swarm, it only shows things in swarm, not regular non-swarm containers, but it will show you the distinct swarms in the cluster
docker service ls

## Check the status of the archiveteam swarm inside of our 'swarm'
docker service ps swarm_archiveteam

## stop and remove the swarm nodes from archiveteam swarm
docker stack rm swarm

Swarm file

###################### docker-swarm.yaml #####################

version: "3"
services: 

## Archive Team Warrior swarm:
  archiveteam:
    image: archiveteam/warrior-dockerfile
    environment:
      - PUID=1002
      - PGID=1002
      - TZ="America/New_York"
      - SELECTED_PROJECT="auto"
      - CONCURRENT_ITEMS="6"
      - DOWNLOADER="<user here>"
      - dns=8.8.8.8
    deploy:
      mode: replicated
      replicas: 20