Liuye Notebook
0%

Docker-Basics

发表于 更新于 分类于 ， 阅读次数： Valine：

阅读更多

1 Installation

1.1 CentOS

1
2
3
4
5
6
7
8
9
10
11
12
13
# --step 1: install dependent softwares
yum install -y yum-utils device-mapper-persistent-data lvm2
# --step 2: add repo
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# --step 3
sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
# --step 4: install docker
yum list docker-ce --showduplicates | sort -r
yum makecache fast
yum -y install docker-ce-20.10.5-3.el7
# --step 5: enable docker service
systemctl enable docker
systemctl start docker

1.2 Ubuntu

Install Docker Engine on Ubuntu

1.3 MacOS (Apple Chip)

Colima runs a minimal Linux VM using Lima under the hood. Docker Engine runs inside that VM. The Docker CLI talks to it through a Unix socket just like Docker Desktop would.

1
2
3
4
5
brew install docker
brew install colima
colima start
# colima start --memory 4 --cpu 2 --disk 60
docker version

1.3.1 colima

Colima (short for Container on Lima) is a fast, lightweight Docker and Kubernetes runtime for macOS and Linux that provides a Docker-compatible environment using a virtual machine under the hood — specifically Lima (Linux virtual machines).

It’s often used as a drop-in replacement for Docker Desktop — especially after Docker Desktop introduced licensing changes in 2022.

Commands:

  • colima status
  • colima list
  • colima ssh
  • colima start --cpu 6 --memory 8 --disk 100
  • colima stop
  • colima start --edit

2 Container Lifecycle

container_lifecycle

3 Filesystem

3.1 overlay2

Example OverlayFS Usage [duplicate]

4 Build Image

Write Dockerfile

  1. FROM: Specifies a base image
    • In most cases, a valid Dockerfile starts with a FROM instruction.
    • FROM must be the first non-comment instruction in a Dockerfile.
    • FROM can appear multiple times in a Dockerfile to create multi-stage builds.
    • If no tag is specified, latest will be used as the default version of the base image.
  2. MAINTAINER: Specifies the author of the image
    • (Note: This instruction is deprecated. Use a LABEL instead to define metadata such as the maintainer.)
  3. RUN: Executes commands during image build
    • RUN instructions create intermediate image layers and are persisted in the final image.
    • Layered RUN instructions support Docker’s core philosophy of image versioning and reproducibility.
    • RUN command cache is not invalidated automatically in the next instruction. For example, a cached RUN apt-get dist-upgrade -y might be reused.
    • Use the --no-cache flag to force disable caching.
  4. ENV: Sets environment variables inside the Docker container
    • Environment variables set by ENV can be viewed using the docker inspect command.
    • These variables can also be overridden at runtime using docker run --env <key>=<value>.
  5. ARG: Defines environment variables that are available only during the image build
    • Available only during docker build, not persisted in the final image.
    • Values can be passed with --build-arg <key>=<value> when building the image.
  6. USER: Changes the user under which the container runs
    • By default, Docker containers run as the root user.
  7. WORKDIR: Sets the working directory for instructions that follow
    • The default working directory is /. While cd can be used within RUN, it only affects the current RUN instruction.
    • WORKDIR changes are persistent and apply to all following instructions, so you don’t need to repeat WORKDIR for each command.
  8. COPY: Copies files or directories from <src> on the host to <dest> in the container
    • <src> must be a path relative to the build context, or a file/dir.
    • <dest> must be an absolute path inside the container.
    • All copied files/folders are assigned UID and GID. If <src> is a remote URL, the destination file permissions default to 600.
  9. ADD: Similar to COPY, but with extended features
    • <src> can be a file, directory, or a remote URL.
    • <dest> is an absolute path inside the container.
    • Files are assigned UID and GID. If <src> is a remote URL, permissions default to 600.
    • ADD can also auto-extract local tar archives, which COPY cannot.
  10. VOLUME: Creates a mount point for external storage (host or other containers)
    • Commonly used for databases or persistent data.
  11. EXPOSE: Informs Docker that the container listens on the specified port
    • Does not publish the port itself; it only serves as documentation or for use with Docker networking.
  12. CMD: Provides default command to run when the container starts
    • Only one CMD instruction is allowed per Dockerfile; if multiple are specified, only the last one is used.
    • Can be overridden by passing a command when running the container: docker run $image $override_command.
  13. ENTRYPOINT: Configures a container to run as an executable
    • The default ENTRYPOINT is /bin/sh -c, but there is no default CMD.
    • For example, in docker run -i -t ubuntu bash, the default ENTRYPOINT is /bin/sh -c and the CMD is bash.
    • CMD essentially acts as arguments to ENTRYPOINT.
  14. ONBUILD: Defers the execution of instructions
    • These instructions are triggered when the resulting image is used as a base image in another build.
    • Each ONBUILD instruction is triggered only once.

4.1 Relative Path in a Dockerfile

In Dockerfiles, relative paths (like ./myfile.txt or src/) are always relative to the build context. The build context is the directory you specify when you run (Here, the <path-to-context> is the build context.):

  • docker build -t <image> <path-to-context>

Docker will throw an error if COPY or ADD refers to something outside the build context, like ../secret.txt.

4.2 Demo

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
mkdir -p friendlyhello
cd friendlyhello

cat > Dockerfile << 'EOF'
# Use an official Python runtime as a parent image
FROM python:2.7-slim

# Set the working directory to /app
WORKDIR /app

# Copy the current directory contents into the container at /app
ADD . /app

# Install any needed packages specified in requirements.txt
RUN pip install --trusted-host pypi.python.org -r requirements.txt

# Make port 80 available to the world outside this container
EXPOSE 80

# Define environment variable
ENV NAME World

# Run app.py when the container launches
CMD ["python", "app.py"]
EOF

cat > requirements.txt << 'EOF'
Flask
Redis
EOF

cat > app.py << 'EOF'
from flask import Flask
from redis import Redis, RedisError
import os
import socket

# Connect to Redis
redis = Redis(host="redis", db=0, socket_connect_timeout=2, socket_timeout=2)

app = Flask(__name__)

@app.route("/")
def hello():
    try:
        visits = redis.incr("counter")
    except RedisError:
        visits = "<i>cannot connect to Redis, counter disabled</i>"

    html = "<h3>Hello {name}!</h3>" \
           "<b>Hostname:</b> {hostname}<br/>" \
           "<b>Visits:</b> {visits}"
    return html.format(name=os.getenv("NAME", "world"), hostname=socket.gethostname(), visits=visits)

if __name__ == "__main__":
    app.run(host='0.0.0.0', port=80)
EOF

docker build -t friendlyhello .

docker run -p 4000:80 friendlyhello

5 Frequently-Used Images

5.1 alpine

Alpine Linux is a lightweight Linux distribution. Unlike typical Linux distributions, Alpine uses musl libc and BusyBox to reduce system size and runtime resource usage. Alpine Linux provides its own package manager: apk.

Here’s an example of how to build a Docker image with bash.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
FROM alpine:3.10.2

MAINTAINER Rethink 
# update repo
RUN echo "https://mirror.tuna.tsinghua.edu.cn/alpine/v3.4/main/" > /etc/apk/repositories

RUN apk update \
        && apk upgrade \
        && apk add --no-cache bash \
        bash-doc \
        bash-completion \
        && rm -rf /var/cache/apk/* \
        && /bin/bash

# Solving the Timezone Issue (For Alpine images, setting the environment variable TZ=Asia/Shanghai alone is not sufficient)
RUN apk add -U tzdata

5.2 jib

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<plugin>
    <groupId>com.google.cloud.tools</groupId>
    <artifactId>jib-maven-plugin</artifactId>
    <version>1.6.1</version>
    <configuration>
        <from>
            <!-- Without this base image, the built image will not include bash -->
            <image>openjdk:8u222-jdk</image> 
        </from>
        <to>
            <image>my-app:v1</image>
            <auth>
                <username>xxx</username>
                <password>xxx</password>
            </auth>
        </to>
        <container>
            <!-- Without this parameter, the built image will be from 49 years ago -->
            <useCurrentTimestamp>true</useCurrentTimestamp>
        </container>
    </configuration>
</plugin>

5.3 minio

1
2
3
4
5
docker run -d --name minio \
  -p 21900:9000 -p 21990:9090 \
  -e MINIO_ROOT_USER=admin \
  -e MINIO_ROOT_PASSWORD=password123 \
  quay.io/minio/minio server /data --console-address ":9090"
1
mc alias set local http://localhost:9000 admin password123

Client tools: aws

  • aws s3 --endpoint http://127.0.0.1:21900 ls s3://my_bucket/file.txt

6 Docker Compose

6.1 Install

1
2
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

6.2 Usage

  • docker-compose [ -f <compose_file> ] [-p <project_name>] up
  • docker-compose [ -f <compose_file> ] [-p <project_name>] up -d
  • docker-compose [ -f <compose_file> ] [-p <project_name>] down
  • docker-compose [ -f <compose_file> ] [-p <project_name>] start
  • docker-compose [ -f <compose_file> ] [-p <project_name>] stop
  • docker-compose [ -f <compose_file> ] [-p <project_name>] restart
  • docker-compose [ -f <compose_file> ] [-p <project_name>] rm
  • docker-compose [ -f <compose_file> ] [-p <project_name>] ps
  • docker-compose [ -f <compose_file> ] [-p <project_name>] exec -it <service_name> bash
  • docker-compose [ -f <compose_file> ] [-p <project_name>] config
  • docker-compose [ -f <compose_file> ] [-p <project_name>] config --services

6.3 Tips

6.3.1 Find docker-compose information for given docker container

  • docker inspect <container_id> | grep 'com.docker.compose'

7 Tips

  1. Start and keep a container running
    • Run a program that doesn’t exit: docker run -dit xxx:v1
    • Start with an interactive bash shell: docker run -it 9a88e0029e3b /bin/bash
  2. Copy files between host and Docker container
    • docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH|-
    • docker cp [OPTIONS] SRC_PATH|- CONTAINER:DEST_PATH
  3. Open a D-Bus connection
    • docker run -d -e "container=docker" --privileged=true [ID] /usr/sbin/init
    • The container’s CMD should include /usr/sbin/init
  4. Start and stop containers:
    • docker start <container-id>
    • docker stop <container-id>
  5. Execute commands in a specific container
    • docker exec -ti my_container /bin/bash -c "echo a && echo b"
  6. View pid and ip of a docker container
    • docker inspect <container-id> | grep Pid
    • docker inspect -f '{{.State.Pid}}' <container-id>
    • docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <container-id>
  7. Delete a specific tag
    • docker rmi <repository>:<tag> — do not use the image ID
  8. List all image IDs
    • docker images -q
  9. Save an image to a file and load from a file
    • docker save -o alpine.tar alpine:3.10.2
    • docker load < alpine.tar
  10. Set timezone
    • docker run -e TZ=Asia/Shanghai ...
  11. Set kernel parameters (requires privileged mode)
    • When writing the Dockerfile, add the kernel parameter changes to the CMD (you can’t modify them with RUN since kernel files are read-only during build)
    • Use privileged mode when starting: docker run --privileged
  12. Use docker commands inside a container
    • --privileged: use privileged mode
    • -v /var/run/docker.sock:/var/run/docker.sock: mount the Docker socket file
    • -v $(which docker):/bin/docker: mount the Docker CLI binary
  13. Join the network namespace of another container when starting
    • docker run [--pid string] [--userns string] [--uts string] [--network string] <other options>
    • docker run -d --net=container:<existing container id> <image id>
  14. After deleting the Docker CLI, Docker’s working directory (including images) still exists. To completely remove it:
    • rm -rf /var/lib/docker
  15. View image build history: docker history <img>
  16. View image details: docker inspect <img>
  17. Check container resource usage: docker stats <container>
  18. Clean up unused images: docker system prune -a
  19. Grant a regular user permission to use docker: sudo usermod -aG docker username

7.1 Modify docker storage path

默认情况下，docker相关的数据会存储在/var/lib/docker。编辑配置文件/etc/docker/daemon.json（没有就新建），增加如下配置项：

1
2
3
{
    "data-root": "<some path>"
}

然后通过systemctl restart docker重启docker即可

7.2 Modify Mirror Address

编辑配置文件/etc/docker/daemon.json（没有就新建），增加如下配置项：

1
2
3
4
5
6
7
8
9
10
11
{
    "registry-mirrors": [
        "http://registry.docker-cn.com",
        "http://docker.mirrors.ustc.edu.cn",
        "http://hub-mirror.c.163.com"
    ],
    "insecure-registries": [
        "registry.docker-cn.com",
        "docker.mirrors.ustc.edu.cn"
    ]
}

然后通过systemctl restart docker重启docker即可

7.3 Run docker across platform

默认情况下，docker是不支持--platform参数的，可以通过修改/etc/docker/daemon.json，添加如下配置项后，重启docker，开启该功能

1
2
3
4
5
{
  ...
  "experimental": true,
  ...
}

如果我们在x86的平台上运行arm64的docker镜像，会得到如下错误信息

1
2
3
4
5
docker run --platform linux/arm64 --rm arm64v8/ubuntu:18.04 uname -a

#-------------------------↓↓↓↓↓↓-------------------------
standard_init_linux.go:219: exec user process caused: exec format error
#-------------------------↑↑↑↑↑↑-------------------------

此时，我们可以借助跨平台工具qemu，下载地址。该方法对内核版本有要求，貌似要4.x以上，我的测试环境的内核版本是4.14.134

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# 下载qemu-aarch64-static
wget 'https://github.com/multiarch/qemu-user-static/releases/download/v5.2.0-2/qemu-aarch64-static.tar.gz'
tar -zxvf qemu-aarch64-static.tar.gz -C /usr/bin

# 安装qemu
docker run --rm --privileged multiarch/qemu-user-static:register

#-------------------------↓↓↓↓↓↓-------------------------
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
#-------------------------↑↑↑↑↑↑-------------------------

# 再次运行
docker run --platform linux/arm64 --rm arm64v8/ubuntu:18.04 uname -a

#-------------------------↓↓↓↓↓↓-------------------------
standard_init_linux.go:219: exec user process caused: no such file or directory
#-------------------------↑↑↑↑↑↑-------------------------

# 将qemu挂到容器内，再次运行
docker run --platform linux/arm64 --rm -v /usr/bin/qemu-aarch64-static:/usr/bin/qemu-aarch64-static arm64v8/ubuntu:18.04 uname -a

#-------------------------↓↓↓↓↓↓-------------------------
Linux c3fb58ea543c 4.14.134 #1 SMP Tue Dec 29 21:27:58 EST 2020 aarch64 aarch64 aarch64 GNU/Linux
#-------------------------↑↑↑↑↑↑-------------------------
  • docker run --rm --privileged multiarch/qemu-user-static:register是向内核注册了各异构平台的binfmt handler，包括aarch64等等，这些注册信息就包括了binfmt handler的路径，比如/usr/bin/qemu-aarch64-static等等，注册信息都在/proc/sys/fs/binfmt_misc目录下，每个注册项都是该目录下的一个文件。实际的qemu-xxx-static文件还得手动放置到对应目录中才能生效

Steps for ubuntu:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
sudo apt-get update

# Install qemu-user-static to enable emulation for different architectures
sudo apt-get install qemu-user-static

# Try to enable QEMU binary formats to support aarch64 and x86_64 architectures
sudo update-binfmts --enable qemu-aarch64
sudo update-binfmts --enable qemu-x86_64

# Install binfmt-support to provide support for the update-binfmts command
sudo apt-get install binfmt-support

# Try again to enable QEMU binary formats to support aarch64 and x86_64 architectures
sudo update-binfmts --enable qemu-aarch64
sudo update-binfmts --enable qemu-x86_64

# Run the multiarch/qemu-user-static Docker image to register QEMU for all supported architectures
sudo docker run --rm --privileged --platform linux/arm64 multiarch/qemu-user-static --reset -p yes

# Register all supported binary formats using tonistiigi/binfmt
docker run --rm --privileged tonistiigi/binfmt --install all

# Test running an amd64 container on the ARM host
docker run --platform linux/amd64 hello-world

7.4 Docker Image Prune Tools - dockerslim

1
docker-slim build --http-probe=false centos:7.6.1810

裁剪之后，镜像的体积从202MB变为3.55MB。但是裁剪之后，大部分的命令都被裁剪了（包括ls这种最基础的命令）

7.5 Be Aware of Running as a Container

一般来说，如果运行环境是容器，那么会存在/.dockerenv这个文件

7.6 Build Image from Container

保留镜像原本的layer，每次commit都会生成一个layer，这样会导致镜像越来越大：

1
docker commit <container-id> <new_image>

将容器导出成单层的镜像：

1
2
docker export <container-id> -o centos7.9.2009-my.tar
docker import centos7.9.2009-my.tar centos7.9.2009-my:latest

7.7 Setup Http Proxy

Add file /etc/systemd/system/docker.service.d/http-proxy.conf with following content:

1
2
3
4
[Service]
Environment="HTTPS_PROXY=xxx"
Environment="HTTP_PROXY=xxx"
Environment="NO_PROXY=localhost,127.0.0.1,yyy"

Check and restart

1
2
3
systemctl daemon-reload
systemctl show docker --property Environment
systemctl restart docker

7.8 Access Host Ip from Container

The host has a changing IP address, or none if you have no network access. We recommend that you connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host.

7.9 How to delete exited containers

1
docker ps -a -f status=exited -q | xargs docker rm -f

7.10 How to create container with host user

1
docker run --user $(id -u):$(id -g) -v /etc/passwd:/etc/passwd

7.11 Find containers using network

1
docker network inspect <ns-name>

7.12 How to map ports with localhost only

1
docker run -dit -p 127.0.0.1:3306:3306 -e MYSQL_ROOT_PASSWORD='Abcd1234' mysql:5.7.37 mysqld --lower_case_table_names=1

7.13 Config Path

  1. Client: ~/.docker/config.json
  2. Daemon: /etc/docker/daemon.json

8 FAQ

8.1 K8S Env docker error

在k8s环境中，若容器运行时用的是docker，那么该docker会依赖containerd，当containerd不正常的时候，docker也就不正常了。恢复containerd的办法：将/var/lib/containerd/io.containerd.metadata.v1.bolt这个文件删掉

8.2 read unix @->/var/run/docker.sock: read: connection reset by peer

  1. 没有权限
  2. 多套docker共用了同一个/var/run/docker.sock套接字文件，可以用lsof -U | grep docker.sock查看。默认情况下只有2个记录，一个是systemd的，另一个是dockerd的

9 Reference