修改Hosts文件加速访问GitHub

在公司使用中国电信的公司宽带访问Github速度虽然不是飞快,但还是能够下载的。不过回家是用的移动宽带,访问github时能打开首页就算时烧高香了。那么首先想到的就是修改Hosts直接访问Github的地址,避免国内的CDN污染和线路瞎JB跳。
先说成果:

# Github
52.74.223.119 github.com
140.82.114.3 github.com
185.199.108.154 github.githubassets.com
185.199.109.154 github.githubassets.com
185.199.110.153 github.io
185.199.111.153 github.io
140.82.113.10 nodeload.github.com
69.171.234.29  github.global.ssl.fastly.net

效果么,只能说能用了。
方法很简单,就是打开站长之家的DNS检测工具,然后观察GitHub访问会用到那些域名,再把域名对应的IP查出来,把TTL值最小的写入hosts文件。虽然无聊,但是有效。

离线部署CDH5.16.2

在实践了CDH6的离线部署之后,发现Cloudera的官方教程是基于CDH5.10的。同时,CDH6的内存占用比较高,我的机器有点遭不住,所以又把CDH5的部署实施走了一遍。

模板部署

  1. 安装CentOS7.10,并完成yum升级;
  2. 所有主机时间与阿里云时间服务器同步;
  3. 关闭selinux和firewalld;
  4. 安装JDK和mysql-connect;
[root@localhost ~]# wget https://repo.huaweicloud.com/java/jdk/8u202-b08/jdk-8u202-linux-x64.rpm
[root@localhost ~]# yum localinstall jdk-8u202-linux-x64.rpm
[root@localhost ~]# mv mysql-connector-java-5.1.39-bin.jar /usr/share/java/mysql-connector-java.jar 
  1. 关闭透明大页;
[root@localhost ~]# vim /etc/rc.d/rc.local

# 在最后一行添加如下内容
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi

if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

# 赋予rc.local具有可执行权限
[root@localhost ~]# chmod +x /etc/rc.d/rc.local
[root@localhost ~]# sh /etc/rc.d/rc.local

# 检查
[root@localhost ~]# cat /sys/kernel/mm/transparent_hugepage/defrag
always madvise [never]
[root@localhost ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
  1. 调整swap策略
[root@localhost ~]# echo 10 > /proc/sys/vm/swappiness
[root@localhost ~]# echo vm.swappiness = 10 >> /etc/sysctl.conf
  1. 修改文件句柄数
[root@localhost ~]# vim /etc/security/limits.conf
# 添加如下内容
* soft nofile 100000
* hard nofile 100000
  1. 收尾
[root@localhost ~]# sys-unconfig
  1. 下载CM和CDH

CM下载地址

CDH下载地址

Agent部署

  1. 配置主机ssh信任
  2. 安装ansible

# 安装 [root@elephant ~]# yum install -y ansible [root@elephant ~]# ls CDH-5.16.2-1.cdh5.16.2.p0.8-el5.parcel.sha1 CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel Mail anaconda-ks.cfg cloudera-manager-agent-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm cloudera-manager-daemons-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm cloudera-manager-server-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm cloudera-manager-server-db-2-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm enterprise-debuginfo-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm hadoop.tar.gz jdk-8u202-linux-x64.rpm manifest.json mysql-connector-java-5.1.39-bin.jar [root@elephant ~]# mkdir client [root@elephant ~]# mv cloudera-manager-daemons-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm client/ [root@elephant ~]# mv cloudera-manager-agent-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm client/ [root@elephant ~]# tar zcvf client.tar.gz client/
  1. 分发
[root@elephant ~]# ansible all -m copy -a 'src=/root/client.tar.gz dest=/root/'
[root@elephant ~]# ansible all -a 'tar zxf /root/client.tar.gz'
  1. 安装
[root@elephant ~]# ansible all -a 'yum localinstall /root/client/cloudera-manager-daemons-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm -y'
[root@elephant ~]# ansible all -a 'yum localinstall /root/client/cloudera-manager-agent-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm -y'

# 修改配置文件
[root@elephant ~]# sed -i ':s/localhost/lion/g' /etc/cloudera-scm-agent/config.ini
[root@elephant ~]# ansible lion,tiger,horse,monkey -m copy -a 'src=/etc/cloudera-scm-agent/config.ini dest=/etc/cloudera-scm-agent/'

# 重启服务
[root@elephant ~]# ansible all -a 'systemctl enable cloudera-scm-agent --now'
[root@elephant ~]# ansible all -a 'systemctl restart cloudera-scm-agent'

管理主机部署

  1. 安装
[root@elephant ~]# ansible lion -m copy -a 'src=/root/cloudera-manager-server-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm dest=/root/'
[root@elephant ~]# ansible lion -m copy -a 'src=/root/CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel dest=/opt/cloudera/parcel-repo'
[root@elephant ~]# ansible lion -m copy -a 'src=/root/manifest.json dest=/opt/cloudera/parcel-repo'
[root@elephant ~]# ansible lion -a 'yum localinstall /root/cloudera-manager-server-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm -y'
  1. 数据库部署
[root@elephant ~]# ansible lion -a 'yum install -y mariadb mariadb-server'
[root@elephant ~]# ssh lion
[root@lion ~]# vim /etc/my.cnf.d/server.cnf
[mysqld]
key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
server_id=1

binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[root@lion ~]# systemctl enable mariadb --now

vim cdh.sql

CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'passwd';

# 导入数据库
[root@lion ~]# mysql -uroot -p < cdh.sql

# 扩展数据库架构和CDH6有区别
[root@lion ~]# cp /usr/share/java/mysql-connector-java.jar /usr/share/cmf/lib/
[root@lion ~]# sh /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm
Enter SCM password:
JAVA_HOME=/usr/java/jdk1.8.0_202-amd64
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/java/jdk1.8.0_202-amd64/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!

# 启动服务
[root@lion parcel-repo]# cd /opt/cloudera/parcel-repo/
[root@lion parcel-repo]# sha1sum CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel | awk '{ print $1 }' > CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel.sha
[root@lion ~]# chkconfig cloudera-scm-server on
[root@lion ~]# systemctl start cloudera-scm-server

# 检查结果
[root@lion ~]# netstat -tlnp |grep 7180
tcp        0      0 0.0.0.0:7180            0.0.0.0:*               LISTEN      3907/java 

GUI安装

接下来的安装就相对简单了,基本比CDH6简单。

start
start
start
start
start
start
start
start
start
start
start
end

wHAT is HDFS?

Hadoop实现了一个分布式文件系统(Hadoop Distributed File System),简称HDFS。HDFS有高容错性的特点,并且设计用来部署在低廉的(low-cost)硬件上;而且它提供高吞吐量(high throughput)来访问应用程序的数据,适合那些有着超大数据集(large data set)的应用程序。HDFS放宽了(relax)POSIX的要求,可以以流的形式访问(streaming access)文件系统中的数据。

概念

  1. HDFS集群分为两大角色:NameNode、DataNode;
  2. NameNode负责管理整个文件系统的元数据,second namenode是namenode的冷备;
  3. DataNode 负责管理用户的文件数据块;
  4. 文件会按照固定的大小(blocksize)(128M)切成若干块后分布式存储在若干台datanode上;
  5. 每一个文件块可以有多个副本,并存放在不同的datanode上;
  6. Datanode会定期向Namenode汇报自身所保存的文件block信息,而namenode则会负责保持文件的副本数量;
  7. HDFS的内部工作机制对客户端保持透明,客户端请求访问HDFS都是通过向namenode申请来进行;

写过程

HDFS_write

  1. Client调用DistributedFileSystem对象的create方法,创建一个文件输出流(FSDataOutputStream)对象
  2. 通过DistributedFileSystem对象与Hadoop集群的NameNode进行一次RPC远程调用,在HDFS的Namespace中创建一个文件条目(Entry),该条目没有任何的Block
  3. 通过FSDataOutputStream对象,向DataNode写入数据,数据首先被写入FSDataOutputStream对象内部的Buffer中,然后数据被分割成一个个Packet数据包
  4. 以Packet最小单位,基于Socket连接发送到按特定算法选择的HDFS集群中一组DataNode(正常是3个,可能大于等于1)中的一个节点上,在这组DataNode组成的Pipeline上依次传输Packet
  5. 这组DataNode组成的Pipeline反方向上,发送ack,最终由Pipeline中第一个DataNode节点将Pipeline ack发送给Client
  6. 完成向文件写入数据,Client在文件输出流(FSDataOutputStream)对象上调用close方法,关闭流
  7. 调用DistributedFileSystem对象的complete方法,通知NameNode文件写入成功

读过程

HDFS_read

  1. Client向NameNode发起RPC请求,来确定请求文件block所在的位置
  2. NameNode会视情况返回文件的部分或者全部block列表,对于每个block,NameNode都会返回含有该block副本的DataNode地址
  3. 这些返回的DN地址,会按照集群拓扑结构得出DataNode与客户端的距离,然后进行排序,排序两个规则:网络拓扑结构中距离Client的排在前;心跳机制中超时汇报的DN状态为STALE,这样的排在后
  4. Clietn选取排序靠前的DataNode来读取block,如果客户端本身就是DataNode,那么将从本地直接获取数据
  5. 底层本质是建立Socket Stream(FSDataInputStream) ,重复调用父类DataInputStream的read方法,知道这个块上的数据读取完毕
  6. 当读完列表的block后,若文件读取还没有结束,客户端会继续想NameNode获取下一批的block列表
  7. 读取完一个Block都会进行checksum验证,如果读取DataNode时出现错误,客户端会通知NameNode,然后再从下一个拥有该block副本的DataNode继续读取。注: 如果在读取过程中DFSInputStream检测到block错误,DFSInputStream也会检查从datanode读取来的数据的校验和,如果发现有数据损坏,它会把坏掉的block报告给namenode同时重新读取其他datanode上的其他block备份
  8. read方法是并行的读取block信息,不是一块一块的读取,NameNode只是返回Client请求包含块的DataNode地址,并不是返回请求块的数据
  9. 最终读取哎所有的block会合并成一个完整的最终文件

副本放置策略

  1. 第一个副本:放置在上传文件的DN;如果是集群外提交,则随机挑选一台磁盘不太满,CPU不太忙的节点
  2. 第二个副本:放置在于第一个副本不同的机架的节点上
  3. 第三个副本:与第二个副本相同机架的节点
  4. 更多副本:随机节点

NameNode 内存使用

NameNode内存数据主要对整个文件系统元数据的管理。Namenode目前元数据管理可以分成两个层次,一个是Namespace的管理层,这一层负责管理HDFS分布式文件系统中的树状目录和文件结构;另一层则为Block管理层,这一层负责管理HDFS分布式文件系统中存储文件到物理块之间的映射关系BlocksMap元数据。其中对Namespace的管理数据除在内存常驻外,会定期Flush到持久化设备中;对BlocksMap元数据的管理只存在内存;当NameNode发生重启,需要从持久化设备中读取Namespace管理数据,并重新构造BlocksMap。

除了对文件系统本身元数据的管理外,NameNode还需要维护DataNode本身的元数据,这部分空间相对固定,且占用空间较小。

从实际Hadoop集群环境历史数据看,当Namespace中包含INode(目录和文件总量)~140M,数据块数量~160M,常驻内存使用量达在~50G。随着数据规模的持续增长,内存占用接近同步线性增长。在整个HDFS服务中,NameNode的核心作用及内存数据结构的重要地位,所以分析内存使用情况对维护HDFS服务稳定性至关重要。

漫画HDFS

HDFS_001
HDFS_002
HDFS_003

离线部署CDH6.3

部署练习的环境有六台主机,主机的配置是4核心4G内存(内存还是低了,建议8G起步):

主机名 IP地址
Elephant 192.168.174.184
Lion 192.168.174.185
Horse 192.168.174.186
Monkey 192.168.174.187
Tiger 192.168.174.188

环境准备

  1. 主机环境配置

按照各个主机配置修改相应值,下面以elephant为例:

[root@localhost ~]# nmcli c m ens33 ipv4.method manual ipv4.addresses 192.168.174.184/24 ipv4.gateway 192.168.174.2 ipv4.dns 192.168.174.2
[root@localhost ~]# nmcli c d ens33 && nmcli c u ens33
[root@localhost ~]# hostnamectl set-hostname elephant
  1. 文件复制
    将离线部署CDH6.3.1的文件复制到每一台主机上。其中包括:
sujx@legion7000:/mnt/d/$ ls Cloudera
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
jdk-8u202-linux-x64.rpm
manifest.json
mysql-connector-java-5.1.39-bin.jar

sujx@legion7000:/mnt/d/$ tar zcvf Cloudera cloudera.tar.gz
sujx@legion7000:/mnt/d/$ scp cloudera.tar.gz root@lion:/root/
  1. 免密登录设置

在lion主机上设置免密:

ssh-keygen -t rsa
ssh-copy-id {elephant\lion\horse\monkey\tiger}

# 使用ansible分发和管理文件,省略配置文件的建立
yum install -y ansible

ansible -m copy -a 'src=/root/cloudera.tar.gz dest=/root'

  1. 主机配置
    建立主机执行文件进行相关配置。
#!/bin/bash

# 安装必要组件
yum update -y
yum install -y mutt net-tools mlocate telnet curl wget ping vim open-vm-tools

systemctl disable firewalld.service --now
sed -i ':s/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
updatedb && sync && ldconfig

# 配置时间服务器
cat >/etc/chrony.conf<<EOF
server ntp.aliyun.com iburst

driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony

EOF

systemctl enable chronyd --now
timedatectl 

# 优化
# 修改swap配置
echo "vm.swappiness = 10" >> /etc/sysctl.conf
sysctl -p

# 修改大页内存的使用
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local
echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local

# 安装JAVA
tar zxvf cloudera.tar.gz
yum localinstall -y Cloudera/jdk-8u202-linux-x64.rpm

java -version
echo $JAVA_HOME

# 建立安装文件源
wget https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/RPM-GPG-KEY-cloudera
rpm --import RPM-GPG-KEY-cloudera
wget https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/cloudera-manager.repo -O /etc/pki/rpm-gpg/RPM-GPG-KEY-cloudera
yum update -y

mkdir /usr/share/java/
cp Cloudera/mysql-connector-java-5.1.39-bin.jar  /usr/share/java/mysql-connector-java.jar

yum localinstall -y Cloudera/cloudera-manager-daemons* 
yum localinstall -y Cloudera/cloudera-manager-agent*

# 修改cloudera-scm客户端的服务器目标地址
sed -i ':s/localhost/lion/g' /etc/cloudera-scm-agent/config.ini
systemctl enable cloudera-scm-agent
systemctl restart cloudera-scm-agent

# 客户端配置完成

数据库安装

数据库将部署在Lion主机之上。
1. 数据库安装

yum install -y mariadb mariadb-server

# 增加MySQL配置文件
vim /etc/my.cnf.d/server.cnf
[mysqld]
key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
server_id=1

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

# 启动数据库服务
systemctl enable mariadb --now

2. 建立数据库配置文件
vim /root/cdh.sql
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'passwd';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'passwd';

mysql -uroot -p < /root/cdh.sql
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm

### 管理节点安装
```shell
yum localinstall -y Clouder/cloudera-manager-server.6*
cp Cloudera/CDH* /opt/cloudera/parcel-repo
cp Cloudera/main* /opt/cloudera/parcel-repo
cd /opt/cloudera/parcel-repo
# wget https://archive.cloudera.com/cdh6/6.3.2/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el6.parcel -P /opt/cloudera/parcel-repo
# wget https://archive.cloudera.com/cdh6/6.3.2/parcels/manifest.json -P /opt/cloudera/parcel-repo
sha1sum CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel | awk '{ print $1 }' > CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha
chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/*
systemctl enable cloudera-scm-server --now
cd

群集配置

访问Lion主机的7180端口,然后使用web-gui界面进行配置。
start
start
start
start
start
start
start
start
start
start
start
start
start
start
start
start
start
start
start
start
start

Docker的快速练习

建立一个三节点的网络环境来进行docker的操作练习,其中master节点存储私有仓库镜像文件。

节点 IP 用途
master 192.168.174.181 管理节点和私有仓库
node1 192.168.174.180 节点一
node1 192.168.174.180 节点二

管理节点安装

docker程序的安装

[root@master ~]# yum install -y docker
[root@master ~]# fdisk -l
Disk /dev/sda: 10.7 GB, 10737418240 bytes, 20971520 sectors
Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors

# 这里我们将使用新增的/dev/sdb磁盘作为docker的存储

[root@master ~]# vim /etc/sysconfig/docker-storage-setup #使用:r 打开docker磁盘驱动模板文件/usr/share/container-storage-setup/container-storage-setup
#STORAGE_DRIVER=overlay2   #注释掉overlay2磁盘驱动
STORAGE_DRIVER=devicemapper #使用默认磁盘驱动
EXTRA_STORAGE_OPTIONS="--storage-opt dm.fs=xfs" #格式化为xfs
DEVS=/dev/sdb   #使用dev/sdb磁盘
CONTAINER_THINPOOL=container-thinpool   #thinpool的容器存储方式,也是lv的名字
VG=docker_VG    #存储的vg名称

[root@master ~]# container-storage-setup 
INFO: Writing zeros to first 4MB of device /dev/sdb
4+0 records in
4+0 records out
4194304 bytes (4.2 MB) copied, 0.00600853 s, 698 MB/s
INFO: Device node /dev/sdb1 exists.
  Physical volume "/dev/sdb1" successfully created.
  Volume group "docker_VG" successfully created
  Rounding up size to full physical extent 12.00 MiB
  Thin pool volume with chunk size 512.00 KiB can address at most 126.50 TiB of data.
  Logical volume "container-thinpool" created.
  Logical volume docker_VG/container-thinpool changed.
[root@master ~]# vgs
  VG        #PV #LV #SN Attr   VSize   VFree
  centos      1   2   0 wz--n-  <9.00g    0 
  docker_VG   1   1   0 wz--n- <10.00g 6.00g
[root@master ~]# lvs
  LV                 VG        Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root               centos    -wi-ao---- <8.00g                                                    
  swap               centos    -wi-ao----  1.00g                                                    
  container-thinpool docker_VG twi-a-t---  3.97g             0.00   10.29   
# 创建成功,启动docker服务
[root@master ~]# systemctl enable docker --now

系统配置

通过配置多个加速器实现不同网络环境下的快速部署。

# Docker加速器配置
cat>/etc/docker/daemon.json<<EOF
{
  "registry-mirrors": ["https://dockerhub.azk8s.cn","http://f1361db2.m.daocloud.io","https://d1a0f2854f4b44c2a3b3af4f5425db1a.mirror.swr.myhuaweicloud.com","https://hub-mirror.c.163.com","https://registry.docker-cn.com"],
  "insecure-registries": ["registry:5000"]
}
EOF
[root@master ~]# systemctl daemon-reload && systemctl restart docker

# 关闭防火墙
[root@master ~]# systemctl disable firewalld.service --now

# 关闭selinux
setenforce 0

Docker环境准备

镜像准备

# 拉取实验镜像
[root@master ~]# docker pull docker.io/centos
Using default tag: latest
Trying to pull repository docker.io/library/centos ... 
latest: Pulling from docker.io/library/centos
8a29a15cefae: Pull complete 
Digest: sha256:fe8d824220415eed5477b63addf40fb06c3b049404242b31982106ac204f6700
Status: Downloaded newer image for docker.io/centos:latest

# 镜像列表
[root@master ~]# docker images
REPOSITORY           TAG                 IMAGE ID            CREATED             SIZE
docker.io/nginx      latest              2073e0bcb60e        2 days ago          127 MB
docker.io/httpd      latest              c562eeace183        3 days ago          165 MB
docker.io/php        latest              7dc31b4f3403        3 days ago          405 MB
docker.io/mysql      latest              791b6e40940c        3 days ago          465 MB
docker.io/debian     latest              a8797652cfd9        3 days ago          114 MB
docker.io/registry   latest              708bc6af7e5e        12 days ago         25.7 MB
docker.io/centos     latest              470671670cac        2 weeks ago         237 MB
docker.io/mysql      5.5                 d404d78aa797        9 months ago        205 MB
docker.io/centos     6.10                48650444e419        10 months ago       194 MB

# 删除镜像
[root@master ~]# docker rmi docker.io/mysql
Untagged: docker.io/mysql:latest
Untagged: docker.io/mysql@sha256:6d0741319b6a2ae22c384a97f4bbee411b01e75f6284af0cce339fee83d7e314
Deleted: sha256:791b6e40940cd550af522eb4ffe995226798204504fe495743445b900e417a51
Deleted: sha256:a3c92ad464abbee6d08856efd404df8c43e9d991b9253bed8281e452d8021dfa
Deleted: sha256:3eb0379ecdc39f86da90c491765187e40dda381e57f319dd21afd0b1e2c40158
Deleted: sha256:fe814f19102e93fd9e2c12b4c864d110bbe4884ff4c5c34e2e1d96341ec17778
Deleted: sha256:f973fa93f201d11a3a6ccf900614fa6e25f4cf899da69f163510560263642d0e
Deleted: sha256:db53286cf6b77826bd35675098bfa76863ace9a04b4e28f4d8340d53c23821e8
Deleted: sha256:477e19600de637164faac8d2e39d4552fac8fbf3c4a9f29efe34072c0fd156e9
Deleted: sha256:2c109aa38ef35164d5adcabac202bde92420867a5839deb75f5ce034aacc00b4
Deleted: sha256:0de337169373e6779cb3ca09485e95fedd4ac98abee19b839cd46e294a64f363
Deleted: sha256:73f1cb0f35d3377b825488e38241d0e12c63e7d30946362402dd8ab2e9467d81
Deleted: sha256:5807022bbb80a63e78831d4dff1ac497a450287ce43fbb0381623b19f5d45c8a
Deleted: sha256:1aaef8d601e09d40fc66f3531268e837f4ae3eedf84f94359fa33177f0be4c6e
Deleted: sha256:e0db3ba0aaea8a01d5cb000aeb449c153be0a47a369cafc4e912b85fb18192cf

# 镜像导出
[root@master ~]# docker save docker.io/centos:6.10 > /tmp/sujxcentos.tar

# 镜像导入
[root@node2 ~]# docker load < /root/sujxcentos.tar
[root@node2 ~]# docker images
REPOSITORY                        TAG                 IMAGE ID            CREATED             SIZE
docker.io/centos                  6.10                48650444e419        10 months ago       194 MB

# 检索镜像
[root@master ~]# docker search oracle
INDEX       NAME                                            DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
docker.io   docker.io/oraclelinux                           Official Docker builds of Oracle Linux.         629       [OK]       
docker.io   docker.io/jaspeen/oracle-11g                    Docker image for Oracle 11g database            144                  [OK]
docker.io   docker.io/oracleinanutshell/oracle-xe-11g                                                       82                   
docker.io   docker.io/oracle/openjdk                        Docker images containing OpenJDK Oracle Linux   60                   [OK]
docker.io   docker.io/oracle/graalvm-ce                     GraalVM Community Edition Official Image        56                   [OK]
docker.io   docker.io/absolutapps/oracle-12c-ee             Oracle 12c EE image with web management co...   38                   
docker.io   docker.io/araczkowski/oracle-apex-ords          Oracle Express Edition 11g Release 2 on Ub...   27                   [OK]
docker.io   docker.io/bofm/oracle12c                        Docker image for Oracle Database                23                   [OK]
docker.io   docker.io/oracle/nosql                          Oracle NoSQL on a Docker Image with Oracle...   22                   [OK]
docker.io   docker.io/datagrip/oracle                       Oracle 11.2 & 12.1.0.2-se2 & 11.2.0.2-xe        14                   [OK]
docker.io   docker.io/oracle/weblogic-kubernetes-operator   Docker images containing the Oracle WebLog...   10                   
docker.io   docker.io/openweb/oracle-tomcat                 A fork off of Official tomcat image with O...   8                    [OK]
docker.io   docker.io/truevoly/oracle-12c                   Copy of sath89/oracle-12c image (https://g...   8                    
docker.io   docker.io/18fgsa/oracle-client                  Hosted version of the Oracle Container Ima...   2                    

建立本地私有仓库

[root@master ~]# docker run -d -p 5000:5000 --name=registry --restart=always docker.io/registry
345e05f68235687b47d2917fd0a86620ac2d6b40fbe7647063b817e0d690cf6b

# 打标
[root@master ~]# docker tag docker.io/mysql:5.5 registry:5000/sujx_images/mysql:5.5

#上传
[root@master ~]# docker push registry:5000/sujx_images/mysql:5.5
The push refers to a repository [registry:5000/sujx_images/mysql]
c9f3545812c8: Pushed 
f49eaacc87a0: Pushed 
a9c5a24e943f: Pushed 
90b4ae8695b5: Pushed 
4054cc666efd: Pushed 
f83622e85376: Pushed 
af84b063c827: Pushed 
ddc265b679cf: Pushed 
647245c554e4: Pushed 
432b5f62e513: Pushed 
6270adb5794c: Pushed 
5.5: digest: sha256:c9c671d0c959183154313d6830d46f9a00d5937f97415c15ebd3c6844f6f1467 size: 2619

# 本地其他客户端拉取
[root@node2 ~]# docker pull registry:5000/sujx_images/mysql:5.5
Trying to pull repository registry:5000/sujx_images/mysql ... 
5.5: Pulling from registry:5000/sujx_images/mysql
743f2d6c1f65: Pull complete 
3f0c413ee255: Pull complete 
aef1ef8f1aac: Pull complete 
f9ee573e34cb: Pull complete 
3f237e01f153: Pull complete 
03da1e065b16: Pull complete 
04087a801070: Pull complete 
7efd5395ab31: Pull complete 
1b5cc03aaac8: Pull complete 
2b7adaec9998: Pull complete 
385b8f96a9ba: Pull complete 
Digest: sha256:c9c671d0c959183154313d6830d46f9a00d5937f97415c15ebd3c6844f6f1467
Status: Downloaded newer image for registry:5000/sujx_images/mysql:5.5

# 本地其他节点上传镜像
[root@node2 ~]# docker tag docker.io/centos:6.10 registry:5000/sujx_images/centos:6.10
[root@node2 ~]# docker push registry:5000/sujx_images/centos:6.10
The push refers to a repository [registry:5000/sujx_images/centos]
8088cb617267: Pushed 
6.10: digest: sha256:7e53308393264c34359fbdf6d15d5c8c4985b8c2a58ee0ad4f7d5cc2e3c1577a size: 529