部署Prometheus + Grafana实现监控数据指标

1.1 Prometheus安装部署

Prometheus监控服务

主机名	IP地址	系统	配置	作用
Prometheus	192.168.110.27/24	CentOS 7.9	4颗CPU 8G内存 100G硬盘	Prometheus服务器
grafana	192.168.110.28/24	CentOS 7.9	4颗CPU 8G内存 100G硬盘	grafana服务器

监控机器

主机名	IP地址	系统	配置
k8s-master-01	192.168.110.21/24	CentOS 7.9	4颗CPU 8G内存 100G硬盘

1.1.1 环境准备

所有机器配置Hosts解析

cat >> /etc/hosts << EOF
192.168.110.21 k8s-master-01
192.168.110.27 Prometheus
192.168.110.28 grafana
EOF

关闭防护墙和SElinux

设置时间同步

sed -i '3,6 s/^/# /' /etc/chrony.conf
sed -i '6 a server ntp.aliyun.com iburst' /etc/chrony.conf
systemctl restart chronyd.service
chronyc sources

1.1.2 二进制方式安装Prometheus（二选一）

从 Download | Prometheus 下载相应版本，安装到服务器上官网提供的是二进制版，解压就能用，不需要编译

[root@Prometheus ~]# wget -c https://github.com/prometheus/prometheus/releases/download/v2.52.0/prometheus-2.52.0.linux-amd64.tar.gz

解压软件

[root@Prometheus ~]# tar xf prometheus-2.52.0.linux-amd64.tar.gz -C /usr/local/
[root@Prometheus ~]# cd /usr/local/
[root@Prometheus local]# ln  -sv prometheus-2.52.0.linux-amd64/ prometheus
‘prometheus’ -> ‘prometheus-2.52.0.linux-amd64/’

[root@Prometheus ~]# mkdir /usr/local/prometheus/data

创建服务运行用户

[root@Prometheus ~]# useradd prometheus -M -s /sbin/nologin
[root@Prometheus ~]# chown -R prometheus.prometheus /usr/local/prometheus/*

创建服务运行脚本

[root@Prometheus ~]# vim /usr/lib/systemd/system/prometheus.service 
[Unit]
Description=prometheus
After=network.target

[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/prometheus/prometheus --config.file=/usr/local/prometheus/prometheus.yml --storage.tsdb.path=/usr/local/prometheus/data
Restart=on-failure
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target

启动服务

[root@Prometheus ~]# systemctl daemon-reload 
[root@Prometheus ~]# systemctl enable --now prometheus.service
[root@Prometheus ~]# systemctl is-active prometheus.service
active
[root@Prometheus ~]# ss -lunpt | grep 9090
tcp    LISTEN     0      128    [::]:9090               [::]:*                   users:(("prometheus",pid=2486,fd=7))

访问 http://本机IP:9090 http://192.168.110.27:9090

1.1.3 容器方式安装 Prometheus（二选一）

安装Docker-ce

[root@Prometheus ~]# wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.huaweicloud.com/docker-ce/linux/centos/docker-ce.repo
[root@Prometheus ~]# sed -i 's+download.docker.com+mirrors.huaweicloud.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
[root@Prometheus ~]# sed -i 's/$releasever/7Server/g' /etc/yum.repos.d/docker-ce.repo
[root@Prometheus ~]# mkdir -p /etc/docker
[root@Prometheus ~]# tee /etc/docker/daemon.json <<-'EOF'
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors": [
"https://dbckerproxy.com",
"https://hub-mirror.c.163.com",
"https://mirror.baidubce.com",
"https://ccr.ccs.tencentyun.com"
]
}
EOF

[root@Prometheus ~]# systemctl daemon-reload
[root@Prometheus ~]# systemctl enable --now docker.service
[root@Prometheus ~]# docker --version
Docker version 20.10.21, build baeda1

拉取Prometheus镜像

[root@Prometheus ~]# docker pull prom/prometheus
[root@Prometheus ~]# docker image ls
REPOSITORY        TAG       IMAGE ID       CREATED       SIZE
prom/prometheus   latest    ecb74a3b23a9   2 weeks ago   272MB
[root@Prometheus ~]# docker save -o prometheus.tar prom/prometheus:latest
[root@Prometheus ~]# ls
anaconda-ks.cfg  prometheus.tar

直接使用官方的镜像启动，并映射prometheus.yml配置文件到本地进行管理

[root@Prometheus ~]# mkdir /data
[root@Prometheus ~]# vim /data/prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'local'
    metrics_path: '/metrics'
    scrape_interval: 5s
    static_configs:
      - targets: ['192.168.110.27:9090']

[root@Prometheus ~]# docker run -d --name prometheus -v /data:/data -p 9090:9090 prom/prometheus --config.file=/data/prometheus.yml
eb149926e5cb74c4a3f0641e82d40c590118647297676c862280b559213f3ca6

1.1.4 主机数据展示

通过http://服务器IP:9090/metrics可以查看到监控的数据

[root@Prometheus ~]# curl http://192.168.110.27:9090/metrics

1.1.5 Grafana安装部署

下载地址:Download Grafana | Grafana Labs

[root@grafana ~]# yum install https://mirrors.tuna.tsinghua.edu.cn/grafana/yum/rpm/Packages/grafana-8.0.4-1.x86_64.rpm -y
[root@grafana ~]# systemctl enable --now grafana-server
[root@grafana ~]# netstat -lnupt | grep 3000
tcp6       0      0 :::3000                 :::*                    LISTEN      13068/grafana-serve

使用grafana-cli 安装插件

[root@grafana ~]# grafana-cli plugins list-remote    # 列出所有可用的插件
[root@grafana ~]# grafana-cli plugins install grafana-worldmap-panel   # 安装世界地图插件
[root@grafana ~]# grafana-cli plugins install grafana-clock-panel   # 安装时间插件
[root@grafana ~]# grafana-cli plugins install grafana-piechart-panel   # 安装圆饼插件
[root@grafana ~]# grafana-cli plugins ls
installed plugins:
grafana-clock-panel @ 2.1.5
grafana-piechart-panel @ 1.6.4
grafana-worldmap-panel @ 1.0.6
Please restart Grafana after installing plugins. Refer to Grafana documentation for instructions if necessary.

[root@grafana ~]# systemctl restart grafana-server