kubekey 离线安装harbor、k8s、kubesphere

目录

参考文献

前提条件

部署准备

下载kubukey

离线包配置和制作

配置离线包

制作离线包

离线安装集群

复制KubeKey 和制品 artifact到离线机器

创建初始换、安装配置文件

 安装镜像仓库harbor

初始化harbor 项目

修改配置文件

安装k8s集群和kubesphere

手动安装依赖包

检查环境是否满足要求

检查防火墙端口是否满足要求

安装完成并登录

安装过程中遇到的问题

制作制品，不能下载github上的操作系统iso

初始化安装harbor报错must specify a CommonName

初始化harbor后，发现harbor的一些模块容器没有正常启动

麒麟系统安装，安装包没有 Fkylin-v10-amd64.iso

离线安装却在线下载calicoctl

参考文献

官网离线安装文档

前提条件

要开始进行多节点安装，您需要参考如下示例准备至少三台主机。

主机 IP主机名称角色
192.168.0.2node1联网主机用于制作离线包
192.168.0.3node2离线环境主节点
192.168.0.4node3离线环境镜像仓库节点

关闭防火墙、selinux、swap、dnsmasq(所有节点)
关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

关闭selinux

sed -i 's/enforcing/disabled/' /etc/selinux/config  #永久
setenforce 0  #临时

关闭swap（k8s禁止虚拟内存以提高性能）

sed -ri 's/.*swap.*/#&/' /etc/fstab #永久
swapoff -a #临时
//关闭dnsmasq（否则可能导致docker容器无法解析域名）

service dnsmasq stop 
systemctl disable dnsmaq

有的机器不允许关闭防火墙可以看下文需要开放的端口

部署准备

下载kubukey

执行以下命令下载 KubeKey 并解压

方式一（可以访问github）：

从 GitHub Release Page 下载 KubeKey 或者直接运行以下命令。

curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.13 sh -

方式二：

首先运行以下命令，以确保您从正确的区域下载 KubeKey。

export KKZONE=cn

运行以下命令来下载 KubeKey：

curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.13 sh -

离线包配置和制作

配置离线包

在联网主机上执行以下命令，并复制示例中的 manifest 内容。

vim manifest.yaml

---

apiVersion: kubekey.kubesphere.io/v1alpha2

kind: Manifest

metadata:

  name: sample

spec:

  arches:

  - amd64

  operatingSystems:

  - arch: amd64

    type: linux

    id: centos

    version: "7"

    repository:

      iso:

        localPath:

        url: https://github.com/kubesphere/kubekey/releases/download/v3.0.10/centos7-rpms-amd64.iso

  - arch: amd64

    type: linux

    id: ubuntu

    version: "20.04"

    repository:

      iso:

        localPath:

        url: https://github.com/kubesphere/kubekey/releases/download/v3.0.10/ubuntu-20.04-debs-amd64.iso

  kubernetesDistributions:

  - type: kubernetes

    version: v1.23.15

  components:

    helm:

      version: v3.9.0

    cni:

      version: v1.2.0

    etcd:

      version: v3.4.13

    calicoctl:

      version: v3.23.2

   ## For now, if your cluster container runtime is containerd, KubeKey will add a docker 20.10.8 container runtime in the below list.

   ## The reason is KubeKey creates a cluster with containerd by installing a docker first and making kubelet connect the socket file of containerd which docker contained.

    containerRuntimes:

    - type: docker

      version: 20.10.8

    - type: containerd

      version: 1.6.4

    crictl:

      version: v1.24.0

    docker-registry:

      version: "2"

    harbor:

      version: v2.5.3

    docker-compose:

      version: v2.2.2

  images:

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.23.15

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.23.15

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.23.15

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.23.15

  - registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.6

  - registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.8.6

  - registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.23.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.23.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.23.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.23.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.23.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/flannel:v0.12.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/provisioner-localpv:3.3.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/linux-utils:3.3.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.3

  - registry.cn-beijing.aliyuncs.com/kubesphereio/nfs-subdir-external-provisioner:v4.0.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.15.12

  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-installer:v3.4.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.4.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-console:v3.4.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-controller-manager:v3.4.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.22.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.21.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.20.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kubefed:v0.8.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/tower:v0.2.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z

  - registry.cn-beijing.aliyuncs.com/kubesphereio/mc:RELEASE.2019-08-07T23-14-43Z

  - registry.cn-beijing.aliyuncs.com/kubesphereio/snapshot-controller:v4.0.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx-ingress-controller:v1.1.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/defaultbackend-amd64:1.4

  - registry.cn-beijing.aliyuncs.com/kubesphereio/metrics-server:v0.4.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine

  - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.0.25-alpine

  - registry.cn-beijing.aliyuncs.com/kubesphereio/alpine:3.14

  - registry.cn-beijing.aliyuncs.com/kubesphereio/openldap:1.3.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/netshoot:v1.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/cloudcore:v1.13.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/iptables-manager:v1.13.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/edgeservice:v0.3.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/gatekeeper:v3.5.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/openpitrix-jobs:v3.3.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-apiserver:ks-v3.4.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-controller:ks-v3.4.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-tools:ks-v3.4.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-jenkins:v3.4.0-2.319.3-1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/inbound-agent:4.10-2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2-podman

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0-podman

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0-podman

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11-podman

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0-podman

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0-podman

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16-podman

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17-podman

  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18-podman

  - registry.cn-beijing.aliyuncs.com/kubesphereio/s2ioperator:v3.2.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/s2irun:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/s2i-binary:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-runtime:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-runtime:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-runtime:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-runtime:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-8-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-6-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-4-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/python-36-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/python-35-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/python-34-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/python-27-centos7:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/argocd:v2.3.3

  - registry.cn-beijing.aliyuncs.com/kubesphereio/argocd-applicationset:v0.4.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/dex:v2.30.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/redis:6.2.6-alpine

  - registry.cn-beijing.aliyuncs.com/kubesphereio/configmap-reload:v0.7.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus:v2.39.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-config-reloader:v0.55.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-operator:v0.55.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.11.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-state-metrics:v2.6.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/node-exporter:v1.3.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/alertmanager:v0.23.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/thanos:v0.31.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/grafana:8.3.3

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.11.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager-operator:v2.3.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager:v2.3.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-tenant-sidecar:v3.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-curator:v5.7.6

  - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-oss:6.8.22

  - registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch:2.6.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch-dashboards:2.6.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch-curator:v0.0.5

  - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentbit-operator:v0.14.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/docker:19.03

  - registry.cn-beijing.aliyuncs.com/kubesphereio/fluent-bit:v1.9.4

  - registry.cn-beijing.aliyuncs.com/kubesphereio/log-sidecar-injector:v1.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/filebeat:6.7.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-operator:v0.6.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-exporter:v0.6.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-ruler:v0.6.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-operator:v0.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-webhook:v0.2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/pilot:1.14.6

  - registry.cn-beijing.aliyuncs.com/kubesphereio/proxyv2:1.14.6

  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-operator:1.29

  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-agent:1.29

  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-collector:1.29

  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-query:1.29

  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-es-index-cleaner:1.29

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali-operator:v1.50.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali:v1.50

  - registry.cn-beijing.aliyuncs.com/kubesphereio/busybox:1.31.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx:1.14-alpine

  - registry.cn-beijing.aliyuncs.com/kubesphereio/wget:1.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/hello:plain-text

  - registry.cn-beijing.aliyuncs.com/kubesphereio/wordpress:4.8-apache

  - registry.cn-beijing.aliyuncs.com/kubesphereio/hpa-example:latest

  - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentd:v1.4.2-2.0

  - registry.cn-beijing.aliyuncs.com/kubesphereio/perl:latest

  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-productpage-v1:1.16.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v1:1.16.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v2:1.16.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-details-v1:1.16.2

  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-ratings-v1:1.16.3

  - registry.cn-beijing.aliyuncs.com/kubesphereio/scope:1.13.0

备注

  • 若需要导出的 artifact 文件中包含操作系统依赖文件（如：conntarck、chrony 等），可在 operationSystem 元素中的 .repostiory.iso.url 中配置相应的 ISO 依赖文件下载地址或者提前下载 ISO 包到本地在 localPath 里填写本地存放路径并删除 url 配置项。

  • 开启 harbor 和 docker-compose 配置项，为后面通过 KubeKey 自建 harbor 仓库推送镜像使用。

  • 默认创建的 manifest 里面的镜像列表从 docker.io 获取。

  • 可根据实际情况修改 manifest-sample.yaml 文件的内容，用于之后导出期望的 artifact 文件。

  • 您可以访问 Release v3.0.7 🌈 · kubesphere/kubekey · GitHub 下载 ISO 文件。

个人备注： 

 在这一步我遇到了操作系统配置在githab上，然后下载不下来的情况

然后我去github上手动下载下来放到了本地，然后下盖下面的配置部分

 operatingSystems:

  - arch: amd64

    type: linux

    id: centos

    version: "7"

    repository:

      iso:

        localPath: 添加你的本地地址

        url: 

  - arch: amd64

    type: linux

    id: ubuntu

    version: "20.04"

    repository:

      iso:

        localPath: 添加你的本地地址

        url:

下载地址

 https://github.com/kubesphere/kubekey/releases/tag/v3.0.10

制作离线包

导出制品 artifact。 

方式一（可以访问github）：

执行以下命令：

./kk artifact export -m manifest-sample.yaml -o kubesphere.tar.gz

方式二：

依次运行以下命令：

export KKZONE=cn

./kk artifact export -m manifest-sample.yaml -o kubesphere.tar.gz

备注

制品（artifact）是一个根据指定的 manifest 文件内容导出的包含镜像 tar 包和相关二进制文件的 tgz 包。在 KubeKey 初始化镜像仓库、创建集群、添加节点和升级集群的命令中均可指定一个 artifact，KubeKey 将自动解包该 artifact 并在执行命令时直接使用解包出来的文件。

  • 导出时请确保网络连接正常。

  • KubeKey 会解析镜像列表中的镜像名，若镜像名中的镜像仓库需要鉴权信息，可在 manifest 文件中的 .registry.auths 字段中进行配置。

离线安装集群

复制KubeKey 和制品 artifact到离线机器

将下载的 KubeKey 和制品 artifact 通过 U 盘等介质拷贝至离线环境安装节点。

创建初始换、安装配置文件

执行以下命令创建离线集群配置文件： 

./kk create config --with-kubesphere v3.4.1 --with-kubernetes v1.23.15 -f config-sample.yaml

执行以下命令修改配置文件：

vim config-sample.yaml

备注

  • 按照实际离线环境配置修改节点信息。
  • 必须指定 registry 仓库部署节点（用于 KubeKey 部署自建 Harbor 仓库）。
  • registry 里必须指定 type 类型为 harbor，否则默认安装 docker registry。

 安装镜像仓库harbor

执行以下命令安装镜像仓库

  1. ./kk init registry -f config-sample.yaml -a kubesphere.tar.gz

    备注

    命令中的参数解释如下：

    • config-sample.yaml 指离线环境集群的配置文件。

    • kubesphere.tar.gz 指源集群打包出来的 tar 包镜像。

个人备注

在执行初始换安装 harbor时报错

11:16:46 UTC success: [rs-node-178-02]
11:16:46 UTC success: [rs-node-177-01]
11:16:46 UTC success: [rs-master-174-01]
11:16:46 UTC success: [rs-node-179-03]
11:16:46 UTC success: [rs-master-175-02]
11:16:46 UTC success: [rs-master-176-03]
11:16:46 UTC success: [devops-180]
11:16:46 UTC [ConfigureOSModule] configure the ntp server for each node
11:16:46 UTC skipped: [rs-node-179-03]
11:16:46 UTC skipped: [rs-master-174-01]
11:16:46 UTC skipped: [rs-master-175-02]
11:16:46 UTC skipped: [rs-master-176-03]
11:16:46 UTC skipped: [devops-180]
11:16:46 UTC skipped: [rs-node-177-01]
11:16:46 UTC skipped: [rs-node-178-02]
11:16:46 UTC [InitRegistryModule] Fetch registry certs
11:16:46 UTC success: [devops-180]
11:16:46 UTC [InitRegistryModule] Generate registry Certs
[certs] Using existing ca certificate authority
11:16:46 UTC message: [LocalHost]
unable to sign certificate: must specify a CommonName
11:16:46 UTC failed: [LocalHost]
error: Pipeline[InitRegistryPipeline] execute failed: Module[InitRegistryModule] exec failed: 
failed: [LocalHost] [GenerateRegistryCerts] exec failed after 1 retries: unable to sign certificate: must specify a CommonName

解决方案：

https://ask.kubesphere.io/forum/d/22879-kubesphere34-unable-to-sign-certificate-must-specify-a-commonname

修改配置文件 

...  
registry:
    type: harbor
    auths:
     "dockerhub.kubekey.local":
       username: admin
       password: Harbor12345
    privateRegistry: "dockerhub.kubekey.local"
    namespaceOverride: ""
    registryMirrors: []
    insecureRegistries: []
  addons: []

然后重新执行即可

 执行安装后去harbor对应服务器检查harbor启动情况，

如果有部分模块启动失败

进入  /opt/harbor  目录

chmod 777 -R ./common

并重启harbor

docker-compose down -v

docker-compose u -d

harbor启动后可以浏览器访问

初始化harbor 项目

备注

由于 Harbor 项目存在访问控制（RBAC）的限制，即只有指定角色的用户才能执行某些操作。如果您未创建项目，则镜像不能被推送到 Harbor。Harbor 中有两种类型的项目：

  • 公共项目（Public）：任何用户都可以从这个项目中拉取镜像。
  • 私有项目（Private）：只有作为项目成员的用户可以拉取镜像。

Harbor 管理员账号：admin，密码：Harbor12345。Harbor 安装文件在 /opt/harbor , 如需运维 Harbor，可至该目录下。

方式一： 

执行脚本创建 Harbor 项目。

a. 执行以下命令下载指定脚本初始化 Harbor 仓库：

curl -O https://raw.githubusercontent.com/kubesphere/ks-installer/master/scripts/create_project_harbor.sh

b. 执行以下命令修改脚本配置文件：

vim create_project_harbor.sh

修改成

#!/usr/bin/env bash



# Copyright 2018 The KubeSphere Authors.

#

# Licensed under the Apache License, Version 2.0 (the "License");

# you may not use this file except in compliance with the License.

# You may obtain a copy of the License at

#

#     http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.



url="https://dockerhub.kubekey.local"  #修改url的值为https://dockerhub.kubekey.local

user="admin"

passwd="Harbor12345"



harbor_projects=(library

    kubesphereio

    kubesphere

    argoproj

    calico

    coredns

    openebs

    csiplugin

    minio

    mirrorgooglecontainers

    osixia

    prom

    thanosio

    jimmidyson

    grafana

    elastic

    istio

    jaegertracing

    jenkins

    weaveworks

    openpitrix

    joosthofman

    nginxdemos

    fluent

    kubeedge

    openpolicyagent

)



for project in "${harbor_projects[@]}"; do

    echo "creating $project"

    curl -u "${user}:${passwd}" -X POST -H "Content-Type: application/json" "${url}/api/v2.0/projects" -d "{ \"project_name\": \"${project}\", \"public\": true}" -k #curl命令末尾加上 -k

done

备注

  • 修改 url 的值为 https://dockerhub.kubekey.local。

  • 需要指定仓库项目名称和镜像列表的项目名称保持一致。

  • 脚本末尾 curl 命令末尾加上 -k。

c. 执行以下命令创建 Harbor 项目：

chmod +x create_project_harbor.sh
./create_project_harbor.sh

 方式二：

登录 Harbor 仓库创建项目。将项目设置为公开以便所有用户都能够拉取镜像。关于如何创建项目，请参阅创建项目。

修改配置文件

再次执行以下命令修改集群配置文件：

vim config-sample.yaml

  ...

  registry:

    type: harbor

    auths:

      "dockerhub.kubekey.local":

        username: admin

        password: Harbor12345

    privateRegistry: "dockerhub.kubekey.local"

    namespaceOverride: "kubesphereio"

    registryMirrors: []

    insecureRegistries: []

  addons: []

备注

  • 新增 auths 配置增加 dockerhub.kubekey.local 和账号密码。
  • privateRegistry 增加 dockerhub.kubekey.local。
  • namespaceOverride 增加 kubesphereio。

安装k8s集群和kubesphere

 执行以下命令安装 KubeSphere 集群:

./kk create cluster -f config-sample.yaml -a kubesphere.tar.gz --with-packages

参数解释如下：

  • config-sample.yaml：离线环境集群的配置文件。
  • kubesphere.tar.gz：源集群打包出来的 tar 包镜像。
  • --with-packages：若需要安装操作系统依赖，需指定该选项。

执行以下命令查看集群状态：

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f

执行命令会看到以下提示：

[root@k8s-master kubekey]# ./kk create cluster -f config-sample.yaml -a kubesphere.tar.gz --with-packages


 _   __      _          _   __           
| | / /     | |        | | / /           
| |/ / _   _| |__   ___| |/ /  ___ _   _ 
|    \| | | | '_ \ / _ \    \ / _ \ | | |
| |\  \ |_| | |_) |  __/ |\  \  __/ |_| |
\_| \_/\__,_|_.__/ \___\_| \_/\___|\__, |
                                    __/ |
                                   |___/

11:07:36 CST [GreetingsModule] Greetings
11:07:37 CST message: [k8s-master]
Greetings, KubeKey!
11:07:37 CST message: [k8s-node]
Greetings, KubeKey!
11:07:37 CST success: [k8s-master]
11:07:37 CST success: [k8s-node]
11:07:37 CST [NodePreCheckModule] A pre-check on nodes
11:07:44 CST success: [k8s-master]
11:07:44 CST success: [k8s-node]
11:07:44 CST [ConfirmModule] Display confirmation form
+------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| name       | sudo | curl | openssl | ebtables | socat | ipset | ipvsadm | conntrack | chrony | docker | containerd | nfs client | ceph client | glusterfs client | time         |
+------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| k8s-node   | y    | y    | y       | y        |       | y     |         |           | y      | 24.0.6 | v1.7.3     | y          |             |                  | CST 11:07:44 |
| k8s-master | y    | y    | y       | y        |       | y     |         |           | y      |        | y          | y          |             |                  | CST 11:07:43 |
+------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+

This is a simple check of your environment.
Before installation, ensure that your machines meet all requirements specified at
https://github.com/kubesphere/kubekey#requirements-and-recommendations

Continue this installation? [yes/no]: no
手动安装依赖包

然后需要手动离线安装 

socat ipvsadm conntrack ceph client glusterfs client

并确保满足 https://github.com/kubesphere/kubekey#requirements-and-recommendations  中的安装条件

检查环境是否满足要求

  • Minimum resource requirements (For Minimal Installation of KubeSphere only)：
    • 2 vCPUs
    • 4 GB RAM
    • 20 GB Storage

/var/lib/docker is mainly used to store the container data, and will gradually increase in size during use and operation. In the case of a production environment, it is recommended that /var/lib/docker mounts a drive separately.

  • OS requirements:
    • SSH can access to all nodes.
    • Time synchronization for all nodes.
    • sudo/curl/openssl should be used in all nodes.
    • docker can be installed by yourself or by KubeKey.
    • Red Hat includes SELinux in its Linux release. It is recommended to close SELinux or switch the mode of SELinux to Permissive
  • It's recommended that Your OS is clean (without any other software installed), otherwise there may be conflicts.
  • A container image mirror (accelerator) is recommended to be prepared if you have trouble downloading images from dockerhub.io. Configure registry-mirrors for the Docker daemon.
  • KubeKey will install OpenEBS to provision LocalPV for development and testing environment by default, this is convenient for new users. For production, please use NFS / Ceph / GlusterFS or commercial products as persistent storage, and install the relevant client in all nodes.
  • If you encounter Permission denied when copying, it is recommended to check SELinux and turn off it first
  • Dependency requirements:

KubeKey can install Kubernetes and KubeSphere together. Some dependencies need to be installed before installing kubernetes after version 1.18. You can refer to the list below to check and install the relevant dependencies on your node in advance.

Kubernetes Version ≥ 1.18
socatRequired
conntrackRequired
ebtablesOptional but recommended
ipsetOptional but recommended
ipvsadmOptional but recommended
  • Networking and DNS requirements:
    • Make sure the DNS address in /etc/resolv.conf is available. Otherwise, it may cause some issues of DNS in cluster.
    • If your network configuration uses Firewall or Security Group，you must ensure infrastructure components can communicate with each other through specific ports. It's recommended that you turn off the firewall or follow the link configuriation: NetworkAccess.
检查防火墙端口是否满足要求

端口开放满足 https://github.com/kubesphere/kubekey/blob/master/docs/network-access.md

If your network configuration uses an firewall，you must ensure infrastructure components can communicate with each other through specific ports that act as communication endpoints for certain processes or services.

servicesprotocolactionstart portend portcomment
sshTCPallow22
etcdTCPallow23792380
apiserverTCPallow6443
calicoTCPallow90999100
bgpTCPallow179
nodeportTCPallow3000032767
masterTCPallow1025010258
dnsTCPallow53
dnsUDPallow53
local-registryTCPallow5000offline environment
local-aptTCPallow5080offline environment
rpcbindTCPallow111use NFS
ipipIPENCAP / IPIPallowcalico needs to allow the ipip protocol

安装完成后，您会看到以下内容：

**************************************************

#####################################################

###              Welcome to KubeSphere!           ###

#####################################################



Console: http://192.168.0.3:30880

Account: admin

Password: P@88w0rd



NOTES：

1. After you log into the console, please check the

monitoring status of service components in

the "Cluster Management". If any service is not

ready, please wait patiently until all components

are up and running.

1. Please change the default password after login.



#####################################################

https://kubesphere.io             2022-02-28 23:30:06

#####################################################
安装完成并登录

通过 http://{IP}:30880 使用默认帐户和密码 admin/P@88w0rd 访问 KubeSphere 的 Web 控制台。

安装过程中遇到的问题

制作制品，不能下载github上的操作系统iso

  在这一步我遇到了操作系统配置在githab上，然后下载不下来的情况

然后我去github上手动下载下来放到了本地，然后下盖下面的配置部分

 operatingSystems:

  - arch: amd64

    type: linux

    id: centos

    version: "7"

    repository:

      iso:

        localPath: 添加你的本地地址

        url: 

  - arch: amd64

    type: linux

    id: ubuntu

    version: "20.04"

    repository:

      iso:

        localPath: 添加你的本地地址

        url:

下载地址

 https://github.com/kubesphere/kubekey/releases/tag/v3.0.10

初始化安装harbor报错must specify a CommonName

11:16:46 UTC success: [rs-node-178-02]
11:16:46 UTC success: [rs-node-177-01]
11:16:46 UTC success: [rs-master-174-01]
11:16:46 UTC success: [rs-node-179-03]
11:16:46 UTC success: [rs-master-175-02]
11:16:46 UTC success: [rs-master-176-03]
11:16:46 UTC success: [devops-180]
11:16:46 UTC [ConfigureOSModule] configure the ntp server for each node
11:16:46 UTC skipped: [rs-node-179-03]
11:16:46 UTC skipped: [rs-master-174-01]
11:16:46 UTC skipped: [rs-master-175-02]
11:16:46 UTC skipped: [rs-master-176-03]
11:16:46 UTC skipped: [devops-180]
11:16:46 UTC skipped: [rs-node-177-01]
11:16:46 UTC skipped: [rs-node-178-02]
11:16:46 UTC [InitRegistryModule] Fetch registry certs
11:16:46 UTC success: [devops-180]
11:16:46 UTC [InitRegistryModule] Generate registry Certs
[certs] Using existing ca certificate authority
11:16:46 UTC message: [LocalHost]
unable to sign certificate: must specify a CommonName
11:16:46 UTC failed: [LocalHost]
error: Pipeline[InitRegistryPipeline] execute failed: Module[InitRegistryModule] exec failed: 
failed: [LocalHost] [GenerateRegistryCerts] exec failed after 1 retries: unable to sign certificate: must specify a CommonName

解决方案：

https://ask.kubesphere.io/forum/d/22879-kubesphere34-unable-to-sign-certificate-must-specify-a-commonname

修改配置文件 

...  
registry:
    type: harbor
    auths:
     "dockerhub.kubekey.local":
       username: admin
       password: Harbor12345
    privateRegistry: "dockerhub.kubekey.local"
    namespaceOverride: ""
    registryMirrors: []
    insecureRegistries: []
  addons: []

然后重新执行即可

初始化harbor后，发现harbor的一些模块容器没有正常启动

 执行安装后去harbor对应服务器检查harbor启动情况，

如果有部分模块容器启动失败

进入  /opt/harbor  目录

chmod 777 -R ./common

并重启harbor

docker-compose down -v

docker-compose u -d

harbor启动后可以浏览器访问

麒麟系统安装，安装包没有 Fkylin-v10-amd64.iso

关于##kylin##上安装KubeSphere的问题，如何解决？ - KubeSphere 开发者社区

离线安装却在线下载calicoctl

离线安装 kubesphere v3.4.1 报错Failed to download calicoctl binary - KubeSphere 开发者社区

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：/a/534130.html

如若内容造成侵权/违法违规/事实不符，请联系我们进行投诉反馈qq邮箱809451989@qq.com，一经查实，立即删除！

相关文章

子域名是什么？有什么作用？

子域名是什么？有什么作用？

在互联网世界中，域名是我们访问网站的关键。每一个公司的网站都需要拥有自己的域名，其中有些大型公司的网站还不止一个域名，除了主域名外还拥有子域名。有些人感到非常困惑，不知道子域名是什么。其实子域名也就是平时所说的二级域…
阅读更多...
AI绘本生成解决方案，快速生成高质量的AI绘本视频

AI绘本生成解决方案，快速生成高质量的AI绘本视频

美摄科技凭借其深厚的技术积累和前瞻性的市场洞察力，近日推出了一款面向企业的AI绘本生成解决方案，旨在通过智能化、自动化的方式，帮助企业快速将文字内容转化为生动有趣的绘本视频，从而提升内容传播效率，增强品牌影响…
阅读更多...
openssl密钥证书管理（Key and Certificate Management）

openssl密钥证书管理（Key and Certificate Management）

前言 前两日应别人要求提供一份CSR文件过去，方便他们生成相关证书，对于这一块本来也不熟，于是找到openssl官网，想找找相关的教程看看，一番小找，果有收获，是个宝藏，源文档在这…
阅读更多...
AI 对话完善【人工智能】

AI 对话完善【人工智能】

AI 对话【人工智能】 前言版权开源推荐AI 对话v0版本：基础v1版本：对话数据表tag.jsTagController v2版本：回复中textarea.jsChatController v3版本：流式输出chatLast.jsChatController v4版本：多轮对话QianfanUtilChat…
阅读更多...
1、Qt UI控件 -- qucsdk

1、Qt UI控件 -- qucsdk

前言：Qt编写的自定义控件插件的sdk集合，包括了各个操作系统的动态库文件以及控件的头文件和sdk使用demo。类似于Wpf中的LivChart2控件库，都是一些编译好的控件，可以直接集成到项目中。该控件是飞扬青云大神多年前开发的&#xff0…
阅读更多...
从零开始：构建、打包并上传个人前端组件库至私有npm仓库的完整指南

从零开始：构建、打包并上传个人前端组件库至私有npm仓库的完整指南

文章目录 一、写组件1、注册全局组件方法2、组件13、组件2 二、测试三、发布1、配置package.json2、生成库包3、配置发布信息4、发布 四、使用1、安装2、使用 五、维护1、维护和更新2、注意事项 一、写组件 确定组件库的需求和功能：在开始构建组件库之前&#xff0c…
阅读更多...
三相整流桥器件选型计算方法-电压与电流计算公式

三相整流桥器件选型计算方法-电压与电流计算公式

三相整流桥的选型主要涉及到两个关键参数：电压和电流。以下是电压与电流的计算公式及选型方法： 电压计算： 输入交流电压有效值（Vrms）是选择整流桥的重要参考。整流桥的额定电压（Vrrm）应至少为输…
阅读更多...
2024QS计算机科学 学科排名

2024QS计算机科学 学科排名

计算机科学
阅读更多...
echarts tooltip提示框显示不全

echarts tooltip提示框显示不全

一、背景： 写在前面： 自行封装。一个可由多个柱形图叠加而成的图表，命名为someHoverLine(可自定义)。 下面罗列了移动端和web端的封装组件代码； 展示了vue2、uniapp、vue3的不同封装和使用案列。 二、问题描述： 三、解…
阅读更多...
数组常用方法

数组常用方法

for循环 使用计数器变量来迭代数组元素 var arr [1,2,3,4,5]for(var i0;i<arr.length;i){console.log(array[i]) }上面的i就是计数器变量&#xff0c;初始值为0&#xff0c;每次循环后加1&#xff0c;直到i等于数组长度为止。在循环体内&#xff0c;可以通过数组索引arr[…
阅读更多...
C 强制类型转换

C 强制类型转换

强制类型转换是把变量从一种类型转换为另一种数据类型。例如&#xff0c;如果您想存储一个 long 类型的值到一个简单的整型中&#xff0c;您需要把 long 类型强制转换为 int 类型。您可以使用强制类型转换运算符来把值显式地从一种类型转换为另一种类型&#xff0c;如下所示&am…
阅读更多...
攻防演练 | redis艰难写shell进入内网

攻防演练 | redis艰难写shell进入内网

背景 某地市级攻防演练 要求 拿到指定单位的靶标系统&#xff08;必须是web后台管理权限数据库服务器&#xff09; 正式开始 redis未授权 首先通过信息收集获取到了一些IP&#xff0c;且发现一个IP中存在redis未授权。 此时兴冲冲的去尝试写入定时任务反弹shell&#xff…
阅读更多...
淘宝评论数据API接口：洞察消费者声音的关键工具

淘宝评论数据API接口：洞察消费者声音的关键工具

淘宝评论数据API接口&#xff1a;洞察消费者声音的关键工具 请求示例&#xff0c;API接口接入Anzexi58 随着电子商务的蓬勃发展&#xff0c;消费者对于商品的评价和反馈成为了购物决策中不可或缺的一部分。淘宝作为中国最大的电商平台之一&#xff0c;汇聚了海量的商品和评论数…
阅读更多...
【SpringBoot】-- mapstruct进行类型转换时Converter实现类不能自动生成代码问题解决

【SpringBoot】-- mapstruct进行类型转换时Converter实现类不能自动生成代码问题解决

问题描述 我的问题如下&#xff1a; 应该在红色区域生成对应的转换细节&#xff0c;但是这里只返回了一个空对象 问题解决 加入lombok-mapstruct-binding依赖,也要注意依赖引用顺序问题 <dependency><groupId>org.projectlombok</groupId><artifactId&…
阅读更多...
photomaker：customizing realistic human photos via stacked id embedding

photomaker：customizing realistic human photos via stacked id embedding

PhotoMaker: 高效个性化定制人像照片文生图 - 知乎今天分享我们团队最新的工作PhotoMaker的技术细节。该工作开源5天Githubstar数已过6千次&#xff0c;已列入Github官方Trending榜第一位&#xff0c;PaperswithCode热度榜第一位&#xff0c;HuggingFace Spaces趋势榜第一位。项…
阅读更多...
小阳同学刷题日记-54. 螺旋矩阵

小阳同学刷题日记-54. 螺旋矩阵

题目&#xff1a;给你一个 m 行 n 列的矩阵 matrix &#xff0c;请按照 顺时针螺旋顺序 &#xff0c;返回矩阵中的所有元素。 思路&#xff1a; 初始化四个边界指针&#xff1a;top, bottom, left, right 分别表示当前螺旋遍历的上边界、下边界、左边界和右边界。不断遍历矩阵直…
阅读更多...
Windows虚拟主机上多个域名访问同一个网站

Windows虚拟主机上多个域名访问同一个网站

近日老板提出了想要多个域名访问同一个网站的想法。这边了解后&#xff0c;由于我们公司使用的是Hostease的Windows虚拟主机产品&#xff0c;因此咨询了Hostease的技术支持&#xff0c;寻求帮助了解到可以实现Windows主机上多个域名访问同一个网站&#xff0c;是需要进入Window…
阅读更多...
win10如何移除此电脑下的“网络”

win10如何移除此电脑下的“网络”

1.按住winr,然后输入regedit打开注册表&#xff0c;在导航栏输入&#xff0c;定位到ShellFolder文件夹 HKEY_CLASSES_ROOT\CLSID\{F02C1A0D-BE21-4350-88B0-7367FC96EF3C}\ShellFolder2.更改权限&#xff0c;在ShellFolder文件夹上右键选择权限->高级->更改->输入Adm…
阅读更多...
antd vue table控件的使用（三）

antd vue table控件的使用（三）

今天就讲讲Ant Design Vue下的控件----table表格&#xff08;分页、编辑和删除功能&#xff09; 结合项目中的需求&#xff0c;看看如何配置,让table即可以展示列表&#xff0c;又可以直接编辑数据。需求&#xff1a; &#xff08;1&#xff09;展示即将检查的数据列表&#…
阅读更多...
通过WebShell登录SQL Server主机并使用SSRS报表服务

通过WebShell登录SQL Server主机并使用SSRS报表服务

背景信息 RDS SQL Server提供了WebShell功能&#xff0c;允许用户通过Web界面登录到RDS SQL Server实例的操作系统中&#xff0c;并在该操作系统中执行命令、上传下载文件等操作。WebShell功能方便用户对RDS SQL Server实例的管理和维护&#xff0c;特别是在无法使用SSH客户端的…
阅读更多...
最新文章

Copyright @ 2022~2023