背景

nas开启nfs存储共享，默认情况下只能给IP/24做限制, 达不到安全效果

需要增加kerberos策略校验，并且持久化kerberos数据，避免容器重启丢失数据

环境描述

宿主机系统：CentOS Linux release 7.9.2009 (Core)

Docker版本：Docker version 20.10.6, build 370c289

一、容器部署kerberos

​# 注意添加 /etc/hosts
192.168.10.10 kerberosclient.wo.com
192.168.10.10 kerberosserver.wo.com

1.kerberos配置文件

mkdir -p /data/kerberos/dockerfile

cd /data/kerberos/dockerfile

@FQDN@ 后面使用sed更改为大写域名 = WO.COM

@fqdn@ 后面使用sed更改为小写域名 = wo.com

@kdc_server@ kdc服务器地址 = kerberosclient.wo.com

krb5.conf

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt
 default_realm = @FQDN@
 default_ccache_name = FILE:/tmp/krb5cc_cli_%{uid}

[realms]
 @FQDN@ = {
  kdc = @kdc_server@
  admin_server = @kdc_server@
  admin_keytab = /var/lib/krb5kdc/admin.keytab
  database_name = /var/lib/krb5kdc/principal
 }

[domain_realm]
 .@fqdn@ = @FQDN@

kadm5.acl

*/admin@@FQDN@ *

kerberos.sh

#!/bin/bash
fqdn="wo.com"
FQDN="WO.COM"
kdc_server="kerberosserver.wo.com"
PASS="Fa1Q@D@N"

KRB5_KTNAME="/var/lib/krb5kdc/admin.keytab"
inited="/var/lib/krb5kdc/inited"

sed -i "s#@kdc_server@#${kdc_server}#g" /etc/krb5kdc/kdc.conf
sed -i "s#@fqdn@#${fqdn}#g" /etc/krb5kdc/kdc.conf
sed -i "s#@FQDN@#${FQDN}#g" /etc/krb5kdc/kdc.conf
sed -i "s#@FQDN@#${FQDN}#g" /etc/krb5kdc/kadm5.acl

function init_user() {
    if [ -f "${inited}" ];then
        # TODO没找到指定该文件的配置项, 将该文件还原到/etc/目录
        cp /var/lib/krb5kdc/.k5.${FQDN} /etc/krb5kdc/.k5.${FQDN}

        echo "kerberos已存在, 跳过初始化"
        return;
    fi

    echo "begin init user"

    # create kerberos database
    echo -e "${PASS}\n${PASS}" | kdb5_util create -s

    # create admin
    echo -e "${PASS}\n${PASS}" | kadmin.local -q "addprinc root/admin"
    kadmin.local -q "ktadd -k /var/lib/krb5kdc/admin.keytab root/admin"

    # create client
    echo -e "${PASS}\n${PASS}" | kadmin.local -q "addprinc -randkey nfs/kerberosclient.${fqdn}"

    # create client keytab
    kadmin.local -q "ktadd -norandkey -k ${KRB5_KTNAME} nfs/kerberosclient.${fqdn}"
    kadmin.local -q "xst -k /app/cert/krb5.keytab -norandkey nfs/kerberosclient.${fqdn}"

    # client使用
    cp /etc/krb5kdc/kdc.conf /app/cert/krb5.conf

    # 将该文件持久化存储
    cp /etc/krb5kdc/.k5.${FQDN} /var/lib/krb5kdc/.k5.${FQDN}

    touch "${inited}"

    echo "user inite success"
}

function main() {
    init_user
    /usr/local/bin/supervisord -n -c /etc/supervisord.conf
}

main

supervisord.conf

[supervisord]
logfile=/var/log/supervisord/supervisord.log    ; supervisord log file
logfile_maxbytes=50MB                           ; maximum size of logfile before rotation
logfile_backups=10                              ; number of backed up logfiles
loglevel=error                                  ; info, debug, warn, trace
pidfile=/var/run/supervisord.pid                ; pidfile location
nodaemon=false                                  ; run supervisord as a daemon
minfds=1024                                     ; number of startup file descriptors
minprocs=200                                    ; number of process descriptors
user=root                                       ; default user
childlogdir=/var/log/supervisord/               ; where child log files will live

[program:krb5-kdc]
command=service krb5-kdc start
autostart=true
autorestart=true

[program:krb5-admin-server]
command=service krb5-admin-server start
autostart=true
autorestart=true

[supervisorctl]

dockerfile

FROM ubuntu:xenial

ENV DEBIAN_FRONTEND noninteractive

RUN sed -i s@/archive.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list \
    && sed -i s@/security.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list \
    && apt update \
    && apt install -y python-dev python-pip python-wheel python-setuptools python-pkg-resources krb5-admin-server krb5-kdc \
    && rm -rf /var/lib/apt/lists/* \
    && mkdir -p /var/log/supervisord /app/cert \
    && pip install supervisor==4.2.4

COPY krb5.conf /etc/krb5kdc/kdc.conf
COPY kadm5.acl /etc/krb5kdc/kadm5.acl
COPY krb5.conf /etc/krb5.conf
COPY kerberos.sh /app/kerberos.sh
COPY supervisord.conf /etc/supervisord.conf

WORKDIR /app

CMD ["/bin/bash", "/app/kerberos.sh"]

2.构建镜像

docker build -t kerberos:1.0.0 .

3.运行镜像

mkdir -p /data/kerberos/data
cd /data/kerberos

start.sh

#!/bin/bash
docker rm -f kerberos

# 持久化数据, 避免容器重启数据库丢失
# /app/cert 用于给client的keytab和conf配置
# /var/lib/krb5kdc 数据库文件存放路径
# /etc/krb5kdc/.k5.xxx master文件, 也需要持久化
# 在kerberos.sh脚本时, 会将.k5文件放到/var/lib/krb5kdc

docker run -itd  \
    -p 88:88 \
    -p 749:749 \
    -v /data/kerberos/data/cert:/app/cert \
    -v /data/kerberos/data/db:/var/lib/krb5kdc \
    --name=kerberos \
    kerberos:1.0.0

二、nas配置krb5.keytab

/data/kerberos/data/cert/krb5.keytab

将krb5.keytab上传到nas

# 开启krb5校验

三、nfs客户端机器

1.安装krb5

yum -y install  krb5-workstation ufs-utils

文件从kerberos服务端获取，拷贝到客户端(注意区分机器)
cp /data/kerberos/data/cert/krb5.conf  /etc/krb5.conf
cp /data/kerberos/data/cert/krb5.keytab /etc/krb5.keytab

客户端启动rpc-gssd
systemctl restart rpc-gssd

2.验证

kinit -kt /etc/krb5.keytab nfs/kerberosclient.wo.com@WO.COM

klist

挂载
mount -o vers=4,sec=krb5 kerberosclient.wo.com:/volume1/data /mnt

四、参考文档

如何配置 NFS 共享文件夹以使用 Kerberos？ - Synology 知识中心

Synology NAS NFS Kerberos 配置与使用 – 个人笔记分享

NFS | DSM - Synology 知识中心

使用Docker快速搭建Kerberos环境 - 知乎

基于Kerberos认证的NFS服务器_nfs kerberos_黑色蒲G英~的博客-CSDN博客

五、其他报错信息

1.用户和组显示nobody

用户和组显示nobody

更改/etc/idmapd.conf 将Domain 改为fqdn 的域名

systemctl restart rpcidmapd

2.创建文件提示权限不足

在nas上,上传kerberos密钥对的配置，增加ID映射，对应用那个user

3.access denied by server while mounting

# 查看kerberos应用日志
tail -f  /var/log/k*

4.mount.nfs an incorrect mount option was specified

# 没有krb5.conf krb5.keytab 2个文件

# 然后启动rpc-gssd
systemctl restart rpc-gssd

 5.挂载后文件都显示777权限

 改为无映射