AI智算-k8s部署DeepSeek Janus-Pro-7B 多模态大模型

文章目录

- - 简介
  - 环境依赖
  - 模型下载
  - 下载Janus库
  - GPU环境镜像
  - 模型manifest
  - 调用Janus
  - - 多模态
    - 文生图

简介

DeepSeek Janus Pro 作为一款强大的多模态理解与生成框架，正在成为研究人员和开发者的热门选择。本文将详细介绍如何在云原生k8s环境中部署配置和使用 DeepSeek Janus Pro，以及它在多模态任务中的强大功能。

环境依赖

Kubernetes、GPU（H100）

模型下载

参考链接：https://www.modelscope.cn/models/deepseek-ai/Janus-Pro-7B/files

modelscope download --model deepseek-ai/Janus-Pro-7B --local_dir file_CPU_01/modelServing/Janus-Pro-7B/

下载Janus库

git clone  https://github.com/deepseek-ai/Janus.git

GPU环境镜像

modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.1.0-py310-torch2.3.1-tf2.16.1-1.22.2

模型manifest

cat Janus-Pro.yaml 
kind: Deployment
apiVersion: apps/v1
metadata:
  name: januspro
  labels:
    app: januspro
spec:
  replicas: 1
  selector:
    matchLabels:
      app: januspro
  template:
    metadata:
      labels:
        app: januspro
    spec:
      schedulerName: volcano
      volumes:
        - name: shm
          emptyDir:
            medium: Memory
            sizeLimit: 512G
        - name: host-time
          hostPath:
            path: /etc/localtime
            type: ''
        - name: model-volume
          hostPath:
            path: /file_CPU_01/modelServing/Janus-Pro-7B/
            type: ''
      containers:
        - name: januspro
          image: modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.1.0-py310-torch2.3.1-tf2.16.1-1.22.2
          command: ["/bin/bash", "-c"]
          args: ["/modelscope_cache/startup.sh"]
          ports:
            - name: http-8000
              containerPort: 8000
              protocol: TCP
          env:
          - name: MODELSCOPE_CACHE
            value: "/modelscope_cache"
          resources:
            limits:
              nvidia.com/gpu: '8'
            requests:
              nvidia.com/gpu: '8'
          volumeMounts:
            - name: host-time
              readOnly: true
              mountPath: /etc/localtime
            - name: shm
              mountPath: /dev/shm
            - name: model-volume
              mountPath: /modelscope_cache
          imagePullPolicy: IfNotPresent
      restartPolicy: Always
---
kind: Service
apiVersion: v1
metadata:
  name: januspro
  labels:
    app: januspro
spec:
  ports:
    - name: http-7860
      protocol: TCP
      port: 7860
      targetPort: 7860
  selector:
    app: januspro
  type: NodePort

###模型启动脚本###
cat startup.sh 
#!/bin/bash
# startup.sh

# 复制文件
cp /modelscope_cache/Janus-main/frpc_linux_amd64_v0.3 /usr/local/lib/python3.10/site-packages/gradio/frpc_linux_amd64_v0.3

# 安装依赖
echo "Installing dependencies..."
cd /modelscope_cache/Janus-main
pip install -e . 

# 启动
python demo/app_januspro.py --device cuda

调用Janus

kubectl get svc -n sre-tools 
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)           AGE
januspro             NodePort    10.233.38.3     <none>        7860:31456/TCP    3h52m