AI智算-k8s部署大语言模型管理工具Ollama

文章目录

- - 简介
  - k8s部署Ollama
  - Open WebUI
  - - 访问Open-WebUI

简介

Github：https://github.com/ollama/ollama
官网：https://ollama.com/
API：https://github.com/ollama/ollama/blob/main/docs/api.md

Ollama 是一个基于 Go 语言开发的可以本地运行大模型的开源框架，同时提供 REST API 管理和使用大模型。

k8s部署Ollama

~# cat ollama-dp.yaml 
kind: Deployment
apiVersion: apps/v1
metadata:
  name: ollama
  labels:
    app: ollama
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ollama
  template:
    metadata:
      labels:
        app: ollama
    spec:
      volumes:
        - name: ollama-models
          hostPath:
            path: /file_CPU_01/cyw_data/ollama
            type: ''
        - name: host-time
          hostPath:
            path: /etc/localtime
            type: ''
      containers:
        - name: ollama
          image: ollama/ollama:0.5.1
          ports:
            - name: http-11434
              containerPort: 11434
              protocol: TCP
          resources:
            limits:
              nvidia.com/gpu: '8'
            requests:
              nvidia.com/gpu: '8'
          volumeMounts:
            - name: ollama-models
              mountPath: /root/.ollama
            - name: host-time
              readOnly: true
              mountPath: /etc/localtime
          imagePullPolicy: IfNotPresent
      restartPolicy: Always
      schedulerName: volcano
---
kind: Service
apiVersion: v1
metadata:
  name: ollama
  labels:
    app: ollama
spec:
  ports:
    - name: http-11434
      protocol: TCP
      port: 11434
      targetPort: 11434
      nodePort: 31434
  selector:
    app: ollama
  type: NodePort

~# kubectl get pods -n sre-tools |grep ollama
ollama-8cd499f48-4b6wv      1/1     Running   0          13m
~# kubectl get svc -n sre-tools |grep ollama
ollama   NodePort   10.233.6.148   <none>        11434:31434/TCP   13m

测试接口：

~# curl -i  http://10.0.xx.xx:31434
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Thu, 12 Dec 2024 08:46:55 GMT
Content-Length: 17

Ollama is running

Open WebUI

Github：https://github.com/open-webui/open-webui
官网：https://openwebui.com/

Open WebUI 是一个可扩展、功能丰富且用户友好的自托管 AI 界面，旨在完全离线运行。它支持各种 LLM 运行器，包括 Ollama 和 OpenAI 兼容 API。

~# cat open-webui.yaml 
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: open-webui-data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  storageClassName: nfs-client

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: open-webui-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: open-webui
  template:
    metadata:
      labels:
        app: open-webui
    spec:
      containers:
      - name: open-webui
        image: ghcr.mirrorify.net/open-webui/open-webui:main
        ports:
        - containerPort: 8080
        env:
        - name: OLLAMA_BASE_URL
          value: "http://10.0.xx.xx:31434"   #ollama地址
        volumeMounts:
        - name: open-webui-data
          mountPath: /app/backend/data
      volumes:
      - name: open-webui-data
        persistentVolumeClaim:
          claimName: open-webui-data-pvc

---
apiVersion: v1
kind: Service
metadata:
  name: open-webui-service
spec:
  type: ClusterIP
  ports:
    - port: 3000
      targetPort: 8080 
  selector:
    app: open-webui
 
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: open-webui-ingress
spec:
  rules:
  - host: open-webui.xxx-sh.com
    http:
      paths:
      - backend:
          service:
            name: open-webui-service
            port:
              number: 3000
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - open-webui.xxx-sh.com
    secretName: xxx-tls