openwebUI访问vllm加载deepseek微调过的本地大模型

文章目录

前言
一、openwebui安装
二、配置openwebui环境
三、安装vllm
四、启动vllm
五、启动openwebui

前言

首先安装vllm，然后加载本地模型，会起一个端口好。
在安装openwebui,去访问这个端口号。下面具体步骤的演示。

一、openwebui安装

root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp# mkdir open-webui
root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp# cd open-webui/
root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# git clone https://github.com/open-webui/open-webui.git
Cloning into 'open-webui'...
remote: Enumerating objects: 90389, done.
remote: Counting objects: 100% (121/121), done.
remote: Compressing objects: 100% (64/64), done.
remote: Total 90389 (delta 80), reused 57 (delta 57), pack-reused 90268 (from 2)
Receiving objects: 100% (90389/90389), 174.91 MiB | 16.20 MiB/s, done.
Resolving deltas: 100% (59438/59438), done.
root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# conda create --name open-webui python=3.12
Channels:
 - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
 - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
 - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

二、配置openwebui环境

在激活虚拟环境时候，下面提示错误，让先run init，但打这个也没啥用，只需要source activate一下即可。
进入虚拟环境就安装openwebui.

root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# conda activate open-webui 

CondaError: Run 'conda init' before 'conda activate'

root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# source activate
(base) root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# conda activate open-webui 
(open-webui) root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# pip install open-webui
Looking in indexes: http://mirrors.aliyun.com/pypi/simple
Collecting open-webui
  Downloading http://mirrors.aliyun.com/pypi/packages/76/f7/89777775051feb35049d70b9119e050b7830ed1eb07cfaa7159bd0c52cc0/open_webui-0.5.18-py3-none-any.whl (131.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 131.0/131.0 MB 17.1 MB/s eta 0:00:00
Collecting aiocache (from open-webui)

三、安装vllm

然后开始装vllm

(open-webui) root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# pip install vllm
Looking in indexes: http://mirrors.aliyun.com/pypi/simple
Collecting vllm
  Downloading http://mirrors.aliyun.com/pypi/packages/4f/d2/18246f43ca730bb81918f87b7e886531eda32d835811ad9f4657c54eee35/sentencepiece-0.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 15.0 MB/s eta 0:00:00
  Downloading http://mirrors.aliyun.com/pypi/packages/8d/cf/9b775a1a1f5fe2f6c2d321396ad41b9849de2c76fa46d78e6294ea13be91/vllm-0.7.3-cp38-abi3-manylinux1_x86_64.whl (264.6 MB)
     ━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━ 136.6/264.6 MB 14.5 MB/s eta 0:00:09

四、启动vllm

启动模型

(open-webui) root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# vllm serve /root/autodl-tmp/llm/deepseek-ai/DeepSeek-R1-Distill-Qwen-1___5B-merged
INFO 03-02 10:00:20 __init__.py:207] Automatically detected platform cuda.
INFO 03-02 10:00:20 api_server.py:912] vLLM API server version 0.7.3
INFO 03-02 10:00:20 api_server.py:913] args: Namespace(subparser='serve', model_tag='/root/autodl-tmp/llm/deepseek-ai/DeepSeek-R1-Distill-Qwen-1___5B-merged', config='', host=None, port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api

五、启动openwebui

调用的微调模型来源
https://blog.csdn.net/weixin_41688410/article/details/145948449
配置openUI环境,因为openwebUI是默认连ollama,所以要设置为false.设置vllm启动的端口号。

(open-webui) root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# export HF_ENDPOINT=https://hf-mirror.com
(open-webui) root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# export ENABLE_OLLAMA_API=False
(open-webui) root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# export OPENAI_API_BASE_URL=http://127.0.0.1:8000/v1
(open-webui) root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui#

上述环境配好后就可以启动open-webui

(open-webui) root@autodl-container-5fd249bc19-6ec28c15:~/autodl-tmp/open-webui# open-webui serve
Loading WEBUI_SECRET_KEY from file, not provided as an environment variable.
Converting 'chat' column to JSON
Renaming 'chat' column to 'old_chat'
Adding new 'chat' column of type JSON
Dropping 'old_chat' column
INFO  [alembic.runtime.migration] Running upgrade 242a2047eae0 -> 1af9b942657b, Migrate tags
INFO  [alembic.runtime.migration] Running upgrade 1af9b942657b -> 3ab32c4b8f59, Update tags
Primary Key: {'name': None, 'constrained_columns': []}
INFO  [alembic.runtime.migration] Running upgrade 3ab32c4b8f59 -> c69f45358db4, Add folder table
INFO  [alembic.runtime.migration] Running upgrade c69f45358db4 -> c29facfe716b, Update file table path
WARNING: CORS_ALLOW_ORIGIN IS SET TO '*' - NOT RECOMMENDED FOR PRODUCTION DEPLOYMENTS.
INFO  [open_webui.env] Embedding model set: sentence-transformers/all-MiniLM-L6-v2
/root/miniconda3/envs/open-webui/lib/python3.12/site-packages/pydub/utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work
  warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning)
WARNI [langchain_community.utils.user_agent] USER_AGENT environment variable not set, consider setting it to identify your requests.

 ██████╗ ██████╗ ███████╗███╗   ██╗    ██╗    ██╗███████╗██████╗ ██╗   ██╗██╗
██╔═══██╗██╔══██╗██╔════╝████╗  ██║    ██║    ██║██╔════╝██╔══██╗██║   ██║██║
██║   ██║██████╔╝█████╗  ██╔██╗ ██║    ██║ █╗ ██║█████╗  ██████╔╝██║   ██║██║
██║   ██║██╔═══╝ ██╔══╝  ██║╚██╗██║    ██║███╗██║██╔══╝  ██╔══██╗██║   ██║██║
╚██████╔╝██║     ███████╗██║ ╚████║    ╚███╔███╔╝███████╗██████╔╝╚██████╔╝██║
 ╚═════╝ ╚═╝     ╚══════╝╚═╝  ╚═══╝     ╚══╝╚══╝ ╚══════╝╚═════╝  ╚═════╝ ╚═╝

v0.5.18 - building the best open-source AI user interface.
https://github.com/open-webui/open-webui

gitattributes: 1.23kB [00:00, 9.08MB/s]                                                                                                          
1_Pooling%2Fconfig.json: 100%|███████████████████████████████████████████████████████████████████████████████████| 190/190 [00:00<00:00, 1.30MB/s]
data_config.json: 39.3kB [00:00, 120MB/s]                                                                               | 0.00/