1 下载CUDA和cudnn

详见文章

CUDA与CUDNN在Windows下的安装与配置（超级详细版）_windows cudnn安装-CSDN博客

我电脑的CUDA下载链接如下

​​​​​https://developer.nvidia.com/cuda-12-1-0-download-archive?target_os=Windows&target_arch=x86_64&target_version=11&target_type=exe_local

安装成功nvcc -V命令显示如下内容

PS D:\Downloads> nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Wed_Feb__8_05:53:42_Coordinated_Universal_Time_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0 

我电脑的cudnn下载链接如下

https://developer.nvidia.com/cudnn-downloads?target_os=Windows&target_arch=x86_64&target_version=Agnostic&cuda_version=12

 

将cudnn-windows-x86_64-9.2.0.82_cuda12-archive里的对应bin、lib、include三个文件内容合并到C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1即可

打开C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\extras\demo_suite，查看是否有文件bandwidthTest.exe以及deviceQuery.exe，若存在则在该文件中打开cmd运行两.exe文件

 

 

2 安装python包

1) Clone or download

https://github.com/oobabooga/text-generation-webui/archive/refs/heads/main.zip

2) Run the start_windows.bat

这里会运行text-generation-webui\one_click.py会显示详细的安装内容

其中会根据cuda的版本安装对于的pytorch 如下所示

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

具体的下载路径如下

https://download.pytorch.org/whl/cu121/torch-2.2.1%2Bcu121-cp311-cp311-win_amd64.whl

3) Select your GPU vendor when asked.

4) Once the installation ends, browse to `http://localhost:7860/?__theme=dark`.