国内Ubuntu环境Docker部署Stable Diffusion入坑记录
本文旨在记录使用docker+python进行部署 stable-diffusion-webui 项目时遇到的一些问题，以及解决方案，原项目地址: https://github.com/AUTOMATIC1111/stable-diffusion-webui
问题一览：
- 国内如何下载大模型？
- 手写Docker运行时遇到的问题记录。
国内如何快速下载大模型？
使用 modelscope 模块进行下载，你可以在官网找到你需要的模型。
官网地址：https://modelscope.cn/models
# SDK模型下载
from modelscope import snapshot_download
# 示例
snapshot_download('iic/CosyVoice2-0.5B', local_dir='pretrained_models/CosyVoice2-0.5B')
Docker运行时遇到的问题
1、安装带gpu版本的torch？
在requirements_versions.txt 和 requirements.txt 的第一行加入以下文本：
国外：--extra-index-url https://download.pytorch.org/whl/cu121
国内：--extra-index-url https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/wheel/cu121/
一般我们安装带GPU的torch时通常会去 download.pytorch.org 找对应的torch版本。
加上上述文本后会自动从该网址查找并下载安装。
2、以下问题需要重新编译python
是因为手动编译python时，系统依赖安装不足，导致有些依赖库找不到指定的系统包。这种情况只能安装相关系统软件后，重新编译python。
ModuleNotFoundError: No module named '_ctypes'
使用 apt-get install libffi-dev 安装 libffi-dev。
ModuleNotFoundError: No module named '_lzma'
使用 apt-get install liblzma-dev 安装 liblzma-dev。
重新编译方法：
wget https://www.python.org/ftp/python/3.10.13/Python-3.10.13.tgz
tar -xzf Python-3.10.13.tgz
cd Python-3.10.13
./configure --with-system-ffi --enable-shared --enable-optimizations && make && make install && echo "/usr/local/lib" | tee /etc/ld.so.conf.d/python3.conf && ldconfig
3、以下问题安装需要安装git
root@58befcdcc1d7:/workspace/automatic1111-stable-diffusion-webui# python3 webui.py
/usr/local/lib/python3.10/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
warnings.warn(
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/git/__init__.py", line 87, in <module>
refresh()
File "/usr/local/lib/python3.10/site-packages/git/__init__.py", line 76, in refresh
if not Git.refresh(path=path):
File "/usr/local/lib/python3.10/site-packages/git/cmd.py", line 341, in refresh
raise ImportError(err)
ImportError: Bad git executable.
The git executable must be specified in one of the following ways:
- be included in your $PATH
- be set via $GIT_PYTHON_GIT_EXECUTABLE
- explicitly set via git.refresh()
All git commands will error until this is rectified.
This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
- quiet|q|silence|s|none|n|0: for no warning or exception
- warn|w|warning|1: for a printed warning
- error|e|raise|r|2: for a raised exception
Example:
export GIT_PYTHON_REFRESH=quiet
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/workspace/automatic1111-stable-diffusion-webui/webui.py", line 15, in <module>
from modules import import_hook, errors, extra_networks, ui_extra_networks_checkpoints
File "/workspace/automatic1111-stable-diffusion-webui/modules/ui_extra_networks_checkpoints.py", line 6, in <module>
from modules import shared, ui_extra_networks, sd_models
File "/workspace/automatic1111-stable-diffusion-webui/modules/shared.py", line 16, in <module>
from modules import localization, extensions, script_loading, errors, ui_components, shared_items
File "/workspace/automatic1111-stable-diffusion-webui/modules/extensions.py", line 6, in <module>
import git
File "/usr/local/lib/python3.10/site-packages/git/__init__.py", line 89, in <module>
raise ImportError('Failed to initialize: {0}'.format(exc)) from exc
ImportError: Failed to initialize: Bad git executable.
The git executable must be specified in one of the following ways:
- be included in your $PATH
- be set via $GIT_PYTHON_GIT_EXECUTABLE
- explicitly set via git.refresh()
All git commands will error until this is rectified.
This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
- quiet|q|silence|s|none|n|0: for no warning or exception
- warn|w|warning|1: for a printed warning
- error|e|raise|r|2: for a raised exception
Example:
export GIT_PYTHON_REFRESH=quiet
使用 apt install git 安装 git即可。
4、以下问题是因为缺少python依赖包或依赖包的版本问题
ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'
这是因为高版本的torch有些包已经移动或者更改了导入方式，可选择降torch版本到1.13以下或者更改高版本的源码的导入方式。
请参考这篇博客：
https://blog.csdn.net/lanxing147/article/details/136625264
这篇博客讲到了有些版本是可以使用的，比如笔者使用的 torch 2.12 + torchvision0.16.2 + torchaudio2.1.2。
可使用以下命令安装:
pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu118
但是国内比较慢，可以选择使用国内镜像源进行提速。使用以下命令：
pip install torch==2.1.2 torchvision torchaudio -f https://mirrors.aliyun.com/pytorch-wheels/cu121/
参考以下文章：
https://docs.infini-ai.com/posts/download-pytorch-from-mirror.html
TypeError: AsyncConnectionPool.__init__() got an unexpected keyword argument 'socket_options'
这是由于 httpcore 版本太低导致的， 使用 pip3 install -U httpcore 升级 httpcore 到最新版本即可。
ModuleNotFoundError: No module named 'clip'
使用 pip3 install clip 安装 clip 库即可。
No module named 'pytorch_lightning.utilities.distributed'
pytorch_lightning的版本较低，安装高版本的 pytorch_lightning 即可。
笔者使用的是 pip3 pytorch_lightning==1.9。
No module named 'gdown'
使用 pip3 install gdown 安装 gdown库即可。
No module named 'open_clip_torch'
使用 pip3 install open_clip_torch 安装 open_clip_torch库即可。
5、启动时遇到的问题
最终启动用命令：
python3 webui.py --listen --port=7860 --no-half --disable-nan-check
写在最后
附上完整的Docker 镜像构筑的相关文件，国内基本上也能够很顺畅的一键构筑好docker镜像，总大小约28G。
请在项目根目录下创建一个 docker 目录，然后将 Dockerfile、compose.yaml、requirements.txt 、 requirements_versions.txt、start.sh等文件放进去。
然后执行 cd docker && docker compose -f compose.yaml up 命令。
如果访问浏览器的 7860 端口出现以下界面，恭喜你成功了。
Dockerfile
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04
ARG VENV_NAME="sd-webui"
ENV VENV=$VENV_NAME
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
ENV DEBIAN_FRONTEN=noninteractive
SHELL ["/bin/bash", "-c"]
RUN apt-get update -y
RUN apt-get install -y libgl1-mesa-glx libglib2.0-0
RUN apt-get install -y net-tools wget curl git
RUN apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev libffi-dev liblzma-dev
# 从国内镜像源下载安装python
# wget https://www.python.org/ftp/python/3.10.13/Python-3.10.13.tar.xz && tar Jxf Python-3.10.13.tar.xz
RUN wget https://mirrors.huaweicloud.com/python/3.10.13/Python-3.10.13.tar.xz && tar Jxf Python-3.10.13.tar.xz
RUN cd Python-3.10.13 && ./configure --with-system-ffi --enable-shared --enable-optimizations && make && make install && echo "/usr/local/lib" | tee /etc/ld.so.conf.d/python3.conf && ldconfig
RUN python3 -V && pip3 -V
# 设置国内镜像源
RUN pip3 config set global.index-url https://mirrors.aliyun.com/pypi/simple/ && pip3 config set install.trusted-host mirrors.aliyun.com
WORKDIR /workspace
COPY ./requirements_versions.txt ./
COPY ./requirements.txt ./
RUN pip3 install -r requirements_versions.txt
RUN pip3 install -r requirements.txt
compose.yaml
services:
sd-webui:
container_name: sd-webui
image: sd-webui:1.0
restart: always
ports:
- 7860:7860
environment:
- TZ=Asia/Tokyo
- NVIDIA_VISIBLE_DEVICES=all
volumes:
- ../../automatic1111-stable-diffusion-webui:/workspace/automatic1111-stable-diffusion-webui
# command: tail -f /dev/null
command: sh -c "sh /workspace/automatic1111-stable-diffusion-webui/docker/start.sh"
deploy:
resources:
reservations:
devices:
- driver: nvidia
capabilities: [gpu]
requirements.txt
--extra-index-url https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/wheel/cu121/
blendmodes
accelerate
basicsr
fonts
font-roboto
gfpgan
gradio
invisible-watermark
numpy
omegaconf
opencv-contrib-python
requests
piexif
Pillow
# pytorch_lightning==1.7.7
pytorch_lightning==1.9
realesrgan
scikit-image>=0.19
timm==0.4.12
transformers==4.25.1
# torch
torch==2.1.2
torchaudio==2.1.2
einops
jsonmerge
clean-fid
resize-right
torchdiffeq
kornia
lark
inflection
GitPython
torchsde
safetensors
psutil
open_clip_torch
gdown
clip
requirements_versions.txt
--extra-index-url https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/wheel/cu121/
blendmodes==2022
transformers==4.25.1
accelerate==0.12.0
basicsr==1.4.2
gfpgan==1.3.8
gradio==3.16.2
numpy==1.23.3
Pillow==9.4.0
realesrgan==0.3.0
torch
omegaconf==2.2.3
# pytorch_lightning==1.7.6
pytorch_lightning
scikit-image==0.19.2
fonts
font-roboto
timm==0.6.7
piexif==1.1.3
einops==0.4.1
jsonmerge==1.8.0
clean-fid==0.1.29
resize-right==0.0.2
torchdiffeq==0.2.3
kornia==0.6.7
lark==1.1.2
inflection==0.5.1
GitPython==3.1.27
torchsde==0.2.5
safetensors==0.2.7
# httpcore<=0.15
httpcore
fastapi==0.90.1
open_clip_torch
start.sh
#! /bin/bash
cd automatic1111-stable-diffusion-webui && python3 webui.py --listen --port=7860 --no-half --disable-nan-check
以上就是本次踩坑的记录。愿看到的小伙伴不迷路。
![[桌面运维]windows自动设置浅深色主题](https://i-blog.csdnimg.cn/direct/1d7e43a6299d48c9b16d8c5c254a06ef.png)
![[ubuntu-22.04]ubuntu不识别rtl8153 usb转网口](https://i-blog.csdnimg.cn/direct/ae1d1eb23292477f9d36a4890edeae6f.png)
