文章目录
- 1.PuLID模型简介:
- 2.PuLID_Flux 工作流的部署流程
- 安装pulid节点
- 3.部署遇到的一些问题
- 加载这个节点错误:PulidFluxInsightFaceLoader
- PulidFluxEvaClipLoader加载错误
- 4.PuLID模型的出图效果
- 5.一些参数的设置
- 用到的提示词
1.PuLID模型简介:
PuLID: Pure and Lightning ID Customization via Contrastive Alignment[NeurIPS 2024]
PuLID 技术的核心优势在于快速换脸,弄起来的时候特别方便,不需要去调整模型。只要上传人脸,就可以把我们想要换的人脸融入到各种场景的人物和风格的人像中,换脸效果非常逼真。
该论文的贡献总结如下。
(1)提出了一种免调优方法,即PuLID,它保留了高ID相似度,同时减轻了对原始模型行为的影响。
(2) 在常规扩散分支旁边引入了闪电 T2I 分支。在此分支中,结合了对比对齐损失和 ID 损失,以最大限度地减少原始模型上 ID 信息的污染,同时确保保真度。与当前改进 ID 编码器或数据集的主流方法相比,提供了新的视角和训练范例。
(3) 实验表明,方法在 ID 保真度和可编辑性方面均实现了 SOTA 性能。此外,与现有方法相比,ID信息对模型的侵入较小,使得方法在实际应用中更加灵活。
2.PuLID_Flux 工作流的部署流程
由于该节点是基于flux 的,所以需要先下载好flux的权重
flux的官方指示文件
精度为fp8的flux-dev,模型大小17个G,打包了vae个clip
安装pulid节点
首先安装pulid节点到comfyui的custom_node
可以在comfyui的custom_node,打开终端,用git命令安装
git clone https://github.com/balazik/ComfyUI-PuLID-Flux.git
然后激活虚拟环境
conda activate env_name
然后把这个节点所需要的包(在这个节点的requirements.txt文件夹里面写了),为自己的虚拟环境安装上
pip install -r requirements.txt
将pulid模型放到/ComfyUI/models/pulid/ 目录下
pulid模型的权重
3.部署遇到的一些问题
主要是连个节点加载错误
加载这个节点错误:PulidFluxInsightFaceLoader
参考的解决办法
将antelopev2.zip解压后的文件放入“ComfyUI\models\insightface\models\antelopev2这个文件夹里面就OK了
PulidFluxEvaClipLoader加载错误
参考这里:参考的经验
clip的下载路径:EVA-CLIP
我们需要把eva-clip的权重文件下载到clip 的文件夹,然后修改加载的绝对路径
要修改的文件在:/your_path/ComfyUI/custom_nodes/ComfyUI-PuLID-Flux-Enhanced/eva_clip/factory.py line 276-281
这里将checkpoint_path改为自己存放eva-clip的路径,然后把下面的4行打上注释就可以了
4.PuLID模型的出图效果
参考图像是小李子的一些出图效果
5.一些参数的设置
用到的提示词
1. 休闲风格
A high-resolution photograph of a 24-year-old Asian man standing casually in a vibrant, sunlit city street. He has short, dark hair with a slightly tousled look and is wearing a loose-fitting, white T-shirt with a subtle graphic print, paired with light blue denim jeans and trendy sneakers. His expression is relaxed, with a playful smile, exuding confidence and youthful energy as he takes up more of the frame. The background is dynamic and full of life, with clear details of a busy street scene—pedestrians strolling, cyclists passing by, food trucks parked nearby, and colorful street art on the walls. Cars and buses move in the distance, and neon signs add a pop of color to the urban environment. The lighting is bright, with sunlight casting sharp shadows, giving the whole scene a sense of vibrancy, movement, and energy. The overall atmosphere is laid-back but filled with the lively pulse of the city.
2. 优雅正式风格
A high-resolution photograph of a 24-year-old Asian man sitting at a sleek wooden desk in a modern office space. He has short, neatly styled black hair and is dressed in a sharp, dark grey suit with a slim black tie. His white shirt is crisp, and his posture is confident, exuding professionalism. He takes up a larger portion of the frame, with his face clearly visible, expressing calm authority. The background is rich with details of the office: glass windows revealing a skyline of tall buildings, a plant on the desk, and a modern coffee machine on a side table. The lighting is soft yet bright, casting a clear glow on his suit, and the sharpness of the background adds to the contemporary and energetic atmosphere. The scene conveys a sense of modern sophistication, with a balanced mix of professionalism and dynamic city energy.
3. 户外运动风格
A high-resolution photograph of a 24-year-old Asian man hiking along a mountain trail, dressed in a fitted grey hiking jacket, black cargo pants, and sturdy hiking boots. His short, dark hair is slightly ruffled from the wind, and his focused expression reveals determination and excitement as he takes in the breathtaking views. He occupies a larger portion of the frame, with the clear details of his gear and features standing out. The background is expansive and sharp, showing vibrant green trees, rocky slopes, and distant snow-capped peaks under a vivid blue sky. A group of hikers can be seen in the distance, and a vibrant sunset begins to cast a warm glow over the landscape. The scene is full of motion and life, with the energy of outdoor adventure captured in every detail.
4. 简约风格
A high-resolution photograph of a 24-year-old Asian man seated on a minimalist wooden bench in a chic, bright studio space. He is dressed in a simple white T-shirt and black jeans, and his short black hair is styled neatly. His expression is calm and reflective, exuding a quiet confidence as he occupies a prominent portion of the frame. The background is sharp and clean, with crisp lines and neutral tones that highlight the simplicity of the setting. Natural light streams through large windows, casting clear, soft shadows and emphasizing the textures of the wood and the minimalist decor. The scene feels open and serene, creating an elegant yet vibrant atmosphere where the focus is on the subject's natural presence.
5. 艺术氛围风格
A high-resolution photograph of a 24-year-old Asian man standing in a modern art gallery, surrounded by colorful abstract paintings. His look is artistic and unconventional, with tousled dark hair and a loose-fitting black sweater. He pairs it with dark jeans and sleek boots, standing tall with one hand in his pocket as he gazes thoughtfully at the artwork. The scene is rich in detail, with vivid colors from the paintings in the background, sharp lines and textures from the gallery’s polished floors, and soft lighting casting dramatic shadows across the room. The vibrant energy of the art fills the space, creating a dynamic atmosphere that reflects the young man’s creative energy and artistic expression.
图片大小:1024,1536(长宽)
采样器设置:heun,beta
fusion:用max (max_token生成的画面会比较小)
添加链接描述
使用的pytorch配置
conda install pytorch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 pytorch-cuda=11.8 -c pytorch -c nvidia
使用的python版本 3.10