文章目录
- KPAC(ICCV 2021)代码单图片推理
- 创建虚拟环境
- 安装依赖包
- 数据集路径设置
- 运行测试
- 单图片推理
KPAC(ICCV 2021)代码单图片推理
论文链接:Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions
该论文研究的问题是散焦去模糊,是图像恢复中经常需要做的一个任务,经常需要找对比图片。如果可以运行该论文,那么论文中可以增加一个对比图片。现在论文中大家都不用TensorFlow,不巧的是该论文原始代码是TF写的。我们并不是改为torch形式,而依然通过安装tf1.15的gpu版本。原始的代码我没法进行测试,改了半天放弃了,结合chatgpt写了一个单图片推理的代码,可以成功运行,但是依然需要大概8.6G的显存。
我的环境:
系统:Ubuntu 22.04(WSL2)
显卡:4070 ti s
wsl2的cuda版本:11.5
论文github链接:https://github.com/HyeongseokSon1/KPAC
TensorFlow的版本:https://developer.download.nvidia.com/compute/redist/nvidia-tensorflow/
我使用的tf版本下载:https://developer.download.nvidia.com/compute/redist/nvidia-tensorflow/nvidia_tensorflow-1.15.5+nv21.11-cp38-cp38-linux_x86_64.whl
该tf1.15.5版本应该是支持30和40系显卡的。
论文的KPAC如下:
创建虚拟环境
通过conda新建虚拟环境,使用如下代码:
conda create -n KPAC python=3.8
激活环境:
conda activate KPAC
再KPAC虚拟环境下安装依赖包,切忌不要再base虚拟环境下操作,通过新建虚拟环境来操作。
安装依赖包
首先下载下来nvidia_tensorflow-1.15.5+nv21.11-cp38-cp38-linux_x86_64.whl,然后本地安装:
pip install nvidia_tensorflow-1.15.5+nv21.11-cp38-cp38-linux_x86_64.whl
安装tf之后,测试gpu是否可用。将下面代码保存为device_gpu.py,然后运行。
import tensorflow as tf
gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
print("Available GPU(s):")
for gpu in gpus:
print(gpu)
else:
print("No GPUs detected.")
输出大概如下:
2024-11-25 11:09:56.018231: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
2024-11-25 11:09:57.076884: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2024-11-25 11:09:57.303655: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1068] could not open file to read NUMA node: /sys/bus/pci/devices/0000:2b:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-11-25 11:09:57.303730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1666] Found device 0 with properties:
name: NVIDIA GeForce RTX 4070 Ti SUPER major: 8 minor: 9 memoryClockRate(GHz): 2.61
pciBusID: 0000:2b:00.0
2024-11-25 11:09:57.303762: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2024-11-25 11:09:57.330370: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2024-11-25 11:09:57.354441: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2024-11-25 11:09:57.357430: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2024-11-25 11:09:57.363308: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2024-11-25 11:09:57.382721: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2024-11-25 11:09:57.383752: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2024-11-25 11:09:57.383854: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1068] could not open file to read NUMA node: /sys/bus/pci/devices/0000:2b:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-11-25 11:09:57.383936: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1068] could not open file to read NUMA node: /sys/bus/pci/devices/0000:2b:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-11-25 11:09:57.383970: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1794] Adding visible gpu devices: 0
Available GPU(s):
PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')
会有一个GPU:0,表示gpu可用。
之后可以安装下面依赖包:
easydict==1.13
matplotlib==3.5.3
numpy==1.18.5
opencv-python==4.10.0.84
protobuf==3.20.3
scikit-image==0.17.2
scipy==1.7.3
数据集路径设置
找到根目录下的config.py文件,修改如下路径:
# for dual
config.TEST.folder_path_c = 'DeblurDataset/DefocusDeblur/test_c/' # need to change
config.TEST.folder_path_l = 'DeblurDataset/DefocusDeblur/test_l/' # need to change
config.TEST.folder_path_r = 'DeblurDataset/DefocusDeblur/test_r/' # need to change
因为我是运行的双像素去看模糊,根据main_eval_dual.py修改的,我就只修改了这个路径。
运行测试
如果安装原论文的方法使用如下代码测试:
CUDA_VISIBLE_DEVICES=0 python main_eval_dual.py
会有如下错误,我是放弃了修改这个代码,scipy=1.2.*的版本好像在python 3.8版本中不可安装。
[*] Load pretrained/dual.npz SUCCESS!
Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
results[i] = fn(data, **kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 26, in get_imgs_fn
return scipy.misc.imread(path + file_name, mode='RGB')
AttributeError: module 'scipy.misc' has no attribute 'imread'
read 1 from DeblurDataset/DefocusDeblur/test_c//source/
Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
results[i] = fn(data, **kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 26, in get_imgs_fn
return scipy.misc.imread(path + file_name, mode='RGB')
AttributeError: module 'scipy.misc' has no attribute 'imread'
read 1 from DeblurDataset/DefocusDeblur/test_l//source/
Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
results[i] = fn(data, **kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 26, in get_imgs_fn
return scipy.misc.imread(path + file_name, mode='RGB')
AttributeError: module 'scipy.misc' has no attribute 'imread'
read 1 from DeblurDataset/DefocusDeblur/test_r//source/
[!] ./Evaluations/dual_results/ exists ...
Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
results[i] = fn(data, **kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 53, in scale_imgs_fn
x = x-127.5
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'
Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
results[i] = fn(data, **kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 53, in scale_imgs_fn
x = x-127.5
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'
Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/tensorlayer/prepro.py", line 99, in apply_fn
results[i] = fn(data, **kwargs)
File "/mnt/e/research/SOTA_Restoration/KPAC/utils.py", line 53, in scale_imgs_fn
x = x-127.5
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'
Traceback (most recent call last):
File "main_eval_dual.py", line 109, in <module>
DefocusDeblur()
File "main_eval_dual.py", line 93, in DefocusDeblur
out = sess.run(result, {t_image_c: valid_ref_img_c, t_image_l: valid_ref_img_l,t_image_r: valid_ref_img_r})
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/site-packages/tensorflow_core/python/client/session.py", line 955, in run
result = self._run(None, fetches, feed_dict, options_ptr,
File "/home/wang/miniconda3/envs/KPAC/lib/python3.8/site-packages/tensorflow_core/python/client/session.py", line 1153, in _run
raise ValueError(
ValueError: Cannot feed value of shape (1,) for Tensor 't_image_c:0', which has shape '(1, 1120, 1680, 3)'
虽说默认的代码无法运行,我们主要为了测试,生成一个去模糊的图片,我们也不是用于训练的,非常不推荐运行。
如果你想复现图像恢复相关的论文,我推荐你NAFNet、Restormer,这两个效果又好,又容易,缺点就是需要显卡,如果你想使用自己的数据集,修改修改batch_size,调小一点。
单图片推理
回到KPAC,默认的代码无法使用,我们修改一下,改为单图片去模糊,使用如下代码,可以随便起一个文件名,例如single_img.py。
import os
import time
import numpy as np
import tensorflow as tf
import tensorlayer as tl
from model import Defocus_Deblur_Net6_ds_dual
from utils import scale_imgs_fn
from PIL import Image
# 加载并预处理图片
def load_and_preprocess_image(img_path):
"""
加载并预处理图片。
:param img_path: 图片路径
:return: 归一化后的图片数组
"""
if not os.path.exists(img_path):
raise FileNotFoundError(f"[Error] Image not found: {img_path}")
img = Image.open(img_path).convert("RGB") # 确保图片为 RGB 格式
img = np.array(img, dtype=np.float32)
img = np.expand_dims(img, axis=0) # 添加 batch 维度
img = tl.prepro.threading_data(img, fn=scale_imgs_fn) # 归一化到 [-1, 1]
return img
# 保存图片
def save_image(image_array, save_path):
"""
保存图片到文件。
:param image_array: 图像数组 ([-1, 1])
:param save_path: 保存路径
"""
# 将 [-1, 1] 转换为 [0, 255]
image_array = ((image_array + 1) * 127.5).astype(np.uint8)
image = Image.fromarray(image_array)
image.save(save_path)
print(f"Deblurred image saved to {save_path}")
# 测试单张图片的去模糊
def test_single_image(image_path_c, image_path_l, image_path_r, save_path):
"""
测试单张图片的去模糊。
:param image_path_c: 中心图像路径
:param image_path_l: 左图像路径
:param image_path_r: 右图像路径
:param save_path: 输出结果保存路径
"""
# 参数定义
H, W = 1120, 1680
checkpoint_dir = "pretrained"
# 加载图片
try:
img_c = load_and_preprocess_image(image_path_c)
img_l = load_and_preprocess_image(image_path_l)
img_r = load_and_preprocess_image(image_path_r)
except FileNotFoundError as e:
print(e)
return
# 构建网络
with tf.device('/device:GPU:0'):
t_image_c = tf.placeholder('float32', [1, H, W, 3], name='t_image_c')
t_image_l = tf.placeholder('float32', [1, H, W, 3], name='t_image_l')
t_image_r = tf.placeholder('float32', [1, H, W, 3], name='t_image_r')
net_g = Defocus_Deblur_Net6_ds_dual(t_image_c, t_image_l, t_image_r, is_train=False, hrg=H, wrg=W)
result = net_g.outputs
# TensorFlow 会话配置
configg = tf.ConfigProto()
configg.gpu_options.allow_growth = True
sess = tf.Session(config=configg)
tl.layers.initialize_global_variables(sess)
tl.files.load_and_assign_npz(sess=sess, name=f'{checkpoint_dir}/dual.npz', network=net_g)
# 运行模型
print("[*] Running deblurring...")
start_time = time.time()
out = sess.run(result, feed_dict={t_image_c: img_c, t_image_l: img_l, t_image_r: img_r})
print("Processing time: %.4fs" % (time.time() - start_time))
# 保存结果
save_image(out[0], save_path)
# 主函数
if __name__ == '__main__':
# 测试单张图片的输入路径
image_path_c = "DeblurDataset/DefocusDeblur/test_c/source/1P0A2417.png" # 替换为实际图片路径
image_path_l = "DeblurDataset/DefocusDeblur/test_l/source/1P0A2417_L.png" # 替换为实际图片路径
image_path_r = "DeblurDataset/DefocusDeblur/test_r/source/1P0A2417_R.png" # 替换为实际图片路径
# 保存结果的路径
save_path = "./deblurred_result.png"
# 调用测试函数
test_single_image(image_path_c, image_path_l, image_path_r, save_path)
之后我们运行该代码:
python single_img.py
大概输出如下内容:
[*] Load pretrained/dual.npz SUCCESS!
[*] Running deblurring...
2024-11-25 11:43:55.798790: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2024-11-25 11:43:58.611393: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: ptxas exited with non-zero error code 65280, output: ptxas fatal : Value 'sm_89' is not defined for option 'gpu-name'
Relying on driver to perform ptx compilation. This message will be only logged once.
2024-11-25 11:43:59.507653: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
Processing time: 33.1819s
Deblurred image saved to ./deblurred_result.png
可以成功使用gpu预测,但是不巧的是,需要显存大概8.6G,所以最好使用12G显存以上的显卡,否则可能无法使用gpu预测,或者直接使用cpu预测。