PyTorch3D是非常好用的3D工具库。但是PyTorch3D对于可用于debug（例如调整cameras参数）的可视化工具并没有进行系统的介绍。这篇文章主要是想介绍我觉得非常使用的PyTorch3D可视化工具。
1. 新建一个Mesh
从hugging face上下载一个glb文件，例如 0025c5e2333949feb1db259d4ff08dbe。用如下代码可以读取并渲染。
from pytorch3d.io import IO
from pytorch3d.io.experimental_gltf_io import MeshGlbFormat
from pytorch3d.renderer import RasterizationSettings, MeshRasterizer, PerspectiveCameras, look_at_view_transform, PointLights, MeshRenderer, SoftPhongShader
import matplotlib.pyplot as plt
'''
https://huggingface.co/datasets/allenai/objaverse/blob/main/glbs/000-000/0025c5e2333949feb1db259d4ff08dbe.glb
'''
device = "cuda"
image_size = (512, 512)
glb_path = "0025c5e2333949feb1db259d4ff08dbe.glb"
io = IO()
io.register_meshes_format(MeshGlbFormat())
with open(glb_path, "rb") as f:
mesh = io.load_mesh(f, include_textures=True).to(device)
# Select the viewpoint using spherical angles
distance = 3 # distance from camera to the object
elevation = 0.0 # angle of elevation in degrees
azimuth = 0.0 # No rotation so the camera is positioned on the +Z axis.
# Get the position of the camera based on the spherical angles
R, T = look_at_view_transform(distance, elevation, azimuth, device=device)
cameras = PerspectiveCameras(device=device, R=R, T=T, image_size=(image_size, ), in_ndc=False)
cameras.to(device)
raster_settings = RasterizationSettings(
image_size=image_size,
blur_radius=0.0,
faces_per_pixel=1,
)
lights = PointLights(device=device, location=[[0.0, 0.0, -3.0]])
renderer = MeshRenderer(
rasterizer=MeshRasterizer(
cameras=cameras,
raster_settings=raster_settings
),
shader=SoftPhongShader(
device=device,
cameras=cameras,
lights=lights
)
)
images = renderer(mesh)
plt.imshow(images[0, ..., :3].cpu())
plt.axis("off")
我们会发现渲染得到的图片是空白的。那么接下来我们可以从利用PyTorch3D可视化工具查看是什么原因导致的。
2. 可视化 Texture
plt.figure(figsize=(7,7))
texture_image=mesh.textures.maps_padded()
plt.imshow(texture_image.squeeze().cpu().numpy())
plt.axis("off")
3. 可视化 cameras 和 meshes
from pytorch3d.vis.plotly_vis import plot_batch_individually
plot_batch_individually([mesh, cameras])
4. 渲染出来空白图像的原因
相机参数不对，可以看到3. 相机视野中压根没出现meshes。
