Open3D
Open3D是一個開源庫，它支援處理3D資料的軟體的快速開發。Open3D前端在C++和Python中公開了一組精心選擇的資料結構和算法。後端經過高度優化，并設定為并行化。
前期也用了一段時間的PCL，但是C++真的啃不動，轉而尋求python版的點雲處理庫，就找到了Open3D。Open3D國内的相關資源還是比較少的，是以我準備在學習過程中，在CSDN上記錄遇到的問題以及解決方法，友善後續查找。
下面代碼可以利用Open3D生成RGB-D資料
import open3d as o3d
import numpy as np
from matplotlib import pyplot as plt
color_raw = o3d.io.read_image("F:/Depth_image_C++/open_and_save/open_and_save/3Dprintcarrot/rgb/(1).png")
depth_raw = o3d.io.read_image("F:/Depth_image_C++/open_and_save/open_and_save/3Dprintcarrot/depth/(1).png")
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(color_raw, depth_raw, convert_rgb_to_intensity=False)
print(rgbd_image)
plt.subplot(1, 2, 1)
plt.title('Redwood grayscale image')
plt.imshow(rgbd_image.color)
plt.subplot(1, 2, 2)
plt.title('Redwood depth image')
plt.imshow(rgbd_image.depth)
plt.show()
inter = o3d.camera.PinholeCameraIntrinsic()
inter.set_intrinsics(1280, 720, 599.795593, 599.633118, 645.792786, 372.238983)
pcd = o3d.geometry.PointCloud().create_from_rgbd_image(
rgbd_image, inter)
# Flip it, otherwise the pointcloud will be upside down
pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
o3d.visualization.draw_geometries([pcd])
遇到的問題
因為我使用的是Azure kinect SDK擷取的rgb圖與深度圖，是以相機的内參需要自己設定下。
内參擷取我看相關資料可以在Azure-Kinect-Sensor-SDK/examples/calibration at develop · microsoft/Azure-Kinect-Sensor-SDK · 找到。隻需在Visual Studio 2017中加載CmakeList.txt并進行建構，将生成Calibration_info.exe。連接配接并執行Azure Kinect後，輸出如下。
C:\GitHub\Azure-Kinect-Sensor-SDK\build\Win-x64-Release-Ninja\bin>calibration_info.exe > out.txt
下面是Azure kinect sdk的一些内參，供大家參考
圖像尺寸為640×576的内參，格式為K4A_DEPTH_MODE_NFOV_UNBINNED
Found 1 connected devices:
===== Device 0: {DeviceID:XXXXXX} =====
resolution width: 640
resolution height: 576
principal point x: 309.301086
principal point y: 330.656738
focal length x: 505.119751
focal length y: 505.163605
radial distortion coefficients:
k1: 3.063722
k2: 1.988396
k3: 0.102371
k4: 3.397454
k5: 2.984188
k6: 0.544097
center of distortion in Z=1 plane, x: 0.000000
center of distortion in Z=1 plane, y: 0.000000
tangential distortion coefficient x: -0.000098
tangential distortion coefficient y: 0.000048
metric radius: 0.000000
也可以進行自動校準
auto calib = calibration.depth_camera_calibration
和
auto calib = calibration.color_camera_calibration
這是1080p的内參
Found 1 connected devices:
===== Device 0: {DeviceID:XXXXXX} =====
resolution width: 1920
resolution height: 1080
principal point x: 968.939209
principal point y: 558.608459
focal length x: 899.693420
focal length y: 899.449646
radial distortion coefficients:
k1: 0.678679
k2: -2.779486
k3: 1.569404
k4: 0.554819
k5: -2.610379
k6: 1.500811
center of distortion in Z=1 plane, x: 0.000000
center of distortion in Z=1 plane, y: 0.000000
tangential distortion coefficient x: 0.000476
tangential distortion coefficient y: 0.000104
metric radius: 0.000000
這是720p内參
Found 1 connected devices:
===== Device 0: {DeviceID:XXXXXX} =====
resolution width: 1280
resolution height: 720
principal point x: 645.792786
principal point y: 372.238983
focal length x: 599.795593
focal length y: 599.633118
radial distortion coefficients:
k1: 0.678679
k2: -2.779486
k3: 1.569404
k4: 0.554819
k5: -2.610379
k6: 1.500811
center of distortion in Z=1 plane, x: 0.000000
center of distortion in Z=1 plane, y: 0.000000
tangential distortion coefficient x: 0.000476
tangential distortion coefficient y: 0.000104
metric radius: 0.000000
有了内參，怎麼用Open3D讀取内參并用于點雲生成中
有兩種方法
第一種
inter = o3d.camera.PinholeCameraIntrinsic()
inter.set_intrinsics(1280, 720, 599.795593, 599.633118, 645.792786, 372.238983)
這裡要注意的是inter = o3d.camera.PinholeCameraIntrinsic()，括号一定不能少，否則會報錯
inter.set_intrinsics(1280, 720, 599.795593, 599.633118, 645.792786, 372.238983)
TypeError: set_intrinsics(): incompatible function arguments. The following argument types are supported:
1. (self: open3d.cpu.pybind.camera.PinholeCameraIntrinsic, width: int, height: int, fx: float, fy: float, cx: float, cy: float) -> None
Invoked with: 1280, 720, 599.795593, 599.633118, 645.792786, 372.238983
第二種
儲存、讀取内參
o3d.io.write_pinhole_camera_intrinsic('./file.json', inter)
pinholeCamera = o3d.io.read_pinhole_camera_intrinsic('./file.json')
最後放幾張處理過程中的圖檔
最後的點雲圖有點失真，可能是因為我的彩色圖是RGBA四通道圖，轉成RGB三通道圖就可以正常顯示了