PointCloud

Supported on:RVC2 RVC4

从深度图计算3D点云。可选配合对齐的颜色输入以生成彩色点云。

默认在主机上运行。在RVC4上，可以通过setRunOnHost(False)将其卸载到设备。在RVC2上，仅支持主机端处理。在DepthAI v3.6.0中新增。

放置方式

Python

1import depthai as dai
2
3with dai.Pipeline() as pipeline:
4    pointCloud = pipeline.create(dai.node.PointCloud)

C++

1dai::Pipeline pipeline;
2auto pointCloud = pipeline.create<dai::node::PointCloud>();

输入与输出

passthroughDepth 转发用于计算给定 PointCloudData 输出的原始深度帧。当输入队列为非阻塞且需要将深度帧与其对应的点云关联时非常有用。

配置

所有设置均可在流水线启动前通过 initialConfig 设置，或在运行时通过 inputConfig 发送。在流中发送新配置是安全的——当配置、标定或帧变换发生变化时，节点会自动重新初始化。有关详细描述和使用示例，请参阅PointCloudConfig。

Python

1pc = pipeline.create(dai.node.PointCloud)
2pc.initialConfig.setOrganized(True)
3pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)
4pc.initialConfig.setTargetCoordinateSystem(dai.CameraBoardSocket.CAM_A)
5pc.setNumFramesPool(8)

C++

1auto pc = pipeline.create<dai::node::PointCloud>();
2pc->initialConfig->setOrganized(true);
3pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);
4pc->initialConfig->setTargetCoordinateSystem(dai::CameraBoardSocket::CAM_A);
5pc->setNumFramesPool(8);

坐标系

默认情况下，节点从深度帧的 ImgTransformation 外参中读取 T_frame_to_ref 并应用它，因此输出点云位于参考（原点）相机坐标系中。此外，您可以应用以下三种额外变换之一：

1. 相机接口坐标系

将点云变换到设备上另一个相机的坐标系（例如 CameraBoardSocket.CAM_A）。原点位于相机传感器处，旋转遵循相机的光学帧。节点利用设备标定自动计算外参变换。

Python

1# 点云在彩色相机（CAM_A）的坐标系中
2pc.initialConfig.setTargetCoordinateSystem(dai.CameraBoardSocket.CAM_A)

2. 外壳坐标系

将点云变换到外壳坐标系。这使用了设备的外壳坐标定义，将所有点表示为相对于设备外壳上某个物理点。

Python

1# 点云在VESA安装坐标系中
2pc.initialConfig.setTargetCoordinateSystem(dai.HousingCoordinateSystem.VESA_A)

可用的外壳坐标系：

HousingCoordinateSystem.CAM_A – CAM_D — 原点与 CameraBoardSocket.CAM_A – CAM_D 相同。区别在于旋转：X-Y平面与设备前玻璃平行，而相机接口坐标系旋转到相机的光学帧。
HousingCoordinateSystem.FRONT_CAM_A – FRONT_CAM_D — 定位与 HousingCoordinateSystem.CAM_A – CAM_D 类似，但向前移动至前玻璃的前侧。
HousingCoordinateSystem.VESA_A – VESA_J — 设备安装点。
HousingCoordinateSystem.IMU — IMU传感器原点。

3. 自定义变换矩阵

应用一个表示T_ref_to_custom的任意4×4变换矩阵——从参考相机到自定义坐标系的变换。节点将其与帧外参组合，得到最终应用的变换：

Python

1# 绕 Z 轴旋转 90°
2transform = [
3    [0.0, -1.0, 0.0, 0.0],
4    [1.0,  0.0, 0.0, 0.0],
5    [0.0,  0.0, 1.0, 0.0],
6    [0.0,  0.0, 0.0, 1.0],
7]
8pc.initialConfig.setTransformationMatrix(transform)

如果未设置坐标系目标且变换矩阵为单位矩阵，则仅应用帧外参（T_frame_to_ref），将点云转换至参考相机坐标系。

选项间的交互

三种模式互斥。设置相机插槽或外壳坐标系会将坐标系类型分别设置为 CAMERA_SOCKET 或 HOUSING —— 在这些模式下，setTransformationMatrix 设置的自定义变换矩阵被忽略。仅当坐标系类型为 DEFAULT（即未调用任一 setTargetCoordinateSystem 重载）时，才会使用自定义矩阵。所有三种方法也可以直接在节点上调用（会转发到 initialConfig）：

Python

1pc.setTargetCoordinateSystem(dai.CameraBoardSocket.CAM_A)
2# 或
3pc.setTargetCoordinateSystem(dai.HousingCoordinateSystem.VESA_A)

用法

从立体深度生成点云

从立体深度生成基础点云。

Python

1import depthai as dai
2
3pipeline = dai.Pipeline()
4
5left = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
6right = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
7stereo = pipeline.create(dai.node.StereoDepth)
8left.requestOutput((640, 400)).link(stereo.left)
9right.requestOutput((640, 400)).link(stereo.right)
10
11pc = pipeline.create(dai.node.PointCloud)
12pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)
13stereo.depth.link(pc.inputDepth)
14
15q = pc.outputPointCloud.createOutputQueue(maxSize=4, blocking=False)
16
17with pipeline:
18    pipeline.start()
19    while pipeline.isRunning():
20        pclData = q.get()
21        points = pclData.getPoints()  # np.ndarray (N, 3) float32
22        print(f"点数: {len(points)}, Z=[{pclData.getMinZ():.2f}, {pclData.getMaxZ():.2f}]")

C++

1#include <iostream>
2#include "depthai/depthai.hpp"
3
4int main() {
5    dai::Pipeline pipeline;
6
7    auto left = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
8    auto right = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
9    auto stereo = pipeline.create<dai::node::StereoDepth>();
10    left->requestOutput(std::make_pair(640, 400))->link(stereo->left);
11    right->requestOutput(std::make_pair(640, 400))->link(stereo->right);
12
13    auto pc = pipeline.create<dai::node::PointCloud>();
14    pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);
15    stereo->depth.link(pc->inputDepth);
16
17    auto q = pc->outputPointCloud.createOutputQueue(4, false);
18
19    pipeline.start();
20    while(pipeline.isRunning()) {
21        auto pclData = q->get<dai::PointCloudData>();
22        auto points = pclData->getPoints();
23        std::cout << "点数: " << points.size()
24                  << ", Z=[" << pclData->getMinZ() << ", " << pclData->getMaxZ() << "]" << std::endl;
25    }
26    pipeline.stop();
27    return 0;
28}

彩色点云

将深度图和彩色图像链接到同一坐标系。通常将深度图对齐到彩色相机。

Python

1import depthai as dai
2
3pipeline = dai.Pipeline()
4
5left = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
6right = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
7color = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_A)
8
9stereo = pipeline.create(dai.node.StereoDepth)
10left.requestFullResolutionOutput().link(stereo.left)
11right.requestFullResolutionOutput().link(stereo.right)
12
13colorOut = color.requestOutput((640, 400), type=dai.ImgFrame.Type.RGB888i,
14                               resizeMode=dai.ImgResizeMode.CROP, enableUndistortion=True)
15
16pc = pipeline.create(dai.node.PointCloud)
17pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)
18
19# 将深度图对齐到彩色相机
20platform = pipeline.getDefaultDevice().getPlatform()
21if platform == dai.Platform.RVC4:
22    imageAlign = pipeline.create(dai.node.ImageAlign)
23    stereo.depth.link(imageAlign.input)
24    colorOut.link(imageAlign.inputAlignTo)
25    imageAlign.outputAligned.link(pc.inputDepth)
26else:
27    colorOut.link(stereo.inputAlignTo)
28    stereo.depth.link(pc.inputDepth)
29
30colorOut.link(pc.inputColor)
31
32q = pc.outputPointCloud.createOutputQueue(maxSize=4, blocking=False)
33
34with pipeline:
35    pipeline.start()
36    while pipeline.isRunning():
37        pcd = q.get()
38        if pcd.isColor():
39            xyz, rgba = pcd.getPointsRGB()
40            print(f"点数: {len(xyz)}, color=yes, Z=[{pcd.getMinZ():.2f}, {pcd.getMaxZ():.2f}]")

C++

1#include <iostream>
2#include "depthai/depthai.hpp"
3
4int main() {
5    dai::Pipeline pipeline;
6
7    auto left = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
8    auto right = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
9    auto color = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_A);
10
11    auto stereo = pipeline.create<dai::node::StereoDepth>();
12    left->requestFullResolutionOutput()->link(stereo->left);
13    right->requestFullResolutionOutput()->link(stereo->right);
14
15    auto colorOut = color->requestOutput(std::make_pair(640, 400), dai::ImgFrame::Type::RGB888i,
16                                         dai::ImgResizeMode::CROP, std::nullopt, true);
17
18    auto pc = pipeline.create<dai::node::PointCloud>();
19    pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);
20
21    auto platform = pipeline.getDefaultDevice()->getPlatform();
22    if(platform == dai::Platform::RVC4) {
23        auto imageAlign = pipeline.create<dai::node::ImageAlign>();
24        stereo->depth.link(imageAlign->input);
25        colorOut->link(imageAlign->inputAlignTo);
26        imageAlign->outputAligned.link(pc->inputDepth);
27    } else {
28        colorOut->link(stereo->inputAlignTo);
29        stereo->depth.link(pc->inputDepth);
30    }
31
32    colorOut->link(pc->getColorInput());
33
34    auto q = pc->outputPointCloud.createOutputQueue(4, false);
35
36    pipeline.start();
37    while(pipeline.isRunning()) {
38        auto pcd = q->get<dai::PointCloudData>();
39        if(pcd->isColor()) {
40            auto points = pcd->getPointsRGB();
41            std::cout << "点数: " << points.size() << ", color=yes"
42                      << ", Z=[" << pcd->getMinZ() << ", " << pcd->getMaxZ() << "]" << std::endl;
43        }
44    }
45    pipeline.stop();
46    return 0;
47}

Examples

PointCloud — 最小化彩色点云示例。
PointCloud Visualizer — 使用Open3D的实时3D可视化。
PointCloud Showcase — 演示过滤、组织、坐标变换、自定义矩阵和彩色模式。

Reference

class

dai::node::PointCloud

#include PointCloud.hpp

variable

std::shared_ptr< PointCloudConfig > initialConfig

Initial config to use when computing the point cloud.

variable

Input inputConfig

Input PointCloudConfig message with ability to modify parameters in runtime. Default queue is non-blocking with size 4.

variable

Subnode< node::Sync > sync

Sync subnode for synchronized depth + color input. When only depth is connected, Sync passes through single-item MessageGroups. When both depth and color are connected, Sync pairs them by timestamp.

variable

InputMap & syncInputs

variable

Input & inputDepth

Input message with depth data used to create the point cloud. Routed through the internal Sync subnode.

variable

Output outputPointCloud

Outputs PointCloudData message

variable

Output passthroughDepth

Passthrough depth from which the point cloud was calculated. Suitable for when input queue is set to non-blocking behavior.

function

PointCloud()

function

PointCloud(std::unique_ptr< Properties > props)

function

~PointCloud()

function

Input & getColorInput()

Get the optional color input for colorized point clouds. Lazily creates the Sync entry so that depth-only mode works without Sync waiting for a color frame that never arrives.Link an aligned color image (RGB888i, same dimensions as depth) to this input to enable colored point cloud output.

function

void setNumFramesPool(int numFramesPool)

Specify number of frames in pool.

Parameters

numFramesPool: How many frames should the pool have

function

void setRunOnHost(bool runOnHost)

Specify whether to run on host or device By default, the node will run on host.

function

void useCPU()

Use single-threaded CPU for processing

function

void useCPUMT(uint32_t numThreads)

Use multi-threaded CPU for processing

function

void useGPU(uint32_t device)

Use GPU for point cloud computation

Parameters

device: GPU device index (default 0)

function

void setTargetCoordinateSystem(CameraBoardSocket targetCamera)

Set target coordinate system to transform point cloud

Parameters

targetCamera: Target camera socket

function

void setTargetCoordinateSystem(HousingCoordinateSystem housingCS)

Set target coordinate system to housing coordinate system Point cloud will be transformed to this housing coordinate system

Parameters

housingCS: Target housing coordinate system

function

void setTargetCoordinateSystem(CameraBoardSocket targetCamera, bool useSpecTranslation)

Deprecated: use setTargetCoordinateSystem(targetCamera) instead.

function

void setTargetCoordinateSystem(HousingCoordinateSystem housingCS, bool useSpecTranslation)

Deprecated: use setTargetCoordinateSystem(housingCS) instead.

function

bool runOnHost()

function

void buildInternal()

class

dai::node::PointCloud::Impl

variable

LengthUnit targetLengthUnit

function

Impl()

function

void setLogger(const std::shared_ptr<::spdlog::logger > & log)

function

void computePointCloudDense(const uint8_t * depthData, std::vector< Point3f > & points)

function

void computePointCloudDenseColored(const uint8_t * depthData, const uint8_t * colorData, std::vector< Point3fRGBA > & points)

function

void applyTransformation(std::vector< PointT > & points)

function

std::vector< PointT > filterValidPoints(const std::vector< PointT > & densePoints)

function

void setLengthUnit(dai::LengthUnit lengthUnit)

function

void useCPU()

function

void useCPUMT(uint32_t numThreads)

function

void useGPU(uint32_t device)

function

void setIntrinsics(float fx, float fy, float cx, float cy, unsigned int width, unsigned int height)

function

void setExtrinsics(const std::vector< std::vector< float >> & transformMatrix)

function

void clearExtrinsics()

需要帮助？

请前往 OAKChina 官网获取技术支持或解答您的任何疑问。

本页目录

PointCloud

放置方式

Python

C++

输入与输出

配置

Python

C++

坐标系

1. 相机接口坐标系

2. 外壳坐标系

3. 自定义变换矩阵

选项间的交互

用法

从立体深度生成点云

Python

C++

彩色点云

Python

C++

Examples

Reference

dai::node::PointCloud

dai::node::PointCloud::Impl

需要帮助？