此页面由 AI 自动翻译。查看英文原版
最后更新

本页目录

  • 如何放置
  • 输入和输出
  • 配置
  • 坐标系
  • 1. 相机插座坐标系
  • 2. 外壳坐标系
  • 3. 自定义变换矩阵
  • 选项之间的交互
  • 用法
  • 从立体深度生成点云
  • 彩色点云
  • 示例
  • 参考
Claude CodeCodexCursor
Luxonis 智能体工具包为你常用的编程智能体加载 Luxonis 上下文与技能,更快地基于 OAK 开发应用。
安装工具包
最后更新
Supported on:RVC2RVC4
从深度图计算 3D 点云。可以选择性地接收对齐的彩色输入以生成彩色点云。

如何放置

Python

Python
1import depthai as dai
2
3with dai.Pipeline() as pipeline:
4    pointCloud = pipeline.create(dai.node.PointCloud)

C++

C++
1dai::Pipeline pipeline;
2auto pointCloud = pipeline.create<dai::node.PointCloud>();

输入和输出

passthroughDepth 转发用于计算给定 PointCloudData 输出的原始深度帧。当输入队列是非阻塞的并且您需要将深度帧与其相应的点云关联时非常有用。

配置

所有设置都可以在管道启动前通过 initialConfig 设置,或者在运行时通过 inputConfig 发送。在流中发送新配置是安全的——当配置、校准或帧转换发生变化时,节点会自动重新初始化。有关详细说明和使用示例,请参阅 PointCloudConfig。

Python

Python
1pc = pipeline.create(dai.node.PointCloud)
2pc.initialConfig.setOrganized(True)
3pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)
4pc.initialConfig.setTargetCoordinateSystem(dai.CameraBoardSocket.CAM_A)
5pc.setNumFramesPool(8)

C++

C++
1auto pc = pipeline.create<dai::node::PointCloud>();
2pc->initialConfig->setOrganized(true);
3pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);
4pc->initialConfig->setTargetCoordinateSystem(dai::CameraBoardSocket::CAM_A);
5pc->setNumFramesPool(8);

坐标系

默认情况下,节点从深度帧的 ImgTransformation 外参中读取 T_frame_to_ref 并应用它,因此输出点云位于参考(原点)相机坐标系中。在此基础上,您可以应用以下三种额外转换之一:

1. 相机插座坐标系

将点云转换到设备上另一个相机的坐标系中(例如 CameraBoardSocket.CAM_A)。原点位于相机传感器处,旋转遵循相机的光学帧。节点使用设备校准自动计算外参转换。
Python
1# 彩色相机 (CAM_A) 坐标系中的点云
2pc.initialConfig.setTargetCoordinateSystem(dai.CameraBoardSocket.CAM_A)

2. 外壳坐标系

将点云转换到外壳坐标系中。这使用设备的外壳坐标定义来表示设备外壳上物理点相关的所有点。
Python
1# VESA 安装坐标系中的点云
2pc.initialConfig.setTargetCoordinateSystem(dai.HousingCoordinateSystem.VESA_A)
可用的外壳坐标系:
  • HousingCoordinateSystem.CAM_A – CAM_D — 原点与 CameraBoardSocket.CAM_A – CAM_D 相同。区别在于旋转:X-Y 平面与设备的前玻璃平行,而相机插座坐标系则旋转到相机的光学帧。
  • HousingCoordinateSystem.FRONT_CAM_A – FRONT_CAM_D — 位置类似于 HousingCoordinateSystem.CAM_A – CAM_D,但向前移动到前玻璃的前侧。
  • HousingCoordinateSystem.VESA_A – VESA_J — 设备安装点。
  • HousingCoordinateSystem.IMU — IMU 传感器原点。

3. 自定义变换矩阵

应用一个任意的 4×4 变换矩阵,表示 T_ref_to_custom — 从参考相机到您的自定义坐标系的变换。节点将其与帧外参组合以获得最终应用的变换:
Python
1# 绕 Z 轴旋转 90°
2transform = [
3    [0.0, -1.0, 0.0, 0.0],
4    [1.0,  0.0, 0.0, 0.0],
5    [0.0,  0.0, 1.0, 0.0],
6    [0.0,  0.0, 0.0, 1.0],
7]
8pc.initialConfig.setTransformationMatrix(transform)
如果未设置坐标系目标且变换矩阵为单位矩阵,则仅应用帧外参 (T_frame_to_ref),将点带入参考相机帧。

选项之间的交互

这三种模式是互斥的。设置相机插座或外壳坐标系会将坐标系类型分别设置为 CAMERA_SOCKET 或 HOUSING — 在这些模式下,setTransformationMatrix 中的自定义变换矩阵将被忽略。自定义矩阵仅在坐标系类型为 DEFAULT 时使用(即未调用 setTargetCoordinateSystem 重载)。所有三种方法也可以直接在节点上调用(转发到 initialConfig):
Python
1pc.setTargetCoordinateSystem(dai.CameraBoardSocket.CAM_A)
2# 或
3pc.setTargetCoordinateSystem(dai.HousingCoordinateSystem.VESA_A)

用法

从立体深度生成点云

从立体深度生成基本点云。

Python

Python
1import depthai as dai
2
3pipeline = dai.Pipeline()
4
5left = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
6right = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
7stereo = pipeline.create(dai.node.StereoDepth)
8left.requestOutput((640, 400)).link(stereo.left)
9right.requestOutput((640, 400)).link(stereo.right)
10
11pc = pipeline.create(dai.node.PointCloud)
12pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)
13stereo.depth.link(pc.inputDepth)
14
15q = pc.outputPointCloud.createOutputQueue(maxSize=4, blocking=False)
16
17with pipeline:
18    pipeline.start()
19    while pipeline.isRunning():
20        pclData = q.get()
21        points = pclData.getPoints()  # np.ndarray (N, 3) float32
22        print(f"Points: {len(points)}, Z=[{pclData.getMinZ():.2f}, {pclData.getMaxZ():.2f}]")

C++

C++
1#include <iostream>
2#include "depthai/depthai.hpp"
3
4int main() {
5    dai::Pipeline pipeline;
6
7    auto left = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
8    auto right = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
9    auto stereo = pipeline.create<dai::node::StereoDepth>();
10    left->requestOutput(std::make_pair(640, 400))->link(stereo->left);
11    right->requestOutput(std::make_pair(640, 400))->link(stereo->right);
12
13    auto pc = pipeline.create<dai::node::PointCloud>();
14    pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);
15    stereo->depth.link(pc->inputDepth);
16
17    auto q = pc->outputPointCloud.createOutputQueue(4, false);
18
19    pipeline.start();
20    while(pipeline.isRunning()) {
21        auto pclData = q->get<dai::PointCloudData>();
22        auto points = pclData->getPoints();
23        std::cout << "Points: " << points.size()
24                  << ", Z=[" << pclData->getMinZ() << ", " << pclData->getMaxZ() << "]" << std::endl;
25    }
26    pipeline.stop();
27    return 0;
28}

彩色点云

将深度和彩色图像对齐到同一坐标系。通常深度会与彩色相机对齐。

Python

Python
1import depthai as dai
2
3pipeline = dai.Pipeline()
4
5left = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
6right = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
7color = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_A)
8
9stereo = pipeline.create(dai.node.StereoDepth)
10left.requestFullResolutionOutput().link(stereo.left)
11right.requestFullResolutionOutput().link(stereo.right)
12
13colorOut = color.requestOutput((640, 400), type=dai.ImgFrame.Type.RGB888i,
14                               resizeMode=dai.ImgResizeMode.CROP, enableUndistortion=True)
15
16pc = pipeline.create(dai.node.PointCloud)
17pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)
18
19# 将深度与彩色相机对齐
20platform = pipeline.getDefaultDevice().getPlatform()
21if platform == dai.Platform.RVC4:
22    imageAlign = pipeline.create(dai.node.ImageAlign)
23    stereo.depth.link(imageAlign.input)
24    colorOut.link(imageAlign.inputAlignTo)
25    imageAlign.outputAligned.link(pc.inputDepth)
26else:
27    colorOut.link(stereo.inputAlignTo)
28    stereo.depth.link(pc.inputDepth)
29
30colorOut.link(pc.inputColor)
31
32q = pc.outputPointCloud.createOutputQueue(maxSize=4, blocking=False)
33
34with pipeline:
35    pipeline.start()
36    while pipeline.isRunning():
37        pcd = q.get()
38        if pcd.isColor():
39            xyz, rgba = pcd.getPointsRGB()
40            print(f"Points: {len(xyz)}, color=yes, Z=[{pcd.getMinZ():.2f}, {pcd.getMaxZ():.2f}]")

C++

C++
1#include <iostream>
2#include "depthai/depthai.hpp"
3
4int main() {
5    dai::Pipeline pipeline;
6
7    auto left = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
8    auto right = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
9    auto color = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_A);
10
11    auto stereo = pipeline.create<dai::node::StereoDepth>();
12    left->requestFullResolutionOutput()->link(stereo->left);
13    right->requestFullResolutionOutput()->link(stereo->right);
14
15    auto colorOut = color->requestOutput(std::make_pair(640, 400), dai::ImgFrame::Type::RGB888i,
16                                         dai::ImgResizeMode::CROP, std::nullopt, true);
17
18    auto pc = pipeline.create<dai::node::PointCloud>();
19    pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);
20
21    auto platform = pipeline.getDefaultDevice()->getPlatform();
22    if(platform == dai::Platform::RVC4) {
23        auto imageAlign = pipeline.create<dai::node::ImageAlign>();
24        stereo->depth.link(imageAlign->input);
25        colorOut->link(imageAlign->inputAlignTo);
26        imageAlign->outputAligned.link(pc->inputDepth);
27    } else {
28        colorOut->link(stereo->inputAlignTo);
29        stereo->depth.link(pc->inputDepth);
30    }
31
32    colorOut->link(pc->getColorInput());
33
34    auto q = pc->outputPointCloud.createOutputQueue(4, false);
35
36    pipeline.start();
37    while(pipeline.isRunning()) {
38        auto pcd = q->get<dai::PointCloudData>();
39        if(pcd->isColor()) {
40            auto points = pcd->getPointsRGB();
41            std::cout << "Points: " << points.size() << ", color=yes"
42                      << ", Z=[" << pcd->getMinZ() << ", " << pcd->getMaxZ() << "]" << std::endl;
43        }
44    }
45    pipeline.stop();
46    return 0;
47}

示例

参考

class

dai::node::PointCloud

#include PointCloud.hpp
variable
std::shared_ptr< PointCloudConfig > initialConfig
Initial config to use when computing the point cloud.
variable
Input inputConfig
Input PointCloudConfig message with ability to modify parameters in runtime. Default queue is non-blocking with size 4.
variable
Subnode< node::Sync > sync
Sync subnode for synchronized depth + color input. When only depth is connected, Sync passes through single-item MessageGroups. When both depth and color are connected, Sync pairs them by timestamp.
variable
InputMap & syncInputs
variable
Input & inputDepth
Input message with depth data used to create the point cloud. Routed through the internal Sync subnode.
variable
Output outputPointCloud
Outputs PointCloudData message
variable
Output passthroughDepth
Passthrough depth from which the point cloud was calculated. Suitable for when input queue is set to non-blocking behavior.
function
PointCloud()
function
PointCloud(std::unique_ptr< Properties > props)
function
~PointCloud()
function
Input & getColorInput()
Get the optional color input for colorized point clouds. Lazily creates the Sync entry so that depth-only mode works without Sync waiting for a color frame that never arrives.Link an aligned color image (RGB888i, same dimensions as depth) to this input to enable colored point cloud output.
function
void setNumFramesPool(int numFramesPool)
Specify number of frames in pool.
Parameters
  • numFramesPool: How many frames should the pool have
function
void setRunOnHost(bool runOnHost)
Specify whether to run on host or device By default, the node will run on host.
function
void useCPU()
Use single-threaded CPU for processing
function
void useCPUMT(uint32_t numThreads)
Use multi-threaded CPU for processing
function
void useGPU(uint32_t device)
Use GPU for point cloud computation
Parameters
  • device: GPU device index (default 0)
function
void setTargetCoordinateSystem(CameraBoardSocket targetCamera)
Set target coordinate system to transform point cloud
Parameters
  • targetCamera: Target camera socket
function
void setTargetCoordinateSystem(HousingCoordinateSystem housingCS)
Set target coordinate system to housing coordinate system Point cloud will be transformed to this housing coordinate system
Parameters
  • housingCS: Target housing coordinate system
function
void setTargetCoordinateSystem(CameraBoardSocket targetCamera, bool useSpecTranslation)
Deprecated: use setTargetCoordinateSystem(targetCamera) instead.
function
void setTargetCoordinateSystem(HousingCoordinateSystem housingCS, bool useSpecTranslation)
Deprecated: use setTargetCoordinateSystem(housingCS) instead.
function
bool runOnHost()
Returns true or false whether the node should be run on host or not.
function
void buildInternal()
Function called from within the create function to build the node. This function is useful for initialization, setting up inputs and outputs = stuff that cannot be perform in the constuctor.
class

dai::node::PointCloud::Impl

variable
LengthUnit targetLengthUnit
function
Impl()
function
void setLogger(const std::shared_ptr<::spdlog::logger > & log)
function
void computePointCloudDense(const uint8_t * depthData, std::vector< Point3f > & points)
function
void computePointCloudDenseColored(const uint8_t * depthData, const uint8_t * colorData, std::vector< Point3fRGBA > & points)
function
void applyTransformation(std::vector< PointT > & points)
function
std::vector< PointT > filterValidPoints(const std::vector< PointT > & densePoints)
function
void setLengthUnit(dai::LengthUnit lengthUnit)
function
void useCPU()
function
void useCPUMT(uint32_t numThreads)
function
void useGPU(uint32_t device)
function
void setIntrinsics(float fx, float fy, float cx, float cy, unsigned int width, unsigned int height)
function
void setDistortion(CameraModel model, std::vector< float > coefficients)
function
void setExtrinsics(const std::vector< std::vector< float >> & transformMatrix)
function
void clearExtrinsics()

需要帮助?

请访问讨论论坛以获取技术支持或您可能有的任何其他问题。