# PointCloud

从深度图计算3D点云。可选配合对齐的颜色输入以生成彩色点云。

> 默认在
> **主机**
> 上运行。在
> **RVC4**
> 上，可以通过
> `setRunOnHost(False)`
> 将其卸载到设备。在
> **RVC2**
> 上，仅支持主机端处理。在DepthAI
> **v3.6.0**
> 中新增。

## 放置方式

#### Python

```python
import depthai as dai

with dai.Pipeline() as pipeline:
    pointCloud = pipeline.create(dai.node.PointCloud)
```

#### C++

```cpp
dai::Pipeline pipeline;
auto pointCloud = pipeline.create<dai::node::PointCloud>();
```

## 输入与输出

passthroughDepth 转发用于计算给定 PointCloudData 输出的原始深度帧。当输入队列为非阻塞且需要将深度帧与其对应的点云关联时非常有用。

## 配置

所有设置均可在流水线启动前通过 initialConfig 设置，或在运行时通过 inputConfig 发送。在流中发送新配置是安全的——当配置、标定或帧变换发生变化时，节点会自动重新初始化。

有关详细描述和使用示例，请参阅[PointCloudConfig](https://docs.luxonis.com/software-v3/depthai/depthai-components/messages/pointcloud_config.md)。

#### Python

```python
pc = pipeline.create(dai.node.PointCloud)
pc.initialConfig.setOrganized(True)
pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)
pc.initialConfig.setTargetCoordinateSystem(dai.CameraBoardSocket.CAM_A)
pc.setNumFramesPool(8)
```

#### C++

```cpp
auto pc = pipeline.create<dai::node::PointCloud>();
pc->initialConfig->setOrganized(true);
pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);
pc->initialConfig->setTargetCoordinateSystem(dai::CameraBoardSocket::CAM_A);
pc->setNumFramesPool(8);
```

## 坐标系

默认情况下，节点从深度帧的 ImgTransformation 外参中读取 T_frame_to_ref 并应用它，因此输出点云位于参考（原点）相机坐标系中。此外，您可以应用以下三种额外变换之一：

### 1. 相机接口坐标系

将点云变换到设备上另一个相机的坐标系（例如 CameraBoardSocket.CAM_A）。原点位于相机传感器处，旋转遵循相机的光学帧。节点利用设备标定自动计算外参变换。

```python
# 点云在彩色相机（CAM_A）的坐标系中
pc.initialConfig.setTargetCoordinateSystem(dai.CameraBoardSocket.CAM_A)
```

### 2. 外壳坐标系

将点云变换到外壳坐标系。这使用了设备的外壳坐标定义，将所有点表示为相对于设备外壳上某个物理点。

```python
# 点云在VESA安装坐标系中
pc.initialConfig.setTargetCoordinateSystem(dai.HousingCoordinateSystem.VESA_A)
```

可用的外壳坐标系：

 * HousingCoordinateSystem.CAM_A – CAM_D — 原点与 CameraBoardSocket.CAM_A – CAM_D 相同。区别在于旋转：X-Y平面与设备前玻璃平行，而相机接口坐标系旋转到相机的光学帧。
 * HousingCoordinateSystem.FRONT_CAM_A – FRONT_CAM_D — 定位与 HousingCoordinateSystem.CAM_A – CAM_D 类似，但向前移动至前玻璃的前侧。
 * HousingCoordinateSystem.VESA_A – VESA_J — 设备安装点。
 * HousingCoordinateSystem.IMU — IMU传感器原点。

### 3. 自定义变换矩阵

应用一个表示T_ref_to_custom的任意4×4变换矩阵——从参考相机到自定义坐标系的变换。节点将其与帧外参组合，得到最终应用的变换：

```python
# 绕 Z 轴旋转 90°
transform = [
    [0.0, -1.0, 0.0, 0.0],
    [1.0,  0.0, 0.0, 0.0],
    [0.0,  0.0, 1.0, 0.0],
    [0.0,  0.0, 0.0, 1.0],
]
pc.initialConfig.setTransformationMatrix(transform)
```

如果未设置坐标系目标且变换矩阵为单位矩阵，则仅应用帧外参（T_frame_to_ref），将点云转换至参考相机坐标系。

### 选项间的交互

三种模式互斥。设置相机插槽或外壳坐标系会将坐标系类型分别设置为 CAMERA_SOCKET 或 HOUSING —— 在这些模式下，setTransformationMatrix 设置的自定义变换矩阵被忽略。仅当坐标系类型为 DEFAULT（即未调用任一
setTargetCoordinateSystem 重载）时，才会使用自定义矩阵。

所有三种方法也可以直接在节点上调用（会转发到 initialConfig）：

```python
pc.setTargetCoordinateSystem(dai.CameraBoardSocket.CAM_A)
# 或
pc.setTargetCoordinateSystem(dai.HousingCoordinateSystem.VESA_A)
```

## 用法

### 从立体深度生成点云

从立体深度生成基础点云。

#### Python

```python
import depthai as dai

pipeline = dai.Pipeline()

left = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
right = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
stereo = pipeline.create(dai.node.StereoDepth)
left.requestOutput((640, 400)).link(stereo.left)
right.requestOutput((640, 400)).link(stereo.right)

pc = pipeline.create(dai.node.PointCloud)
pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)
stereo.depth.link(pc.inputDepth)

q = pc.outputPointCloud.createOutputQueue(maxSize=4, blocking=False)

with pipeline:
    pipeline.start()
    while pipeline.isRunning():
        pclData = q.get()
        points = pclData.getPoints()  # np.ndarray (N, 3) float32
        print(f"点数: {len(points)}, Z=[{pclData.getMinZ():.2f}, {pclData.getMaxZ():.2f}]")
```

#### C++

```cpp
#include <iostream>
#include "depthai/depthai.hpp"

int main() {
    dai::Pipeline pipeline;

    auto left = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
    auto right = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
    auto stereo = pipeline.create<dai::node::StereoDepth>();
    left->requestOutput(std::make_pair(640, 400))->link(stereo->left);
    right->requestOutput(std::make_pair(640, 400))->link(stereo->right);

    auto pc = pipeline.create<dai::node::PointCloud>();
    pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);
    stereo->depth.link(pc->inputDepth);

    auto q = pc->outputPointCloud.createOutputQueue(4, false);

    pipeline.start();
    while(pipeline.isRunning()) {
        auto pclData = q->get<dai::PointCloudData>();
        auto points = pclData->getPoints();
        std::cout << "点数: " << points.size()
                  << ", Z=[" << pclData->getMinZ() << ", " << pclData->getMaxZ() << "]" << std::endl;
    }
    pipeline.stop();
    return 0;
}
```

### 彩色点云

将深度图和彩色图像链接到同一坐标系。通常将深度图对齐到彩色相机。

#### Python

```python
import depthai as dai

pipeline = dai.Pipeline()

left = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
right = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
color = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_A)

stereo = pipeline.create(dai.node.StereoDepth)
left.requestFullResolutionOutput().link(stereo.left)
right.requestFullResolutionOutput().link(stereo.right)

colorOut = color.requestOutput((640, 400), type=dai.ImgFrame.Type.RGB888i,
                               resizeMode=dai.ImgResizeMode.CROP, enableUndistortion=True)

pc = pipeline.create(dai.node.PointCloud)
pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)

# 将深度图对齐到彩色相机
platform = pipeline.getDefaultDevice().getPlatform()
if platform == dai.Platform.RVC4:
    imageAlign = pipeline.create(dai.node.ImageAlign)
    stereo.depth.link(imageAlign.input)
    colorOut.link(imageAlign.inputAlignTo)
    imageAlign.outputAligned.link(pc.inputDepth)
else:
    colorOut.link(stereo.inputAlignTo)
    stereo.depth.link(pc.inputDepth)

colorOut.link(pc.inputColor)

q = pc.outputPointCloud.createOutputQueue(maxSize=4, blocking=False)

with pipeline:
    pipeline.start()
    while pipeline.isRunning():
        pcd = q.get()
        if pcd.isColor():
            xyz, rgba = pcd.getPointsRGB()
            print(f"点数: {len(xyz)}, color=yes, Z=[{pcd.getMinZ():.2f}, {pcd.getMaxZ():.2f}]")
```

#### C++

```cpp
#include <iostream>
#include "depthai/depthai.hpp"

int main() {
    dai::Pipeline pipeline;

    auto left = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
    auto right = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
    auto color = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_A);

    auto stereo = pipeline.create<dai::node::StereoDepth>();
    left->requestFullResolutionOutput()->link(stereo->left);
    right->requestFullResolutionOutput()->link(stereo->right);

    auto colorOut = color->requestOutput(std::make_pair(640, 400), dai::ImgFrame::Type::RGB888i,
                                         dai::ImgResizeMode::CROP, std::nullopt, true);

    auto pc = pipeline.create<dai::node::PointCloud>();
    pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);

    auto platform = pipeline.getDefaultDevice()->getPlatform();
    if(platform == dai::Platform::RVC4) {
        auto imageAlign = pipeline.create<dai::node::ImageAlign>();
        stereo->depth.link(imageAlign->input);
        colorOut->link(imageAlign->inputAlignTo);
        imageAlign->outputAligned.link(pc->inputDepth);
    } else {
        colorOut->link(stereo->inputAlignTo);
        stereo->depth.link(pc->inputDepth);
    }

    colorOut->link(pc->getColorInput());

    auto q = pc->outputPointCloud.createOutputQueue(4, false);

    pipeline.start();
    while(pipeline.isRunning()) {
        auto pcd = q->get<dai::PointCloudData>();
        if(pcd->isColor()) {
            auto points = pcd->getPointsRGB();
            std::cout << "点数: " << points.size() << ", color=yes"
                      << ", Z=[" << pcd->getMinZ() << ", " << pcd->getMaxZ() << "]" << std::endl;
        }
    }
    pipeline.stop();
    return 0;
}
```

## Examples

 * [PointCloud](https://docs.luxonis.com/software-v3/depthai/examples/pointcloud/point_cloud.md) — 最小化彩色点云示例。
 * [PointCloud Visualizer](https://docs.luxonis.com/software-v3/depthai/examples/pointcloud/point_cloud_visualizer.md) —
   使用Open3D的实时3D可视化。
 * [PointCloud Showcase](https://docs.luxonis.com/software-v3/depthai/examples/pointcloud/point_cloud_showcase.md) —
   演示过滤、组织、坐标变换、自定义矩阵和彩色模式。

## Reference

### dai::node::PointCloud

Kind: class

PointCloud node. Computes point cloud from depth frames.

#### dai::node::PointCloud::Impl

Kind: class

##### LengthUnit targetLengthUnit

Kind: variable

##### Impl()

Kind: function

##### void setLogger(const std::shared_ptr<::spdlog::logger > & log)

Kind: function

##### void computePointCloudDense(const uint8_t * depthData, std::vector< Point3f > & points)

Kind: function

##### void computePointCloudDenseColored(const uint8_t * depthData, const uint8_t * colorData, std::vector< Point3fRGBA > &
points)

Kind: function

##### void applyTransformation(std::vector< PointT > & points)

Kind: function

##### std::vector< PointT > filterValidPoints(const std::vector< PointT > & densePoints)

Kind: function

##### void setLengthUnit(dai::LengthUnit lengthUnit)

Kind: function

##### void useCPU()

Kind: function

##### void useCPUMT(uint32_t numThreads)

Kind: function

##### void useGPU(uint32_t device)

Kind: function

##### void setIntrinsics(float fx, float fy, float cx, float cy, unsigned int width, unsigned int height)

Kind: function

##### void setExtrinsics(const std::vector< std::vector< float >> & transformMatrix)

Kind: function

##### void clearExtrinsics()

Kind: function

#### std::shared_ptr< PointCloudConfig > initialConfig

Kind: variable

Initial config to use when computing the point cloud.

#### Input inputConfig

Kind: variable

Input PointCloudConfig message with ability to modify parameters in runtime. Default queue is non-blocking with size 4.

#### Subnode < node::Sync > sync

Kind: variable

Sync subnode for synchronized depth + color input. When only depth is connected, Sync passes through single-item MessageGroups.
When both depth and color are connected, Sync pairs them by timestamp.

#### InputMap & syncInputs

Kind: variable

#### Input & inputDepth

Kind: variable

Input message with depth data used to create the point cloud. Routed through the internal Sync subnode.

#### Output outputPointCloud

Kind: variable

Outputs PointCloudData message

#### Output passthroughDepth

Kind: variable

Passthrough depth from which the point cloud was calculated. Suitable for when input queue is set to non-blocking behavior.

#### PointCloud()

Kind: function

#### PointCloud(std::unique_ptr< Properties > props)

Kind: function

#### ~PointCloud()

Kind: function

#### Input & getColorInput()

Kind: function

Get the optional color input for colorized point clouds. Lazily creates the Sync entry so that depth-only mode works without Sync
waiting for a color frame that never arrives. Link an aligned color image (RGB888i, same dimensions as depth) to this input to
enable colored point cloud output.

#### void setNumFramesPool(int numFramesPool)

Kind: function

Specify number of frames in pool. parameters: numFramesPool: How many frames should the pool have

#### void setRunOnHost(bool runOnHost)

Kind: function

Specify whether to run on host or device By default, the node will run on host.

#### void useCPU()

Kind: function

Use single-threaded CPU for processing

#### void useCPUMT(uint32_t numThreads)

Kind: function

Use multi-threaded CPU for processing

#### void useGPU(uint32_t device)

Kind: function

Use GPU for point cloud computation parameters: device: GPU device index (default 0)

#### void setTargetCoordinateSystem(CameraBoardSocket targetCamera)

Kind: function

Set target coordinate system to transform point cloud parameters: targetCamera: Target camera socket

#### void setTargetCoordinateSystem(HousingCoordinateSystem housingCS)

Kind: function

Set target coordinate system to housing coordinate system Point cloud will be transformed to this housing coordinate system
parameters: housingCS: Target housing coordinate system

#### void setTargetCoordinateSystem(CameraBoardSocket targetCamera, bool useSpecTranslation)

Kind: function

Deprecated: use setTargetCoordinateSystem(targetCamera) instead.

#### void setTargetCoordinateSystem(HousingCoordinateSystem housingCS, bool useSpecTranslation)

Kind: function

Deprecated: use setTargetCoordinateSystem(housingCS) instead.

#### bool runOnHost()

Kind: function

Returns true or false whether the node should be run on host or not.

#### void buildInternal()

Kind: function

Function called from within the

### 需要帮助？

请前往 [OAKChina 官网](https://www.oakchina.cn/) 获取技术支持或解答您的任何疑问。