# 空间检测网络

空间检测节点类似于 [DetectionNetwork](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/detection_network.md) 和
[SpatialLocationCalculator](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/spatial_location_calculator.md)
的组合。

## 如何放置

#### Python

```python
modelDescription = dai.NNModelDescription("yolov6-nano")
pipeline = dai.Pipeline()
spatialDetectionNetwork = pipeline.create(dai.node.SpatialDetectionNetwork).build(camRgb, stereo, modelDescription)
```

#### C++

```cpp
dai::NNModelDescription modelDescription("yolov6-nano");
dai::Pipeline pipeline;
auto spatialDetectionNetwork = pipeline.create<dai::node::SpatialDetectionNetwork>()->build(camRgb, stereo, modelDescription);
```

## 输入与输出

## 配置空间检测

SpatialDetectionNetwork 节点的流程如下图所示：

空间检测节点本质上是对检测网络（[DetectionNetwork](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/detection_network.md)）和[SpatialLocationCalculator](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/spatial_location_calculator.md)的抽象。

其工作原理是将每个检测到的物体的边界框与空间位置计算器关联起来。流程如下：

### 检测

检测网络负责在输入帧中检测物体。它输出一个检测到的物体列表，每个物体由一个边界框、标签和置信度分数表示。

### 对齐

深度图与输入帧对齐。这是必需的，因为 DetectionNetwork 在输入帧上操作，而 SpatialLocationCalculator 在深度图上操作。

### 边界框缩放

来自网络的边界框被发送到 SpatialLocationCalculator，并根据 BoundingBoxScaleFactor 进行缩放。这样做是为了确保边界框包含整个物体。然后边界框结合深度用于计算物体的空间坐标。

### 空间坐标计算

 * X 和 Y 坐标取自边界框中心。它们是根据帧中心的偏移量以及该点的深度计算得出的。
 * 对于深度（Z），缩放后的边界框（ROI）内的每个像素都会被考虑。这为我们提供了一组深度值，然后对这些值进行平均以得到最终的深度值。

### 平均方法

 * 平均值/均值：使用 ROI 的平均值进行计算。
 * 最小值：使用 ROI 内的最小值进行计算。
 * 最大值：使用 ROI 内的最大值进行计算。
 * 众数：使用 ROI 内出现最频繁的值进行计算。
 * 中位数：使用 ROI 内的中位数进行计算。

默认方法是中位数。

## 常见错误

大多数错误源于边界框重叠不正确。缩放的边界框可能包含背景部分，这可能会使深度计算产生偏差。

 * 细长物体（如杆子）的空间坐标可能不准确，因为边界框只有一小部分实际覆盖在检测到的物体上。在这种情况下，如果可以的话，最好使用较小的 BoundingBoxScaleFactor。
 * 有孔的物体——篮筐、圆环等。为了获得正确的深度，边界框应包含整个物体。不使用中位数深度，而是使用最小值深度方法来排除背景对计算的影响。或者，在静态环境中可以设置深度阈值来忽略背景。

## 使用方法

#### Python

```python
with dai.Pipeline() as p:
    camRgb = p.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_A)
    monoLeft = p.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
    monoRight = p.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
    stereo = p.create(dai.node.StereoDepth)
    spatialDetectionNetwork = p.create(dai.node.SpatialDetectionNetwork).build(camRgb, stereo, modelDescription, fps=FPS)

    spatialDetectionNetwork.input.setBlocking(False)
    spatialDetectionNetwork.setBoundingBoxScaleFactor(0.5)
    spatialDetectionNetwork.setDepthLowerThreshold(100)
    spatialDetectionNetwork.setDepthUpperThreshold(5000)

    labelMap = spatialDetectionNetwork.getClasses()
```

#### C++

```cpp
dai::Pipeline pipeline;
auto camRgb = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_A);
auto monoLeft = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
auto monoRight = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
auto stereo = pipeline.create<dai::node::StereoDepth>();
auto spatialDetectionNetwork = pipeline.create<dai::node::SpatialDetectionNetwork>()->build(camRgb, stereo, modelDescription, FPS);
spatialDetectionNetwork->input.setBlocking(false);
spatialDetectionNetwork->setBoundingBoxScaleFactor(0.5);
spatialDetectionNetwork->setDepthLowerThreshold(100);
spatialDetectionNetwork->setDepthUpperThreshold(5000);
auto labelMap = spatialDetectionNetwork->getClasses();
```

## 功能示例

 * [空间检测网络](https://docs.luxonis.com/software-v3/depthai/examples/spatial_detection_network/spatial_detection.md)

## 空间坐标系

OAK相机使用RDF（右-下-前）坐标系处理所有空间坐标。

帧中心是X,Y坐标的(0,0)点。向下移动Y增加，向右移动X增加。

## 参考

### dai::node::SpatialDetectionNetwork

Kind: class

SpatialDetectionNetwork node. Runs a neural inference on input image and calculates spatial location data.

#### SpatialDetectionNetworkProperties Properties

Kind: enum

#### NeuralNetwork::Model Model

Kind: enum

#### Properties & properties

Kind: variable

#### Subnode < NeuralNetwork > neuralNetwork

Kind: variable

#### Subnode < DetectionParser > detectionParser

Kind: variable

#### Subnode < SpatialLocationCalculator > spatialLocationCalculator

Kind: variable

#### std::unique_ptr< Subnode < ImageAlign > > depthAlign

Kind: variable

#### Input & input

Kind: variable

Input message with data to be inferred upon Default queue is blocking with size 5

#### Output & outNetwork

Kind: variable

Outputs unparsed inference results.

#### Output & passthrough

Kind: variable

Passthrough message on which the inference was performed. Suitable for when input queue is set to non-blocking behavior.

#### Input & inputDepth

Kind: variable

Input message with depth data used to retrieve spatial information about detected object Default queue is non-blocking with size 4

#### Output & out

Kind: variable

Outputs ImgDetections message that carries parsed detection results.

#### Output & passthroughDepth

Kind: variable

Passthrough message for depth frame on which the spatial location calculation was performed. Suitable for when input queue is set
to non-blocking behavior.

#### SpatialDetectionNetwork(const std::shared_ptr< Device > & device)

Kind: function

#### SpatialDetectionNetwork(std::unique_ptr< Properties > props)

Kind: function

#### SpatialDetectionNetwork(std::unique_ptr< Properties > props, bool confMode)

Kind: function

#### SpatialDetectionNetwork(const std::shared_ptr< Device > & device, std::unique_ptr< Properties > props, bool confMode)

Kind: function

#### std::shared_ptr< SpatialDetectionNetwork > build(const std::shared_ptr< Camera > & inputRgb, const DepthSource & depthSource,
const Model & model, std::optional< float > fps, std::optional< dai::ImgResizeMode > resizeMode)

Kind: function

Build SpatialDetectionNetwork node with specified depth source. Connect Camera and depth source outputs to this node's inputs and
configure the inference model.

parameters: inputRgb: Camera node; depthSource: Depth source node ( StereoDepth , NeuralDepth , ToF , or Depth ); model: Neural
network model description, NNArchive or HubAI model id string; fps: Desired frames per second; resizeMode: Resize mode for input
color frames return: Shared pointer to SpatialDetectionNetwork node

#### std::shared_ptr< SpatialDetectionNetwork > build(const std::shared_ptr< Camera > & inputRgb, const DepthSource & depthSource,
const Model & model, const ImgFrameCapability & capability)

Kind: function

Build SpatialDetectionNetwork node with specified depth source. Connect Camera and depth source outputs to this node's inputs and
configure the inference model.

parameters: inputRgb: Camera node; depthSource: Depth source node ( StereoDepth , NeuralDepth , ToF , or Depth ); model: Neural
network model description, NNArchive or HubAI model id string; capability: Camera capabilities return: Shared pointer to
SpatialDetectionNetwork node

#### void setNNArchive(const NNArchive & nnArchive)

Kind: function

Set NNArchive for this Node . If the archive's type is SUPERBLOB, use default number of shaves.

parameters: nnArchive: NNArchive to set

#### void setFromModelZoo(NNModelDescription description, bool useCached)

Kind: function

Download model from zoo and set it for this Node .

parameters: description: Model description to download; useCached: Use cached model if available

#### void setNNArchive(const NNArchive & nnArchive, int numShaves)

Kind: function

Set NNArchive for this Node , throws if the archive's type is not SUPERBLOB.

parameters: nnArchive: NNArchive to set; numShaves: Number of shaves to use

#### void setBlobPath(const std::filesystem::path & path)

Kind: function

Backwards compatibility interface Load network blob into assets and use once pipeline is started. parameters: Error: if file
doesn't exist or isn't a valid network blob. parameters: path: Path to network blob

#### void setBlob(const OpenVINO::Blob & blob)

Kind: function

Load network blob into assets and use once pipeline is started. parameters: blob: Network blob

#### void setBlob(const std::filesystem::path & path)

Kind: function

Same functionality as the setBlobPath() . Load network blob into assets and use once pipeline is started. parameters: Error: if
file doesn't exist or isn't a valid network blob. parameters: path: Path to network blob

#### void setModelPath(const std::filesystem::path & modelPath)

Kind: function

Load network file into assets. parameters: modelPath: Path to the model file.

#### void setNumPoolFrames(int numFrames)

Kind: function

Specifies how many frames will be available in the pool parameters: numFrames: How many frames will pool have

#### void setNumInferenceThreads(int numThreads)

Kind: function

How many threads should the node use to run the network. parameters: numThreads: Number of threads to dedicate to this node

#### void setNumNCEPerInferenceThread(int numNCEPerThread)

Kind: function

How many Neural Compute Engines should a single thread use for inference parameters: numNCEPerThread: Number of NCE per thread

#### void setNumShavesPerInferenceThread(int numShavesPerThread)

Kind: function

How many Shaves should a single thread use for inference parameters: numShavesPerThread: Number of shaves per thread

#### void setBackend(const std::string & backend)

Kind: function

Specifies backend to use parameters: backend: String specifying backend to use

#### void setBackendProperties(const std::map< std::string, std::string > & properties)

Kind: function

Set backend properties parameters: backendProperties: backend properties map

#### int getNumInferenceThreads()

Kind: function

How many inference threads will be used to run the network return: Number of threads, 0, 1 or 2. Zero means AUTO

#### void setConfidenceThreshold(float thresh)

Kind: function

Specifies confidence threshold at which to filter the rest of the detections. parameters: thresh: Detection confidence must be
greater than specified threshold to be added to the list

#### float getConfidenceThreshold()

Kind: function

Retrieves threshold at which to filter the rest of the detections. return: Detection confidence

#### void setBoundingBoxScaleFactor(float scaleFactor)

Kind: function

Custom interface Specifies scale factor for detected bounding boxes. parameters: scaleFactor: Scale factor must be in the interval
(0,1].

#### void setDepthLowerThreshold(uint32_t lowerThreshold)

Kind: function

Specifies lower threshold in depth units (millimeter by default) for depth values which will used to calculate spatial data
parameters: lowerThreshold: LowerThreshold must be in the interval [0,upperThreshold] and less than upperThreshold.

#### void setDepthUpperThreshold(uint32_t upperThreshold)

Kind: function

Specifies upper threshold in depth units (millimeter by default) for depth values which will used to calculate spatial data
parameters: upperThreshold: UpperThreshold must be in the interval (lowerThreshold,65535].

#### void setSpatialCalculationAlgorithm(dai::SpatialLocationCalculatorAlgorithm calculationAlgorithm)

Kind: function

Specifies spatial location calculator algorithm: Average/Min/Max parameters: calculationAlgorithm: Calculation algorithm.

#### void setSpatialCalculationStepSize(int stepSize)

Kind: function

Specifies spatial location calculator step size for depth calculation. Step size 1 means that every pixel is taken into
calculation, size 2 means every second etc. parameters: stepSize: Step size.

#### std::optional< std::vector< std::string > > getClasses()

Kind: function

Get classes labels.

#### void buildInternal()

Kind: function

Function called from within the

### 需要帮助？

请前往 [OAKChina 官网](https://www.oakchina.cn/) 获取技术支持或解答您的任何疑问。