概览硬件软件云平台
OAK 应用
DepthAI
AI
软件栈
故障排除

本页目录

  • 如何放置
  • 输入与输出
  • 配置空间检测
  • 常见错误
  • 使用方法
  • 功能示例
  • 空间坐标系
  • 参考

空间检测网络

Supported on:RVC2RVC4
空间检测节点类似于 DetectionNetwork 和 SpatialLocationCalculator 的组合。

如何放置

Python

Python
1modelDescription = dai.NNModelDescription("yolov6-nano")
2pipeline = dai.Pipeline()
3spatialDetectionNetwork = pipeline.create(dai.node.SpatialDetectionNetwork).build(camRgb, stereo, modelDescription)

C++

C++
1dai::NNModelDescription modelDescription("yolov6-nano");
2dai::Pipeline pipeline;
3auto spatialDetectionNetwork = pipeline.create<dai::node::SpatialDetectionNetwork>()->build(camRgb, stereo, modelDescription);

输入与输出

配置空间检测

SpatialDetectionNetwork 节点的流程如下图所示：
空间检测节点本质上是对检测网络（DetectionNetwork）和SpatialLocationCalculator的抽象。其工作原理是将每个检测到的物体的边界框与空间位置计算器关联起来。流程如下：
检测网络负责在输入帧中检测物体。它输出一个检测到的物体列表，每个物体由一个边界框、标签和置信度分数表示。
深度图与输入帧对齐。这是必需的，因为 DetectionNetwork 在输入帧上操作，而 SpatialLocationCalculator 在深度图上操作。
来自网络的边界框被发送到 SpatialLocationCalculator，并根据 BoundingBoxScaleFactor 进行缩放。这样做是为了确保边界框包含整个物体。然后边界框结合深度用于计算物体的空间坐标。
  • X 和 Y 坐标取自边界框中心。它们是根据帧中心的偏移量以及该点的深度计算得出的。
  • 对于深度（Z），缩放后的边界框（ROI）内的每个像素都会被考虑。这为我们提供了一组深度值，然后对这些值进行平均以得到最终的深度值。

平均方法

  • 平均值/均值：使用 ROI 的平均值进行计算。
  • 最小值：使用 ROI 内的最小值进行计算。
  • 最大值：使用 ROI 内的最大值进行计算。
  • 众数：使用 ROI 内出现最频繁的值进行计算。
  • 中位数：使用 ROI 内的中位数进行计算。
默认方法是中位数。

常见错误

大多数错误源于边界框重叠不正确。缩放的边界框可能包含背景部分，这可能会使深度计算产生偏差。
  • 细长物体（如杆子）的空间坐标可能不准确，因为边界框只有一小部分实际覆盖在检测到的物体上。在这种情况下，如果可以的话，最好使用较小的 BoundingBoxScaleFactor。
  • 有孔的物体——篮筐、圆环等。为了获得正确的深度，边界框应包含整个物体。不使用中位数深度，而是使用最小值深度方法来排除背景对计算的影响。或者，在静态环境中可以设置深度阈值来忽略背景。

使用方法

Python

Python
1with dai.Pipeline() as p:
2    camRgb = p.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_A)
3    monoLeft = p.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
4    monoRight = p.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
5    stereo = p.create(dai.node.StereoDepth)
6    spatialDetectionNetwork = p.create(dai.node.SpatialDetectionNetwork).build(camRgb, stereo, modelDescription, fps=FPS)
7
8    spatialDetectionNetwork.input.setBlocking(False)
9    spatialDetectionNetwork.setBoundingBoxScaleFactor(0.5)
10    spatialDetectionNetwork.setDepthLowerThreshold(100)
11    spatialDetectionNetwork.setDepthUpperThreshold(5000)
12
13    labelMap = spatialDetectionNetwork.getClasses()

C++

C++
1dai::Pipeline pipeline;
2auto camRgb = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_A);
3auto monoLeft = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
4auto monoRight = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
5auto stereo = pipeline.create<dai::node::StereoDepth>();
6auto spatialDetectionNetwork = pipeline.create<dai::node::SpatialDetectionNetwork>()->build(camRgb, stereo, modelDescription, FPS);
7spatialDetectionNetwork->input.setBlocking(false);
8spatialDetectionNetwork->setBoundingBoxScaleFactor(0.5);
9spatialDetectionNetwork->setDepthLowerThreshold(100);
10spatialDetectionNetwork->setDepthUpperThreshold(5000);
11auto labelMap = spatialDetectionNetwork->getClasses();

功能示例

空间坐标系

OAK相机使用RDF（右-下-前）坐标系处理所有空间坐标。
帧中心是X,Y坐标的(0,0)点。向下移动Y增加，向右移动X增加。

参考

class

dai::node::SpatialDetectionNetwork

#include SpatialDetectionNetwork.hpp
variable
Properties & properties
variable
Subnode< NeuralNetwork > neuralNetwork
variable
Subnode< DetectionParser > detectionParser
variable
Subnode< SpatialLocationCalculator > spatialLocationCalculator
variable
std::unique_ptr< Subnode< ImageAlign > > depthAlign
variable
Input & input
Input message with data to be inferred upon Default queue is blocking with size 5
variable
Output & outNetwork
Outputs unparsed inference results.
variable
Output & passthrough
Passthrough message on which the inference was performed.Suitable for when input queue is set to non-blocking behavior.
variable
Input & inputDepth
Input message with depth data used to retrieve spatial information about detected object Default queue is non-blocking with size 4
variable
Output & out
Outputs ImgDetections message that carries parsed detection results.
variable
Output & passthroughDepth
Passthrough message for depth frame on which the spatial location calculation was performed. Suitable for when input queue is set to non-blocking behavior.
explicit function
SpatialDetectionNetwork(const std::shared_ptr< Device > & device)
explicit function
SpatialDetectionNetwork(std::unique_ptr< Properties > props)
function
SpatialDetectionNetwork(std::unique_ptr< Properties > props, bool confMode)
function
SpatialDetectionNetwork(const std::shared_ptr< Device > & device, std::unique_ptr< Properties > props, bool confMode)
function
std::shared_ptr< SpatialDetectionNetwork > build(const std::shared_ptr< Camera > & inputRgb, const DepthSource & depthSource, const Model & model, std::optional< float > fps, std::optional< dai::ImgResizeMode > resizeMode)
function
std::shared_ptr< SpatialDetectionNetwork > build(const std::shared_ptr< Camera > & inputRgb, const DepthSource & depthSource, const Model & model, const ImgFrameCapability & capability)
function
void setNNArchive(const NNArchive & nnArchive)
function
void setFromModelZoo(NNModelDescription description, bool useCached)
function
void setNNArchive(const NNArchive & nnArchive, int numShaves)
function
void setBlobPath(const std::filesystem::path & path)
Backwards compatibility interface Load network blob into assets and use once pipeline is started.
Parameters
  • Error: if file doesn't exist or isn't a valid network blob.
Parameters
  • path: Path to network blob
function
void setBlob(const OpenVINO::Blob & blob)
Load network blob into assets and use once pipeline is started.
Parameters
  • blob: Network blob
function
void setBlob(const std::filesystem::path & path)
Same functionality as the setBlobPath(). Load network blob into assets and use once pipeline is started.
Parameters
  • Error: if file doesn't exist or isn't a valid network blob.
Parameters
  • path: Path to network blob
function
void setModelPath(const std::filesystem::path & modelPath)
Load network file into assets.
Parameters
  • modelPath: Path to the model file.
function
void setNumPoolFrames(int numFrames)
Specifies how many frames will be available in the pool
Parameters
  • numFrames: How many frames will pool have
function
void setNumInferenceThreads(int numThreads)
How many threads should the node use to run the network.
Parameters
  • numThreads: Number of threads to dedicate to this node
function
void setNumNCEPerInferenceThread(int numNCEPerThread)
How many Neural Compute Engines should a single thread use for inference
Parameters
  • numNCEPerThread: Number of NCE per thread
function
void setNumShavesPerInferenceThread(int numShavesPerThread)
How many Shaves should a single thread use for inference
Parameters
  • numShavesPerThread: Number of shaves per thread
function
void setBackend(const std::string & backend)
Specifies backend to use
Parameters
  • backend: String specifying backend to use
function
void setBackendProperties(const std::map< std::string, std::string > & properties)
Set backend properties
Parameters
  • backendProperties: backend properties map
function
int getNumInferenceThreads()
How many inference threads will be used to run the network
Returns
Number of threads, 0, 1 or 2. Zero means AUTO
function
void setConfidenceThreshold(float thresh)
Specifies confidence threshold at which to filter the rest of the detections.
Parameters
  • thresh: Detection confidence must be greater than specified threshold to be added to the list
function
float getConfidenceThreshold()
Retrieves threshold at which to filter the rest of the detections.
Returns
Detection confidence
function
void setBoundingBoxScaleFactor(float scaleFactor)
Custom interface Specifies scale factor for detected bounding boxes.
Parameters
  • scaleFactor: Scale factor must be in the interval (0,1].
function
void setDepthLowerThreshold(uint32_t lowerThreshold)
Specifies lower threshold in depth units (millimeter by default) for depth values which will used to calculate spatial data
Parameters
  • lowerThreshold: LowerThreshold must be in the interval [0,upperThreshold] and less than upperThreshold.
function
void setDepthUpperThreshold(uint32_t upperThreshold)
Specifies upper threshold in depth units (millimeter by default) for depth values which will used to calculate spatial data
Parameters
  • upperThreshold: UpperThreshold must be in the interval (lowerThreshold,65535].
function
void setSpatialCalculationAlgorithm(dai::SpatialLocationCalculatorAlgorithm calculationAlgorithm)
Specifies spatial location calculator algorithm: Average/Min/Max
Parameters
  • calculationAlgorithm: Calculation algorithm.
function
void setSpatialCalculationStepSize(int stepSize)
Specifies spatial location calculator step size for depth calculation. Step size 1 means that every pixel is taken into calculation, size 2 means every second etc.
Parameters
  • stepSize: Step size.
function
std::optional< std::vector< std::string > > getClasses()
function
void buildInternal()
enum

enum

NeuralNetwork::Model Model

需要帮助？

请前往 OAKChina 官网 获取技术支持或解答您的任何疑问。