# ObjectTracker

ObjectTracker 使用卡尔曼滤波和匈牙利算法跟踪来自 [ImgDetections](https://docs.luxonis.com/software/depthai-components/messages/img_detections.md)
的检测对象。

## 如何放置

#### Python

```python
pipeline = dai.Pipeline()
objectTracker = pipeline.create(dai.node.ObjectTracker)
```

#### C++

```cpp
dai::Pipeline pipeline;
auto objectTracker = pipeline.create<dai::node::ObjectTracker>();
```

## 输入和输出

## 零项跟踪

零项跟踪执行对象关联，即不基于之前的跟踪历史进行预测和跟踪。对象关联意味着来自外部检测器的检测对象与之前帧中检测到并正在跟踪的跟踪对象进行映射。

## 短期跟踪

短期跟踪允许在帧之间跟踪对象，从而减少每帧运行对象检测的需求。这非常适合无法达到 30FPS 的 NN 模型（例如
[YoloV5](https://github.com/luxonis/oak-examples/tree/master/gen2-yolo)）；跟踪器可以在没有推理时提供轨迹，因此整个系统可以以 30FPS 运行。

## 支持的对象跟踪器类型

 * SHORT_TERM_KCF：核相关滤波跟踪。KCF 利用循环矩阵的特性来提高处理速度。论文见[此处](https://www.robots.ox.ac.uk/~joao/publications/henriques_tpami2015.pdf)。
 * SHORT_TERM_IMAGELESS：短期跟踪允许在跳过了对象检测的帧上，通过外推先前检测的目标轨迹来跟踪对象。
 * ZERO_TERM_COLOR_HISTOGRAM：利用位置、形状和输入图像信息（如 RGB 直方图）进行对象跟踪。
 * ZERO_TERM_IMAGELESS：仅利用检测对象的矩形形状和位置信息进行对象跟踪，不使用跟踪对象的颜色信息。其吞吐量高于 ZERO_TERM_COLOR_HISTOGRAM。用户在选择对象跟踪器类型时需权衡吞吐量与精度。

关于对象跟踪器的更详细比较可参见[此处](https://github.com/openvinotoolkit/dlstreamer_gst/wiki/Object-tracking#object-tracking-types)。

## 最大跟踪对象数量

ObjectTracker 节点最多可同时跟踪 60 个对象。目前，如果跟踪对象超过 60 个，固件会崩溃。

## 使用示例

#### Python

```python
pipeline = dai.Pipeline()
objectTracker = pipeline.create(dai.node.ObjectTracker)

objectTracker.setDetectionLabelsToTrack([15])  # 仅跟踪人
# 可能的跟踪类型：ZERO_TERM_COLOR_HISTOGRAM, ZERO_TERM_IMAGELESS, SHORT_TERM_IMAGELESS, SHORT_TERM_KCF
objectTracker.setTrackerType(dai.TrackerType.ZERO_TERM_COLOR_HISTOGRAM)
# 跟踪新对象时采用最小 ID，可选：SMALLEST_ID, UNIQUE_ID
objectTracker.setTrackerIdAssignmentPolicy(dai.TrackerIdAssignmentPolicy.SMALLEST_ID)

# 必须将对象跟踪器与检测网络以及图像帧源（单目/彩色相机或 xlinkIn 节点）结合使用
```

#### C++

```cpp
dai::Pipeline pipeline;
auto objectTracker = pipeline.create<dai::node::ObjectTracker>();

objectTracker->setDetectionLabelsToTrack({15});  // 仅跟踪人
// 可能的跟踪类型：ZERO_TERM_COLOR_HISTOGRAM, ZERO_TERM_IMAGELESS, SHORT_TERM_IMAGELESS, SHORT_TERM_KCF
objectTracker->setTrackerType(dai::TrackerType::ZERO_TERM_COLOR_HISTOGRAM);
// 跟踪新对象时采用最小 ID，可选：SMALLEST_ID, UNIQUE_ID
objectTracker->setTrackerIdAssignmentPolicy(dai::TrackerIdAssignmentPolicy::SMALLEST_ID);

// 必须将对象跟踪器与检测网络以及图像帧源（单目/彩色相机或 xlinkIn 节点）结合使用
```

## 功能示例

 * [RGB 上的对象跟踪](https://docs.luxonis.com/software/depthai/examples/object_tracker.md)
 * [RGB 上的空间对象跟踪](https://docs.luxonis.com/software/depthai/examples/spatial_object_tracker.md)
 * [视频上的对象跟踪](https://docs.luxonis.com/software/depthai/examples/object_tracker_video.md)

## 参考

### depthai.node.ObjectTracker(depthai.Node)

Kind: Class

ObjectTracker node. Performs object tracking using Kalman filter and hungarian
algorithm.

#### setDetectionLabelsToTrack(self, labels: collections.abc.Sequence [ typing.SupportsInt ])

Kind: Method

Specify detection labels to track.

Parameter ``labels``:
Detection labels to track. Default every label is tracked from image
detection network output.

#### setMaxObjectsToTrack(self, maxObjectsToTrack: typing.SupportsInt)

Kind: Method

Specify maximum number of object to track.

Parameter ``maxObjectsToTrack``:
Maximum number of object to track. Maximum 60 in case of SHORT_TERM_KCF,
otherwise 1000.

#### setOcclusionRatioThreshold(self, occlusionRatioThreshold: typing.SupportsFloat)

Kind: Method

Specify occlusion ratio threshold.

Parameter ``occlusionRatioThreshold``:
Occlusion ratio threshold. Used to filter out overlapping tracklets. Default
0.4.

#### setTrackerIdAssignmentPolicy(self, type: depthai.TrackerIdAssignmentPolicy)

Kind: Method

Specify tracker ID assignment policy.

Parameter ``type``:
Tracker ID assignment policy.

#### setTrackerThreshold(self, threshold: typing.SupportsFloat)

Kind: Method

Specify tracker threshold.

Parameter ``threshold``:
Above this threshold the detected objects will be tracked. Default 0, all
image detections are tracked.

#### setTrackerType(self, type: depthai.TrackerType)

Kind: Method

Specify tracker type algorithm.

Parameter ``type``:
Tracker type.

#### setTrackingPerClass(self, trackingPerClass: bool)

Kind: Method

Whether tracker should take into consideration class label for tracking.

#### setTrackletBirthThreshold(self, threshold: typing.SupportsInt)

Kind: Method

Specify tracklet birth threshold.

Parameter ``threshold``:
Tracklet birth threshold. Minimum consecutive tracked frames required to
consider a tracklet as a new instance. Default 3.

#### setTrackletMaxLifespan(self, lifespan: typing.SupportsInt)

Kind: Method

Specify tracklet lifespan.

Parameter ``lifespan``:
Tracklet lifespan in number of frames. Number of frames after which a LOST
tracklet is removed. Default 120.

#### inputConfig

Kind: Property

Input ObjectTrackerConfig message with ability to modify parameters at runtime.
Default queue is non-blocking with size 4.

#### inputDetectionFrame

Kind: Property

Input ImgFrame message on which object detection was performed. Default queue is
non-blocking with size 4.

#### inputDetections

Kind: Property

Input message with image detection from neural network. Default queue is non-
blocking with size 4.

#### inputTrackerFrame

Kind: Property

Input ImgFrame message on which tracking will be performed. RGBp, BGRp, NV12,
YUV420p types are supported. Default queue is non-blocking with size 4.

#### out

Kind: Property

Outputs Tracklets message that carries object tracking results.

#### passthroughDetectionFrame

Kind: Property

Passthrough ImgFrame message on which object detection was performed. Suitable
for when input queue is set to non-blocking behavior.

#### passthroughDetections

Kind: Property

Passthrough image detections message from neural network output. Suitable for
when input queue is set to non-blocking behavior.

#### passthroughTrackerFrame

Kind: Property

Passthrough ImgFrame message on which tracking was performed. Suitable for when
input queue is set to non-blocking behavior.

### 需要帮助？

请前往 [OAKChina 官网](https://www.oakchina.cn/) 获取技术支持或解答您的任何疑问。