DepthAI v2 has been superseded by DepthAI v3. You are viewing legacy documentation.
概览硬件软件云平台
DepthAI 组件
DepthAI 教程
DepthAI Perception
ROS
神经网络
DepthAI API 参考

本页目录

  • 如何放置
  • 输入和输出
  • 使用
  • 3A 算法
  • 限制
  • 功能示例
  • 参考

摄像头

Camera 节点是 ImgFrame 的数据源。您可以通过 InputControl 和 InputConfig 在运行时对其进行控制。它旨在将 Color Camera 和 MonoCamera 统一为一个节点。与 Color Camera 节点相比，Camera 节点：
  • 支持 cam.setSize()，它取代了 cam.setResolution() 和 cam.setIspScale()。Camera 节点会自动找到最合适的分辨率，并应用正确的缩放以实现用户选择的大小。
  • 支持 cam.setCalibrationAlpha()，示例如下：消除相机畸变
  • 支持 cam.loadMeshData() 和 cam.setMeshStep()，可用于自定义图像扭曲（去畸变、透视校正等）。
  • 如果相机的 HFOV 大于 85°，则会自动消除相机畸变。您可以通过 cam.setMeshSource(dai.CameraProperties.WarpMeshSource.NONE) 禁用此功能。
除上述几点外，与 MonoCamera 节点相比，Camera 节点：
  • 没有 out 输出，因为它具有与 ColorCamera 相同的输出（raw、isp、still、preview、video）。这意味着 preview 将输出同一灰度帧的 3 个平面（3 倍开销），而 isp / video / still 将输出亮度（有用的灰度信息）+ 色度（所有值均为 128），这将导致 1.5 倍的带宽开销。

如何放置

Python

Python
1pipeline = dai.Pipeline()
2cam = pipeline.create(dai.node.Camera)

C++

C++
1dai::Pipeline pipeline;
2auto cam = pipeline.create<dai::node::Camera>();

输入和输出

消息类型
  • inputConfig - ImageManipConfig
  • inputControl - CameraControl
  • raw - ImgFrame - RAW10 Bayer 数据。解包演示代码参见此处
  • isp - ImgFrame - YUV420 平面格式（与 YU12/IYUV/I420 相同）
  • still - ImgFrame - NV12，适用于较大尺寸的帧。当向 Camera 发送捕获事件时创建图像，类似于拍照。
  • preview - ImgFrame - RGB（如果配置了 BGR 平面或交错格式），主要用于小尺寸预览并馈送至 NeuralNetwork
  • video - ImgFrame - NV12，适用于较大尺寸的帧
ISP（图像信号处理器）用于 Bayer 变换、去马赛克、降噪以及其他图像增强。 它与 3A 算法交互：自动对焦、自动曝光和自动白平衡，这些算法在运行时处理图像传感器 的调整，例如曝光时间、感光度（ISO）以及镜头位置（如果相机模块具有电动镜头）。 点击此处了解更多信息。图像后处理将 ISP 输出的 YUV420 平面帧转换为 video/preview/still 帧。still（当触发捕获时）和 isp 以最高相机分辨率工作，而 video 和 preview 限制为最大 4K（3840 x 2160）分辨率，该分辨率从 isp 中裁剪得到。 对于 IMX378（1200 万像素），后处理的工作原理如下：
software/depthai/nodes/isp.webp
上图显示了 Camera 的 isp 输出（IMX378 的 1200 万像素分辨率）。如果您不缩小 ISP， video 输出将裁剪为 4K（最大 3840x2160，受 video 输出限制），如蓝色矩形所示。 黄色矩形表示 preview 输出在预览大小设置为 1:1 纵横比时的裁剪（例如，为 MobileNet-SSD NN 模型使用 300x300 的预览大小），因为 preview 输出源自 video 输出。

使用

Python

Python
1pipeline = dai.Pipeline()
2cam = pipeline.create(dai.node.Camera)
3cam.setPreviewSize(300, 300)
4cam.setBoardSocket(dai.CameraBoardSocket.CAM_A)
5# Instead of setting the resolution, user can specify size, which will set
6# sensor resolution to best fit, and also apply scaling
7cam.setSize(1280, 720)

C++

C++
1dai::Pipeline pipeline;
2auto cam = pipeline.create<dai::node::Camera>();
3cam->setPreviewSize(300, 300);
4cam->setBoardSocket(dai::CameraBoardSocket::CAM_A);
5// Instead of setting the resolution, user can specify size, which will set
6// sensor resolution to best fit, and also apply scaling
7cam->setSize(1280, 720);

3A 算法

3A（自动曝光 AE、自动白平衡 AWB 和自动对焦 AF）算法用于优化图像质量，并直接在 RVC 上运行。默认情况下，这些设置处于自动模式，每个传感器都有特定的限制（例如最小/最大曝光），详见支持的传感器。您可以通过以下方式手动控制这些设置：按照 RGB 相机控制示例 中的步骤操作，或使用 cam_test.py 脚本。
  • 立体相机：传感器共享同一条 I2C 总线，可自动同步 3A 设置（AWB、AE）。
  • 独立传感器：在 OAK FFC 或 OAK-D-LR 等配置中，每个传感器拥有独立的 I2C，可使用 3a-follow 功能将 3A 设置从一个传感器同步到其他传感器。
示例用法
Python
1cam['cam_b'].initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_A)
2cam['cam_c'].initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_A)
3a-follow 功能将主摄像头（例如 CAM_A）的 3A 设置（曝光、ISO 和白平衡）复制到配置中的其他摄像头（例如 CAM_B 和 CAM_C）。

限制

以下是已知的 RVC2 相机限制：
  • ISP 每秒可处理约 600 MP，当流水线同时运行神经网络和视频编码器时，约 500 MP/s。
  • 3A 算法总体可处理约 200..250 FPS（针对所有相机流）。这是我们当前实现的限制，我们计划通过一种变通方法让 3A 算法每隔 X 帧运行一次，暂无预计时间。

功能示例

参考

class

depthai.node.Camera(depthai.Node)

method
getBoardSocket(self) -> depthai.CameraBoardSocket: depthai.CameraBoardSocket
Retrieves which board socket to use  Returns:     Board socket to use
method
getCalibrationAlpha(self) -> float|None: float|None
Get calibration alpha parameter that determines FOV of undistorted frames
method
getCamera(self) -> str: str
Retrieves which camera to use by name  Returns:     Name of the camera to use
method
getFps(self) -> float: float
Get rate at which camera should produce frames  Returns:     Rate in frames per second
method
getHeight(self) -> int: int
Get sensor resolution height
method
method
method
getMeshStep(self) -> tuple[int, int]: tuple[int, int]
Gets the distance between mesh points
method
method
method
method
method
method
method
method
method
method
method
getWidth(self) -> int: int
Get sensor resolution width
method
loadMeshData(self, warpMesh: typing_extensions.Buffer)
Specify mesh calibration data for undistortion See `loadMeshFiles` for the expected data format
method
loadMeshFile(self, warpMesh: Path)
Specify local filesystem paths to the undistort mesh calibration files.  When a mesh calibration is set, it overrides the camera intrinsics/extrinsics matrices. Overrides useHomographyRectification behavior. Mesh format: a sequence of (y,x) points as 'float' with coordinates from the input image to be mapped in the output. The mesh can be subsampled, configured by `setMeshStep`.  With a 1280x800 resolution and the default (16,16) step, the required mesh size is:  width: 1280 / 16 + 1 = 81  height: 800 / 16 + 1 = 51
method
setBoardSocket(self, boardSocket: depthai.CameraBoardSocket)
Specify which board socket to use  Parameter ``boardSocket``:     Board socket to use
method
setCalibrationAlpha(self, alpha: typing.SupportsFloat)
Set calibration alpha parameter that determines FOV of undistorted frames
method
setCamera(self, name: str)
Specify which camera to use by name  Parameter ``name``:     Name of the camera to use
method
setFps(self, fps: typing.SupportsFloat)
Set rate at which camera should produce frames  Parameter ``fps``:     Rate in frames per second
method
method
setIsp3aFps(self, isp3aFps: typing.SupportsInt)
Isp 3A rate (auto focus, auto exposure, auto white balance, camera controls etc.). Default (0) matches the camera FPS, meaning that 3A is running on each frame. Reducing the rate of 3A reduces the CPU usage on CSS, but also increases the convergence rate of 3A. Note that camera controls will be processed at this rate. E.g. if camera is running at 30 fps, and camera control is sent at every frame, but 3A fps is set to 15, the camera control messages will be processed at 15 fps rate, which will lead to queueing.
method
method
setMeshStep(self, width: typing.SupportsInt, height: typing.SupportsInt)
Set the distance between mesh points. Default: (32, 32)
method
method
setRawOutputPacked(self, packed: bool)
Configures whether the camera `raw` frames are saved as MIPI-packed to memory. The packed format is more efficient, consuming less memory on device, and less data to send to host: RAW10: 4 pixels saved on 5 bytes, RAW12: 2 pixels saved on 3 bytes. When packing is disabled (`false`), data is saved lsb-aligned, e.g. a RAW10 pixel will be stored as uint16, on bits 9..0: 0b0000'00pp'pppp'pppp. Default is auto: enabled for standard color/monochrome cameras where ISP can work with both packed/unpacked, but disabled for other cameras like ToF.
method
method
method
property
frameEvent
Outputs metadata-only ImgFrame message as an early indicator of an incoming frame.  It's sent on the MIPI SoF (start-of-frame) event, just after the exposure of the current frame has finished and before the exposure for next frame starts. Could be used to synchronize various processes with camera capture. Fields populated: camera id, sequence number, timestamp
property
initialControl
Initial control options to apply to sensor
property
inputConfig
Input for ImageManipConfig message, which can modify crop parameters in runtime  Default queue is non-blocking with size 8
property
inputControl
Input for CameraControl message, which can modify camera parameters in runtime  Default queue is blocking with size 8
property
isp
Outputs ImgFrame message that carries YUV420 planar (I420/IYUV) frame data.  Generated by the ISP engine, and the source for the 'video', 'preview' and 'still' outputs
property
preview
Outputs ImgFrame message that carries BGR/RGB planar/interleaved encoded frame data.  Suitable for use with NeuralNetwork node
property
raw
Outputs ImgFrame message that carries RAW10-packed (MIPI CSI-2 format) frame data.  Captured directly from the camera sensor, and the source for the 'isp' output.
property
still
Outputs ImgFrame message that carries NV12 encoded (YUV420, UV plane interleaved) frame data.  The message is sent only when a CameraControl message arrives to inputControl with captureStill command set.
property
video
Outputs ImgFrame message that carries NV12 encoded (YUV420, UV plane interleaved) frame data.  Suitable for use with VideoEncoder node

需要帮助？

请前往 OAKChina 官网 获取技术支持或解答您的任何疑问。