Add multi camera sync verification node tutorial

This commit is contained in:
slz
2025-10-31 15:38:07 +08:00
parent 4d02cd129e
commit 2c6a3b54e2
3 changed files with 276 additions and 260 deletions
@@ -1,161 +1,177 @@
# Multi_camera_synced_verification_tool # Multi-Camera Synchronization Verification Node
> This article will introduce how to verify the synchronization accuracy of multi-camera synchronization. **File path:** [image_sync_example_node.cpp](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/multi_camera_time_sync/image_sync_example_node.cpp)
>
> First, please see how to use [multi_camera_synced](./multi_camera_synced.md).
You can find example usage code in the [example](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples). This example node is designed for **synchronized capture and timestamp verification** across **four Orbbec cameras**.
It can be used to validate frame alignment accuracy under the multi-camera **Primary / Secondary Synced** mode.
## Directory Structure ---
## Usage Guide
1. **Modify `multi_camera_synced.launch.py` as follows**
- Add `launch_include` to support four cameras.
- Select the appropriate launch file based on your camera model.
For example, use `gemini_330_series.launch.py` for the Gemini 330 series.
- Naming convention:
`camera_name` should follow the format `camera_01`, `camera_02`, `camera_03`, ...
- Set `device_num` to **4**.
- Configure the USB ports using:
`ros2 run orbbec_camera list_devices_node`
to view and bind the correct ports.
- Set synchronization mode to **Primary/Secondary Synced**,
with one camera as **primary** and the others as **secondary_synced**.
- **Startup sequence:**
The **Primary camera should always be launched last** to ensure the synchronization signal is established correctly.
---
2. **Modify configuration files**
Edit the following two files under the `config` directory:
- `camera_params.yaml`
- `camera_secondary_params.yaml`
Ensure the following settings are consistent across all cameras:
- Enable both **depth** and **color** streams.
- Use the same **frame rate (fps)** for all cameras.
---
3. **Launch and Verification**
- Start the multi-camera synchronization launch file:
`ros2 launch orbbec_camera multi_camera_synced.launch.py`
- In a new terminal, run the synchronization verification node:
`ros2 run orbbec_camera image_sync_example_node`
This node outputs timestamp differences between multiple camera streams for synchronization validation.
---
4. **Reference Launch File**
```plaintext
.
├── multicamera_sync/
│ ├── output/
│ ├──20250218102900/
│ ├──DevicesInfo.txt
│ ├──StreamProfileInfo.txt
│ ├── Python/
│ ├──Config.ini
├── gemini_330_series_synced_verify.launch.py
├── multi_camera_synced_verify.launch.py
``` ```
import os
from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch_ros.actions import Node
from launch.actions import IncludeLaunchDescription, GroupAction, TimerAction
from launch.launch_description_sources import PythonLaunchDescriptionSource
from launch_ros.actions import Node, LoadComposableNodes
**multicamera_sync**
* `gemini_330_series_synced_verify.launch.py` : Single camera runs launch, which provides the camera running node for multi_camera_synced_verify.launch.py.
* `multi_camera_synced_verify.launch.py` : Multi camer synced + launch of save_rgbir tool.
**output** def generate_launch_description():
# Include launch files
package_dir = get_package_share_directory("orbbec_camera")
launch_file_dir = os.path.join(package_dir, "launch")
config_file_dir = os.path.join(package_dir, "config")
config_file_path = os.path.join(config_file_dir, "camera_params.yaml")
secondary_config_file_path = os.path.join(config_file_dir, "camera_secondary_params.yaml")
> The `ouput` folder is the folder where the camera pictures are output launch1_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
),
launch_arguments={
"camera_name": "camera_01",
"usb_port": "2-2.3",
"device_num": "4",
"sync_mode": "primary",
"config_file_path": config_file_path,
"trigger_out_enabled": "true"
}.items(),
)
In the output example provided launch2_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
),
launch_arguments={
"camera_name": "camera_02",
"usb_port": "2-1",
"device_num": "4",
"sync_mode": "secondary_synced",
"config_file_path": secondary_config_file_path,
"trigger_out_enabled": "false"
}.items(),
)
* `20250218102900` : Represents the camera image information collected at 10:29:00 on February 18, 2025. launch3_include = IncludeLaunchDescription(
* `TotalModeFrames` : Camera image information storage directory. PythonLaunchDescriptionSource(
* `results-2025-02-25_163648` : The result after analyzing and matching the image information of TotalModeFrames. os.path.join(launch_file_dir, "gemini_330_series.launch.py")
* `DevicesInfo.txt` : Camera equipment basic information (need to be modified). ),
* `StreamProfileInfo.txt` : Camera video stream information (no need to modify). launch_arguments={
"camera_name": "camera_03",
"usb_port": "2-3",
"device_num": "4",
"sync_mode": "secondary_synced",
"config_file_path": secondary_config_file_path,
"trigger_out_enabled": "false"
}.items(),
)
**Python** launch4_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
),
launch_arguments={
"camera_name": "camera_04",
"usb_port": "2-4",
"device_num": "4",
"sync_mode": "secondary_synced",
"config_file_path": secondary_config_file_path,
"trigger_out_enabled": "false"
}.items(),
)
* `Config.ini` : Configuration file for Python analysis script (need to be modified).
## Preparation for operation
**save_rgbir node** # Launch description
Edit multi_camera_synced_verify.launch.py and fill in the activated camera device,we can find that save_rgbir is started at the end.
> IMPORTANT
>
> If you are using ROS 2 Foxy, you need to launch the save_rgbir node first.
```python
# Launch description
ld = LaunchDescription( ld = LaunchDescription(
[ [
shared_orbbec_container,
TimerAction(period=0.0, actions=[GroupAction([launch2_include])]), TimerAction(period=0.0, actions=[GroupAction([launch2_include])]),
TimerAction(period=2.0, actions=[GroupAction([launch1_include])]), TimerAction(period=2.0, actions=[GroupAction([launch3_include])]),
TimerAction(period=4.0, actions=[GroupAction([launch4_include])]),
TimerAction(period=6.0, actions=[GroupAction([launch1_include])]),
# The primary camera should be launched at last # The primary camera should be launched at last
TimerAction(period=6.0, actions=[GroupAction([save_rgbir])]),
# save_rgbir is synced verification tool
] ]
) )
return ld return ld
``` ```
save_rgbir is a tool for saving images. The configuration file of this tool is in [multi_save_rgbir_params.json](../../config/tools/multisavergbir/multi_save_rgbir_params.json).
```json
{
"save_rgbir_params": {
"time_domain": "global",
"usb_ports": [
"2-1",
"2-3"
],
"camera_name": [
"camera_01",
"camera_02"
]
}
}
```
* `time_domain` : Timestamp Type ---
* `usb_ports` : "primary", "secondary 1", "secondary 2", "secondary 3", fill in as many usb_ports as there are cameras
* `camera_name` : The name of the camera setting, for example: camera_01
**DevicesInfo.txt** ## System Configuration Requirements
Edit DevicesInfo.txt. Only the `primarySerialNumber`, `index` and `serialNumber` parameters need to be changed. Other parameters do not need to be changed.Refer to the example of [20250218102900](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900). 1. **Increase USB Buffer Memory**
* `primarySerialNumber` : The SN serial number of the primary camera Prevent frame drops caused by concurrent data transmission:
* `index` : Camera index
* `serialNumber` : The SN serial number of the camera
**Config.ini** `echo 512 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb`
Edit Config.ini.Modify `frameRate` and `tspRangeThreshold`. 2. **Configure Fast DDS**
## Run this example Optimize ROS2 node communication latency to reduce image transmission delay.
See [this section](../performance/fastdds_tuning.md) for detailed configuration instructions.**
**Run launch and save camera pictures**
* First terminal
```bash
ros2 launch orbbec_camera multi_camera_synced_verify.launch.py
```
* Second terminal
```bash
ros2 service call /save_rgbir/start_capture orbbec_camera_msgs/srv/SetInt32 '{data: 100}'
```
When the terminal displays "over", the image is saved.A new multicamera_sync folder will be generated under the workspace.
**Camera pictures naming format**
Take [color_SNCP1E5420006D_Index0_g1739874543227_f0_s1739874543327_e50_d16_.jpg](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900/TotalModeFrames/SNCP1E5420006D_Index0/color_SNCP1E5420006D_Index0_g1739874543227_f0_s1739874543327_e50_d16_.jpg) as an example
* `color` : This image is a snapshot of the color stream.
* `SNCP1E5420006D` : The camera's SN serial number is CP1E5420006D.
* `Index0` : Camera 0 (usually referred to primary camera).
* `g1739874543227` : “g” represents the global timestamp, which means the global timestamp of this frame is 1739874543227.
* `f0` : The 0th picture of the color stream acquisition of this camera.
* `s1739874543327` : "s" represents the timestamp of the current system, which means the current system timestamp of this frame is 1739874543327.
* `e50` : The exposure of this frame is 50.
* `d16` : The gain of this frame is 16.
**Analyzing camera image data**
You need to copy the modified [Python folder](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/Python) to the new multi_camera_synced subdirectory, and copy the modified [DevicesInfo.txt ](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900/DevicesInfo.txt)and [StreamProfileInfo.txt](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/output/20250218102900/StreamProfileInfo.txt) to the same level directory as the TotalModeFrames folder.
* Finally everything is ready, run the python script
```bash
cd multicamera_sync/Python
python3 SyncFramesMain.py
```
After the operation is successful, you can view the synchronization effect in the `results folder`
## Files that need to be changed
**Analysis tools**
* [multi_save_rgbir_params.json](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/config/tools/multisavergbir/multi_save_rgbir_params.json)
* [DevicesInfo.txt ](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/output/20250218102900/DevicesInfo.txt)
* [Config.ini](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/Python/Config.ini)
**Camera Configuration**
[camera_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/config/camera_params.yaml)(Camera startup parameter settings)
[multi_camera_synced_verify.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multi_camera_synced_verify.launch.py)
@@ -1,161 +1,161 @@
# 多相机同步验证工具 # 多相机同步验证节点
> 本文将介绍如何验证多相机同步的同步精度。 文件路径:[image_sync_example_node.cpp](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/multi_camera_time_sync/image_sync_example_node.cpp)
>
> 首先,请查看如何使用 [multi_camera_synced](./multi_camera_synced.md)。
您可以在 [example](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples) 中找到示例使用代码。 此示例节点用于 **4 台 Orbbec 相机** 的同步采集与时间戳验证。
它可用于验证多相机主从同步(Primary / Secondary Synced)模式下的帧对齐精度。
## 目录结构 ---
## 使用教程
1. **在 `multi_camera_synced.launch.py` 的基础上做以下修改**
- 增加 `launch_include` 以适配 4 台相机
- 根据设备型号选择对应的 launch 文件,如 330 系列选择 gemini_330_series.launch.py
- 命名规范:camera_name 命名方式 camera_01、camera_02、camera_03 ...
- 设置 device_num 数量为 4
- USB 端口配置 usb_port,使用以下命令查看并绑定正确的端口
`ros2 run orbbec_camera list_devices_node`
- 同步模式设置为主从模式,一个 primary 相机,其他设置为 secondary_synced
- 启动顺序:主相机(Primary)应最后启动,以确保同步信号建立正确。
---
2. **参数文件修改**
修改 `config` 目录下的以下两个配置文件:
- `camera_params.yaml`
- `camera_secondary_params.yaml`
确保以下内容统一:
- 启用 **depth** 与 **color** 流;
- 统一各相机的 **帧率(fps)**。
---
3. **启动与验证**
- 启动多机同步 launch: `multi_camera_synced.launch.py`
- 打开新终端运行同步验证节点: `ros2 run orbbec_camera image_sync_example_node`
该节点会输出多相机图像的时间戳差异信息,用于验证同步效果。
---
4. **参考 launch 文件**
```plaintext
.
├── multicamera_sync/
│ ├── output/
│ ├──20250218102900/
│ ├──DevicesInfo.txt
│ ├──StreamProfileInfo.txt
│ ├── Python/
│ ├──Config.ini
├── gemini_330_series_synced_verify.launch.py
├── multi_camera_synced_verify.launch.py
``` ```
import os
from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch_ros.actions import Node
from launch.actions import IncludeLaunchDescription, GroupAction, TimerAction
from launch.launch_description_sources import PythonLaunchDescriptionSource
from launch_ros.actions import Node, LoadComposableNodes
**multicamera_sync**
* `gemini_330_series_synced_verify.launch.py`:单相机运行启动文件,为 multi_camera_synced_verify.launch.py 提供相机运行节点。
* `multi_camera_synced_verify.launch.py`:多相机同步 + save_rgbir 工具的启动文件。
**output** def generate_launch_description():
# Include launch files
package_dir = get_package_share_directory("orbbec_camera")
launch_file_dir = os.path.join(package_dir, "launch")
config_file_dir = os.path.join(package_dir, "config")
config_file_path = os.path.join(config_file_dir, "camera_params.yaml")
secondary_config_file_path = os.path.join(config_file_dir, "camera_secondary_params.yaml")
> `ouput` 文件夹是相机图片输出的文件夹 launch1_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
),
launch_arguments={
"camera_name": "camera_01",
"usb_port": "2-2.3",
"device_num": "4",
"sync_mode": "primary",
"config_file_path": config_file_path,
"trigger_out_enabled": "true"
}.items(),
)
在提供的输出示例中 launch2_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
),
launch_arguments={
"camera_name": "camera_02",
"usb_port": "2-1",
"device_num": "4",
"sync_mode": "secondary_synced",
"config_file_path": secondary_config_file_path,
"trigger_out_enabled": "false"
}.items(),
)
* `20250218102900`:表示在 2025 年 2 月 18 日 10:29:00 收集的相机图像信息。 launch3_include = IncludeLaunchDescription(
* `TotalModeFrames`:相机图像信息存储目录。 PythonLaunchDescriptionSource(
* `results-2025-02-25_163648`:分析和匹配 TotalModeFrames 图像信息后的结果。 os.path.join(launch_file_dir, "gemini_330_series.launch.py")
* `DevicesInfo.txt`:相机设备基本信息(需要修改)。 ),
* `StreamProfileInfo.txt`:相机视频流信息(无需修改)。 launch_arguments={
"camera_name": "camera_03",
"usb_port": "2-3",
"device_num": "4",
"sync_mode": "secondary_synced",
"config_file_path": secondary_config_file_path,
"trigger_out_enabled": "false"
}.items(),
)
**Python** launch4_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
),
launch_arguments={
"camera_name": "camera_04",
"usb_port": "2-4",
"device_num": "4",
"sync_mode": "secondary_synced",
"config_file_path": secondary_config_file_path,
"trigger_out_enabled": "false"
}.items(),
)
* `Config.ini`:Python 分析脚本的配置文件(需要修改)。
## 运行准备
**save_rgbir 节点** # Launch description
编辑 multi_camera_synced_verify.launch.py 并填写激活的相机设备,我们可以看到 save_rgbir 在最后启动。
> 重要
>
> 如果您使用 ROS 2 Foxy,需要先启动 save_rgbir 节点。
```python
# Launch description
ld = LaunchDescription( ld = LaunchDescription(
[ [
shared_orbbec_container,
TimerAction(period=0.0, actions=[GroupAction([launch2_include])]), TimerAction(period=0.0, actions=[GroupAction([launch2_include])]),
TimerAction(period=2.0, actions=[GroupAction([launch1_include])]), TimerAction(period=2.0, actions=[GroupAction([launch3_include])]),
TimerAction(period=4.0, actions=[GroupAction([launch4_include])]),
TimerAction(period=6.0, actions=[GroupAction([launch1_include])]),
# The primary camera should be launched at last # The primary camera should be launched at last
TimerAction(period=6.0, actions=[GroupAction([save_rgbir])]),
# save_rgbir is synced verification tool
] ]
) )
return ld return ld
``` ```
save_rgbir 是一个用于保存图像的工具。该工具的配置文件在 [multi_save_rgbir_params.json](../../config/tools/multisavergbir/multi_save_rgbir_params.json) 中。 ---
```json ## 必要系统配置
{
"save_rgbir_params": { 1. **提升 USB 缓冲区容量**
"time_domain": "global",
"usb_ports": [ ```
"2-1", echo 512 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb
"2-3"
],
"camera_name": [
"camera_01",
"camera_02"
]
}
}
``` ```
* `time_domain`:时间戳类型 2. **配置 Fast DDS**
* `usb_ports`:"primary"、"secondary 1"、"secondary 2"、"secondary 3",根据相机数量填写 usb_ports
* `camera_name`:相机设置的名称,例如:camera_01
**DevicesInfo.txt** 优化 ROS2 节点间通信延迟,可以有效减少图像传输延迟,[配置过程见此章节](../performance/fastdds_tuning.md)。
编辑 DevicesInfo.txt。只需要更改 `primarySerialNumber`、`index` 和 `serialNumber` 参数。其他参数不需要更改。请参考 [20250218102900](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900) 的示例。
* `primarySerialNumber`:主相机的 SN 序列号
* `index`:相机索引
* `serialNumber`:相机的 SN 序列号
**Config.ini**
编辑 Config.ini。修改 `frameRate` 和 `tspRangeThreshold`。
## 运行此示例
**运行启动文件并保存相机图片**
* 第一个终端
```bash
ros2 launch orbbec_camera multi_camera_synced_verify.launch.py
```
* 第二个终端
```bash
ros2 service call /save_rgbir/start_capture orbbec_camera_msgs/srv/SetInt32 '{data: 100}'
```
当终端显示"over"时,图像已保存。将在工作空间下生成一个新的 multicamera_sync 文件夹。
**相机图片命名格式**
以 [color_SNCP1E5420006D_Index0_g1739874543227_f0_s1739874543327_e50_d16_.jpg](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900/TotalModeFrames/SNCP1E5420006D_Index0/color_SNCP1E5420006D_Index0_g1739874543227_f0_s1739874543327_e50_d16_.jpg) 为例
* `color`:此图像是彩色流的快照。
* `SNCP1E5420006D`:相机的 SN 序列号是 CP1E5420006D。
* `Index0`:相机 0(通常指主相机)。
* `g1739874543227`:"g"表示全局时间戳,意味着此帧的全局时间戳是 1739874543227。
* `f0`:此相机彩色流采集的第 0 张图片。
* `s1739874543327`:"s"表示当前系统的时间戳,意味着此帧的当前系统时间戳是 1739874543327。
* `e50`:此帧的曝光为 50。
* `d16`:此帧的增益为 16。
**分析相机图像数据**
您需要将修改后的 [Python 文件夹](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/Python) 复制到新的 multi_camera_synced 子目录,并将修改后的 [DevicesInfo.txt](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900/DevicesInfo.txt) 和 [StreamProfileInfo.txt](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/output/20250218102900/StreamProfileInfo.txt) 复制到与 TotalModeFrames 文件夹同级的目录。
* 最后一切准备就绪,运行 python 脚本
```bash
cd multicamera_sync/Python
python3 SyncFramesMain.py
```
操作成功后,您可以在 `results 文件夹` 中查看同步效果
## 需要更改的文件
**分析工具**
* [multi_save_rgbir_params.json](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/config/tools/multisavergbir/multi_save_rgbir_params.json)
* [DevicesInfo.txt](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/output/20250218102900/DevicesInfo.txt)
* [Config.ini](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/Python/Config.ini)
**相机配置**
[camera_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/config/camera_params.yaml)(相机启动参数设置)
[multi_camera_synced_verify.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multi_camera_synced_verify.launch.py)