mirror of
https://github.com/orbbec/OrbbecSDK_ROS2.git
synced 2026-10-04 03:57:46 +08:00
Add Chinese documentation for camera and lidar devices
This commit is contained in:
ob-yalian
@@ -0,0 +1,43 @@
|
||||
Advanced Guide
|
||||
======================================================
|
||||
|
||||
This chapter covers advanced features of the SDK, including multi-camera usage, and special configuration modes.
|
||||
|
||||
|
||||
|
||||
Performance & Optimization
|
||||
------------------------------------------------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
performance/lower_cpu_usage.md
|
||||
performance/efficient_intra_process_communication.md
|
||||
performance/fastdds_tuning.md
|
||||
|
||||
|
||||
Multi-Camera
|
||||
------------------------------------------------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
multi_camera/multi_camera.md
|
||||
multi_camera/multi_camera_synced.md
|
||||
multi_camera/multi_camera_synced_verification_tool.md
|
||||
multi_camera/gmsl_camera.md
|
||||
|
||||
|
||||
Configuration & Modes
|
||||
------------------------------------------------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
configuration/align_depth_color.md
|
||||
configuration/configuration_of_depth_NFOV_and_WFOV_modes.md
|
||||
configuration/depth_work_mode_switch.md
|
||||
configuration/disparity_search_offset.md
|
||||
configuration/interleave_ae_mode.md
|
||||
configuration/predefined_presets.md
|
||||
configuration/net_camera.md
|
||||
@@ -0,0 +1,39 @@
|
||||
## Aligning Depth to Color in ROS 2
|
||||
|
||||
This section explains how to align depth images with color images to create an overlay image using ROS 2. This is particularly useful for applications requiring synchronized visual information from different sensor modalities.
|
||||
|
||||
### Commands to Align and View Depth and Color Images
|
||||
|
||||
1. **Basic Depth to Color Alignment:**
|
||||
|
||||
To simply align the depth image to the color image, use the following command:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true
|
||||
```
|
||||
|
||||
This command activates the depth registration feature without opening a viewer.
|
||||
2. **Viewing Depth to Color Overlay:**
|
||||
|
||||
If you wish to view the depth to color overlay, you need to enable the viewer by using the command below:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true enable_d2c_viewer:=true
|
||||
```
|
||||
|
||||
This launches the camera node with depth to color registration and opens a viewer to display the overlay image.
|
||||
|
||||
### Selecting Topics in RViz2
|
||||
|
||||
To visualize the aligned images in RViz2:
|
||||
|
||||
1. Launch RViz2 after running one of the above commands.
|
||||
2. Select the topic for the depth to color overlay image. An example topic selection is shown here:
|
||||
|
||||
|
||||
|
||||
### Example of Depth to Color Overlay
|
||||
|
||||
After selecting the appropriate topic in RViz2, you will be able to see the depth to color overlay image. Here's what it might look like:
|
||||
|
||||
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
# Configuration of depth NFOV and WFOV modes
|
||||
|
||||
For the Femto Mega and Femto Bolt devices, the NFOV and WFOV modes are implemented by configuring the resolution of
|
||||
Depth and IR in the launch file.
|
||||
In launch file, depth_width, depth_height, ir_width, ir_height represents the resolution of the depth and the resolution of
|
||||
the IR.
|
||||
The frame fps and resolution of IR must be consistent with the depth. The correspondence between different modes and
|
||||
resolutions is as follows:
|
||||
|
||||
- NFOV unbinned: 640 x 576.
|
||||
- NFOV binned: 320 x 288.
|
||||
- WFOV unbinned: 1024 x 1024.
|
||||
- WFOV binned: 512 x 512.
|
||||
+32
@@ -0,0 +1,32 @@
|
||||
# Depth work mode switch
|
||||
|
||||
Orbbec SDK ROS 2 supports the depth work mode switch. The depth work mode switch is supported by Gemini 2, Gemini 2 L,
|
||||
and Femto and Femto Bolt cameras.
|
||||
|
||||
- Before starting the camera, depth work mode (depth_work_mode) can be configured for the corresponding xxx.launch.py
|
||||
file's support.
|
||||
- The depth work mode switch is supported by Gemini 2, Gemini 2 L, and Gemini 2 XL cameras.
|
||||
- The default depth work mode configuration of xxx.launch.py is the camera's default configuration. If you need to
|
||||
modify it, you can switch to the corresponding mode as needed.
|
||||
- The specific camera depth work mode support types can be found in the comments of the depth mode.
|
||||
|
||||
```python
|
||||
# Depth work mode support is as follows:
|
||||
# Unbinned Dense Default
|
||||
# Unbinned Sparse Default
|
||||
# Binned Sparse Default
|
||||
# Obstacle Avoidance
|
||||
DeclareLaunchArgument('depth_work_mode', default_value='')
|
||||
```
|
||||
|
||||
- View depth work modes:
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_depth_work_mode_node
|
||||
```
|
||||
|
||||
* Example:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini2L.launch.py depth_work_mode:="Unbinned Dense Default"
|
||||
```
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Disparity_search_offset
|
||||
|
||||
> This section describes how to use the disparity_search_offset function in the Gemini330 series cameras (minimum camera firmware version [1.4.60](https://www.orbbec.com/docs/g330-firmware-release/)).Disparity_search_offset is effective only for1280×720, 1280×800 and 640×400 resolutions of depth stream.
|
||||
|
||||
## Function Introduction
|
||||
|
||||
The definition of disparity search range: For any pixel *(u_l, v)* in the left image, by default, the corresponding disparity search range in the right image is *[ (u_l - 255, v)*, *(u_l, v) ]*, where the disparity search length is 256 and the maximum integer disparity is 255. If the starting point of the search is adjusted to *[ (u_l - 255 - offset, v)*, *(u_l - offset, v) ]*, the offset is defined as the disparity shift. Therefore, our disparity search range configuration includes both the disparity search length and the search position offset (which can also be referred to as the disparity shift).
|
||||
|
||||
|
||||
|
||||
## Parameter Introduction
|
||||
|
||||
The disparity_search_offset related parameters are set in [gemini_330_series.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/launch/gemini_330_series.launch.py)
|
||||
|
||||
* `disparity_range_mode` : Disparity search length,can only be set to 64, 128 and 256.
|
||||
|
||||
* `disparity_search_offset` : Disparity search offset value,Disparity search offset value, can be set from 0 to 127.
|
||||
|
||||
* `disparity_offset_config` : Disparity search offset interleave frames.
|
||||
|
||||
* `offset_index0` : Frame 0 disparity search offset value.
|
||||
|
||||
* `offset_index1` : Frame 1 disparity search offset value.
|
||||
|
||||
| disparity range mode | disparity search offset | Minimum depth of inclined wall (mm) |
|
||||
| :------------------: | :---------------------: | :---------------------------------: |
|
||||
| 64 | 85 | Gemini 335L 388-406 |
|
||||
| 64 | 127 | Gemini 335L 302-317 |
|
||||
|
||||
| disparity range mode | disparity search offset | Minimum depth of inclined wall (mm) |
|
||||
| :------------------: | :---------------------: | :---------------------------------------------: |
|
||||
| 128 | 0 | Gemini 335 233-249<br />Gemini 335L 453-475 |
|
||||
| 128 | 45 | Gemini 335 172-184<br />Gemini 335L 334-349 |
|
||||
| 128 | 127 | Gemini 335 117-125<br />Gemini 335L 226-236 |
|
||||
|
||||
| disparity range mode | disparity search offset | Minimum depth of inclined wall (mm) |
|
||||
| :------------------: | :---------------------: | :---------------------------------: |
|
||||
| 256 | 85 | Gemini 335L 169-178 |
|
||||
| 256 | 127 | Gemini 335L 151-158 |
|
||||
|
||||
## Run the launch
|
||||
|
||||
Setting the disparity_search_offset parameter,`colcon build` again and run launch
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py
|
||||
```
|
||||
@@ -0,0 +1,81 @@
|
||||
# Using interleave_ae with Gemini330 series cameras
|
||||
|
||||
> This section describes how to use interleave_ae in Gemini 330 series cameras (minimum camera firmware version [1.4.00](https://www.orbbec.com/docs/g330-firmware-release/))
|
||||
|
||||
## Parameter Introduction
|
||||
|
||||
The interleave_ae related parameters are set in [gemini_330_series.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/launch/gemini_330_series.launch.py)
|
||||
|
||||
* `interleave_ae_mode` : Set laser or hdr interleave.
|
||||
|
||||
* `interleave_frame_enable` : enable interleave frame mode.
|
||||
|
||||
* `interleave_skip_enable` : enable skip frame mode.
|
||||
|
||||
* `interleave_skip_index` : Set 0 for skip pattern ir, set 1 for skip flood ir.
|
||||
|
||||
**interleave hdr**
|
||||
|
||||
When the `interleave_ae_mode` parameter is set to `hdr` and `interleave_frame_enable `is set to `true`, interleave hdr will be enabled
|
||||
|
||||
* `hdr_index1_laser_control` : Frame 1 laser switch settings.
|
||||
|
||||
* `hdr_index1_depth_exposure` : Frame 1 depth exposure value setting, not in AE mode.
|
||||
|
||||
* `hdr_index1_depth_gain` : Frame 1 depth gain value setting, not in AE mode.
|
||||
|
||||
* `hdr_index1_ir_brightness` : Frame 1 ir gain value setting.
|
||||
|
||||
* `hdr_index1_ir_ae_max_exposure` : Frame 1 ir maximum exposure value setting in AE (auto exposure).
|
||||
|
||||
* `hdr_index0_laser_control`: Frame 0 laser switch settings.
|
||||
|
||||
* `hdr_index0_depth_exposure`: Frame 0 depth exposure value setting, not in AE mode.
|
||||
|
||||
* `hdr_index0_depth_gain` : Frame 0 depth gain value setting, not in AE mode.
|
||||
|
||||
* `hdr_index0_ir_brightness` : Frame 0 ir gain value setting.
|
||||
|
||||
* `hdr_index0_ir_ae_max_exposure` : Frame 0 ir maximum exposure value setting in AE (auto exposure).
|
||||
|
||||
**interleave laser**
|
||||
|
||||
When the `interleave_ae_mode` parameter is set to `laser` and `interleave_frame_enable `is set to `true`, interleave laser will be enabled
|
||||
|
||||
* `laser_index1_laser_control` : Frame 1 laser switch settings.
|
||||
|
||||
* `laser_index1_depth_exposure` : Frame 1 depth exposure value setting, not in AE mode.
|
||||
|
||||
* `laser_index1_depth_gain` : Frame 1 depth gain value setting, not in AE mode.
|
||||
|
||||
* `laser_index1_ir_brightness` : Frame 1 ir gain value setting.
|
||||
|
||||
* `laser_index1_ir_ae_max_exposure` : Frame 1 ir maximum exposure value setting in AE (auto exposure).
|
||||
|
||||
* `laser_index0_laser_control` : Frame 0 laser switch settings.
|
||||
|
||||
* `laser_index0_depth_exposure` : Frame 0 depth exposure value setting, not in AE mode.
|
||||
|
||||
* `laser_index0_depth_gain` : Frame 0 depth gain value setting, not in AE mode.
|
||||
|
||||
* `laser_index0_ir_brightness` : Frame 0 ir gain value setting.
|
||||
|
||||
* `laser_index0_ir_ae_max_exposure` : Frame 0 ir maximum exposure value setting in AE (auto exposure).
|
||||
|
||||
## Run the launch
|
||||
|
||||
Setting the interleave_ae parameter,`colcon build` again and run launch
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py
|
||||
```
|
||||
|
||||
**Example Visualization**
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Multi_camera_synced + Interleave_ae
|
||||
|
||||
Please refer to [multi_camera_synced](./multi_camera_synced.md) and [Parameter Introduction](#parameter-introduction)
|
||||
@@ -0,0 +1,101 @@
|
||||
# Net_camera
|
||||
|
||||
> This section describes how to use Net camera in OrbbecSDK_ROS2.Currently, only Femto_Mega, Gemini 335Le and Gemini 435Le devices are supported, and other Net devices will be supported in the near future.
|
||||
|
||||
You can find example usage code in the [example](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples).
|
||||
|
||||
## Femto Mega & Gemini 435Le & Gemini 335Le
|
||||
|
||||
**Parameter Introduction**
|
||||
|
||||
Network device settings: `enumerate_net_device` is set to true, which will automatically enumerate network devices.
|
||||
|
||||
If you do not want to automatically enumerate network devices,you can set `enumerate_net_device` to false, `net_device_ip` to the device's IP address, and `net_device_port` to the default value of 8090.
|
||||
|
||||
* `enumerate_net_device` : Enable automatically enumerate network devices.
|
||||
* `net_device_ip` : Setting net device's IP address.
|
||||
* `net_device_port` : Setting net device's port.Usually, you can set it to 8090.
|
||||
|
||||
**Single Net camera**
|
||||
|
||||
> If you need to run Gemini 435Le/Gemini 335Le, you only need to replace [femto_mega.launch.py ](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/launch/femto_mega.launch.py)in the run command with [gemini435_le.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/launch/gemini435_le.launch.py)/[gemini_330_series.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/launch/gemini_330_series.launch.py)
|
||||
|
||||
For [femto_mega.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/launch/femto_mega.launch.py) as an example:
|
||||
|
||||
- **automatically enumerate network devices:**
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera femto_mega.launch.py enumerate_net_device:=true
|
||||
```
|
||||
|
||||
- **Specify IP address to start the device:**
|
||||
|
||||
Note: `net_device_ip` needs to be changed to the IP address of the device, here it is 192.168.1.10
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera femto_mega.launch.py enumerate_net_device:=false net_device_ip:=192.168.1.10 net_device_port:=8090
|
||||
```
|
||||
|
||||
**Multi Net camera**
|
||||
|
||||
For [multi_net_camera.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/net_camera/multi_net_camera.launch.py) as an example:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_net_camera.launch.py
|
||||
```
|
||||
|
||||
## set_device_ip Utility
|
||||
|
||||
The **`set_device_ip`** executable allows you to configure the IP settings of a network camera directly from ROS 2, including switching between DHCP and static IP, and setting subnet mask and gateway. This is useful for quickly assigning or updating IP addresses without modifying launch files.
|
||||
|
||||
> **Note:** The IP settings applied with `set_device_ip` are **permanent** and **will not be reset** if the device is powered off or restarted.
|
||||
|
||||
**Example Usage**
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera set_device_ip --ros-args \
|
||||
-p old_ip:=192.168.1.10 \
|
||||
-p dhcp:=false \
|
||||
-p new_ip:=192.168.1.11 \
|
||||
-p mask:=255.255.255.0 \
|
||||
-p gateway:=192.168.1.1
|
||||
```
|
||||
|
||||
**Parameters**
|
||||
|
||||
- **`old_ip`** – Current IP address of the device.
|
||||
- **`dhcp`** – Set to `true` to use DHCP or `false` for static IP.
|
||||
- **`new_ip`** – Static IP address to assign when DHCP is disabled.
|
||||
- **`mask`** – Subnet mask for the new IP.
|
||||
- **`gateway`** – Gateway address for the new IP.
|
||||
|
||||
## Force IP Function
|
||||
|
||||
The **Force IP** feature allows you to assign a **static IP address** to a network camera, overriding DHCP settings. This is useful when multiple network cameras are connected, and you need each device to have a fixed IP for reliable communication.
|
||||
|
||||
> **Note:** The Force IP configuration **will be reset if the device is powered off or restarted**. You need to reapply the settings after reboot.
|
||||
|
||||
**Parameters**
|
||||
|
||||
- **`force_ip_enable`** – Enable the Force IP function. **Default:** `false`
|
||||
- **`force_ip_mac`** – Target device MAC address when multiple cameras are connected (e.g., `"54:14:FD:06:07:DA"`). You can use the `list_devices_node` to find the MAC of each device. **Default:** `""`
|
||||
- **`force_ip_address`** – Static IP address to assign . **Default:** `192.168.1.10`
|
||||
- **`force_ip_subnet_mask`** – Subnet mask for the static IP. **Default:** `255.255.255.0`
|
||||
- **`force_ip_gateway`** – Gateway address for the static IP. **Default:** `192.168.1.1`
|
||||
|
||||
**Example Usage**
|
||||
|
||||
- **Enable Force IP for a specific device:**
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py \
|
||||
force_ip_enable:=true \
|
||||
force_ip_mac:=54:14:FD:06:07:DA \
|
||||
force_ip_address:=192.168.1.50 \
|
||||
force_ip_subnet_mask:=255.255.255.0 \
|
||||
force_ip_gateway:=192.168.1.1 \
|
||||
net_device_ip:=192.168.1.50 \
|
||||
net_device_port:=8090
|
||||
```
|
||||
|
||||
> Tip: Make sure the camera is connected and its MAC address is correct before enabling Force IP. Use `list_devices_node` to check the MAC address of all connected cameras.
|
||||
@@ -0,0 +1,15 @@
|
||||
# Predefined presets
|
||||
|
||||
| Preset | Features | Recommended use cases |
|
||||
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Default | - Best visual perception``- Overall good performance in accuracy, fill rate, tiny objects, etc. | - Generic `<br>`- Robotics |
|
||||
| Hand | - Clear hand and finger edges | - Gesture recognition |
|
||||
| High Accuracy | - Depth of high confidence `<br>`- Barely noise depth values `<br>`- Lower fill rate | - Collision avoidance `<br>`- Object scanning |
|
||||
| High Density | - Higher fill rate `<br>`- More tiny objects `<br>`- May suffer from noise depth values | - Object recognition `<br>`- Pick & place `<br>`- Foreground & background animation |
|
||||
| Medium Density | - Balanced performance in fill rate and accuracy `<br>`- In comparison to Default: lower fill rate, better edge quality | - Generic and alternative to Default |
|
||||
| Custom | - User defined Preset `<br>`- Derived from Presets above, with customized modifications, e.g. a new configuration for the post-processing pipeline, modified mean intensity set point of depth AE function, etc. | - Better depth performance achieved using customized configurations in comparison to using predefined presets `<br>`- For well-established custom configurations |
|
||||
|
||||
Choose the appropriate preset name based on your specific use case and set it as the value for the `device_preset`
|
||||
parameter.
|
||||
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
# GMSL_camera
|
||||
|
||||
> This section describes how to use GMSL camera in OrbbecSDK_ROS2.Currently, only Gemini 335Lg and Gemini 345Lg, other GMSL devices will be supported in the near future.
|
||||
|
||||
You can find example usage code in the [example](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples).
|
||||
|
||||
## Single GMSL camera
|
||||
|
||||
The usage of GMSL camera in OrbbecSDK_ROS2 is the same as that of Gemini 330 series camera via USB.
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_gmsl.launch.py
|
||||
```
|
||||
|
||||
## Multi GMSL camera
|
||||
|
||||
To get the `usb_port` of the GMSL camera, plug in the camera and run the following command in the terminal:
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_devices_node
|
||||
```
|
||||
|
||||
For example, the obtained gmsl camera `usb_port`: `gmsl2-1`
|
||||
|
||||
Go to the [multi_gmsl_camera.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/gmsl_camera/multi_gmsl_camera.launch.py) file and change the `usb_port`.
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_gmsl_camera.launch.py
|
||||
```
|
||||
|
||||
> Note: By default, multi_gmsl_camera.launch.py only starts color and depth. If you want to start other sensors, please go to [camera_secondary_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/camera_secondary_params.yaml) to modify them.
|
||||
|
||||
## Multi GMSL camera synced
|
||||
|
||||
First, please see how to use [multi_camera_synced](./multi_camera_synced.md).
|
||||
|
||||
In addition, GMSL multi-camera synced does not require Multi-Camera Sync Hub Pro, so there is no need to set the `primary` mode. Each GMSL camera is `secondary`.
|
||||
|
||||
**Additional Parameter Settings**
|
||||
|
||||
* `gmsl_trigger_fps` : set hardware soc trigger source frame rate.
|
||||
* `enable_gmsl_trigger` : enable hardware soc trigger.
|
||||
|
||||
**Run the launch**
|
||||
|
||||
Please refer to the configuration in [multi_gmsl_camera_synced.launch.py.](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/gmsl_camera/multi_gmsl_camera_synced.launch.py)
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_gmsl_camera_synced.launch.py
|
||||
```
|
||||
|
||||
> Note: By default, multi_gmsl_camera_synced.launch.py only starts color and depth. If you want to start other sensors, please go to [camera_secondary_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/camera_secondary_params.yaml) and [camera_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/camera_params.yaml) to modify them.
|
||||
|
||||
## Usage Limitations of GMSL Cameras
|
||||
|
||||
[](https://github.com/orbbec/MIPI_Camera_Platform_Driver/tree/main?tab=readme-ov-file#usage-limitations-of-gmsl-cameras)
|
||||
|
||||
GMSL cameras interface with various deserializer chips such as MAX9296 and MAX92716. Orbbec GMSL cameras support multiple streams including depth, color, IR, and IMU data, but certain usage limitations apply:
|
||||
|
||||
- GMSL only supports V4L2 and YUYV format; MJPG format is not supported. RGB output is derived from YUYV format conversion.
|
||||
- Metadata for Gemini-335Lg is provided via a separate node, while metadata for other models is embedded within video frames, which remains transparent to users.
|
||||
- When using the Max96712 as a deserializer chip, due to the characteristics of the Max96712 chip, a multi - machine synchronous trigger signal must be provided in the secondary_synced mode. Otherwise, data flow interruption will occur when switching the data stream
|
||||
- Two cameras connected on the same MAX9296, MAX96712 LinkA/B, or MAX96712 LinkC/D have the following limitations:
|
||||
- Before driver version v1.2.02, there was a restriction that the RGB of one camera and the right IR of another camera could not stream simultaneously. After driver version v1.2.02, the restriction was modified to that the RGB of one camera and the left IR of another camera cannot stream simultaneously.
|
||||
- Before driver version v1.2.02, there was a restriction that the DEPTH of one camera and the left IR of another camera could not stream simultaneously. After driver version v1.2.02, the restriction was modified to that the DEPTH of one camera and the right IR of another camera cannot stream simultaneously.
|
||||
- The combined maximum number of active streams from both cameras is limited to four (satisfying the above two conditions ensures compliance).
|
||||
|
||||
For further known limitations, please refer to [Usage Limitations of Orbbec GMSL Cameras](https://github.com/orbbec/MIPI_Camera_Platform_Driver/blob/main/doc/Instructions%20for%20Using%20GMSL%20Camera.md)
|
||||
@@ -0,0 +1,91 @@
|
||||
# Multi-Camera
|
||||
|
||||
- To get the `usb_port` of the camera, plug in the camera and run the following command in the terminal:
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_devices_node
|
||||
```
|
||||
|
||||
- Set the `device_num` parameter to the number of cameras you have.
|
||||
- Go to the `OrbbecSDK_ROS2/launch/multi_xxx.launch.py` file and change the `usb_port`.
|
||||
- Don't forget to put the `include` tag inside the `group` tag.
|
||||
Otherwise, the parameter values of different cameras may become contaminated.
|
||||
|
||||
```python
|
||||
from launch import LaunchDescription
|
||||
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription, GroupAction, ExecuteProcess
|
||||
from launch.launch_description_sources import PythonLaunchDescriptionSource
|
||||
from launch_ros.actions import Node
|
||||
from ament_index_python.packages import get_package_share_directory
|
||||
import os
|
||||
|
||||
|
||||
def generate_launch_description():
|
||||
# Include launch files
|
||||
package_dir = get_package_share_directory('orbbec_camera')
|
||||
launch_file_dir = os.path.join(package_dir, 'launch')
|
||||
launch1_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, 'gemini2L.launch.py')
|
||||
),
|
||||
launch_arguments={
|
||||
'camera_name': 'camera_01',
|
||||
'usb_port': '6-2.4.4.2', # replace your usb port here
|
||||
'device_num': '2'
|
||||
}.items()
|
||||
)
|
||||
|
||||
launch2_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, 'gemini2L.launch.py')
|
||||
),
|
||||
launch_arguments={
|
||||
'camera_name': 'camera_02',
|
||||
'usb_port': '6-2.4.1', # replace your usb port here
|
||||
'device_num': '2'
|
||||
}.items()
|
||||
)
|
||||
|
||||
# If you need more cameras, just add more launch_include here, and change the usb_port and device_num
|
||||
|
||||
# Launch description
|
||||
ld = LaunchDescription([
|
||||
GroupAction([launch1_include]),
|
||||
GroupAction([launch2_include]),
|
||||
])
|
||||
|
||||
return ld
|
||||
|
||||
```
|
||||
|
||||
- To launch the cameras, run the following command:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_camera.launch.py
|
||||
```
|
||||
|
||||
## No Data Stream from Multiple Cameras
|
||||
|
||||
**Insufficient Power Supply**:
|
||||
|
||||
- Ensure that each camera is connected to a separate hub.
|
||||
- Use a powered hub to provide sufficient power to each camera.
|
||||
|
||||
**High Resolution**:
|
||||
|
||||
- Try lowering the resolution to resolve data stream issues.
|
||||
|
||||
**Increase usbfs_memory_mb Value**:
|
||||
|
||||
- Increase the `usbfs_memory_mb` value to 128MB (this is a reference value and can be adjusted based on your system’s needs)
|
||||
by running the following command:
|
||||
|
||||
```bash
|
||||
echo 128 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb
|
||||
```
|
||||
|
||||
- To make this change permanent, check [this link](https://github.com/OpenKinect/libfreenect2/issues/807).
|
||||
|
||||
## Image topic frame rate too low from Multiple Cameras
|
||||
|
||||
Refer to the [Fast DDS Configuration](../performance/fastdds_tuning.md) file.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Multi_camera synced Instructions
|
||||
|
||||
> The purpose of this document is to explain how to use multi-camera synced with OrbbecSDK_ROS2
|
||||
|
||||
### Setup instructions
|
||||
|
||||
* Please read the Multi-Camera Synchronization Setup Guide:[Multi-Camera Synchronization Setup](https://www.orbbec.com/docs/set-up-cameras-for-external-synchronization_v1-2/)
|
||||
* Make sure the camera is correctly connected to the multi-camera synchronizer.
|
||||
|
||||
|
||||
|
||||
### Checking camera port with OrbbecSDK_ROS2
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_devices_node
|
||||
```
|
||||
|
||||
### OrbbecSDK_ROS2 multi-camera synced configuration
|
||||
|
||||
Open multi_camera_synced.launch.py, and configure the camera settings as shown below:
|
||||
|
||||
|
||||
|
||||
1. `gemini_330_series.launch.py` is the launch file for starting the camera.
|
||||
|
||||
2. Set `camera_name` to `G330_0`. For example, the published color image topic will be `/G330_0/color/image_raw`.
|
||||
|
||||
3. Set `usb_port` to `2-2`, indicating that the camera device on port `2-2` is being used. This value can be found in the output of the `ros2 run orbbec_camera list_devices_node` command.
|
||||
|
||||
4. Set `device_num` to `2`, meaning two cameras will be used.
|
||||
|
||||
5. Set `sync_mode` to `primary` to indicate that the `2-7` camera device is in primary mode. The multi-camera sync mode options can be found in the figure below.
|
||||
|
||||
6. Parameters from the `config_file_path` can override the parameters set in `gemini_330_series.launch.py` (optional).
|
||||
|
||||
7. For slave cameras, set `trigger_out_enabled` to false.
|
||||
|
||||
| **Pattern Nam**e | **Setting effect description** |
|
||||
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| free_run | -Support different frame rate settings<br />The -8-pin synchronization interface does not support external output of synchronization-related signals |
|
||||
| standalone(default) | ● Same as Primary by default<br />● Built-in RGBD frame synchronization<br />● 8-pin synchronous interface does not output signals to the outside by default |
|
||||
| primary | ● Set as primary camera<br />● 8-pin synchronous interface output signal to external device |
|
||||
| secondary | ● Set as secondary (passive synchronization; When there is a hardware continuous trigger signal input from the outside and the continuous trigger signal matches the currently set frame rate, the image is collected according to the external trigger signal; When there is no external trigger signal, the flow is stopped)<br />● 8-pin synchronous interface output signal to external device |
|
||||
| secondary_synced | ● Set to secondary synchronization (passive synchronization; When there is a hardware continuous trigger signal input from the outside and the continuous trigger signal matches the currently set frame rate, the image is collected according to the external trigger signal; When there is no external trigger signal, the image is collected according to the internal trigger signal at the set frame rate)<br />● 8-pin synchronous interface output signal to external device |
|
||||
| hardware_triggering | ● Set as hardware trigger (passive trigger; When there is a hardware trigger signal input from the outside and the trigger signal time interval is not less than the current upper limit, the image is collected according to the external trigger signal; When there is no external trigger signal, the image is not collected)<br />● 8-pin synchronous interface output signal to external device |
|
||||
| software_triggering | ● Set as software trigger (passive trigger; When there is a trigger command input from the host computer and the trigger command time interval is not less than the current upper limit, the image is collected according to the trigger command; When there is no trigger command, the image is not collected)<br />● 8-pin synchronous interface output signal to external device |
|
||||
|
||||
* The master camera should be launched last.
|
||||
* Ideally, there should be a 2-second delay between starting each camera.
|
||||
|
||||
### Run the following command to start the multi-camera synced
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_camera_synced.launch.py
|
||||
```
|
||||
+177
@@ -0,0 +1,177 @@
|
||||
# Multi-Camera Synchronization Verification Node
|
||||
|
||||
**File path:** [image_sync_example_node.cpp](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/multi_camera_time_sync/image_sync_example_node.cpp)
|
||||
|
||||
This example node is designed for **synchronized capture and timestamp verification** across **four Orbbec cameras**.
|
||||
It can be used to validate frame alignment accuracy under the multi-camera **Primary / Secondary Synced** mode.
|
||||
|
||||
---
|
||||
|
||||
## Usage Guide
|
||||
|
||||
### **Modify `multi_camera_synced.launch.py` as follows**
|
||||
|
||||
- Add `launch_include` to support four cameras.
|
||||
|
||||
- Select the appropriate launch file based on your camera model.
|
||||
For example, use `gemini_330_series.launch.py` for the Gemini 330 series.
|
||||
|
||||
- Naming convention:
|
||||
`camera_name` should follow the format `camera_01`, `camera_02`, `camera_03`, ...
|
||||
|
||||
- Set `device_num` to **4**.
|
||||
|
||||
- Configure the USB ports using:
|
||||
|
||||
`ros2 run orbbec_camera list_devices_node`
|
||||
|
||||
to view and bind the correct ports.
|
||||
|
||||
- Set synchronization mode to **Primary/Secondary Synced**,
|
||||
with one camera as **primary** and the others as **secondary_synced**.
|
||||
|
||||
- **Startup sequence:**
|
||||
The **Primary camera should always be launched last** to ensure the synchronization signal is established correctly.
|
||||
|
||||
---
|
||||
|
||||
### **Modify configuration files**
|
||||
|
||||
Edit the following two files under the `config` directory:
|
||||
|
||||
- `camera_params.yaml`
|
||||
|
||||
- `camera_secondary_params.yaml`
|
||||
|
||||
Ensure the following settings are consistent across all cameras:
|
||||
|
||||
- Enable both **depth** and **color** streams.
|
||||
|
||||
- Use the same **frame rate (fps)** for all cameras.
|
||||
|
||||
---
|
||||
|
||||
### **Launch and Verification**
|
||||
|
||||
- Start the multi-camera synchronization launch file:
|
||||
|
||||
`ros2 launch orbbec_camera multi_camera_synced.launch.py`
|
||||
|
||||
- In a new terminal, run the synchronization verification node:
|
||||
|
||||
`ros2 run orbbec_camera image_sync_example_node`
|
||||
|
||||
This node outputs timestamp differences between multiple camera streams for synchronization validation.
|
||||
|
||||
---
|
||||
|
||||
### **Reference Launch File**
|
||||
|
||||
```
|
||||
import os
|
||||
from ament_index_python.packages import get_package_share_directory
|
||||
from launch import LaunchDescription
|
||||
from launch_ros.actions import Node
|
||||
from launch.actions import IncludeLaunchDescription, GroupAction, TimerAction
|
||||
from launch.launch_description_sources import PythonLaunchDescriptionSource
|
||||
from launch_ros.actions import Node, LoadComposableNodes
|
||||
|
||||
|
||||
|
||||
def generate_launch_description():
|
||||
# Include launch files
|
||||
package_dir = get_package_share_directory("orbbec_camera")
|
||||
launch_file_dir = os.path.join(package_dir, "launch")
|
||||
config_file_dir = os.path.join(package_dir, "config")
|
||||
config_file_path = os.path.join(config_file_dir, "camera_params.yaml")
|
||||
secondary_config_file_path = os.path.join(config_file_dir, "camera_secondary_params.yaml")
|
||||
|
||||
launch1_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
|
||||
),
|
||||
launch_arguments={
|
||||
"camera_name": "camera_01",
|
||||
"usb_port": "2-2.3",
|
||||
"device_num": "4",
|
||||
"sync_mode": "primary",
|
||||
"config_file_path": config_file_path,
|
||||
"trigger_out_enabled": "true"
|
||||
}.items(),
|
||||
)
|
||||
|
||||
launch2_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
|
||||
),
|
||||
launch_arguments={
|
||||
"camera_name": "camera_02",
|
||||
"usb_port": "2-1",
|
||||
"device_num": "4",
|
||||
"sync_mode": "secondary_synced",
|
||||
"config_file_path": secondary_config_file_path,
|
||||
"trigger_out_enabled": "false"
|
||||
}.items(),
|
||||
)
|
||||
|
||||
launch3_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
|
||||
),
|
||||
launch_arguments={
|
||||
"camera_name": "camera_03",
|
||||
"usb_port": "2-3",
|
||||
"device_num": "4",
|
||||
"sync_mode": "secondary_synced",
|
||||
"config_file_path": secondary_config_file_path,
|
||||
"trigger_out_enabled": "false"
|
||||
}.items(),
|
||||
)
|
||||
|
||||
launch4_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, "gemini_330_series.launch.py")
|
||||
),
|
||||
launch_arguments={
|
||||
"camera_name": "camera_04",
|
||||
"usb_port": "2-4",
|
||||
"device_num": "4",
|
||||
"sync_mode": "secondary_synced",
|
||||
"config_file_path": secondary_config_file_path,
|
||||
"trigger_out_enabled": "false"
|
||||
}.items(),
|
||||
)
|
||||
|
||||
|
||||
|
||||
# Launch description
|
||||
ld = LaunchDescription(
|
||||
[
|
||||
|
||||
TimerAction(period=0.0, actions=[GroupAction([launch2_include])]),
|
||||
TimerAction(period=2.0, actions=[GroupAction([launch3_include])]),
|
||||
TimerAction(period=4.0, actions=[GroupAction([launch4_include])]),
|
||||
TimerAction(period=6.0, actions=[GroupAction([launch1_include])]),
|
||||
# The primary camera should be launched at last
|
||||
]
|
||||
)
|
||||
|
||||
return ld
|
||||
```
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## System Configuration Requirements
|
||||
|
||||
### **Increase USB Buffer Memory**
|
||||
|
||||
Prevent frame drops caused by concurrent data transmission:
|
||||
|
||||
`echo 512 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb`
|
||||
|
||||
### **Configure Fast DDS**
|
||||
|
||||
Optimize ROS2 node communication latency to reduce image transmission delay.
|
||||
See [this section](../performance/fastdds_tuning.md) for detailed configuration instructions.
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
# Efficient intra-process communication:
|
||||
|
||||
### Introduction
|
||||
|
||||
Our ROS2 Wrapper node supports zero-copy communications if loaded in the same process as a subscriber node. This can reduce copy times on image/pointcloud topics, especially with big frame resolutions and high FPS.
|
||||
|
||||
You will need to launch a component container and launch our node as a component together with other component nodes. Further details on "Composing multiple nodes in a single process" can be found [here](https://docs.ros.org/en/rolling/Tutorials/Composition.html).
|
||||
|
||||
Further details on efficient intra-process communication can be found [here](https://docs.ros.org/en/humble/Tutorials/Intra-Process-Communication.html#efficient-intra-process-communication).
|
||||
|
||||
### Example
|
||||
|
||||
**Manually loading multiple components into the same process**
|
||||
|
||||
* Start the component:
|
||||
|
||||
```bash
|
||||
ros2 run rclcpp_components component_container
|
||||
```
|
||||
|
||||
* Add the wrapper:
|
||||
|
||||
```bash
|
||||
ros2 component load /ComponentManager orbbec_camera orbbec_camera::OBCameraNodeDriver -e use_intra_process_comms:=true
|
||||
```
|
||||
|
||||
Load other component nodes (consumers of the wrapper topics) in the same way.
|
||||
|
||||
**Using a launch file**
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_intra_process_demo_launch.py
|
||||
```
|
||||
|
||||
**Limitations**
|
||||
|
||||
* Node components are currently not supported on RCLPY
|
||||
|
||||
* Compressed images using `image_transport` will be disabled as this isn't supported with intra-process communication
|
||||
@@ -0,0 +1,154 @@
|
||||
# Fast DDS Optimization for Orbbec Camera with ROS2
|
||||
|
||||
When operating with the default configuration, Fast DDS exhibits suboptimal transmission efficiency, resulting in
|
||||
significant image transmission delays when used with the Orbbec camera in ROS2. This document provides guidance on
|
||||
optimizing Fast DDS to enhance image transfer efficiency.
|
||||
|
||||
## Adjusting System Parameters
|
||||
|
||||
**IP Fragmentation Time**
|
||||
|
||||
- **Path**: `/proc/sys/net/ipv4/ipfrag_time` (default: 30 seconds)
|
||||
- **Purpose**: Defines the duration that IP fragments are kept in memory.
|
||||
- **Adjustment**: Decrease this value to reduce the time window where no fragments are received, which can help reduce
|
||||
delays. Consider the specific needs of your environment as this setting affects all incoming fragments.
|
||||
|
||||
**Example**: Set to 3 seconds.
|
||||
|
||||
```bash
|
||||
sudo sysctl net.ipv4.ipfrag_time=3
|
||||
```
|
||||
|
||||
**IP Fragmentation Memory Threshold**
|
||||
|
||||
- **Path**: `/proc/sys/net/ipv4/ipfrag_high_thresh` (default: 262144 bytes)
|
||||
- **Purpose**: Sets the maximum memory used to reassemble IP fragments.
|
||||
- **Adjustment**: Increase this value to allow more memory for fragment reassembly, which can improve handling of larger
|
||||
data packets.
|
||||
|
||||
**Example**: Increase to 128 MB.
|
||||
|
||||
```bash
|
||||
sudo sysctl net.ipv4.ipfrag_high_thresh=134217728
|
||||
```
|
||||
|
||||
**Maximum Buffer Sizes**
|
||||
|
||||
- **Purpose**: Configures the maximum buffer sizes for receiving and sending data, which is critical for high-throughput
|
||||
data transmission.
|
||||
- **Adjustment**: Set the maximum buffer sizes for both receiving and sending operations.
|
||||
|
||||
**Commands**:
|
||||
|
||||
```bash
|
||||
sudo sysctl -w net.core.rmem_max=2147483647
|
||||
sudo sysctl -w net.core.rmem_default=2147483647
|
||||
sudo sysctl -w net.core.wmem_max=2147483647
|
||||
sudo sysctl -w net.core.wmem_default=2147483647
|
||||
```
|
||||
|
||||
Alternatively, make these settings permanent by adding them to the `/etc/sysctl.d/10-fastrtps-max.conf` file.
|
||||
|
||||
```bash
|
||||
sudo gedit /etc/sysctl.d/10-fastrtps-max.conf
|
||||
```
|
||||
|
||||
add blow lines to the file:
|
||||
|
||||
```bash
|
||||
net.core.rmem_max=2147483647
|
||||
net.core.rmem_default=2147483647
|
||||
net.core.wmem_max=2147483647
|
||||
net.core.wmem_default=2147483647
|
||||
```
|
||||
|
||||
then save and exit the file. run `sudo sysctl -p` to apply the changes.
|
||||
|
||||
For detailed guidance, refer
|
||||
to [ROS 2 DDS Tuning Documentation](https://docs.ros.org/en/foxy/How-To-Guides/DDS-tuning.html).
|
||||
|
||||
## Fast DDS Configuration
|
||||
|
||||
Below is an example of a Fast DDS configuration file optimized for ROS2 usage with the Orbbec camera. This configuration
|
||||
enhances the overall data transmission by adjusting buffer sizes and transport settings.
|
||||
|
||||
**Configuration File:** `shm_fastdds.xml`
|
||||
|
||||
Place this file in the `$HOME` directory.
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
|
||||
<transport_descriptors>
|
||||
<transport_descriptor>
|
||||
<transport_id>UDP_transport</transport_id>
|
||||
<type>UDPv4</type>
|
||||
<maxInitialPeersRange>10</maxInitialPeersRange>
|
||||
<maxMessageSize>65000</maxMessageSize>
|
||||
<sendBufferSize>1048576</sendBufferSize>
|
||||
<receiveBufferSize>1048576</receiveBufferSize>
|
||||
</transport_descriptor>
|
||||
</transport_descriptors>
|
||||
<participant profile_name="participant_profile_ros2" is_default_profile="true">
|
||||
<rtps>
|
||||
<name>profile_for_ros2_context</name>
|
||||
<userTransports>
|
||||
<transport_id>UDP_transport</transport_id>
|
||||
</userTransports>
|
||||
<useBuiltinTransports>false</useBuiltinTransports>
|
||||
<sendSocketBufferSize>1048576</sendSocketBufferSize>
|
||||
<listenSocketBufferSize>1048576</listenSocketBufferSize>
|
||||
<builtin>
|
||||
<initialPeersList>
|
||||
<locator>
|
||||
<udpv4>
|
||||
<address>127.0.0.1</address>
|
||||
</udpv4>
|
||||
</locator>
|
||||
</initialPeersList>
|
||||
</builtin>
|
||||
</rtps>
|
||||
</participant>
|
||||
<data_writer profile_name="default publisher profile" is_default_profile="true">
|
||||
<qos>
|
||||
<publishMode>
|
||||
<kind>ASYNCHRONOUS</kind>
|
||||
</publishMode>
|
||||
<latencyBudget>
|
||||
<duration>
|
||||
<sec>0</sec>
|
||||
<nanosec>1000000</nanosec>
|
||||
</duration>
|
||||
</latencyBudget>
|
||||
</qos>
|
||||
<historyMemoryPolicy>PREALLOCATED_WITH_REALLOC</historyMemoryPolicy>
|
||||
</data_writer>
|
||||
<data_reader profile_name="default subscription profile" is_default_profile="true">
|
||||
<qos>
|
||||
<data_sharing>
|
||||
<kind>AUTOMATIC</kind>
|
||||
</data_sharing>
|
||||
<latencyBudget>
|
||||
<duration>
|
||||
<sec>0</sec>
|
||||
<nanosec>1000000</nanosec>
|
||||
</duration>
|
||||
</latencyBudget>
|
||||
</qos>
|
||||
<historyMemoryPolicy>PREALLOCATED_WITH_REALLOC</historyMemoryPolicy>
|
||||
</data_reader>
|
||||
</profiles>
|
||||
```
|
||||
|
||||
**Environment Variables**
|
||||
|
||||
Set the following environment variables to use the custom Fast DDS profile:
|
||||
|
||||
```bash
|
||||
export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
|
||||
export FASTRTPS_DEFAULT_PROFILES_FILE=$HOME/shm_fastdds.xml
|
||||
export RMW_FASTRTPS_USE_QOS_FROM_XML=1
|
||||
```
|
||||
|
||||
This configuration aims to optimize the data flow and reduce transmission delays, improving the responsiveness and
|
||||
reliability of the Orbbec camera system in a ROS2 environment.
|
||||
@@ -0,0 +1,96 @@
|
||||
## Reducing CPU Usage with Orbbec ROS Package
|
||||
|
||||
You can find example usage code in the [example](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples).
|
||||
|
||||
This document outlines strategies for minimizing CPU usage in the **OrbbecSDK_ROS2 v2** environment when using **Gemini 330 series cameras**. The firmware version must be **no lower than 1.4.10**, and `device` should be set to **Default**.
|
||||
|
||||
### Recommended Settings for Lower CPU Usage
|
||||
|
||||
To achieve the lowest possible CPU usage in OrbbecSDK_ROS2, it is recommended to configure the following parameters.
|
||||
|
||||
| Parameter | Recommendation | Note |
|
||||
| :--------------: | :------------------------------------: | :--------------------------------------------: |
|
||||
| `uvc_backend` | `v4l2` | Lower CPU usage compared to `libuvc` |
|
||||
| `color_format` | `RGB` | Lower CPU usage than `MJPG` |
|
||||
| `filter` | Only `hardware_noise_removal_filter` | Other filters significantly increase CPU usage |
|
||||
|
||||
### Launch Files Used for Testing
|
||||
|
||||
* `gemini_330_series_lower_cpu_usage.launch.py`
|
||||
* `multi_camera_lower_cpu_usage.launch.py`
|
||||
|
||||
### Test environment
|
||||
|
||||
**Hardware Configuration**
|
||||
|
||||
* **CPU**: Intel i7-8700 @ 3.20GHz
|
||||
|
||||
* **Memory**: 24 GB
|
||||
|
||||
* **Storage**: Micron 2200S NVMe 256GB
|
||||
|
||||
* **GPU**: NVIDIA GeForce GTX 1660Ti
|
||||
|
||||
* **OS**: Ubuntu22.04
|
||||
|
||||
**ROS Configuration**
|
||||
|
||||
* **ROS Version**: ROS2 Humble
|
||||
|
||||
* **SDK Version**: OrbbecSDK_ROS2 v2.2.1
|
||||
|
||||
**Camera Setup**
|
||||
|
||||
* Devices: 2x Gemini 335, 1x Gemini 336, 1x Gemini 336L
|
||||
|
||||
* Firmware Version: 1.4.10
|
||||
|
||||
|
||||
### Test Setup
|
||||
|
||||
**Stream Settings:**
|
||||
|
||||
* Depth / IR Left / IR Right: 848×480 @ 30fps
|
||||
* Color: 848×480 @ 30fps
|
||||
|
||||
Note: The following CPU usage data focuses on `uvc_backend`, `color_format` and various filter combinations.
|
||||
|
||||
### Test Results
|
||||
|
||||
**`uvc_backend` Comparison (RGB format)**
|
||||
|
||||
| libuvc CPU Usage | v4l2 CPU Usage | Absolute Change |
|
||||
| :--------------: | :------------: | :-------------: |
|
||||
| 182.8% | 118.8% | -64.0% |
|
||||
|
||||
The CPU usage can be significantly reduced with v4l2 backend. In our implementation, v4l2 works without requiring any patches to the Linux kernel, allowing users to easily switch between v4l2 and libuvc and maintaining full compatibility with standard Linux distributions.
|
||||
|
||||
**`color_format` Comparison (MJPG vs RGB)**
|
||||
|
||||
| Backend | MJPG CPU Usage | RGB CPU Usage | Absolute Change |
|
||||
| :-----: | :------------: | :-----------: | :-------------: |
|
||||
| libuvc | 347.7% | 182.8% | -164.9% |
|
||||
| v4l2 | 170.0% | 118.8% | -51.2% |
|
||||
|
||||
The CPU usage can be reduced if the RGB format is selected instead of MJPG, since the decoding of MJPG image will consume the host CPU resource.
|
||||
|
||||
**Filter Configuration Impact**
|
||||
|
||||
| Filters Applied | libuvc CPU Usage | CPU Usage Increase | v4l2 CPU Usage | CPU Usage Increase |
|
||||
| ----------------------------------------------------- | ---------------- | ------------------ | -------------- | ------------------ |
|
||||
| No Filter (benchmark) | 182.8% | 0.0%(benchmark) | 118.8% | 0.0%(benchmark) |
|
||||
| `(software)noise_removal_filter` | 218.0% | +35.2% | 128.5% | +9.7% |
|
||||
| `(software)noise_removal_filter + spatial_filter` | 469.6% | +286.8% | 336.7% | +217.9% |
|
||||
| `hardware_noise_removal_filter` | 186.3% | +3.5% | 115.4% | -3.4% |
|
||||
| `hardware_noise_removal_filter + spatial_filter` | 251.3% | +68.5% | 152.5% | +33.7% |
|
||||
|
||||
Based on the test results, using only the `hardware_noise_removal_filter` results in a negligible change in CPU usage for both `libuvc` (+3.5%) and `v4l2` (-3.4%) compared to the no-filter benchmark, as this filter runs internally on the camera hardware. In contrast, other filters execute on the host system. Adding the `spatial_filter` to the hardware filter leads to a moderate increase in CPU usage, while applying the software-based `noise_removal_filter` —either alone or combined with `spatial_filter` —significantly increases CPU load. To maintain low CPU usage, it is recommended to avoid software-based filters and rely solely on the `hardware_noise_removal_filter`.
|
||||
|
||||
### Further Optimizationa
|
||||
|
||||
| Parameter | Recommendation | Note |
|
||||
| :----------------------------: | :----------------------------------------------: | :---------------------------------------------: |
|
||||
| `depth_registration` | `false` or `true` with `align_mode=HW` | Software alignment consumes more CPU |
|
||||
| `enable_point_cloud` | `false` | Disabling point cloud reduces CPU usage |
|
||||
| `enable_colored_point_cloud` | `false` | Disabling colored point cloud reduces CPU usage |
|
||||
|
||||
Reference in New Issue
Block a user