diff --git a/README.MD b/README.MD index 642d6a5b..3653c9d0 100644 --- a/README.MD +++ b/README.MD @@ -23,18 +23,18 @@ Here is the device support list of main branch (v1.x) and v2-main branch (v2.x): Gemini 330 Gemini 335 - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Gemini 336 - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Gemini 330 - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Gemini 335L @@ -43,77 +43,77 @@ Here is the device support list of main branch (v1.x) and v2-main branch (v2.x): Gemini 336L - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Gemini 330L - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Gemini 335Lg - not supported - recommended for new designs + Not supported + Recommended for new designs Gemini 335Le - not supported - recommended for new designs + Not supported + Recommended for new designs Gemini 2 Gemini 2 - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Gemini 2 L - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Gemini 2 XL - recommended for new designs - to be supported + Recommended for new designs + To be supported Femto Femto Bolt - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Femto Mega - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Femto Mega I - full maintenance - to be supported + Full maintenance + To be supported Astra Astra 2 - full maintenance - recommended for new designs + Full maintenance + Recommended for new designs Astra+ - limited maintenance - not supported + Limited maintenance + Not supported Astra Pro Plus - limited maintenance - not supported + Limited maintenance + Not supported Astra Mini Astra Mini Pro - full maintenance - not supported + Full maintenance + Not supported @@ -122,11 +122,11 @@ Here is the device support list of main branch (v1.x) and v2-main branch (v2.x): **Definition**: -1. recommended for new designs: we will provide full supports with new features, bug fix and performance optimization; -2. full maintenance: we will provide bug fix support; -3. limited maintenance: we will provide critical bug fix support; -4. not supported: we will not support specific device in this version; -5. to be supported: we will add support in the near future. +1. Recommended for new designs: we will provide full supports with new features, bug fix and performance optimization; +2. Full maintenance: we will provide bug fix support; +3. Limited maintenance: we will provide critical bug fix support; +4. Not supported: we will not support specific device in this version; +5. To be supported: we will add support in the near future. **Documentation Note:** @@ -199,6 +199,7 @@ cd ~/ros2_ws/src git clone https://github.com/orbbec/OrbbecSDK_ROS2.git cd OrbbecSDK_ROS2 git checkout v2-main +git branch #Check whether the branch switch is successful ``` Install deb dependencies @@ -226,7 +227,7 @@ Currently, the following devices are supported by the OrbbecSDK ROS2 Wrapper v2- For optimal performance, we strongly recommend updating to the latest firmware version. This ensures that you benefit from the most recent enhancements and bug fixes. -| Product List | Minimal Firmware Version | **Launch File** | +| Product List | Minimal Firmware Version | **Launch File** | | :----------- | :----------------------- | :-------------------------- | | Gemini 330 | 1.2.20 | gemini_330_series.launch.py | | Gemini 330L | 1.2.20 | gemini_330_series.launch.py | @@ -259,6 +260,7 @@ Launch camera node ```bash . ./install/setup.bash +ros2 run orbbec_camera list_devices_node #Check if the camera is connected ros2 launch orbbec_camera gemini_330_series.launch.py # or other launch file, see below table ``` @@ -441,7 +443,6 @@ When you have multiple cameras, you can activate all of them at the same time. For more information about Multi camera.Reference:[Multi camera documentation](./orbbec_camera/examples/multi_camera/multi_camera.MD) - ### Check which profiles the camera supports ```bash @@ -476,6 +477,12 @@ For the Femto Mega and Femto Bolt devices setting NFOV and WFOV modes.Reference: Note: The V4L2 backend is not enabled by default. +* Example: + +```bash +ros2 launch orbbec_camera gemini_330_series.launch.py uvc_backend:=v4l2 +``` + ### DDS Tuning The default DDS settings (Galactic) may not be optimal for data transmission. Different DDS settings can have varying diff --git a/docs/all_available_topics.md b/docs/all_available_topics.md index 87c67160..c3155312 100644 --- a/docs/all_available_topics.md +++ b/docs/all_available_topics.md @@ -7,7 +7,7 @@ - `/camera/color/camera_info` : The color camera info. - `/camera/color/image_raw`: The color stream image. - `/camera/color/metadata`: The color stream firmware data. -- `/camera/depth/camera_info`: The depth stream image. +- `/camera/depth/camera_info`: The depth camera info. - `/camera/depth/image_raw`: The depth stream image. - `/camera/depth/metadata`:The depth stream firmware data. - `/camera/depth/points` : The point cloud, only available when `enable_point_cloud` is `true`. diff --git a/docs/depth_work_mode_switch.md b/docs/depth_work_mode_switch.md index 253ff6a6..bdd82bb8 100644 --- a/docs/depth_work_mode_switch.md +++ b/docs/depth_work_mode_switch.md @@ -24,3 +24,9 @@ DeclareLaunchArgument('depth_work_mode', default_value='') ```bash ros2 run orbbec_camera list_depth_work_mode_node ``` + +* Example: + +```bash +ros2 launch orbbec_camera gemini2L.launch.py depth_work_mode:="Unbinned Dense Default" +``` diff --git a/orbbec_camera/examples/lower_cpu_usage/README.MD b/orbbec_camera/examples/lower_cpu_usage/README.MD index 37afe8ad..08b77a69 100644 --- a/orbbec_camera/examples/lower_cpu_usage/README.MD +++ b/orbbec_camera/examples/lower_cpu_usage/README.MD @@ -1,99 +1,86 @@ -## Using the Orbbec ROS package to get lower CPU usage +## Reducing CPU Usage with Orbbec ROS Package -This section shows you how to reduce CPU usage for one or more cameras in a ROS 2 environment. This method only works with Gemini 330 series cameras, and the firmware version needs to be above 1.4.10(`device_preset `is set to Default). +This document outlines strategies for minimizing CPU usage in the **OrbbecSDK_ROS2 v2** environment when using **Gemini 330 series cameras**. The firmware version must be **no lower than 1.4.10**, and `device` should be set to **Default**. -The example launch files used in this section are `gemini_330_series_lower_cpu_usage.launch.py` and `multi_camera_lower_cpu_usage.launch.py`. +### Recommended Settings for Lower CPU Usage -### Parameters that affect CPU usage +To achieve the lowest possible CPU usage in OrbbecSDK_ROS2, it is recommended to configure the following parameters. -* `uvc_backend`:Global UVC Backend select on Linux,optional values: libuvc, v4l2.(v4l2 is lower) +| Parameter | Recommendation | Note | +| :--------------: | :------------------------------------: | :--------------------------------------------: | +| `uvc_backend` | `v4l2` | Lower CPU usage compared to `libuvc` | +| `color_format` | `RGB` | Lower CPU usage than `MJPG` | +| `filter` | Only `hardware_noise_removal_filter` | Other filters significantly increase CPU usage | -* `color_format`:Color Stream coding format.(RGB is lower) +### Launch Files Used for Testing -* `depth_registration`: Enables alignment of the depth frame to the color frame.(align_mode is set to HW lower) +* `gemini_330_series_lower_cpu_usage.launch.py` +* `multi_camera_lower_cpu_usage.launch.py` -* `enable_point_cloud`:Disables the point cloud +### Test environment -* `enable_colored_point_cloud`:Disables the RGB point cloud +#### Hardware Configuration -* `filter`:decimation_filter,hdr_merge,sequenced_filter,threshold_filter,hardware_noise_removal_filter,noise_removal_filter,spatial_filter,temporal_filter,hole_filling_filter.(hardware_noise_removal_filter is lower) +* **CPU**: Intel i7-8700 @ 3.20GHz +* **Memory**: 24 GB +* **Storage**: Micron 2200S NVMe 256GB +* **GPU**: NVIDIA GeForce GTX 1660Ti +* **OS**: Ubuntu22.04 -### Data comparison +#### ROS Configuration -> Since different machines and different software environments will result in different CPU usage, the CPU improvement rate is calculated based on the device's CPU benchmark. -> A comparing B: -> `CPU improvement rate=（CPUA-CPUB）/CPUB*100` +* **ROS Version**: ROS2 Humble +* **SDK Version**: OrbbecSDK_ROS2 v2.2.1 -In this example, we used two different video stream resolutions as test benchmarks to provide a performance comparison between lower and higher resolutions.(Currently, data is only provided for `uvc_backend`, `color_format`, and some `filter`) +#### Camera Setup -#### Depth&Left_ir&Right_ir 424 * 266 15fps,Color 848 * 480 15fps +* Devices: 2x Gemini 335, 1x Gemini 336, 1x Gemini 336L +* Firmware Version: 1.4.10 -##### uvc_backend +### Test Setup -libuvc comparing v4l2 In the same environment, CPU improvement rate is ***13.1%*** +* **Stream Settings:** + * Depth / IR Left / IR Right: 848×480 @ 30fps + * Color: 848×480 @ 30fps -* libuvc is ***72.3%*** -* v4l2 is ***62.8%*** +Note: The following CPU usage data focuses on `uvc_backend`, `color_format` and various filter combinations. -##### color_format +### Test Results -ROS2 color stream sets MJPG encoding format and decodes it to RGB comparing RGB encoding format (including YUYV format to RGB conversion operation cost).The CPU improvement rate under the libuvc protocol is ***14.2%*** the CPU improvement rate under the v4l2 protocol is ***30.8%*** +### 1. `uvc_backend` Comparison (RGB format) -* libuvc:RGB encoding format is ***63.3%***，MJPG encoding format and decoded into RGB is ***72.3%*** -* v4l2:RGB encoding format is ***48%***，MJPG encoding format and decoded into RGB is ***62.8%*** +| libuvc CPU Usage | v4l2 CPU Usage | Absolute Change | +| :--------------: | :------------: | :-------------: | +| 182.8% | 118.8% | -64.0% | -##### filter +The CPU usage can be significantly reduced with v4l2 backend. In our implementation, v4l2 works without requiring any patches to the Linux kernel, allowing users to easily switch between v4l2 and libuvc and maintaining full compatibility with standard Linux distributions. -The color stream setting RGB encoding and MJPG have no effect on filtering and are basically the same. The CPU usage of the depth stream is basically the same under the configuration of 424*266 15fps. +### 2. `color_format` Comparison (MJPG vs RGB) -| Filter Configuration | libuvc(CPU improvement rate) | v4l2(CPU improvement rate) | -| :------------------------------------------: | :--------------------------: | :------------------------: | -| No filter | 63.3%(0%) | 48.0%(0%) | -| hardware_noise_removal_filter | 61.2%(-3.3%) | 47.9%(-0.2%) | -| hardware_noise_removal_filter+spatial_filter | 65.3%(3.2%) | 51.2%(6.7%) | -| noise_removal_filter | 77.0%(21.6%) | 53.3%(11.0%) | -| noise_removal_filter+spatial_filter | 78.4%(23.9%) | 54.8%(14.2%) | +| Backend | MJPG CPU Usage | RGB CPU Usage | Absolute Change | +| :-----: | :------------: | :-----------: | :-------------: | +| libuvc | 347.7% | 182.8% | -164.9% | +| v4l2 | 170.0% | 118.8% | -51.2% | -#### Depth&Left_ir&Right_ir 848 * 480 30fps,Color 848 * 480 30fps +The CPU usage can be reduced if the RGB format is selected instead of MJPG, since the decoding of MJPG image will consume the host CPU resource. -##### uvc_backend +### 3. Filter Configuration Impact -libuvc comparing v4l2 In the same environment, CPU improvement rate is ***53.9%*** +| Filters Applied | libuvc CPU Usage | CPU Usage Increase | v4l2 CPU Usage | CPU Usage Increase | +| ----------------------------------------------------- | ---------------- | ------------------ | -------------- | ------------------ | +| No Filter (benchmark) | 182.8% | 0.0%(benchmark) | 118.8% | 0.0%(benchmark) | +| `(software)noise_removal_filter` | 218.0% | +35.2% | 128.5% | +9.7% | +| `(software)noise_removal_filter + spatial_filter` | 469.6% | +286.8% | 336.7% | +217.9% | +| `hardware_noise_removal_filter` | 186.3% | +3.5% | 115.4% | -3.4% | +| `hardware_noise_removal_filter + spatial_filter` | 251.3% | +68.5% | 152.5% | +33.7% | -* libuvc is ***182.8%*** -* v4l2 is ***118.8%*** +Based on the test results, using only the `hardware_noise_removal_filter` results in a negligible change in CPU usage for both `libuvc` (+3.5%) and `v4l2` (-3.4%) compared to the no-filter benchmark, as this filter runs internally on the camera hardware. In contrast, other filters execute on the host system. Adding the `spatial_filter` to the hardware filter leads to a moderate increase in CPU usage, while applying the software-based `noise_removal_filter` —either alone or combined with `spatial_filter` —significantly increases CPU load. To maintain low CPU usage, it is recommended to avoid software-based filters and rely solely on the `hardware_noise_removal_filter`. -##### color_format +## Further Optimization -ROS2 color stream sets MJPG encoding format and decodes it to RGB comparing RGB encoding format (including YUYV format to RGB conversion operation cost).The CPU improvement rate under the libuvc protocol is ***90.2%*** the CPU improvement rate under the v4l2 protocol is ***43.1%*** +| Parameter | Recommendation | Note | +| :----------------------------: | :----------------------------------------------: | :---------------------------------------------: | +| `depth_registration` | `false` or `true` with `align_mode=HW` | Software alignment consumes more CPU | +| `enable_point_cloud` | `false` | Disabling point cloud reduces CPU usage | +| `enable_colored_point_cloud` | `false` | Disabling colored point cloud reduces CPU usage | -* libuvc:RGB encoding format is ***182.8%***，MJPG encoding format and decoded into RGB is ***347.7%*** -* v4l2:RGB encoding format is ***118.8%***，MJPG encoding format and decoded into RGB is ***170.0%*** - -##### filter - -When the depth stream is configured at 848*480 30fps, the CPU usage is significantly different. - -| Filter Configuration | libuvc(CPU improvement rate) | v4l2(CPU improvement rate) | -| :------------------------------------------: | :--------------------------: | :------------------------: | -| No filter | 182.8%(0%) | 118.8%(0%) | -| hardware_noise_removal_filter | 186.3%(1.9%) | 115.4%(-2.9%) | -| hardware_noise_removal_filter+spatial_filter | 251.3%(37.5%) | 152.5%(28.4%) | -| noise_removal_filter | 218.0%(16.1%) | 128.5%(8.2%) | -| noise_removal_filter+spatial_filter | 469.6%(156.9%) | 336.7%(183.4%) | - -### Example launch - -This is the current example file for running the camera to achieve the lowest CPU usage, excluding the impact of filters. - -Use the following command to start the single-camera configuration: - -```bash -ros2 launch orbbec_camera gemini_330_series_lower_cpu_usage.launch.py -``` - -Use the following command to start the multi-camera configuration: - -```bash -roslaunch orbbec_camera multi_camera_lower_cpu_usage.launch.py -``` diff --git a/orbbec_camera/examples/multi_camera/multi_camera.MD b/orbbec_camera/examples/multi_camera/multi_camera.MD index 479ea44f..a6d514fd 100644 --- a/orbbec_camera/examples/multi_camera/multi_camera.MD +++ b/orbbec_camera/examples/multi_camera/multi_camera.MD @@ -1,4 +1,3 @@ - # Multi-Camera - To get the `usb_port` of the camera, plug in the camera and run the following command in the terminal: @@ -64,3 +63,29 @@ def generate_launch_description(): ```bash ros2 launch orbbec_camera multi_camera.launch.py ``` + +## No Data Stream from Multiple Cameras + +**Insufficient Power Supply**: + +- Ensure that each camera is connected to a separate hub. +- Use a powered hub to provide sufficient power to each camera. + +**High Resolution**: + +- Try lowering the resolution to resolve data stream issues. + +**Increase usbfs_memory_mb Value**: + +- Increase the `usbfs_memory_mb` value to 128MB (this is a reference value and can be adjusted based on your system’s needs) + by running the following command: + +```bash + echo 128 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb +``` + +- To make this change permanent, check [this link](https://github.com/OpenKinect/libfreenect2/issues/807). + +## Image topic frame rate too low from Multiple Cameras + +Refer to the [Fast DDS Configuration](./docs/fastdds_tuning.md) file. diff --git a/orbbec_camera/examples/multi_camera_synced_verification_tool/README.MD b/orbbec_camera/examples/multi_camera_synced_verification_tool/README.MD index 3cd38dbf..8a65dc7b 100644 --- a/orbbec_camera/examples/multi_camera_synced_verification_tool/README.MD +++ b/orbbec_camera/examples/multi_camera_synced_verification_tool/README.MD @@ -46,6 +46,10 @@ In the output example provided Edit multi_camera_synced_verify.launch.py and fill in the activated camera device,we can find that save_rgbir is started at the end. +> IMPORTANT +> +> If you are using ROS 2 Foxy, you need to launch the save_rgbir node first. + ```python # Launch description ld = LaunchDescription( @@ -138,7 +142,7 @@ cd multicamera_sync/Python python3 SyncFramesMain.py ``` -After the operation is successful, you can view the synchronization effect in the r `esults folder` +After the operation is successful, you can view the synchronization effect in the `results folder` ## Files that need to be changed @@ -153,3 +157,4 @@ After the operation is successful, you can view the synchronization effect in th [camera_params.yaml](../../config/camera_params.yaml)(Camera startup parameter settings) [multi_camera_synced_verify.launch.py](./multi_camera_synced_verify.launch.py) + diff --git a/orbbec_camera/examples/start_camera_node/README.MD b/orbbec_camera/examples/start_camera_node/README.MD index 4bf49c09..9350c5b0 100644 --- a/orbbec_camera/examples/start_camera_node/README.MD +++ b/orbbec_camera/examples/start_camera_node/README.MD @@ -7,7 +7,7 @@ This guide provides instructions on how to launch the camera node with a colored To start the camera node, execute the following command in your terminal: ```bash -ros2 launch orbbec_camera gemini_330_series.launch.py enable_colored_point_cloud:=true +ros2 launch orbbec_camera gemini_330_series.launch.py enable_colored_point_cloud:=true enable_d2c_viewer:=true enable_right_ir:=true enable_left_ir:=true enable_sync_output_accel_gyro:=true enable_accel:=true enable_gyro:=true ``` This command initiates the camera node and enables the colored point cloud.