mirror of
https://github.com/orbbec/OrbbecSDK_ROS2.git
synced 2026-10-04 12:07:46 +08:00
deploy: orbbec/OrbbecSDK_ROS2@2460077f7c
This commit is contained in:
winf
@@ -0,0 +1,39 @@
|
||||
# Introduction
|
||||
|
||||
OrbbecSDK ROS2 Wrapper provides seamless integration of Orbbec cameras with ROS 2 environment. It supports ROS2 Foxy, Humble, and Jazzy distributions.
|
||||
|
||||
With a major update in October 2024, we release the [OrbbecSDK ROS2 Wrapper v2](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main) connected to the open source [OrbbecSDK v2](https://github.com/orbbec/OrbbecSDK_v2/releases) with enhanced flexibility and extensibility. This update ensures compatibility with all Orbbec USB products adhering to UVC standard. However, it no longer supports Orbbec's traditional OpenNI protocol devices. We strongly encourage you to use the v2-main branch if your device is supported.
|
||||
|
||||
If you are a user in China, it is recommended to use [gitee Repo](https://gitee.com/orbbecdeveloper/OrbbecSDK_ROS2).
|
||||
|
||||
## Support Hardware Products
|
||||
|
||||
The following devices are supported by the OrbbecSDK ROS2 Wrapper v2-main branch. More devices support will be added in the near future. If you can not find your device in the table below, try the [main](https://github.com/orbbec/OrbbecSDK_ROS2) branch.
|
||||
|
||||
For optimal performance, we strongly recommend updating to the latest firmware version. This ensures that you benefit from the most recent enhancements and bug fixes.
|
||||
|
||||
| Product List | Minimal Firmware Version | **Launch File** |
|
||||
| :------------- | :----------------------- | :-------------------------- |
|
||||
| Gemini 435Le | 1.2.04 | gemini435_le.launch.py |
|
||||
| Gemini 335 | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 336 | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 335L | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 336L | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 335Lg | 1.3.46 | gemini_330_series.launch.py |
|
||||
| Gemini 335Le | 1.5.31 | gemini_330_series.launch.py |
|
||||
| Gemini 330 | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 330L | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 2 | 1.4.92 | gemini2.launch.py |
|
||||
| Gemini 2 L | 1.4.53 | gemini2L.launch.py |
|
||||
| Femto Bolt | 1.1.2 | femto_bolt.launch.py |
|
||||
| Femto Mega | 1.3.0 | femto_mega.launch.py |
|
||||
| Astra 2 | 2.8.20 | astra2.launch.py |
|
||||
| Astra Mini Pro | 2.0.01 | astra.launch.py |
|
||||
|
||||
All launch files are essentially similar, with the primary difference being the default values of the parameters set
|
||||
for different models within the same series. Differences in USB standards, such as USB 2.0 versus USB 3.0, may require adjustments to these parameters. If you encounter a startup failure, please carefully review the specification manual. Pay special attention to the resolution settings in the launch file, as well as other parameters, to ensure compatibility and optimal performance.
|
||||
|
||||
## Support Platforms
|
||||
|
||||
- Linux x64: tested on Ubuntu 22.04
|
||||
- Linux ARM64: tested on NVIDIA Jetson AGX Orin , NVIDIA Jetson Orin NX , NVIDIA Jetson Orin Nano , NVIDIA Jetson AGX Xavier , NVIDIA Jetson Xavier NX
|
||||
@@ -0,0 +1,116 @@
|
||||
# Orbbec SDK Overview
|
||||
|
||||
This section introduces the Orbbec SDK in C++. Its architecture and concepts are consistent with those of the Python Wrapper.
|
||||
|
||||
## Terms
|
||||
|
||||
| ID | Name | Explain |
|
||||
| --- | --- | --- |
|
||||
| 1 | USB | Universal Serial Bus |
|
||||
| 2 | UVC | USB Video Class |
|
||||
| 3 | Firmware | Firmware of 3D camera |
|
||||
| 4 | Disparity | Disparity is to observe the direction difference of the same target from two points with a certain distance. |
|
||||
| 5 | D2D (Disparity to depth) | Disparity to depth is an image processing technique used to convert disparity information into depth information. |
|
||||
| 6 | Hardware D2D | Disparity to depth is implemented internally in the device, without occupying the computational power of the host computer. |
|
||||
| 7 | Software D2D | Disparity to depth, implemented in Orbbec SDK |
|
||||
| 8 | Depth point cloud | Depth point cloud, the coordinates of points in a three-dimensional world coordinate system, can be transformed into a point cloud using the intrinsic parameters of a Depth camera. |
|
||||
| 9 | RGBD point cloud | Point cloud with overlaid RGB information |
|
||||
| 10 | D2C | The translation of "Depth to Color" is a feature that performs per-pixel geometric transformation on a depth image. Its result is aligning the depth image with its corresponding color image through the D2C transformation, allowing us to locate the depth information of a color pixel by using the same image coordinate position of that pixel in the transformed depth image. After the D2C transformation, we generate a depth image of the same size as the target color image, where the image content represents depth data in the coordinate system of the color camera. In other words, it reconstructs a depth image "captured" using the origin and dimensions of the color camera, where each pixel matches the corresponding pixel coordinates of the color camera. |
|
||||
| 11 | Hardware D2C | Hardware D2C refers to the functionality of performing Depth to Color transformation within the camera itself, with the camera directly outputting the result of the D2C transformation. |
|
||||
| 12 | Software D2C | Performing D2C computation on the host computer side using an SDK. |
|
||||
| 13 | Frame aggregation (FrameSet) | Combining Depth, IR, and Color frames into a Frameset and invoking it through a pipeline. |
|
||||
| 14 | C2D | The translation of "Color to Depth" is a feature that performs per-pixel geometric transformation on a color image. Its result is aligning the color image with its corresponding depth image through the C2D transformation. |
|
||||
| 15 | MetaData | Frame metadata is a set of parameters (or attributes) that provide a snapshot of the sensor configuration and/or system state present during the frame’s generation. |
|
||||
| 16 | HDR | High Dynamic Range (HDR) imaging allows imaging systems to capture images in extremely dark and bright scenes alike. We propose a software solution running on the host CPU to implement this feature. It utilizes data from two consecutive frames and directly synthesizes these two depth images, thereby enhances the dynamic range of 16-bit depth images. |
|
||||
| 17 | LDP | Laser close-range protection |
|
||||
|
||||
## Orbbec SDK v2 Architecture Overview
|
||||
|
||||
|
||||
|
||||
- Application
|
||||
|
||||
OrbbecViewer, Sample, and User Application Implementation.
|
||||
|
||||
- Interfaces and Encapsulation Layer
|
||||
|
||||
OrbbecSDK Interface Encapsulation and Wrapper Encapsulation.
|
||||
|
||||
- High-level Layer
|
||||
|
||||
HighLevel encapsulates the core business components and provides interfaces to the outside using a pipeline.
|
||||
|
||||
- Basic business layer
|
||||
|
||||
The realization of the core business logic framework.
|
||||
|
||||
- Platform abstraction layer
|
||||
|
||||
Cross-platform components abstract operating system differences and provide a unified access interface.
|
||||
|
||||
|
||||
- Platform implementation layer
|
||||
|
||||
The driver implementation of each platform.
|
||||
|
||||
## SDK Concept Overview
|
||||
|
||||
- Context
|
||||
|
||||
|
||||
Context which provides a set of settings includes settings such as device state change callbacks, log levels, and more. The Context can access multiple devices.
|
||||
|
||||
- Device
|
||||
|
||||
|
||||
One actual hardware device corresponds to one Device object, which is used to obtain relevant information of the device and control its attributes.
|
||||
|
||||
- Pipeline
|
||||
|
||||
|
||||
The HighLevel corresponding object encapsulates the interface for quick access to the SDK. It has simple functions that allow users to quickly get started and use the SDK.
|
||||
|
||||
- Config
|
||||
|
||||
|
||||
Provides configuration for enabling data streams, alignment modes, and frame aggregation modes, It is used to control the behavior of the data output.
|
||||
|
||||
- StreamProfile
|
||||
|
||||
|
||||
Stream configuration that defines parameters such as resolution, frame rate, and encoding format, It also provides management of camera parameters.
|
||||
|
||||
- Frame
|
||||
|
||||
|
||||
Represents a frame of data in the Stream, and also contains relevant information about that frame of data, such as timestamp, type, etc.
|
||||
|
||||
- Filter
|
||||
|
||||
|
||||
It mainly refers to some algorithmic processing modules for the composite stream FrameSet, such as point cloud algorithm processing.
|
||||
|
||||
- Record
|
||||
|
||||
|
||||
Recording functionality that captures data streams and saves them as files for later analysis or playback.
|
||||
|
||||
- Playback
|
||||
|
||||
|
||||
Playback functionality that plays recorded files and supports control over playback speed and other related parameters.
|
||||
|
||||
|
||||
## SDK Programming Model
|
||||
|
||||
Here is the C++ programming logic flow chart. Python's programming logic is the same as it.
|
||||
|
||||
- Standard Flowchart:
|
||||
|
||||
|
||||
|
||||
The standard flowchart demonstrates how to create a device from the device list, set and get parameters, and apply post-processing filters.
|
||||
|
||||
- Flowchart using default configuration (stream acquisition based on the default settings in OrbbecSDKConfig.xml):
|
||||
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
Overview
|
||||
======================================================
|
||||
|
||||
This chapter provides an overview of the Orbbec SDK, including supported products, main features, and architecture.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
introduction.md
|
||||
orbbecsdk_overview.md
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
### Build from Source
|
||||
|
||||
#### Environment
|
||||
|
||||
Install ROS 2 according to the official guide:
|
||||
|
||||
* [ROS 2 installation (Ubuntu)](https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debians.html)
|
||||
|
||||
Enable ROS 2 auto-completion:
|
||||
|
||||
```bash
|
||||
eval "$(register-python-argcomplete3 ros2)"
|
||||
eval "$(register-python-argcomplete3 colcon)"
|
||||
```
|
||||
|
||||
Create a `colcon` workspace:
|
||||
|
||||
```bash
|
||||
mkdir -p ~/ros2_ws/src
|
||||
```
|
||||
|
||||
#### Linux ROS2 Wrapper Compilation
|
||||
|
||||
Clone source and checkout `v2-main` branch:
|
||||
|
||||
```bash
|
||||
cd ~/ros2_ws/src
|
||||
git clone https://github.com/orbbec/OrbbecSDK_ROS2.git
|
||||
cd OrbbecSDK_ROS2
|
||||
git checkout v2-main
|
||||
```
|
||||
|
||||
Install dependencies:
|
||||
|
||||
```bash
|
||||
sudo apt install libgflags-dev nlohmann-json3-dev libgoogle-glog-dev libgoogle-glog0v5 libssl-dev \
|
||||
ros-$ROS_DISTRO-image-transport ros-${ROS_DISTRO}-image-transport-plugins ros-${ROS_DISTRO}-compressed-image-transport \
|
||||
ros-$ROS_DISTRO-image-publisher ros-$ROS_DISTRO-camera-info-manager \
|
||||
ros-$ROS_DISTRO-diagnostic-updater ros-$ROS_DISTRO-diagnostic-msgs ros-$ROS_DISTRO-statistics-msgs \
|
||||
ros-$ROS_DISTRO-backward-ros libdw-dev
|
||||
```
|
||||
|
||||
Build:
|
||||
|
||||
```bash
|
||||
cd ~/ros2_ws
|
||||
colcon build --event-handlers console_direct+ --cmake-args -DCMAKE_BUILD_TYPE=Release
|
||||
```
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
Installation
|
||||
======================================================
|
||||
|
||||
This chapter explains how to install the Orbbec ROS2 Python SDK, including building from source, installing dependencies, and using registration scripts.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
build_the_package.md
|
||||
registration_script.md
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
## Registration script (required)
|
||||
|
||||
To allow the Orbbec cameras to be recognized correctly on Linux, install the udev rules:
|
||||
|
||||
```bash
|
||||
cd ~/ros2_ws/src/OrbbecSDK_ROS2/orbbec_camera/scripts
|
||||
sudo bash install_udev_rules.sh
|
||||
sudo udevadm control --reload-rules && sudo udevadm trigger
|
||||
```
|
||||
|
||||
This step is **mandatory** for Linux users.
|
||||
|
||||
`Notes:` If this script is not executed, open the device will fail due to permission issues. You need to run the sample with sudo (administrator privileges).
|
||||
@@ -0,0 +1,92 @@
|
||||
## QuickStarts
|
||||
|
||||
### Introduction
|
||||
|
||||
This section provides a quick start to using the Orbbec ROS 2 wrapper.
|
||||
You will learn how to:
|
||||
|
||||
* Launch a camera node.
|
||||
* Visualize depth/color streams in **RViz2**.
|
||||
* Interact with topics and services using **ROS 2 CLI tools**.
|
||||
|
||||
---
|
||||
|
||||
### Build your First Camera Application
|
||||
|
||||
#### Step 1: Source ROS 2 and Workspace
|
||||
|
||||
Make sure ROS 2 and your workspace environment are sourced:
|
||||
|
||||
```bash
|
||||
source /opt/ros/$ROS_DISTRO/setup.bash
|
||||
source ~/ros2_ws/install/setup.bash
|
||||
```
|
||||
|
||||
#### Step 2: Launch the Camera Node
|
||||
|
||||
- On terminal 1
|
||||
|
||||
```bash
|
||||
. ./install/setup.bash
|
||||
ros2 run orbbec_camera list_devices_node #Check if the camera is connected
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py # Or other launch file, see below table
|
||||
```
|
||||
|
||||
If you have multiple cameras connected, you can specify the **serial number**:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py serial_number:=<YourCameraSN>
|
||||
```
|
||||
|
||||
#### Step 3: Visualize in RViz2
|
||||
|
||||
Launch RViz2 and load the default config:
|
||||
|
||||
- On terminal 2
|
||||
|
||||
```bash
|
||||
rviz2
|
||||
```
|
||||
|
||||
* Add an **Image** display, set topic to `/camera/color/image_raw`.
|
||||
* Add another **Image** display for `/camera/depth/image_raw`.
|
||||
* Optionally, add a **PointCloud2** display for `/camera/depth/points`.
|
||||
|
||||
You should now see the color stream, depth stream, and 3D point cloud in RViz2.
|
||||
|
||||
---
|
||||
|
||||
### Sample Features
|
||||
|
||||
After the node is running, try some ROS 2 CLI commands:
|
||||
|
||||
#### List available topics / services/ parameters
|
||||
|
||||
```bash
|
||||
ros2 topic list
|
||||
ros2 service list
|
||||
ros2 param list
|
||||
```
|
||||
|
||||
#### Echo a topic
|
||||
|
||||
View depth camera data:
|
||||
|
||||
```bash
|
||||
ros2 topic echo /camera/depth/camera_info
|
||||
```
|
||||
|
||||
#### Call a service
|
||||
|
||||
For example, get device Information:
|
||||
|
||||
```bash
|
||||
ros2 service call /camera/get_device_info orbbec_camera_msgs/srv/GetDeviceInfo '{}'
|
||||
```
|
||||
|
||||
#### Record with rosbag2
|
||||
|
||||
```bash
|
||||
ros2 bag record /camera/color/image_raw /camera/depth/image_raw
|
||||
```
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
Quickstarts
|
||||
======================================================
|
||||
|
||||
This chapter provides quick start guides for the SDK, allowing users to run basic example programs quickly.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
quickstart.md
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
Application Guide
|
||||
======================================================
|
||||
|
||||
This chapter introduces application development with the SDK, including launch parameter configuration, ROS2 services, and topics usage.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
launch_parameters.md
|
||||
services.md
|
||||
topics.md
|
||||
|
||||
@@ -0,0 +1,264 @@
|
||||
# Launch parameters
|
||||
|
||||
> If you are not sure how to set the parameters, you can connect the orbbec camera and open the [OrbbecViewer](https://github.com/orbbec/OrbbecSDK/releases).
|
||||
|
||||
The following are the launch parameters available:
|
||||
|
||||
### Core & Stream Configuration
|
||||
|
||||
* **`camera_name`**
|
||||
* Start the node namespace.
|
||||
|
||||
* **`depth_registration`**
|
||||
* Enable alignment of the depth frame to the color frame. This field is required when the `enable_colored_point_cloud` is set to `true`.
|
||||
|
||||
* **`serial_number`**
|
||||
* The serial number of the camera. This is required when multiple cameras are used.
|
||||
|
||||
* **`usb_port`**
|
||||
* The USB port of the camera. This is required when multiple cameras are used.
|
||||
|
||||
* **`device_num`**
|
||||
* The number of devices. This must be filled in if multiple cameras are required.
|
||||
|
||||
* **`[color|depth|left_ir|right_ir|ir]_[width|height|fps|format]`**
|
||||
* The resolution and frame rate of the sensor stream.
|
||||
|
||||
* **`[color|depth|left_ir|right_ir|ir]_rotation`**
|
||||
* Set stream image rotation.
|
||||
* The possible values are `0`, `90`, `180`, `270`.
|
||||
|
||||
* **`[color|depth|left_ir|right_ir|ir]_flip`**
|
||||
* Enable the stream image flip.
|
||||
|
||||
* **`[color|depth|left_ir|right_ir|ir]_mirror`**
|
||||
* Enable the stream image mirror.
|
||||
|
||||
* **`enable_point_cloud`**
|
||||
* Enable the point cloud.
|
||||
|
||||
* **`enable_colored_point_cloud`**
|
||||
* Enable the RGB point cloud.
|
||||
|
||||
* **`cloud_frame_id`**
|
||||
* Modify the `frame_id` name within the ros message.
|
||||
|
||||
* **`ordered_pc`**
|
||||
* Enable filtering of invalid point clouds.
|
||||
|
||||
* **`align_mode`**
|
||||
* The alignment mode to be used. Options are `HW` for hardware alignment and `SW` for software alignment.
|
||||
|
||||
* **`align_target_stream`**
|
||||
* Set align target stream mode.
|
||||
* The possible values are `COLOR`, `DEPTH`.
|
||||
* `COLOR`: Align depth to color.
|
||||
* `DEPTH`: Align color to depth.
|
||||
|
||||
* **`point_cloud_qos`, `[stream]_qos`, `[stream]_camera_info_qos`**
|
||||
* ROS 2 Message Quality of Service (QoS) settings. The possible values are `SYSTEM_DEFAULT`, `DEFAULT`, `PARAMETER_EVENTS`, `SERVICES_DEFAULT`, `PARAMETERS`, `SENSOR_DATA` and are case-insensitive. These correspond to `rmw_qos_profile_system_default`, `rmw_qos_profile_default`, `rmw_qos_profile_parameter_events`, `rmw_qos_profile_services_default`, `rmw_qos_profile_parameters`, and `SENSOR_DATA`, respectively.
|
||||
|
||||
### Sensor Controls
|
||||
|
||||
#### Color Stream
|
||||
* **`enable_color_auto_exposure`**
|
||||
* Enable the Color auto exposure.
|
||||
* **`enable_color_auto_exposure_priority`**
|
||||
* Enable the Color auto exposure priority.
|
||||
* **`color_exposure`**
|
||||
* Set the Color exposure.
|
||||
* **`color_gain`**
|
||||
* Set the Color gain.
|
||||
* **`enable_color_auto_white_balance`**
|
||||
* Enable the Color auto white balance.
|
||||
* **`color_white_balance`**
|
||||
* Set the Color white balance.
|
||||
* **`color_ae_max_exposure`**
|
||||
* Set the maximum exposure value for Color auto exposure.
|
||||
* **`color_brightness`**, **`color_sharpness`**, **`color_gamma`**, **`color_saturation`**, **`color_contrast`**, **`color_hue`**
|
||||
* Set the Color brightness, sharpness, gamma, saturation, contrast, and hue.
|
||||
* **`enable_color_backlight_compensation`**
|
||||
* Enable the Color backlight compensation.
|
||||
* **`color_powerline_freq`**
|
||||
* Set the power line freq. The possible values are `disable`, `50hz`, `60hz`, `auto`.
|
||||
* **`enable_color_decimation_filter`** / **`color_decimation_filter_scale`**
|
||||
* Enable the Color decimation filter and set its scale.
|
||||
* **`color_ae_roi_[left|right|top|bottom]`**
|
||||
* Set Color auto exposure ROI.
|
||||
|
||||
#### Depth Stream
|
||||
* **`enable_depth_auto_exposure_priority`**
|
||||
* Enable the Depth auto exposure priority.
|
||||
* **`mean_intensity_set_point`**
|
||||
* Set the target mean intensity of the Depth image. For example: `mean_intensity_set_point:=100`.
|
||||
> **Note:** This replaces the deprecated `depth_brightness`, which is still supported for backward compatibility.
|
||||
* **`enable_depth_scale`**
|
||||
* Enable the depth scale.
|
||||
* **`depth_precision`**
|
||||
* The depth precision should be in the format `1mm`. The default value is `1mm`.
|
||||
* **`depth_ae_roi_[left|right|top|bottom]`**
|
||||
* Set Depth auto exposure ROI.
|
||||
|
||||
#### IR Stream
|
||||
* **`enable_ir_auto_exposure`**
|
||||
* Enable the IR auto exposure.
|
||||
* **`ir_exposure`** / **`ir_gain`**
|
||||
* Set the IR exposure and gain.
|
||||
* **`ir_ae_max_exposure`**
|
||||
* Set the maximum exposure value for IR auto exposure.
|
||||
* **`ir_brightness`**
|
||||
* Set the IR brightness.
|
||||
|
||||
#### Laser / LDP
|
||||
* **`enable_laser`**
|
||||
* Enable the laser. The default value is `true`.
|
||||
* **`laser_energy_level`**
|
||||
* Set the laser energy level.
|
||||
* **`enable_ldp`** / **`ldp_power_level`**
|
||||
* Enable the LDP and set its power level.
|
||||
|
||||
### Device, Sync & Advanced Features
|
||||
|
||||
#### Multi-Camera Synchronization
|
||||
* **`sync_mode`**
|
||||
* Set sync mode. The default value is `standalone`.
|
||||
* **`depth_delay_us`** / **`color_delay_us`**
|
||||
* The delay time (microseconds) of the depth/color image capture after receiving the capture command or trigger signal.
|
||||
* **`trigger2image_delay_us`**
|
||||
* The delay time (microseconds) of the image capture after receiving the capture command or trigger signal. Us
|
||||
* **`trigger_out_delay_us`**
|
||||
* The delay time (microseconds) of the trigger signal output after receiving the capture command or trigger signal.
|
||||
* **`trigger_out_enabled`**
|
||||
* Enable the trigger out signal.
|
||||
* **`software_trigger_enabled`** / **`software_trigger_period`**
|
||||
* Enable the software trigger out signal / set the software trigger period in ms.
|
||||
* **`frames_per_trigger`**
|
||||
* The frame number of each stream after each trigger in triggering mode.
|
||||
|
||||
> Used for [multi camera synced](../5_advanced_guide/multi_camera_synced.md).
|
||||
|
||||
#### Network Cameras
|
||||
* **`enumerate_net_device`**
|
||||
* Enable automatically enumerate network devices.
|
||||
* **`net_device_ip`** / **`net_device_port`**
|
||||
* Set net device's IP address and port (Usually `8090`).
|
||||
|
||||
> Used for [net camera](../5_advanced_guide/net_camera.md).
|
||||
|
||||
#### Device-Specific
|
||||
* **`device_preset`**
|
||||
* The default value is `Default`. Only the G330 series is supported. For more information, refer to the [G330 documentation](https://www.orbbec.com/docs/g330-use-depth-presets/). The value should be one of the preset names listed [in the table](../5_advanced_guide/predefined_presets.md).
|
||||
* **`enable_gmsl_trigger`** / **`gmsl_trigger_fps`**
|
||||
* Enable the gmsl trigger out signal / set gmsl trigger fps. Used for [gmsl camera](../5_advanced_guide/gmsl_camera.md).
|
||||
|
||||
#### Disparity
|
||||
* **`disparity_to_depth_mode`**
|
||||
* `HW`: use hardware disparity to depth conversion. `SW`: use software disparity to depth conversion.
|
||||
* **`disparity_range_mode`**, **`disparity_search_offset`**, **`disparity_offset_config`**
|
||||
* Parameters for disparity search offset. Used for [disparity search offset](../5_advanced_guide/disparity_search_offset.md).
|
||||
|
||||
#### Interleave AE Mode
|
||||
* **`interleave_ae_mode`**
|
||||
* Set `laser` or `hdr` interleave.
|
||||
* **`interleave_frame_enable`**, **`interleave_skip_enable`**, **`interleave_skip_index`**
|
||||
* Parameters to control interleave frame mode.
|
||||
* **`[hdr|laser]_index[0|1]_[...]`**
|
||||
* In interleave frame mode, set the 0th and 1st frame parameters of hdr or laser interleaving frames.
|
||||
* *All interleave parameters are used for [interleave ae mode](../5_advanced_guide/interleave_ae_mode.md).*
|
||||
|
||||
### Basic & General Parameters
|
||||
|
||||
#### Firmware & Backend
|
||||
* **`upgrade_firmware`**
|
||||
* The input parameter is the firmware path.
|
||||
* **`preset_firmware_path`**
|
||||
* The input parameter is the preset firmware path. If multiple paths are input, each path needs to be separated by `,` and a maximum of 3 firmware paths can be input.
|
||||
* **`uvc_backend`**
|
||||
* Optional values: `v4l2`, `libuvc`.
|
||||
* **`connection_delay`**
|
||||
* The delay time in milliseconds for reopening the device. Some devices, such as Astra mini, require a longer time to initialize and reopening the device immediately can cause firmware crashes when hot plugging.
|
||||
* **`retry_on_usb3_detection_failure`**
|
||||
* If the camera is connected to a USB 2.0 port and is not detected, the system will attempt to reset the camera up to three times. It is recommended to set this parameter to `false` when using a USB 2.0 connection to avoid unnecessary resets.
|
||||
|
||||
#### TF, Extrinsics & Calibration
|
||||
* **`publish_tf`** / **`tf_publish_rate`**
|
||||
* Enable the TF publish and set its publication rate.
|
||||
* **`enable_publish_extrinsic`**
|
||||
* Enable the extrinsics publish.
|
||||
* **`ir_info_url`** / **`color_info_url`**
|
||||
* Set URL of the IR/color camera info.
|
||||
* **`enable_color_undistortion`**
|
||||
* Enable the Color undistortion.
|
||||
|
||||
#### Time Synchronization
|
||||
* **`enable_sync_host_time`**
|
||||
* Enable synchronization of the host time with the camera time. The default value is `true`. If using global time, set to `false`.
|
||||
* **`time_domain`**
|
||||
* Select timestamp type: `device`, `global`, and `system`.
|
||||
* **`time_sync_period`**
|
||||
* Interval (in seconds) for synchronizing the camera time with the host system.
|
||||
> **Note**: This parameter only needs to be set when **`enable_sync_host_time = true`** and **`time_domain = device`**.
|
||||
* **`enable_ptp_config`**
|
||||
* Enable PTP time synchronization. Only for Gemini 335Le. Requires `enable_sync_host_time` to be `false`.
|
||||
* **`enable_frame_sync`**
|
||||
* Enable the frame synchronization.
|
||||
|
||||
#### Logging & Diagnostics
|
||||
* **`log_level`**
|
||||
* SDK log level. Default is `info`. Optional values: `debug`, `info`, `warn`, `error`, `fatal`.
|
||||
* **`diagnostic_period`**
|
||||
* Diagnostic period in seconds.
|
||||
* **`enable_heartbeat`**
|
||||
* Enable the heartbeat function. Default is `false`. If `true`, the camera node will send heartbeat signals to the firmware.
|
||||
|
||||
#### Miscellaneous
|
||||
* **`config_file_path`**
|
||||
* The path to the YAML configuration file. Default is `""`. If not specified, default parameters from the launch file will be used.
|
||||
* **`frame_aggregate_mode`**
|
||||
* Set frame aggregate output mode. Optional values: `full_frame`, `color_frame`, `ANY`, `disable`.
|
||||
* **`enable_d2c_viewer`**
|
||||
* Publishes the D2C overlay image (for testing only).
|
||||
|
||||
### IMU
|
||||
|
||||
* **`enable_accel`** / **`enable_gyro`**
|
||||
* Enable the Accelerometer/gyroscope and output its info topic data.
|
||||
* **`enable_sync_output_accel_gyro`**
|
||||
* Enable the sync `accel_gyro`, and output IMU topic real-time data.
|
||||
* **`accel_rate`** / **`gyro_rate`**
|
||||
* The frequency of the accelerometer/gyroscope. Values range from `1.5625hz` to `32khz`.
|
||||
* **`accel_range`** / **`gyro_range`**
|
||||
* The range of the accelerometer (`2g`, `4g`, `8g`, `16g`) and gyroscope (`16dps` to `2000dps`).
|
||||
* **`enable_accel_data_correction`** / **`enable_gyro_data_correction`**
|
||||
* Enable data correction for the accelerometer/gyroscope.
|
||||
* **`linear_accel_cov`** / **`angular_vel_cov`**
|
||||
* Covariance of the linear acceleration and angular velocity.
|
||||
|
||||
### Depth Filters
|
||||
|
||||
* **`enable_decimation_filter`**
|
||||
* Enable the Depth decimation filter. Set with `decimation_filter_scale`.
|
||||
* **`enable_hdr_merge`**
|
||||
* Enable the Depth hdr merge filter. Set with `hdr_merge_exposure_1`, etc.
|
||||
* **`enable_sequence_id_filter`**
|
||||
* Enable the Depth sequence id filter. Set with `sequence_id_filter_id`.
|
||||
* **`enable_threshold_filter`**
|
||||
* Enable the Depth threshold filter. Set with `threshold_filter_max`, `threshold_filter_min`.
|
||||
* **`enable_hardware_noise_removal_filter`**
|
||||
* Enable the Depth hardware noise removal filter.
|
||||
* **`enable_noise_removal_filter`**
|
||||
* Enable the Depth software noise removal filter. Set with `noise_removal_filter_min_diff`, etc.
|
||||
* **`enable_spatial_filter`**
|
||||
* Enable the Depth spatial filter. Set with `spatial_filter_alpha`, etc.
|
||||
* **`enable_temporal_filter`**
|
||||
* Enable the Depth temporal filter. Set with `temporal_filter_diff_threshold`, etc.
|
||||
* **`enable_hole_filling_filter`**
|
||||
* Enable the Depth hole filling filter. Set with `hole_filling_filter_mode`.
|
||||
* **`enable_spatial_fast_filter`**
|
||||
* Enable the Depth spatial fast filter. Set with `spatial_fast_filter_radius`.
|
||||
* **`enable_spatial_moderate_filter`**
|
||||
* Enable the Depth spatial moderate filter. Set with `spatial_moderate_filter_diff_threshold`, etc.
|
||||
|
||||
---
|
||||
|
||||
> **_IMPORTANT_**: Please carefully read the instructions regarding software filtering settings at [this link](https://www.orbbec.com/docs/g330-use-depth-post-processing-blocks/). If you are uncertain, do not modify these settings.
|
||||
@@ -0,0 +1,209 @@
|
||||
# All available services for camera control
|
||||
|
||||
> **Note:** Services related to a specific stream (e.g., `/camera/set_color_*`) are only available if that stream is enabled in the launch file (e.g., `enable_color:=true`).
|
||||
|
||||
### Stream Control
|
||||
|
||||
#### Color Stream
|
||||
* `/camera/toggle_color`
|
||||
```bash
|
||||
ros2 service call /camera/toggle_color std_srvs/srv/SetBool '{data: true}'
|
||||
```
|
||||
* `/camera/get_color_exposure` & `/camera/get_color_gain`
|
||||
```bash
|
||||
ros2 service call /camera/get_color_exposure orbbec_camera_msgs/srv/GetInt32 '{}'
|
||||
ros2 service call /camera/get_color_gain orbbec_camera_msgs/srv/GetInt32 '{}'
|
||||
```
|
||||
* `/camera/set_color_auto_exposure`
|
||||
```bash
|
||||
ros2 service call /camera/set_color_auto_exposure std_srvs/srv/SetBool '{data: true}'
|
||||
```
|
||||
* `/camera/set_color_exposure` & `/camera/set_color_gain`
|
||||
```bash
|
||||
ros2 service call /camera/set_color_exposure orbbec_camera_msgs/srv/SetInt32 '{data: 1}'
|
||||
ros2 service call /camera/set_color_gain orbbec_camera_msgs/srv/SetInt32 '{data: 64}'
|
||||
```
|
||||
* `/camera/set_color_mirror`, `/camera/set_color_flip`, `/camera/set_color_rotation`
|
||||
```bash
|
||||
ros2 service call /camera/set_color_mirror std_srvs/srv/SetBool '{data: true}'
|
||||
ros2 service call /camera/set_color_flip std_srvs/srv/SetBool '{data: true}'
|
||||
ros2 service call /camera/set_color_rotation orbbec_camera_msgs/srv/SetInt32 '{data: 180}'
|
||||
```
|
||||
* `/camera/set_color_ae_roi`
|
||||
```bash
|
||||
# data_param: [Left, Right, Top, Bottom]
|
||||
ros2 service call /camera/set_color_ae_roi orbbec_camera_msgs/srv/SetArrays '{data_param: [0,1279,0,719]}'
|
||||
```
|
||||
|
||||
#### Depth Stream
|
||||
* `/camera/toggle_depth`
|
||||
```bash
|
||||
ros2 service call /camera/toggle_depth std_srvs/srv/SetBool '{data: true}'
|
||||
```
|
||||
* `/camera/get_depth_exposure` & `/camera/get_depth_gain`
|
||||
```bash
|
||||
ros2 service call /camera/get_depth_exposure orbbec_camera_msgs/srv/GetInt32 '{}'
|
||||
ros2 service call /camera/get_depth_gain orbbec_camera_msgs/srv/GetInt32 '{}'
|
||||
```
|
||||
* `/camera/set_depth_auto_exposure`
|
||||
```bash
|
||||
ros2 service call /camera/set_depth_auto_exposure std_srvs/srv/SetBool '{data: true}'
|
||||
```
|
||||
* `/camera/set_depth_exposure` & `/camera/set_depth_gain`
|
||||
```bash
|
||||
ros2 service call /camera/set_depth_exposure orbbec_camera_msgs/srv/SetInt32 '{data: 3000}'
|
||||
ros2 service call /camera/set_depth_gain orbbec_camera_msgs/srv/SetInt32 '{data: 64}'
|
||||
```
|
||||
* `/camera/set_depth_mirror`, `/camera/set_depth_flip`, `/camera/set_depth_rotation`
|
||||
```bash
|
||||
ros2 service call /camera/set_depth_mirror std_srvs/srv/SetBool '{data: true}'
|
||||
ros2 service call /camera/set_depth_flip std_srvs/srv/SetBool '{data: true}'
|
||||
ros2 service call /camera/set_depth_rotation orbbec_camera_msgs/srv/SetInt32 '{data: 180}'
|
||||
```
|
||||
* `/camera/set_depth_ae_roi`
|
||||
```bash
|
||||
# data_param: [Left, Right, Top, Bottom]
|
||||
ros2 service call /camera/set_depth_ae_roi orbbec_camera_msgs/srv/SetArrays '{data_param: [0,847,0,479]}'
|
||||
```
|
||||
|
||||
#### IR Stream
|
||||
* `/camera/toggle_ir`
|
||||
|
||||
```bash
|
||||
ros2 service call /camera/toggle_ir std_srvs/srv/SetBool '{data: true}'
|
||||
```
|
||||
* `/camera/get_ir_exposure` & `/camera/get_ir_gain`
|
||||
```bash
|
||||
ros2 service call /camera/get_ir_exposure orbbec_camera_msgs/srv/GetInt32 '{}'
|
||||
ros2 service call /camera/get_ir_gain orbbec_camera_msgs/srv/GetInt32 '{}'
|
||||
```
|
||||
* `/camera/set_ir_auto_exposure`
|
||||
```bash
|
||||
ros2 service call /camera/set_ir_auto_exposure std_srvs/srv/SetBool '{data: true}'
|
||||
```
|
||||
* `/camera/set_ir_exposure` & `/camera/set_ir_gain`
|
||||
```bash
|
||||
ros2 service call /camera/set_ir_exposure orbbec_camera_msgs/srv/SetInt32 '{data: 3000}'
|
||||
ros2 service call /camera/set_ir_gain orbbec_camera_msgs/srv/SetInt32 '{data: 64}'
|
||||
```
|
||||
* `/camera/switch_ir`
|
||||
```bash
|
||||
ros2 service call /camera/switch_ir orbbec_camera_msgs/srv/SetString '{data: left}'
|
||||
```
|
||||
|
||||
#### All Streams
|
||||
* `/camera/get_streams_enable` & `/camera/set_streams_enable`
|
||||
```bash
|
||||
ros2 service call /camera/get_streams_enable orbbec_camera_msgs/srv/GetBool '{}'
|
||||
ros2 service call /camera/set_streams_enable std_srvs/srv/SetBool '{data: false}'
|
||||
```
|
||||
|
||||
### Sensor & Emitter Control
|
||||
|
||||
* `/camera/set_auto_white_balance` & `/camera/get_auto_white_balance`
|
||||
```bash
|
||||
ros2 service call /camera/set_auto_white_balance std_srvs/srv/SetBool '{data: true}'
|
||||
ros2 service call /camera/get_auto_white_balance orbbec_camera_msgs/srv/GetInt32 '{}'
|
||||
```
|
||||
* `/camera/set_white_balance` & `/camera/get_white_balance`
|
||||
```bash
|
||||
ros2 service call /camera/set_white_balance orbbec_camera_msgs/srv/SetInt32 '{data: 2800}'
|
||||
ros2 service call /camera/get_white_balance orbbec_camera_msgs/srv/GetInt32 '{}'
|
||||
```
|
||||
* `/camera/set_laser_enable`
|
||||
```bash
|
||||
ros2 service call /camera/set_laser_enable std_srvs/srv/SetBool '{data: true}'
|
||||
```
|
||||
* `/camera/set_ldp_enable` & `/camera/get_ldp_status`
|
||||
```bash
|
||||
ros2 service call /camera/set_ldp_enable std_srvs/srv/SetBool '{data: true}'
|
||||
ros2 service call /camera/get_ldp_status orbbec_camera_msgs/srv/GetBool '{}'
|
||||
```
|
||||
* `/camera/set_fan_work_mode`
|
||||
```bash
|
||||
ros2 service call /camera/set_fan_work_mode orbbec_camera_msgs/srv/SetInt32 '{data: 0}'
|
||||
```
|
||||
* `/camera/set_floor_enable`
|
||||
```bash
|
||||
ros2 service call /camera/set_floor_enable std_srvs/srv/SetBool '{data: true}'
|
||||
```
|
||||
|
||||
### Device Information & Management
|
||||
|
||||
* `/camera/get_device_info`
|
||||
```bash
|
||||
ros2 service call /camera/get_device_info orbbec_camera_msgs/srv/GetDeviceInfo
|
||||
```
|
||||
* `/camera/get_sdk_version`
|
||||
```bash
|
||||
ros2 service call /camera/get_sdk_version orbbec_camera_msgs/srv/GetString
|
||||
```
|
||||
* `/camera/reboot_device`
|
||||
```bash
|
||||
ros2 service call /camera/reboot_device std_srvs/srv/Empty '{}'
|
||||
```
|
||||
|
||||
### Synchronization & Triggering
|
||||
|
||||
* `/camera/send_software_trigger`
|
||||
```bash
|
||||
ros2 service call /camera/send_software_trigger std_srvs/srv/SetBool '{data: true}'
|
||||
```* `/camera/set_reset_timestamp`
|
||||
```bash
|
||||
# Only available when time_domain param is set to device
|
||||
ros2 service call /camera/set_reset_timestamp std_srvs/srv/SetBool '{data: true}'
|
||||
```
|
||||
* `/camera/set_sync_interleaverlaser`
|
||||
```bash
|
||||
# Only available if interleave_ae_mode is 'laser' and interleave_frame_enable is true
|
||||
ros2 service call /camera/set_sync_interleaverlaser orbbec_camera_msgs/srv/SetInt32 '{data: 0}'
|
||||
```
|
||||
|
||||
### Depth Filter Configuration
|
||||
|
||||
* `/camera/set_filter`
|
||||
```bash
|
||||
# Set DecimationFilter
|
||||
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: DecimationFilter, filter_enable: false, filter_param: [5]}'
|
||||
# Set SpatialAdvancedFilter
|
||||
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: SpatialAdvancedFilter, filter_enable: true, filter_param: [0.5,160,1,8]}'
|
||||
```
|
||||
|
||||
### Data Capture & Calibration Management
|
||||
|
||||
* `/camera/save_images`
|
||||
```bash
|
||||
ros2 service call /camera/save_images std_srvs/srv/Empty '{}'
|
||||
```
|
||||
* `/camera/save_point_cloud`
|
||||
```bash
|
||||
ros2 service call /camera/save_point_cloud std_srvs/srv/Empty '{}'
|
||||
```
|
||||
|
||||
> **Note**: The following services are currently supported only on the 435Le module. Each service can store only one set of data or string at a time.
|
||||
|
||||
* `/camera/write_customer_data` & `/camera/read_customer_data`
|
||||
```bash
|
||||
ros2 service call /camera/write_customer_data orbbec_camera_msgs/srv/SetString '{data: "string"}'
|
||||
ros2 service call /camera/read_customer_data orbbec_camera_msgs/srv/GetString '{}'
|
||||
```
|
||||
* `/camera/set_user_calib_params` & `/camera/get_user_calib_params`
|
||||
```bash
|
||||
ros2 service call /camera/set_user_calib_params orbbec_camera_msgs/srv/SetUserCalibParams \
|
||||
'{k: [614.9613647460938, 0.0, 634.91552734375,
|
||||
0.0, 614.65771484375, 391.407470703125,
|
||||
0.0, 0.0, 1.0],
|
||||
d: [-0.03131488710641861,
|
||||
0.032955970615148544,
|
||||
9.096559369936585e-05,
|
||||
-0.0003368517500348389,
|
||||
-0.01115430984646082,
|
||||
0.0, 0.0, 0.0],
|
||||
rotation: [0.9999880790710449, 0.0003024190664291382, -0.004874417092651129,
|
||||
-0.0002965621242765337, 0.9999992251396179, 0.001202247804030776,
|
||||
0.004874777048826218, -0.0012007878394797444, 0.9999874234199524],
|
||||
translation: [-0.023897956848144532,
|
||||
-9.439220279455185e-05,
|
||||
-6.804073229432106e-06]}'
|
||||
ros2 service call /camera/get_user_calib_params orbbec_camera_msgs/srv/GetUserCalibParams '{}'
|
||||
```
|
||||
@@ -0,0 +1,67 @@
|
||||
# Available Topics
|
||||
|
||||
Topics are organized by stream and function. By default, all topics are published under the `/camera` namespace, which can be changed with the `camera_name` launch parameter.
|
||||
|
||||
> **Note:** Topics for a specific stream (e.g., `/camera/color/...`) are only published if their corresponding launch parameter (e.g., `enable_color`) is set to `true`.
|
||||
|
||||
### Image Streams
|
||||
|
||||
These topics provide the raw image data and corresponding calibration information for each enabled camera stream. The pattern is consistent for `color`, `depth`, `ir`, `left_ir`, and `right_ir` streams.
|
||||
|
||||
* `/camera/color/image_raw`
|
||||
* Raw image data from the color stream.
|
||||
* `/camera/color/camera_info`
|
||||
* Camera calibration data and metadata for the color stream.
|
||||
* `/camera/color/metadata`
|
||||
* Low-level metadata from the color stream firmware.
|
||||
|
||||
* `/camera/depth/image_raw`
|
||||
* Raw image data from the depth stream.
|
||||
* `/camera/depth/camera_info`
|
||||
* Camera calibration data and metadata for the depth stream.
|
||||
* `/camera/depth/metadata`
|
||||
* Low-level metadata from the depth stream firmware.
|
||||
|
||||
* `/camera/ir/image_raw`
|
||||
* Raw image data from the infrared (IR) stream.
|
||||
* `/camera/ir/camera_info`
|
||||
* Camera calibration data and metadata for the IR stream.
|
||||
* `/camera/ir/metadata`
|
||||
* Low-level metadata from the IR stream firmware.
|
||||
|
||||
### Point Cloud Topics
|
||||
|
||||
* `/camera/depth/points`
|
||||
* Point cloud data generated from the depth stream.
|
||||
* **Condition:** Published only when `enable_point_cloud` is `true`.
|
||||
|
||||
* `/camera/depth_registered/points`
|
||||
* Colored point cloud data, where the depth points are registered to the color image frame.
|
||||
* **Condition:** Published only when `enable_colored_point_cloud` is `true`.
|
||||
|
||||
### IMU Topics
|
||||
|
||||
The Inertial Measurement Unit (IMU) topics provide accelerometer and gyroscope data. Their behavior depends on the synchronization setting.
|
||||
|
||||
* `/camera/accel/sample`
|
||||
* Individual accelerometer data stream.
|
||||
* **Condition:** Published when `enable_accel` is `true` AND `enable_sync_output_accel_gyro` is `false`.
|
||||
|
||||
* `/camera/gyro/sample`
|
||||
* Individual gyroscope data stream.
|
||||
* **Condition:** Published when `enable_gyro` is `true` AND `enable_sync_output_accel_gyro` is `false`.
|
||||
|
||||
* `/camera/gyro_accel/sample`
|
||||
* Synchronized data stream containing both accelerometer and gyroscope data in a single message.
|
||||
* **Condition:** Published when `enable_sync_output_accel_gyro` is `true`.
|
||||
|
||||
### Device Status & Diagnostics
|
||||
|
||||
* `/camera/device_status`
|
||||
* Reports the current status of the camera device.
|
||||
|
||||
* `/camera/depth_filter_status`
|
||||
* Reports the status of the depth sensor's post-processing filters.
|
||||
|
||||
* `/diagnostics`
|
||||
* Publishes diagnostic information about the camera node. Currently, this includes the device temperature.
|
||||
@@ -0,0 +1,26 @@
|
||||
Advanced Guide
|
||||
======================================================
|
||||
|
||||
This chapter covers advanced features of the SDK, including multi-camera synchronization, depth-color alignment, performance tuning, and more.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
align_depth_color.md
|
||||
benchmark.md
|
||||
building_a_Debian_Package.md
|
||||
configuration_of_depth_NFOV_and_WFOV_modes.md
|
||||
depth_work_mode_switch.md
|
||||
disparity_search_offset.md
|
||||
efficient_intra_process_communication.md
|
||||
fastdds_tuning.md
|
||||
gmsl_camera.md
|
||||
interleave_ae_mode.md
|
||||
lower_cpu_usage.md
|
||||
multi_camera.md
|
||||
multi_camera_synced.md
|
||||
multi_camera_synced_verification_tool.md
|
||||
net_camera.md
|
||||
point_cloud.md
|
||||
predefined_presets.md
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
## Aligning Depth to Color in ROS 2
|
||||
|
||||
This section explains how to align depth images with color images to create an overlay image using ROS 2. This is particularly useful for applications requiring synchronized visual information from different sensor modalities.
|
||||
|
||||
### Commands to Align and View Depth and Color Images
|
||||
|
||||
1. **Basic Depth to Color Alignment:**
|
||||
|
||||
To simply align the depth image to the color image, use the following command:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true
|
||||
```
|
||||
|
||||
This command activates the depth registration feature without opening a viewer.
|
||||
2. **Viewing Depth to Color Overlay:**
|
||||
|
||||
If you wish to view the depth to color overlay, you need to enable the viewer by using the command below:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true enable_d2c_viewer:=true
|
||||
```
|
||||
|
||||
This launches the camera node with depth to color registration and opens a viewer to display the overlay image.
|
||||
|
||||
### Selecting Topics in RViz2
|
||||
|
||||
To visualize the aligned images in RViz2:
|
||||
|
||||
1. Launch RViz2 after running one of the above commands.
|
||||
2. Select the topic for the depth to color overlay image. An example topic selection is shown here:
|
||||
|
||||
|
||||
|
||||
### Example of Depth to Color Overlay
|
||||
|
||||
After selecting the appropriate topic in RViz2, you will be able to see the depth to color overlay image. Here's what it might look like:
|
||||
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# Ob_benchmark tool
|
||||
|
||||
> The goal of this tool is to benchmark the performance of various OrbbecSDK_ROS2 camera configurations. The benchmark results depend on the camera and settings used.(Currently only works with ROS2 Humble)
|
||||
|
||||
## Usage Instructions
|
||||
|
||||
### Tool Configuration ([start_benchmark_params.json](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/tools/startbenchmark/start_benchmark_params.json))
|
||||
|
||||
```json
|
||||
{
|
||||
"start_benchmark_params": {
|
||||
"camera_name": [
|
||||
"camera_01",
|
||||
"camera_02",
|
||||
"camera_03",
|
||||
"camera_04"
|
||||
],
|
||||
"process_name": "component_conta",
|
||||
"switch_cycle": 300,
|
||||
"test_cycle": 1,
|
||||
"skip_number": 30
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
* `camera_name`: Names of the cameras to be configured. Example: `"camera_01"`, `"camera_02"`, etc.
|
||||
* `process_name`: The name of the process to be monitored. For example, `"component_conta"` will monitor the data of the container process.
|
||||
* `switch_cycle`: The cycle time for switching configurations, in seconds. For example, setting it to `300` means the configuration will switch every 300 seconds.
|
||||
* `test_cycle`: The testing cycle, in seconds. For example, setting it to `1` means the tool will collect data for the monitored process every 1 second.
|
||||
* `skip_number`: The number of data points to skip. For example, setting it to `30` means that the first 30 data points will be ignored.
|
||||
|
||||
### Camera configuration (launch files)
|
||||
|
||||
In the launch folder, there are multiple\.launch.py files (`ob_benchmark_0.launch.py`, `ob_benchmark_1.launch.py`, ..., `ob_benchmark_19.launch.py`). Each file corresponds to a different camera configuration.
|
||||
|
||||
### Running the ob_benchmark tool
|
||||
|
||||
To run the tool, use the following commands:
|
||||
|
||||
```bash
|
||||
source install/setup.bash
|
||||
ros2 run orbbec_camera ob_benchmark_node
|
||||
```
|
||||
|
||||
### Output Data Files
|
||||
|
||||
The output data files will be stored in the ob_benchmark folder with filenames like `0.csv`, `1.csv`, ..., 19.csv. For example:
|
||||
|
||||
* `0.csv` contains data from the `ob_benchmark_0.launch.py` configuration.
|
||||
* `1.csv` contains data from the `ob_benchmark_1.launch.py` configuration.
|
||||
@@ -0,0 +1,45 @@
|
||||
# Building a Debian Package
|
||||
|
||||
## Preparing the Environment
|
||||
|
||||
Before starting, install the required tools:
|
||||
|
||||
```bash
|
||||
sudo apt install debhelper fakeroot python3-bloom
|
||||
```
|
||||
|
||||
## Configuring ROS Dependencies
|
||||
|
||||
Add the following YAML file to your system at `/etc/ros/rosdep/sources.list.d/00-orbbec.yaml`. Make sure to
|
||||
replace `focal` with the codename of your Ubuntu version and `humble` with your ROS2 distribution name:
|
||||
|
||||
```yaml
|
||||
orbbec_camera_msgs:
|
||||
ubuntu:
|
||||
focal: [ ros-humble-orbbec-camera-msgs ]
|
||||
```
|
||||
|
||||
Next, create a new file `/etc/ros/rosdep/sources.list.d/50-orbbec.list` and add this line to specify the path to the
|
||||
YAML file:
|
||||
|
||||
```bash
|
||||
yaml file:///etc/ros/rosdep/sources.list.d/00-orbbec.yaml
|
||||
```
|
||||
|
||||
Update the rosdep database to reflect these changes:
|
||||
|
||||
```bash
|
||||
rosdep update
|
||||
```
|
||||
|
||||
## Building the Package
|
||||
|
||||
Navigate to your workspace and build the project:
|
||||
|
||||
```bash
|
||||
cd ~/ros2_ws/
|
||||
colcon build --event-handlers console_direct+ --cmake-args -DCMAKE_BUILD_TYPE=Release
|
||||
. install/setup.bash
|
||||
cd src/OrbbecSDK_ROS2/
|
||||
bash .make_deb.sh
|
||||
```
|
||||
@@ -0,0 +1,13 @@
|
||||
# Configuration of depth NFOV and WFOV modes
|
||||
|
||||
For the Femto Mega and Femto Bolt devices, the NFOV and WFOV modes are implemented by configuring the resolution of
|
||||
Depth and IR in the launch file.
|
||||
In launch file, depth_width, depth_height, ir_width, ir_height represents the resolution of the depth and the resolution of
|
||||
the IR.
|
||||
The frame fps and resolution of IR must be consistent with the depth. The correspondence between different modes and
|
||||
resolutions is as follows:
|
||||
|
||||
- NFOV unbinned: 640 x 576.
|
||||
- NFOV binned: 320 x 288.
|
||||
- WFOV unbinned: 1024 x 1024.
|
||||
- WFOV binned: 512 x 512.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Depth work mode switch
|
||||
|
||||
Orbbec SDK ROS 2 supports the depth work mode switch. The depth work mode switch is supported by Gemini 2, Gemini 2 L,
|
||||
and Femto and Femto Bolt cameras.
|
||||
|
||||
- Before starting the camera, depth work mode (depth_work_mode) can be configured for the corresponding xxx.launch.py
|
||||
file's support.
|
||||
- The depth work mode switch is supported by Gemini 2, Gemini 2 L, and Gemini 2 XL cameras.
|
||||
- The default depth work mode configuration of xxx.launch.py is the camera's default configuration. If you need to
|
||||
modify it, you can switch to the corresponding mode as needed.
|
||||
- The specific camera depth work mode support types can be found in the comments of the depth mode.
|
||||
|
||||
```python
|
||||
# Depth work mode support is as follows:
|
||||
# Unbinned Dense Default
|
||||
# Unbinned Sparse Default
|
||||
# Binned Sparse Default
|
||||
# Obstacle Avoidance
|
||||
DeclareLaunchArgument('depth_work_mode', default_value='')
|
||||
```
|
||||
|
||||
- View depth work modes:
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_depth_work_mode_node
|
||||
```
|
||||
|
||||
* Example:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini2L.launch.py depth_work_mode:="Unbinned Dense Default"
|
||||
```
|
||||
@@ -0,0 +1,47 @@
|
||||
# Disparity_search_offset
|
||||
|
||||
> This section describes how to use the disparity_search_offset function in the Gemini330 series cameras (minimum camera firmware version [1.4.60](https://www.orbbec.com/docs/g330-firmware-release/)).Disparity_search_offset is effective only for1280×720, 1280×800 and 640×400 resolutions of depth stream.
|
||||
|
||||
## Function Introduction
|
||||
|
||||
The definition of disparity search range: For any pixel *(u_l, v)* in the left image, by default, the corresponding disparity search range in the right image is *[ (u_l - 255, v)*, *(u_l, v) ]*, where the disparity search length is 256 and the maximum integer disparity is 255. If the starting point of the search is adjusted to *[ (u_l - 255 - offset, v)*, *(u_l - offset, v) ]*, the offset is defined as the disparity shift. Therefore, our disparity search range configuration includes both the disparity search length and the search position offset (which can also be referred to as the disparity shift).
|
||||
|
||||
|
||||
|
||||
## Parameter Introduction
|
||||
|
||||
The disparity_search_offset related parameters are set in [gemini_330_series.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/launch/gemini_330_series.launch.py)
|
||||
|
||||
* `disparity_range_mode` : Disparity search length,can only be set to 64, 128 and 256.
|
||||
|
||||
* `disparity_search_offset` : Disparity search offset value,Disparity search offset value, can be set from 0 to 127.
|
||||
|
||||
* `disparity_offset_config` : Disparity search offset interleave frames.
|
||||
|
||||
* `offset_index0` : Frame 0 disparity search offset value.
|
||||
|
||||
* `offset_index1` : Frame 1 disparity search offset value.
|
||||
|
||||
| disparity range mode | disparity search offset | Minimum depth of inclined wall (mm) |
|
||||
| :------------------: | :---------------------: | :---------------------------------: |
|
||||
| 64 | 85 | Gemini 335L 388-406 |
|
||||
| 64 | 127 | Gemini 335L 302-317 |
|
||||
|
||||
| disparity range mode | disparity search offset | Minimum depth of inclined wall (mm) |
|
||||
| :------------------: | :---------------------: | :---------------------------------------------: |
|
||||
| 128 | 0 | Gemini 335 233-249<br />Gemini 335L 453-475 |
|
||||
| 128 | 45 | Gemini 335 172-184<br />Gemini 335L 334-349 |
|
||||
| 128 | 127 | Gemini 335 117-125<br />Gemini 335L 226-236 |
|
||||
|
||||
| disparity range mode | disparity search offset | Minimum depth of inclined wall (mm) |
|
||||
| :------------------: | :---------------------: | :---------------------------------: |
|
||||
| 256 | 85 | Gemini 335L 169-178 |
|
||||
| 256 | 127 | Gemini 335L 151-158 |
|
||||
|
||||
## Run the launch
|
||||
|
||||
Setting the disparity_search_offset parameter,`colcon build` again and run launch
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py
|
||||
```
|
||||
@@ -0,0 +1,35 @@
|
||||
# Efficient intra-process communication:
|
||||
|
||||
Our ROS2 Wrapper node supports zero-copy communications if loaded in the same process as a subscriber node. This can reduce copy times on image/pointcloud topics, especially with big frame resolutions and high FPS.
|
||||
|
||||
You will need to launch a component container and launch our node as a component together with other component nodes. Further details on "Composing multiple nodes in a single process" can be found [here](https://docs.ros.org/en/rolling/Tutorials/Composition.html).
|
||||
|
||||
Further details on efficient intra-process communication can be found [here](https://docs.ros.org/en/humble/Tutorials/Intra-Process-Communication.html#efficient-intra-process-communication).
|
||||
|
||||
## Example
|
||||
|
||||
### Manually loading multiple components into the same process
|
||||
|
||||
* Start the component:
|
||||
|
||||
```bash
|
||||
ros2 run rclcpp_components component_container
|
||||
```
|
||||
* Add the wrapper:
|
||||
|
||||
```bash
|
||||
ros2 component load /ComponentManager orbbec_camera orbbec_camera::OBCameraNodeDriver -e use_intra_process_comms:=true
|
||||
```
|
||||
|
||||
Load other component nodes (consumers of the wrapper topics) in the same way.
|
||||
|
||||
### Using a launch file
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_intra_process_demo_launch.py
|
||||
```
|
||||
|
||||
### Limitations
|
||||
|
||||
* Node components are currently not supported on RCLPY
|
||||
* Compressed images using `image_transport` will be disabled as this isn't supported with intra-process communication
|
||||
@@ -0,0 +1,154 @@
|
||||
# Fast DDS Optimization for Orbbec Camera with ROS2
|
||||
|
||||
When operating with the default configuration, Fast DDS exhibits suboptimal transmission efficiency, resulting in
|
||||
significant image transmission delays when used with the Orbbec camera in ROS2. This document provides guidance on
|
||||
optimizing Fast DDS to enhance image transfer efficiency.
|
||||
|
||||
## Adjusting System Parameters
|
||||
|
||||
### IP Fragmentation Time
|
||||
|
||||
- **Path**: `/proc/sys/net/ipv4/ipfrag_time` (default: 30 seconds)
|
||||
- **Purpose**: Defines the duration that IP fragments are kept in memory.
|
||||
- **Adjustment**: Decrease this value to reduce the time window where no fragments are received, which can help reduce
|
||||
delays. Consider the specific needs of your environment as this setting affects all incoming fragments.
|
||||
|
||||
**Example**: Set to 3 seconds.
|
||||
|
||||
```bash
|
||||
sudo sysctl net.ipv4.ipfrag_time=3
|
||||
```
|
||||
|
||||
### IP Fragmentation Memory Threshold
|
||||
|
||||
- **Path**: `/proc/sys/net/ipv4/ipfrag_high_thresh` (default: 262144 bytes)
|
||||
- **Purpose**: Sets the maximum memory used to reassemble IP fragments.
|
||||
- **Adjustment**: Increase this value to allow more memory for fragment reassembly, which can improve handling of larger
|
||||
data packets.
|
||||
|
||||
**Example**: Increase to 128 MB.
|
||||
|
||||
```bash
|
||||
sudo sysctl net.ipv4.ipfrag_high_thresh=134217728
|
||||
```
|
||||
|
||||
### Maximum Buffer Sizes
|
||||
|
||||
- **Purpose**: Configures the maximum buffer sizes for receiving and sending data, which is critical for high-throughput
|
||||
data transmission.
|
||||
- **Adjustment**: Set the maximum buffer sizes for both receiving and sending operations.
|
||||
|
||||
**Commands**:
|
||||
|
||||
```bash
|
||||
sudo sysctl -w net.core.rmem_max=2147483647
|
||||
sudo sysctl -w net.core.rmem_default=2147483647
|
||||
sudo sysctl -w net.core.wmem_max=2147483647
|
||||
sudo sysctl -w net.core.wmem_default=2147483647
|
||||
```
|
||||
|
||||
Alternatively, make these settings permanent by adding them to the `/etc/sysctl.d/10-fastrtps-max.conf` file.
|
||||
|
||||
```bash
|
||||
sudo gedit /etc/sysctl.d/10-fastrtps-max.conf
|
||||
```
|
||||
|
||||
add blow lines to the file:
|
||||
|
||||
```bash
|
||||
net.core.rmem_max=2147483647
|
||||
net.core.rmem_default=2147483647
|
||||
net.core.wmem_max=2147483647
|
||||
net.core.wmem_default=2147483647
|
||||
```
|
||||
|
||||
then save and exit the file. run `sudo sysctl -p` to apply the changes.
|
||||
|
||||
For detailed guidance, refer
|
||||
to [ROS 2 DDS Tuning Documentation](https://docs.ros.org/en/foxy/How-To-Guides/DDS-tuning.html).
|
||||
|
||||
## Fast DDS Configuration
|
||||
|
||||
Below is an example of a Fast DDS configuration file optimized for ROS2 usage with the Orbbec camera. This configuration
|
||||
enhances the overall data transmission by adjusting buffer sizes and transport settings.
|
||||
|
||||
### Configuration File: `shm_fastdds.xml`
|
||||
|
||||
Place this file in the `$HOME` directory.
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
|
||||
<transport_descriptors>
|
||||
<transport_descriptor>
|
||||
<transport_id>UDP_transport</transport_id>
|
||||
<type>UDPv4</type>
|
||||
<maxInitialPeersRange>10</maxInitialPeersRange>
|
||||
<maxMessageSize>65000</maxMessageSize>
|
||||
<sendBufferSize>1048576</sendBufferSize>
|
||||
<receiveBufferSize>1048576</receiveBufferSize>
|
||||
</transport_descriptor>
|
||||
</transport_descriptors>
|
||||
<participant profile_name="participant_profile_ros2" is_default_profile="true">
|
||||
<rtps>
|
||||
<name>profile_for_ros2_context</name>
|
||||
<userTransports>
|
||||
<transport_id>UDP_transport</transport_id>
|
||||
</userTransports>
|
||||
<useBuiltinTransports>false</useBuiltinTransports>
|
||||
<sendSocketBufferSize>1048576</sendSocketBufferSize>
|
||||
<listenSocketBufferSize>1048576</listenSocketBufferSize>
|
||||
<builtin>
|
||||
<initialPeersList>
|
||||
<locator>
|
||||
<udpv4>
|
||||
<address>127.0.0.1</address>
|
||||
</udpv4>
|
||||
</locator>
|
||||
</initialPeersList>
|
||||
</builtin>
|
||||
</rtps>
|
||||
</participant>
|
||||
<data_writer profile_name="default publisher profile" is_default_profile="true">
|
||||
<qos>
|
||||
<publishMode>
|
||||
<kind>ASYNCHRONOUS</kind>
|
||||
</publishMode>
|
||||
<latencyBudget>
|
||||
<duration>
|
||||
<sec>0</sec>
|
||||
<nanosec>1000000</nanosec>
|
||||
</duration>
|
||||
</latencyBudget>
|
||||
</qos>
|
||||
<historyMemoryPolicy>PREALLOCATED_WITH_REALLOC</historyMemoryPolicy>
|
||||
</data_writer>
|
||||
<data_reader profile_name="default subscription profile" is_default_profile="true">
|
||||
<qos>
|
||||
<data_sharing>
|
||||
<kind>AUTOMATIC</kind>
|
||||
</data_sharing>
|
||||
<latencyBudget>
|
||||
<duration>
|
||||
<sec>0</sec>
|
||||
<nanosec>1000000</nanosec>
|
||||
</duration>
|
||||
</latencyBudget>
|
||||
</qos>
|
||||
<historyMemoryPolicy>PREALLOCATED_WITH_REALLOC</historyMemoryPolicy>
|
||||
</data_reader>
|
||||
</profiles>
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Set the following environment variables to use the custom Fast DDS profile:
|
||||
|
||||
```bash
|
||||
export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
|
||||
export FASTRTPS_DEFAULT_PROFILES_FILE=$HOME/shm_fastdds.xml
|
||||
export RMW_FASTRTPS_USE_QOS_FROM_XML=1
|
||||
```
|
||||
|
||||
This configuration aims to optimize the data flow and reduce transmission delays, improving the responsiveness and
|
||||
reliability of the Orbbec camera system in a ROS2 environment.
|
||||
@@ -0,0 +1,50 @@
|
||||
# GMSL_camera
|
||||
|
||||
> This section describes how to use GMSL camera in OrbbecSDK_ROS2.Currently, only Gemini 335Lg GMSL devices are supported, and other GMSL devices will be supported in the near future.
|
||||
|
||||
## Single GMSL camera
|
||||
|
||||
The usage of GMSL camera in OrbbecSDK_ROS2 is the same as that of Gemini 330 series camera via USB.
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_gmsl.launch.py
|
||||
```
|
||||
|
||||
## Multi GMSL camera
|
||||
|
||||
To get the `usb_port` of the GMSL camera, plug in the camera and run the following command in the terminal:
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_devices_node
|
||||
```
|
||||
|
||||
For example, the obtained gmsl camera `usb_port`: `gmsl2-1`
|
||||
|
||||
Go to the [multi_gmsl_camera.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/gmsl_camera/multi_gmsl_camera.launch.py) file and change the `usb_port`.
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_gmsl_camera.launch.py
|
||||
```
|
||||
|
||||
> Note: By default, multi_gmsl_camera.launch.py only starts color and left_ir. If you want to start other sensors, please go to [camera_secondary_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/camera_secondary_params.yaml) to modify them.
|
||||
|
||||
## Multi GMSL camera synced
|
||||
|
||||
First, please see how to use [multi_camera_synced](./multi_camera_synced.md).
|
||||
|
||||
In addition, GMSL multi-camera synced does not require Multi-Camera Sync Hub Pro, so there is no need to set the `primary` mode. Each GMSL camera is `secondary`.
|
||||
|
||||
### Additional Parameter Settings
|
||||
|
||||
* `gmsl_trigger_fps` : set hardware soc trigger source frame rate.
|
||||
* `enable_gmsl_trigger` : enable hardware soc trigger.
|
||||
|
||||
### Run the launch
|
||||
|
||||
Please refer to the configuration in [multi_gmsl_camera_synced.launch.py.](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/gmsl_camera/multi_gmsl_camera_synced.launch.py)
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_gmsl_camera_synced.launch.py
|
||||
```
|
||||
|
||||
> Note: By default, multi_gmsl_camera_synced.launch.py only starts color and left_ir. If you want to start other sensors, please go to [camera_secondary_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/camera_secondary_params.yaml) and [camera_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/camera_params.yaml) to modify them.
|
||||
@@ -0,0 +1,81 @@
|
||||
# Using interleave_ae with Gemini330 series cameras
|
||||
|
||||
> This section describes how to use interleave_ae in Gemini 330 series cameras (minimum camera firmware version [1.4.00](https://www.orbbec.com/docs/g330-firmware-release/))
|
||||
|
||||
## Parameter Introduction
|
||||
|
||||
The interleave_ae related parameters are set in [gemini_330_series.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/launch/gemini_330_series.launch.py)
|
||||
|
||||
* `interleave_ae_mode` : Set laser or hdr interleave.
|
||||
|
||||
* `interleave_frame_enable` : enable interleave frame mode.
|
||||
|
||||
* `interleave_skip_enable` : enable skip frame mode.
|
||||
|
||||
* `interleave_skip_index` : Set 0 for skip pattern ir, set 1 for skip flood ir.
|
||||
|
||||
### interleave hdr
|
||||
|
||||
When the `interleave_ae_mode` parameter is set to `hdr` and `interleave_frame_enable `is set to `true`, interleave hdr will be enabled
|
||||
|
||||
* `hdr_index1_laser_control` : Frame 1 laser switch settings.
|
||||
|
||||
* `hdr_index1_depth_exposure` : Frame 1 depth exposure value setting, not in AE mode.
|
||||
|
||||
* `hdr_index1_depth_gain` : Frame 1 depth gain value setting, not in AE mode.
|
||||
|
||||
* `hdr_index1_ir_brightness` : Frame 1 ir gain value setting.
|
||||
|
||||
* `hdr_index1_ir_ae_max_exposure` : Frame 1 ir maximum exposure value setting in AE (auto exposure).
|
||||
|
||||
* `hdr_index0_laser_control`: Frame 0 laser switch settings.
|
||||
|
||||
* `hdr_index0_depth_exposure`: Frame 0 depth exposure value setting, not in AE mode.
|
||||
|
||||
* `hdr_index0_depth_gain` : Frame 0 depth gain value setting, not in AE mode.
|
||||
|
||||
* `hdr_index0_ir_brightness` : Frame 0 ir gain value setting.
|
||||
|
||||
* `hdr_index0_ir_ae_max_exposure` : Frame 0 ir maximum exposure value setting in AE (auto exposure).
|
||||
|
||||
### interleave laser
|
||||
|
||||
When the `interleave_ae_mode` parameter is set to `laser` and `interleave_frame_enable `is set to `true`, interleave laser will be enabled
|
||||
|
||||
* `laser_index1_laser_control` : Frame 1 laser switch settings.
|
||||
|
||||
* `laser_index1_depth_exposure` : Frame 1 depth exposure value setting, not in AE mode.
|
||||
|
||||
* `laser_index1_depth_gain` : Frame 1 depth gain value setting, not in AE mode.
|
||||
|
||||
* `laser_index1_ir_brightness` : Frame 1 ir gain value setting.
|
||||
|
||||
* `laser_index1_ir_ae_max_exposure` : Frame 1 ir maximum exposure value setting in AE (auto exposure).
|
||||
|
||||
* `laser_index0_laser_control` : Frame 0 laser switch settings.
|
||||
|
||||
* `laser_index0_depth_exposure` : Frame 0 depth exposure value setting, not in AE mode.
|
||||
|
||||
* `laser_index0_depth_gain` : Frame 0 depth gain value setting, not in AE mode.
|
||||
|
||||
* `laser_index0_ir_brightness` : Frame 0 ir gain value setting.
|
||||
|
||||
* `laser_index0_ir_ae_max_exposure` : Frame 0 ir maximum exposure value setting in AE (auto exposure).
|
||||
|
||||
## Run the launch
|
||||
|
||||
Setting the interleave_ae parameter,`colcon build` again and run launch
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py
|
||||
```
|
||||
|
||||
#### Example Visualization
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Multi_camera_synced + Interleave_ae
|
||||
|
||||
Please refer to [multi_camera_synced](./multi_camera_synced.md) and [Parameter Introduction](#parameter-introduction)
|
||||
@@ -0,0 +1,86 @@
|
||||
## Reducing CPU Usage with Orbbec ROS Package
|
||||
|
||||
This document outlines strategies for minimizing CPU usage in the **OrbbecSDK_ROS2 v2** environment when using **Gemini 330 series cameras**. The firmware version must be **no lower than 1.4.10**, and `device` should be set to **Default**.
|
||||
|
||||
### Recommended Settings for Lower CPU Usage
|
||||
|
||||
To achieve the lowest possible CPU usage in OrbbecSDK_ROS2, it is recommended to configure the following parameters.
|
||||
|
||||
| Parameter | Recommendation | Note |
|
||||
| :--------------: | :------------------------------------: | :--------------------------------------------: |
|
||||
| `uvc_backend` | `v4l2` | Lower CPU usage compared to `libuvc` |
|
||||
| `color_format` | `RGB` | Lower CPU usage than `MJPG` |
|
||||
| `filter` | Only `hardware_noise_removal_filter` | Other filters significantly increase CPU usage |
|
||||
|
||||
### Launch Files Used for Testing
|
||||
|
||||
* `gemini_330_series_lower_cpu_usage.launch.py`
|
||||
* `multi_camera_lower_cpu_usage.launch.py`
|
||||
|
||||
### Test environment
|
||||
|
||||
#### Hardware Configuration
|
||||
|
||||
* **CPU**: Intel i7-8700 @ 3.20GHz
|
||||
* **Memory**: 24 GB
|
||||
* **Storage**: Micron 2200S NVMe 256GB
|
||||
* **GPU**: NVIDIA GeForce GTX 1660Ti
|
||||
* **OS**: Ubuntu22.04
|
||||
|
||||
#### ROS Configuration
|
||||
|
||||
* **ROS Version**: ROS2 Humble
|
||||
* **SDK Version**: OrbbecSDK_ROS2 v2.2.1
|
||||
|
||||
#### Camera Setup
|
||||
|
||||
* Devices: 2x Gemini 335, 1x Gemini 336, 1x Gemini 336L
|
||||
* Firmware Version: 1.4.10
|
||||
|
||||
### Test Setup
|
||||
|
||||
* **Stream Settings:**
|
||||
* Depth / IR Left / IR Right: 848×480 @ 30fps
|
||||
* Color: 848×480 @ 30fps
|
||||
|
||||
Note: The following CPU usage data focuses on `uvc_backend`, `color_format` and various filter combinations.
|
||||
|
||||
### Test Results
|
||||
|
||||
#### `uvc_backend` Comparison (RGB format)
|
||||
|
||||
| libuvc CPU Usage | v4l2 CPU Usage | Absolute Change |
|
||||
| :--------------: | :------------: | :-------------: |
|
||||
| 182.8% | 118.8% | -64.0% |
|
||||
|
||||
The CPU usage can be significantly reduced with v4l2 backend. In our implementation, v4l2 works without requiring any patches to the Linux kernel, allowing users to easily switch between v4l2 and libuvc and maintaining full compatibility with standard Linux distributions.
|
||||
|
||||
#### `color_format` Comparison (MJPG vs RGB)
|
||||
|
||||
| Backend | MJPG CPU Usage | RGB CPU Usage | Absolute Change |
|
||||
| :-----: | :------------: | :-----------: | :-------------: |
|
||||
| libuvc | 347.7% | 182.8% | -164.9% |
|
||||
| v4l2 | 170.0% | 118.8% | -51.2% |
|
||||
|
||||
The CPU usage can be reduced if the RGB format is selected instead of MJPG, since the decoding of MJPG image will consume the host CPU resource.
|
||||
|
||||
#### Filter Configuration Impact
|
||||
|
||||
| Filters Applied | libuvc CPU Usage | CPU Usage Increase | v4l2 CPU Usage | CPU Usage Increase |
|
||||
| ----------------------------------------------------- | ---------------- | ------------------ | -------------- | ------------------ |
|
||||
| No Filter (benchmark) | 182.8% | 0.0%(benchmark) | 118.8% | 0.0%(benchmark) |
|
||||
| `(software)noise_removal_filter` | 218.0% | +35.2% | 128.5% | +9.7% |
|
||||
| `(software)noise_removal_filter + spatial_filter` | 469.6% | +286.8% | 336.7% | +217.9% |
|
||||
| `hardware_noise_removal_filter` | 186.3% | +3.5% | 115.4% | -3.4% |
|
||||
| `hardware_noise_removal_filter + spatial_filter` | 251.3% | +68.5% | 152.5% | +33.7% |
|
||||
|
||||
Based on the test results, using only the `hardware_noise_removal_filter` results in a negligible change in CPU usage for both `libuvc` (+3.5%) and `v4l2` (-3.4%) compared to the no-filter benchmark, as this filter runs internally on the camera hardware. In contrast, other filters execute on the host system. Adding the `spatial_filter` to the hardware filter leads to a moderate increase in CPU usage, while applying the software-based `noise_removal_filter` —either alone or combined with `spatial_filter` —significantly increases CPU load. To maintain low CPU usage, it is recommended to avoid software-based filters and rely solely on the `hardware_noise_removal_filter`.
|
||||
|
||||
### Further Optimizationa
|
||||
|
||||
| Parameter | Recommendation | Note |
|
||||
| :----------------------------: | :----------------------------------------------: | :---------------------------------------------: |
|
||||
| `depth_registration` | `false` or `true` with `align_mode=HW` | Software alignment consumes more CPU |
|
||||
| `enable_point_cloud` | `false` | Disabling point cloud reduces CPU usage |
|
||||
| `enable_colored_point_cloud` | `false` | Disabling colored point cloud reduces CPU usage |
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
# Multi-Camera
|
||||
|
||||
- To get the `usb_port` of the camera, plug in the camera and run the following command in the terminal:
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_devices_node
|
||||
```
|
||||
|
||||
- Set the `device_num` parameter to the number of cameras you have.
|
||||
- Go to the `OrbbecSDK_ROS2/launch/multi_xxx.launch.py` file and change the `usb_port`.
|
||||
- Don't forget to put the `include` tag inside the `group` tag.
|
||||
Otherwise, the parameter values of different cameras may become contaminated.
|
||||
|
||||
```python
|
||||
from launch import LaunchDescription
|
||||
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription, GroupAction, ExecuteProcess
|
||||
from launch.launch_description_sources import PythonLaunchDescriptionSource
|
||||
from launch_ros.actions import Node
|
||||
from ament_index_python.packages import get_package_share_directory
|
||||
import os
|
||||
|
||||
|
||||
def generate_launch_description():
|
||||
# Include launch files
|
||||
package_dir = get_package_share_directory('orbbec_camera')
|
||||
launch_file_dir = os.path.join(package_dir, 'launch')
|
||||
launch1_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, 'gemini2L.launch.py')
|
||||
),
|
||||
launch_arguments={
|
||||
'camera_name': 'camera_01',
|
||||
'usb_port': '6-2.4.4.2', # replace your usb port here
|
||||
'device_num': '2'
|
||||
}.items()
|
||||
)
|
||||
|
||||
launch2_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, 'gemini2L.launch.py')
|
||||
),
|
||||
launch_arguments={
|
||||
'camera_name': 'camera_02',
|
||||
'usb_port': '6-2.4.1', # replace your usb port here
|
||||
'device_num': '2'
|
||||
}.items()
|
||||
)
|
||||
|
||||
# If you need more cameras, just add more launch_include here, and change the usb_port and device_num
|
||||
|
||||
# Launch description
|
||||
ld = LaunchDescription([
|
||||
GroupAction([launch1_include]),
|
||||
GroupAction([launch2_include]),
|
||||
])
|
||||
|
||||
return ld
|
||||
|
||||
```
|
||||
|
||||
- To launch the cameras, run the following command:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_camera.launch.py
|
||||
```
|
||||
|
||||
## No Data Stream from Multiple Cameras
|
||||
|
||||
**Insufficient Power Supply**:
|
||||
|
||||
- Ensure that each camera is connected to a separate hub.
|
||||
- Use a powered hub to provide sufficient power to each camera.
|
||||
|
||||
**High Resolution**:
|
||||
|
||||
- Try lowering the resolution to resolve data stream issues.
|
||||
|
||||
**Increase usbfs_memory_mb Value**:
|
||||
|
||||
- Increase the `usbfs_memory_mb` value to 128MB (this is a reference value and can be adjusted based on your system’s needs)
|
||||
by running the following command:
|
||||
|
||||
```bash
|
||||
echo 128 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb
|
||||
```
|
||||
|
||||
- To make this change permanent, check [this link](https://github.com/OpenKinect/libfreenect2/issues/807).
|
||||
|
||||
## Image topic frame rate too low from Multiple Cameras
|
||||
|
||||
Refer to the [Fast DDS Configuration](./fastdds_tuning.md) file.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Multi_camera synced Instructions
|
||||
|
||||
> The purpose of this document is to explain how to use multi-camera synced with OrbbecSDK_ROS2
|
||||
|
||||
## Setup instructions
|
||||
|
||||
* Please read the Multi-Camera Synchronization Setup Guide:[Multi-Camera Synchronization Setup](https://www.orbbec.com/docs/set-up-cameras-for-external-synchronization_v1-2/)
|
||||
* Make sure the camera is correctly connected to the multi-camera synchronizer.
|
||||
|
||||
|
||||
|
||||
### Checking camera port with OrbbecSDK_ROS2
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_devices_node
|
||||
```
|
||||
|
||||
### OrbbecSDK_ROS2 multi-camera synced configuration
|
||||
|
||||
Open multi_camera_synced.launch.py, and configure the camera settings as shown below:
|
||||
|
||||
|
||||
|
||||
1. `gemini_330_series.launch.py` is the launch file for starting the camera.
|
||||
|
||||
2. Set `camera_name` to `G330_0`. For example, the published color image topic will be `/G330_0/color/image_raw`.
|
||||
|
||||
3. Set `usb_port` to `2-2`, indicating that the camera device on port `2-2` is being used. This value can be found in the output of the `ros2 run orbbec_camera list_devices_node` command.
|
||||
|
||||
4. Set `device_num` to `2`, meaning two cameras will be used.
|
||||
|
||||
5. Set `sync_mode` to `primary` to indicate that the `2-7` camera device is in primary mode. The multi-camera sync mode options can be found in the figure below.
|
||||
|
||||
6. Parameters from the `config_file_path` can override the parameters set in `gemini_330_series.launch.py` (optional).
|
||||
|
||||
7. For slave cameras, set `trigger_out_enabled` to false.
|
||||
|
||||
| **Pattern Nam**e | **Setting effect description** |
|
||||
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| free_run | -Support different frame rate settings<br />The -8-pin synchronization interface does not support external output of synchronization-related signals |
|
||||
| standalone(default) | ● Same as Primary by default<br />● Built-in RGBD frame synchronization<br />● 8-pin synchronous interface does not output signals to the outside by default |
|
||||
| primary | ● Set as primary camera<br />● 8-pin synchronous interface output signal to external device |
|
||||
| secondary | ● Set as secondary (passive synchronization; When there is a hardware continuous trigger signal input from the outside and the continuous trigger signal matches the currently set frame rate, the image is collected according to the external trigger signal; When there is no external trigger signal, the flow is stopped)<br />● 8-pin synchronous interface output signal to external device |
|
||||
| secondary_synced | ● Set to secondary synchronization (passive synchronization; When there is a hardware continuous trigger signal input from the outside and the continuous trigger signal matches the currently set frame rate, the image is collected according to the external trigger signal; When there is no external trigger signal, the image is collected according to the internal trigger signal at the set frame rate)<br />● 8-pin synchronous interface output signal to external device |
|
||||
| hardware_triggering | ● Set as hardware trigger (passive trigger; When there is a hardware trigger signal input from the outside and the trigger signal time interval is not less than the current upper limit, the image is collected according to the external trigger signal; When there is no external trigger signal, the image is not collected)<br />● 8-pin synchronous interface output signal to external device |
|
||||
| software_triggering | ● Set as software trigger (passive trigger; When there is a trigger command input from the host computer and the trigger command time interval is not less than the current upper limit, the image is collected according to the trigger command; When there is no trigger command, the image is not collected)<br />● 8-pin synchronous interface output signal to external device |
|
||||
|
||||
* The master camera should be launched last.
|
||||
* Ideally, there should be a 2-second delay between starting each camera.
|
||||
|
||||
### Run the following command to start the multi-camera synced
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_camera_synced.launch.py
|
||||
```
|
||||
@@ -0,0 +1,159 @@
|
||||
# Multi_camera_synced_verification_tool
|
||||
|
||||
> This article will introduce how to verify the synchronization accuracy of multi-camera synchronization.
|
||||
>
|
||||
> First, please see how to use [multi_camera_synced](./multi_camera_synced.md).
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```plaintext
|
||||
.
|
||||
├── multicamera_sync/
|
||||
│ ├── output/
|
||||
│ ├──20250218102900/
|
||||
│ ├──DevicesInfo.txt
|
||||
│ ├──StreamProfileInfo.txt
|
||||
│ ├── Python/
|
||||
│ ├──Config.ini
|
||||
├── gemini_330_series_synced_verify.launch.py
|
||||
├── multi_camera_synced_verify.launch.py
|
||||
```
|
||||
|
||||
### multicamera_sync
|
||||
|
||||
* `gemini_330_series_synced_verify.launch.py` : Single camera runs launch, which provides the camera running node for multi_camera_synced_verify.launch.py.
|
||||
* `multi_camera_synced_verify.launch.py` : Multi camer synced + launch of save_rgbir tool.
|
||||
|
||||
#### output
|
||||
|
||||
> The `ouput` folder is the folder where the camera pictures are output
|
||||
|
||||
In the output example provided
|
||||
|
||||
* `20250218102900` : Represents the camera image information collected at 10:29:00 on February 18, 2025.
|
||||
* `TotalModeFrames` : Camera image information storage directory.
|
||||
* `results-2025-02-25_163648` : The result after analyzing and matching the image information of TotalModeFrames.
|
||||
* `DevicesInfo.txt` : Camera equipment basic information (need to be modified).
|
||||
* `StreamProfileInfo.txt` : Camera video stream information (no need to modify).
|
||||
|
||||
#### Python
|
||||
|
||||
* `Config.ini` : Configuration file for Python analysis script (need to be modified).
|
||||
|
||||
## Preparation for operation
|
||||
|
||||
### save_rgbir node
|
||||
|
||||
Edit multi_camera_synced_verify.launch.py and fill in the activated camera device,we can find that save_rgbir is started at the end.
|
||||
|
||||
> IMPORTANT
|
||||
>
|
||||
> If you are using ROS 2 Foxy, you need to launch the save_rgbir node first.
|
||||
|
||||
```python
|
||||
# Launch description
|
||||
ld = LaunchDescription(
|
||||
[
|
||||
shared_orbbec_container,
|
||||
TimerAction(period=0.0, actions=[GroupAction([launch2_include])]),
|
||||
TimerAction(period=2.0, actions=[GroupAction([launch1_include])]),
|
||||
# The primary camera should be launched at last
|
||||
TimerAction(period=6.0, actions=[GroupAction([save_rgbir])]),
|
||||
# save_rgbir is synced verification tool
|
||||
]
|
||||
)
|
||||
|
||||
return ld
|
||||
```
|
||||
|
||||
save_rgbir is a tool for saving images. The configuration file of this tool is in [multi_save_rgbir_params.json](../../config/tools/multisavergbir/multi_save_rgbir_params.json).
|
||||
|
||||
```json
|
||||
{
|
||||
"save_rgbir_params": {
|
||||
"time_domain": "global",
|
||||
"usb_ports": [
|
||||
"2-1",
|
||||
"2-3"
|
||||
],
|
||||
"camera_name": [
|
||||
"camera_01",
|
||||
"camera_02"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
* `time_domain` : Timestamp Type
|
||||
* `usb_ports` : "primary", "secondary 1", "secondary 2", "secondary 3", fill in as many usb_ports as there are cameras
|
||||
* `camera_name` : The name of the camera setting, for example: camera_01
|
||||
|
||||
### DevicesInfo.txt
|
||||
|
||||
Edit DevicesInfo.txt. Only the `primarySerialNumber`, `index` and `serialNumber` parameters need to be changed. Other parameters do not need to be changed.Refer to the example of [20250218102900](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900).
|
||||
|
||||
* `primarySerialNumber` : The SN serial number of the primary camera
|
||||
* `index` : Camera index
|
||||
* `serialNumber` : The SN serial number of the camera
|
||||
|
||||
### Config.ini
|
||||
|
||||
Edit Config.ini.Modify `frameRate` and `tspRangeThreshold`.
|
||||
|
||||
## Run this example
|
||||
|
||||
### Run launch and save camera pictures
|
||||
|
||||
* First terminal
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_camera_synced_verify.launch.py
|
||||
```
|
||||
|
||||
* Second terminal
|
||||
|
||||
```bash
|
||||
ros2 service call /save_rgbir/start_capture orbbec_camera_msgs/srv/SetInt32 '{data: 100}'
|
||||
```
|
||||
|
||||
When the terminal displays "over", the image is saved.A new multicamera_sync folder will be generated under the workspace.
|
||||
|
||||
### Camera pictures naming format
|
||||
|
||||
Take [color_SNCP1E5420006D_Index0_g1739874543227_f0_s1739874543327_e50_d16_.jpg](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900/TotalModeFrames/SNCP1E5420006D_Index0/color_SNCP1E5420006D_Index0_g1739874543227_f0_s1739874543327_e50_d16_.jpg) as an example
|
||||
|
||||
* `color` : This image is a snapshot of the color stream.
|
||||
* `SNCP1E5420006D` : The camera's SN serial number is CP1E5420006D.
|
||||
* `Index0` : Camera 0 (usually referred to primary camera).
|
||||
* `g1739874543227` : “g” represents the global timestamp, which means the global timestamp of this frame is 1739874543227.
|
||||
* `f0` : The 0th picture of the color stream acquisition of this camera.
|
||||
* `s1739874543327` : "s" represents the timestamp of the current system, which means the current system timestamp of this frame is 1739874543327.
|
||||
* `e50` : The exposure of this frame is 50.
|
||||
* `d16` : The gain of this frame is 16.
|
||||
|
||||
### Analyzing camera image data
|
||||
|
||||
You need to copy the modified [Python folder](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/Python) to the new multi_camera_synced subdirectory, and copy the modified [DevicesInfo.txt ](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900/DevicesInfo.txt)and [StreamProfileInfo.txt](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/output/20250218102900/StreamProfileInfo.txt) to the same level directory as the TotalModeFrames folder.
|
||||
|
||||
* Finally everything is ready, run the python script
|
||||
|
||||
```bash
|
||||
cd multicamera_sync/Python
|
||||
python3 SyncFramesMain.py
|
||||
```
|
||||
|
||||
After the operation is successful, you can view the synchronization effect in the `results folder`
|
||||
|
||||
## Files that need to be changed
|
||||
|
||||
### Analysis tools
|
||||
|
||||
* [multi_save_rgbir_params.json](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/config/tools/multisavergbir/multi_save_rgbir_params.json)
|
||||
* [DevicesInfo.txt ](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/output/20250218102900/DevicesInfo.txt)
|
||||
* [Config.ini](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/Python/Config.ini)
|
||||
|
||||
### Camera Configuration
|
||||
|
||||
[camera_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/config/camera_params.yaml)(Camera startup parameter settings)
|
||||
|
||||
[multi_camera_synced_verify.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multi_camera_synced_verify.launch.py)
|
||||
@@ -0,0 +1,69 @@
|
||||
# Net_camera
|
||||
|
||||
> This section describes how to use Net camera in OrbbecSDK_ROS2.Currently, only Femto_Mega, Gemini 335Le and Gemini 435Le devices are supported, and other Net devices will be supported in the near future.
|
||||
|
||||
## Femto Mega & Gemini 435Le
|
||||
|
||||
### Parameter Introduction
|
||||
|
||||
Network device settings: `enumerate_net_device` is set to true, which will automatically enumerate network devices.
|
||||
|
||||
If you do not want to automatically enumerate network devices,you can set `enumerate_net_device` to false, `net_device_ip` to the device's IP address, and `net_device_port` to the default value of 8090.
|
||||
|
||||
* `enumerate_net_device` : Enable automatically enumerate network devices.
|
||||
* `net_device_ip` : Setting net device's IP address.
|
||||
* `net_device_port` : Setting net device's port.Usually, you can set it to 8090.
|
||||
|
||||
### Single Net camera
|
||||
|
||||
> If you need to run Gemini 435Le, you only need to replace [femto_mega.launch.py ](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/launch/femto_mega.launch.py)in the run command with [gemini435_le.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/launch/gemini435_le.launch.py)
|
||||
|
||||
For [femto_mega.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/launch/femto_mega.launch.py) as an example:
|
||||
|
||||
- **automatically enumerate network devices:**
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera femto_mega.launch.py enumerate_net_device:=true
|
||||
```
|
||||
|
||||
- **Specify IP address to start the device:**
|
||||
|
||||
Note: `net_device_ip` needs to be changed to the IP address of the device, here it is 192.168.1.10
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera femto_mega.launch.py enumerate_net_device:=false net_device_ip:=192.168.1.10 net_device_port:=8090
|
||||
```
|
||||
|
||||
## Gemini 335Le
|
||||
|
||||
Network device settings: `enumerate_net_device` must be set to true, set `net_device_ip` to the IP address of the device, and set `net_device_port` to the default value of 8090.
|
||||
|
||||
* `enumerate_net_device` : Enable automatically enumerate network devices.
|
||||
* `net_device_ip` : Setting net device's IP address.
|
||||
* `net_device_port` : Setting net device's port.Usually, you can set it to 8090.
|
||||
|
||||
### Single Net camera
|
||||
|
||||
For [gemini_330_series.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/launch/gemini_330_series.launch.py) as an example:
|
||||
|
||||
- **automatically enumerate network devices:**
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py enumerate_net_device:=true
|
||||
```
|
||||
|
||||
- **Specify IP address to start the device:**
|
||||
|
||||
Note: `net_device_ip` needs to be changed to the IP address of the device, here it is 192.168.1.10
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py enumerate_net_device:=true net_device_ip:=192.168.1.10 net_device_port:=8090
|
||||
```
|
||||
|
||||
## Multi Net camera
|
||||
|
||||
For [multi_net_camera.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/net_camera/multi_net_camera.launch.py) as an example:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_net_camera.launch.py
|
||||
```
|
||||
@@ -0,0 +1,53 @@
|
||||
## Enabling and Visualizing Point Cloud in ROS 2
|
||||
|
||||
This section demonstrates how to enable point cloud data output from the camera node and visualize it using RViz2.
|
||||
|
||||
### Enabling Depth Point Cloud
|
||||
|
||||
#### Command to Enable Depth Point Cloud
|
||||
|
||||
To activate the point cloud data stream for depth information, use the following command:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py enable_point_cloud:=true
|
||||
```
|
||||
|
||||
#### Visualizing Depth Point Cloud in RViz2
|
||||
|
||||
After running the above command, perform the following steps to visualize the depth point cloud:
|
||||
|
||||
1. Open RViz2.
|
||||
2. Add a `PointCloud2` display.
|
||||
3. Select the `/camera/depth/points` topic for visualization.
|
||||
4. Set the fixed frame to `camera_link` to properly align the data.
|
||||
|
||||
- **Example Visualization**
|
||||
|
||||
Here is what the depth point cloud might look like in RViz2:
|
||||
|
||||
|
||||
|
||||
### Enabling Colored Point Cloud
|
||||
|
||||
#### Command to Enable Colored Point Cloud
|
||||
|
||||
To enable the colored point cloud feature, enter the following command:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_series.launch.py enable_colored_point_cloud:=true
|
||||
```
|
||||
|
||||
#### Visualizing Colored Point Cloud in RViz2
|
||||
|
||||
To visualize the colored point cloud data:
|
||||
|
||||
1. Launch RViz2 following the command execution.
|
||||
2. Add a `PointCloud2` display panel.
|
||||
3. Choose the `/camera/depth_registered/points` topic from the list.
|
||||
4. Ensure the fixed frame is set to `camera_link`.
|
||||
|
||||
- **Example Visualization**
|
||||
|
||||
The result of the colored point cloud in RViz2 should look similar to this:
|
||||
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
# Predefined presets
|
||||
|
||||
| Preset | Features | Recommended use cases |
|
||||
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Default | - Best visual perception``- Overall good performance in accuracy, fill rate, tiny objects, etc. | - Generic `<br>`- Robotics |
|
||||
| Hand | - Clear hand and finger edges | - Gesture recognition |
|
||||
| High Accuracy | - Depth of high confidence `<br>`- Barely noise depth values `<br>`- Lower fill rate | - Collision avoidance `<br>`- Object scanning |
|
||||
| High Density | - Higher fill rate `<br>`- More tiny objects `<br>`- May suffer from noise depth values | - Object recognition `<br>`- Pick & place `<br>`- Foreground & background animation |
|
||||
| Medium Density | - Balanced performance in fill rate and accuracy `<br>`- In comparison to Default: lower fill rate, better edge quality | - Generic and alternative to Default |
|
||||
| Custom | - User defined Preset `<br>`- Derived from Presets above, with customized modifications, e.g. a new configuration for the post-processing pipeline, modified mean intensity set point of depth AE function, etc. | - Better depth performance achieved using customized configurations in comparison to using predefined presets `<br>`- For well-established custom configurations |
|
||||
|
||||
Choose the appropriate preset name based on your specific use case and set it as the value for the `device_preset`
|
||||
parameter.
|
||||
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
# Frequently Asked Questions
|
||||
|
||||
### Unexpected Crash
|
||||
|
||||
If the camera node crashes unexpectedly, it will generate a crash log in the current running directory: `Log/camera_crash_stack_trace_xx.log`. Please send this log to the support team or submit it to a GitHub issue for further assistance.
|
||||
|
||||
### No Data Stream from Multiple Cameras
|
||||
|
||||
**Insufficient Power Supply**:
|
||||
|
||||
- Ensure that each camera is connected to a separate hub.
|
||||
- Use a powered hub to provide sufficient power to each camera.
|
||||
|
||||
**High Resolution**:
|
||||
|
||||
- Try lowering the resolution to resolve data stream issues.
|
||||
|
||||
**Increase usbfs_memory_mb Value**:
|
||||
|
||||
- Increase the `usbfs_memory_mb` value to 128MB (this is a reference value and can be adjusted based on your system’s needs) by running the following command:
|
||||
|
||||
```
|
||||
echo 128 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb
|
||||
```
|
||||
|
||||
- To make this change permanent, check [this link](https://github.com/OpenKinect/libfreenect2/issues/807).
|
||||
|
||||
### Additional Troubleshooting
|
||||
|
||||
- If you encounter other issues, set the `log_level` parameter to `debug`. This will generate an SDK log file in the running directory: `Log/OrbbecSDK.log.txt`. Please provide this file to the support team for further assistance.
|
||||
- If firmware logs are required, set `enable_heartbeat` to `true` to activate this feature.
|
||||
|
||||
### Why Are There So Many Launch Files?
|
||||
|
||||
- Different cameras have varying default resolutions and image formats.
|
||||
- To simplify usage, each camera has its own launch file.
|
||||
@@ -0,0 +1,14 @@
|
||||
## License
|
||||
|
||||
Copyright 2024 Orbbec INC.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this project except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "
|
||||
AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific
|
||||
language governing permissions and limitations under the License.
|
||||
|
||||
Other names and brands may be claimed as the property of others.
|
||||
Reference in New Issue
Block a user