# orbbec_camera [![stable](http://badges.github.io/stability-badges/dist/stable.svg)](http://github.com/badges/stability-badges) ![version](https://img.shields.io/badge/version-1.4.6-green) --- OrbbecSDK ROS 2 is a wrapper for the Orbbec 3D camera that provides seamless integration with the ROS 2 environment. It supports ROS 2 Foxy, Galactic, and Humble distributions. ## Installation Instructions Install ROS 2 - Please refer to the official [ROS 2 installation guide](https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debians.html) guidance > If your ROS 2 command does not auto-complete, put the following two lines into your `.bashrc` > or `.zshrc` ```bash eval "$(register-python-argcomplete3 ros2)" eval "$(register-python-argcomplete3 colcon)" ``` Create `colcon` workspace ```bash mkdir -p ~/ros2_ws/src ``` Get source code ```bash cd ~/ros2_ws/src git clone https://github.com/orbbec/OrbbecSDK_ROS2.git ``` Install deb dependencies ```bash # assume you have sourced ROS environment, same blow sudo apt install libgflags-dev nlohmann-json3-dev libgoogle-glog-dev \ ros-$ROS_DISTRO-image-transport ros-$ROS_DISTRO-image-publisher ros-$ROS_DISTRO-camera-info-manager \ ros-$ROS_DISTRO-diagnostic-updater ros-$ROS_DISTRO-diagnostic-msgs ``` Install udev rules. ```bash cd ~/ros2_ws/src/OrbbecSDK_ROS2/orbbec_camera/scripts sudo bash install_udev_rules.sh sudo udevadm control --reload-rules && sudo udevadm trigger ``` ## Getting start ```bash cd ~/ros2_ws/ # build release, Default is Debug colcon build --event-handlers console_direct+ --cmake-args -DCMAKE_BUILD_TYPE=Release ``` Launch camera node - On terminal 1 ```bash . ./install/setup.bash ros2 launch orbbec_camera gemini2R.launch.py ``` - On terminal 2 ```bash . ./install/setup.bash rviz2 ``` Select the topic you want to display - List topics / services/ parameters ( on terminal 3) ```bash ros2 topic list ros2 service list ros2 param list ``` - Get device info ```bash ros2 service call /camera/get_device_info orbbec_camera_msgs/srv/GetDeviceInfo '{}' ``` - Get SDK version ```bash ros2 service call /camera/get_sdk_version orbbec_camera_msgs/srv/GetString '{}' ``` - Get exposure ```bash ros2 service call /camera/get_color_exposure orbbec_camera_msgs/srv/GetInt32 '{}' ``` > If your check `ir` or `depth`, please change `/camera/get_color_exposure` > to `/camera/get_ir_exposure` or `/camera/get_depth_exposure`, Same below. - Get gain ```bash ros2 service call /camera/get_color_gain orbbec_camera_msgs/srv/GetInt32 '{}' ``` - Get white balance ```bash ros2 service call /camera/get_white_balance orbbec_camera_msgs/srv/GetInt32 '{}' ``` - Set auto exposure ```bash ros2 service call /camera/set_color_auto_exposure std_srvs/srv/SetBool '{data: false}' ``` - Set white balance ```bash ros2 service call /camera/set_white_balance orbbec_camera_msgs/srv/SetInt32 '{data: 4600}' ``` - Set laser enable ```bash ros2 service call /camera/set_laser_enable std_srvs/srv/SetBool "{data: true}" ``` - toggle sensor ```bash ros2 service call /camera/toggle_left_ir std_srvs/srv/SetBool "{data : true}" # left ir ros2 service call /camera/toggle_left_ir std_srvs/srv/SetBool "{data : true}" # right ir ``` - save point cloud ```bash ros2 service call /camera/save_point_cloud std_srvs/srv/Empty "{}" ``` ### All available service for camera control The name of the following service already expresses its function. However, it should be noted that the corresponding `set_[left_ir|right_ir|depth|color]*` and `get[left_ir|right_ir|depth|color]*` **services are only available if you set** `enable[left_ir|right_ir|depth|color]` to `true` in the stream that corresponds to the argument of the launch file. - `/camera/get_auto_white_balance` - `/camera/get_color_exposure` - `/camera/get_color_gain` - `/camera/get_depth_exposure` - `/camera/get_depth_gain` - `/camera/get_device_info` - `/camera/get_left_ir_exposure` - `/camera/get_left_ir_gain` - `/camera/get_right_ir_exposure` - `/camera/get_right_ir_gain` - `/camera/get_sdk_version` - `/camera/get_white_balance` - `/camera/set_auto_white_balance` - `/camera/set_color_auto_exposure` - `/camera/set_color_exposure` - `/camera/set_color_gain` - `/camera/set_depth_auto_exposure` - `/camera/set_depth_exposure` - `/camera/set_depth_gain` - `/camera/set_fan_work_mode` - `/camera/set_floor_enable` - `/camera/set_left_ir_auto_exposure` - `/camera/set_left_ir_exposure` - `/camera/set_left_ir_gain` - `/camera/set_right_ir_auto_exposure` - `/camera/set_right_ir_exposure` - `/camera/set_right_ir_gain` - `/camera/set_laser_enable` - `/camera/set_ldp_enable` - `/camera/set_white_balance` - `/camera/toggle_color` - `/camera/toggle_depth` - `/camera/toggle_left_ir` - `/camera/toggle_right_ir` ### All available topics - `/camera/color/camera_info` : The color camera info. - `/camera/color/image_raw`: The color stream image. - `/camera/depth/camera_info`: The depth stream image. - `/camera/depth/image_raw`: The depth stream image - `/camera/depth/points` : The point cloud, only available when `enable_point_cloud` is `true`. - `/camera/depth_registered/points`: The colored point cloud, only available when `enable_colored_point_cloud` is `true`. - `/camera/left_ir/camera_info`: The left IR camera info. - `/camera/left_ir/image_raw`: The left IR stream image - `/camera/right_ir/camera_info`: The right IR camera info. - `/camera/right_ir/image_raw`: The right IR stream image - `/camera/accel/sample`: Acceleration data stream `enable_sync_output_accel_gyro`turned off，`enable_accel`turned on - `/camera/gyro/sample`: Gyroscope data stream，enable_sync_output_accel_gyro`turned off，`enable_gyro`turned on - `camera/gyro_accel/sample`: Synchronized data stream of acceleration and gyroscope，`enable_sync_output_accel_gyro`turned on ### Multi-Camera - To get the `usb_port` of the camera, plug in the camera and run the following command in the terminal: ```bash ros2 run orbbec_camera list_devices_node ``` - Set the `device_num` parameter to the number of cameras you have. - Go to the `OrbbecSDK_ROS2/launch/multi_xxx.launch.py` file and change the `usb_port`. - Don't forget to put the `include` tag inside the `group` tag. Otherwise, the parameter values of different cameras may become contaminated. ```python from launch import LaunchDescription from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription, GroupAction, ExecuteProcess from launch.launch_description_sources import PythonLaunchDescriptionSource from launch_ros.actions import Node from ament_index_python.packages import get_package_share_directory import os def generate_launch_description(): # Include launch files package_dir = get_package_share_directory('orbbec_camera') launch_file_dir = os.path.join(package_dir, 'launch') launch1_include = IncludeLaunchDescription( PythonLaunchDescriptionSource( os.path.join(launch_file_dir, 'gemini2R.launch.py') ), launch_arguments={ 'camera_name': 'camera_01', 'usb_port': '6-2.4.4.2', # replace your usb port here 'device_num': '2' }.items() ) launch2_include = IncludeLaunchDescription( PythonLaunchDescriptionSource( os.path.join(launch_file_dir, 'gemini2R.launch.py') ), launch_arguments={ 'camera_name': 'camera_02', 'usb_port': '6-2.4.1', # replace your usb port here 'device_num': '2' }.items() ) # If you need more cameras, just add more launch_include here, and change the usb_port and device_num # Launch description ld = LaunchDescription([ GroupAction([launch1_include]), GroupAction([launch2_include]), ]) return ld ``` - To launch the cameras, run the following command: ```bash ros2 launch orbbec_camera multi_camera.launch.py ``` ## Use hardware decoder to decode JPEG ### rockchip and Amlogic Depends on `rockchip-mpp-dev` and `rockchip-rga-dev`, not all systems have these two packages, the names may be different, please search by yourself. Open `CMakeLists.txt` and set `USE_RK_HW_DECODER` to `ON`. ### Nvidia Jetson Depends on: `jetson_multimedia_api`,`libyuv`. Open `CMakeLists.txt` and set `USE_NV_HW_DECODER` to `ON`. ## Launch parameters The following are the launch parameters available: - `connection_delay`: The delay time in milliseconds for reopening the device. Some devices, such as Astra mini, require a longer time to initialize and reopening the device immediately can cause firmware crashes when hot plugging. - `enable_point_cloud`: Enables the point cloud. - `enable_colored_point_cloud`: Enables the RGB point cloud. - `point_cloud_qos`, `[color|depth|ir]_qos,``[color|depth|ir]_camera_info_qos`: ROS 2 Message Quality of Service (QoS) settings. The possible values are `SYSTEM_DEFAULT`, `DEFAULT`,`PARAMETER_EVENTS`, `SERVICES_DEFAULT`, `PARAMETERS`, `SENSOR_DATA` and are case-insensitive. These correspond to `rmw_qos_profile_system_default`, `rmw_qos_profile_default`, `rmw_qos_profile_parameter_events`, `rmw_qos_profile_services_default`, `rmw_qos_profile_parameters`, and `SENSOR_DATA`, respectively. - `enable_d2c_viewer`: Publishes the D2C overlay image (for testing only). - `device_num`: The number of devices. This must be filled in if multiple cameras are required. - `color_width`, `color_height`, `color_fps`: The resolution and frame rate of the color stream. - `left_ir_width`, `left_ir_height`, `left_ir_fps`: The resolution and frame rate of the left IR stream. - `right_ir_width`, `right_ir_height`, `right_ir_fps`: The resolution and frame rate of the right IR stream. - `depth_width`, `depth_height`, `depth_fps`: The resolution and frame rate of the depth stream. - `enable_color`: Enables the RGB camera. - `enable_depth`: Enables the depth camera. - `enable_left_ir`: Enables the left IR camera. - `enable_right_ir`: Enables the right camera. - `depth_registration`: Enables hardware alignment the depth frame to color frame. This field is required when the `enable_colored_point_cloud` is set to `true`. - `usb_port`: The USB port of the camera. This is required when multiple cameras are used. - `enable_accel` : Enables the accelerometer. - `accel_rate`: The frequency of the accelerometer, the optional values are `1.5625hz`,`3.125hz`,`6.25hz`,`12.5hz`,`25hz`,`50hz`, `100hz`,`200hz`,`500hz`,`1khz`,`2khz`,`4khz`,`8khz`,`16khz`,`32khz`. The specific value depends on the current camera. - `accel_range` : The range of the accelerometer, the optional values are `2g`,`4g`,`8g`,`16g`. The specific value depends on the current camera. - `enable_gyro`: Whether to enable the gyroscope. - `gyro_rate` : The frequency of the gyroscope, the optional values are `1.5625hz`,`3.125hz`,`6.25hz`,`12.5hz`,`25hz`,`50hz`, `100hz`,`200hz`,`500hz`,`1khz`,`2khz`,`4khz`,`8khz`,`16khz`,`32khz`. The specific value depends on the current camera. - `gyro_range` : The range of the gyroscope, the optional values are `16dps`,`31dps`,`62dps`,`125dps`,`250dps`,`500dps`,`1000dps`,`2000dps`. The specific value depends on the current camera. - `depth_precision` : The depth precision, should be like `1mm` format. The default value is `1mm`. - `enable_laser` : Whether to enable the laser. The default value is `true`. - `laser_on_off_mode`: Laser on/off alternate mode, 0: off, 1: on-off alternate, 2: off-on alternate. The default value is `0`. - `device_preset`: The device preset options are `Default` or `High Accuracy`. - `enable_decimation_filter`: This filter effectively reduces the depth scene complexity. The filter runs on kernel sizes [2x2] to [8x8] pixels. The image size is scaled down proportionally in both dimensions to preserve the aspect ratio. - `enable_hdr_merge`: This filter is used jointly with depth HDR function. By merging consecutive depth images of alternating exposure values, we can over come challenges in acquiring depth values for under-illuminated and over-illuminated objects simultaneously. - `enable_sequence_id_filter`: This filter is used jointly with depth HDR function and outputs only the sequence with specified sequence ID. - `enable_threshold_filter`: This filter preserves depth values of interest and omits depth values out of scope. - `enable_noise_removal_filter`: This filter removes speckle noise in clusters and gives rise to a less-filled depth map. - `enable_spatial_filter`: This filter performs multiple iterations of processing as specified by the magnitude parameter, to enhance the smoothness of depth data. It is also capable of filling small holes in depth maps. - `enable_temporal_filter`:This filter is intended to improve the depth data persistency by manipulating per-pixel values based on previous frames. The filter performs a single pass on the data, adjusting the depth values while also updating the tracking history. - `enable_hole_filling_filter`: This filter fills all holes in the depth map using the specified mode. | ## DDS Tuning The default DDS settings (Galactic) may not be optimal for data transmission. Different DDS settings can have varying performance. In this example, we use CycloneDDS. For more detailed information, please refer to the [ROS DDS Tuning](https://docs.ros.org/en/humble/How-To-Guides/DDS-tuning.html)。 ● Edit cyclonedds configuration file ```bash sudo gedit /etc/cyclonedds/config.xml ``` Add ```xml lo false 16MB auto 30 ``` ● Set the environment variables, add to `.zshrc` or `.bashrc` ```bash export ROS_DOMAIN_ID=42 # Numbers from 0 to 232 export ROS_LOCALHOST_ONLY=1 export CYCLONEDDS_URI=file:///etc/cyclonedds/config.xml ``` Tip：to understand why the maximum ROS_DOMAIN_ID is 232, please visit [The ROS DOMAIN ID](https://docs.ros.org/en/humble/Concepts/About-Domain-ID.html) ● Increase UDP receive buffer size Edit ```bash /etc/sysctl.d/10-cyclone-max.conf ``` Add ```bash net.core.rmem_max=2147483647 net.core.rmem_default=2147483647 ``` ## Frequently Asked Questions No Picture from Multiple Cameras - it's possible that the power supply is insufficient. To avoid this, do not connect all cameras to the same hub and use a powered hub instead. - It's also possible that the resolution is too high. To resolve this, try lowering the resolution. Why are there so many launch files here - The reason for the presence of multiple launch files is due to the fact that the default resolutions and image formats of different cameras vary. To make it easier to use, the launch files have been separated for each camera. ## License Copyright 2023 Orbbec Ltd. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this project except in compliance with the License. You may obtain a copy of the License at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0) Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an " AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. **Other names and brands may be claimed as the property of others**