Add Chinese documentation for camera and lidar devices

This commit is contained in:
ob-yalian
2025-12-31 17:42:46 +08:00
parent c59af3d328
commit fd13aba31f
172 changed files with 466 additions and 8 deletions
@@ -0,0 +1,14 @@
Application Guide
======================================================
This chapter introduces application development with the SDK, including launch parameter configuration, ROS2 services, and topics usage.
.. toctree::
:maxdepth: 2
launch_parameters.md
services.md
topics.md
coordinate_and_tf.md
compressed_image.md
point_cloud.md
@@ -0,0 +1,11 @@
### Compressed Image
You can use `image_transport` to compress the image using `jpeg`. Below is an example of how to use it:
To access the compressed color image, you can use the following command:
```bash
ros2 topic echo /camera/color/image_raw/compressed --no-arr
```
This command will allow you to receive the compressed color image from the specified topic.
@@ -0,0 +1,353 @@
## Coordinate Systems and TF Transforms
### Camera sensor structure
![module in rviz2](../image/application_guide/image3.png)
![module in rviz2](../image/application_guide/image1.png)
### TF from coordinate A to coordinate B:
In Orbbec cameras, the origin point (0,0,0) is taken from the camera_link position.
You can view the camera's URDF model and coordinate system structure using the following command:
```bash
ros2 launch orbbec_description view_model.launch.py model:=gemini_335_336.urdf.xacro
```
![module in rviz2](../image/application_guide/image2.png)
### ROS2 Robot Coordinate System vs Camera Optical Coordinate System
* Point of View:
* Imagine standing behind the camera and looking forward.
* Always use this point of view when discussing coordinates, left vs right IR, sensor positions, etc.
![ROS2 and Camera Coordinate System](../image/application_guide/image0.png)
* ROS2 Coordinate System: (X: Forward, Y: Left, Z: Up)
* Camera Optical Coordinate System: (X: Right, Y: Down, Z: Forward)
* All data published in the wrapper topics is optical data taken directly from the camera sensors.
* Static and dynamic TF topics publish optical and ROS coordinate systems so users can transform between them.
### Using ROS2 TF Tools
#### View TF Tree Structure
Use the following ROS2 commands to print and visualize the TF tree published by the camera package:
**Print all TF relationships:**
```bash
ros2 run tf2_tools view_frames
```
This command generates a `frames.pdf` file showing the hierarchy between all frames.
![image-20251027111351870](../image/application_guide/image4.png)
**View all currently published TF information:**
```bash
ros2 topic echo /tf_static
```
**View the TF transform between two specified frames:**
Use the following command to view the transform between two specific frames:
```bash
ros2 run tf2_ros tf2_echo [source_frame] [target_frame]
```
Example, view transform from `camera_link` to `camera_depth_optical_frame`:
```bash
ros2 run tf2_ros tf2_echo camera_link camera_depth_optical_frame
```
The command continuously outputs the real-time transform between the two frames, including:
- Translation: x, y, z (meters)
- Rotation (Quaternion): x, y, z, w
- Rotation (RPY): roll, pitch, yaw (radians and degrees)
- Transform Matrix: 4×4 matrix with rotation and translation
Sample output:
```
At time 0.0
- Translation: [0.000, 0.000, 0.000]
- Rotation: in Quaternion [-0.500, 0.500, -0.500, 0.500]
- Rotation: in RPY (radian) [-1.571, -0.000, -1.571]
- Rotation: in RPY (degree) [-90.000, -0.000, -90.000]
- Matrix:
0.000 0.000 1.000 0.000
-1.000 0.000 0.000 0.000
0.000 -1.000 0.000 0.000
0.000 0.000 0.000 1.000
```
#### Visualize TF Tree in rviz2
Use rviz2 to visualize the TF tree and relative frame poses in real time:
```bash
rviz2
```
In rviz2:
- Add the `TF` display plugin
- Set the Fixed Frame to `camera_link` or `camera_depth_optical_frame`
- Select which TF frames to display
![image-20251027140652727](../image/application_guide/image5.png)
### Camera TF Calculation and Publishing Mechanism
#### Core Function: [OBCameraNode::calcAndPublishStaticTransform()](https://github.com/orbbec/OrbbecSDK_ROS2/blob/166c35b4ea211c60265ca9b38b1b15519d1ea3dd/orbbec_camera/src/ob_camera_node.cpp#L3475)
The camera node uses this function to calculate and publish all static transforms between sensors.
```cpp
void OBCameraNode::calcAndPublishStaticTransform() {
tf2::Quaternion quaternion_optical, zero_rot;
zero_rot.setRPY(0.0, 0.0, 0.0);
quaternion_optical.setRPY(-M_PI / 2, 0.0, -M_PI / 2);
tf2::Vector3 zero_trans(0, 0, 0);
auto base_stream_profile = stream_profile_[base_stream_];
auto device_info = device_->getDeviceInfo();
CHECK_NOTNULL(device_info);
auto pid = device_info->getPid();
if (!base_stream_profile) {
RCLCPP_ERROR_STREAM(logger_, "Failed to get base stream profile");
return;
}
CHECK_NOTNULL(base_stream_profile.get());
for (const auto &item : stream_profile_) {
auto stream_index = item.first;
auto stream_profile = item.second;
if (!stream_profile) {
continue;
}
OBExtrinsic ex;
try {
ex = stream_profile->getExtrinsicTo(base_stream_profile);
} catch (const ob::Error &e) {
RCLCPP_ERROR_STREAM(logger_, "Failed to get " << stream_name_[stream_index]
<< " extrinsic: " << e.getMessage());
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
}
auto Q = rotationMatrixToQuaternion(ex.rot);
Q = quaternion_optical * Q * quaternion_optical.inverse();
tf2::Vector3 trans(ex.trans[0], ex.trans[1], ex.trans[2]);
auto timestamp = node_->now();
if (stream_index.first != base_stream_.first) {
if (stream_index.first == OB_STREAM_IR_RIGHT && base_stream_.first == OB_STREAM_DEPTH) {
trans[0] = std::abs(trans[0]); // because left and right ir calibration is error
}
publishStaticTF(timestamp, trans, Q, frame_id_[base_stream_], frame_id_[stream_index]);
}
publishStaticTF(timestamp, zero_trans, quaternion_optical, frame_id_[stream_index],
optical_frame_id_[stream_index]);
RCLCPP_INFO_STREAM(logger_, "Publishing static transform from " << stream_name_[stream_index]
<< " to "
<< stream_name_[base_stream_]);
RCLCPP_INFO_STREAM(logger_, "Translation " << trans[0] << ", " << trans[1] << ", " << trans[2]);
RCLCPP_INFO_STREAM(logger_, "Rotation " << Q.getX() << ", " << Q.getY() << ", " << Q.getZ()
<< ", " << Q.getW());
}
if ((pid == FEMTO_BOLT_PID || pid == FEMTO_MEGA_PID) && enable_stream_[DEPTH] &&
enable_stream_[COLOR] && enable_publish_extrinsic_) {
// calc depth to color
CHECK_NOTNULL(stream_profile_[COLOR]);
auto depth_to_color_extrinsics = base_stream_profile->getExtrinsicTo(stream_profile_[COLOR]);
auto Q = rotationMatrixToQuaternion(depth_to_color_extrinsics.rot);
Q = quaternion_optical * Q * quaternion_optical.inverse();
publishStaticTF(node_->now(), zero_trans, Q, camera_link_frame_id_, frame_id_[base_stream_]);
} else {
publishStaticTF(node_->now(), zero_trans, zero_rot, camera_link_frame_id_,
frame_id_[base_stream_]);
}
if (enable_stream_[DEPTH] && enable_stream_[COLOR] && enable_publish_extrinsic_) {
static const char *frame_id = "depth_to_color_extrinsics";
OBExtrinsic ex;
try {
ex = base_stream_profile->getExtrinsicTo(stream_profile_[COLOR]);
} catch (const ob::Error &e) {
RCLCPP_ERROR_STREAM(logger_,
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
}
depth_to_other_extrinsics_[COLOR] = ex;
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[COLOR]);
depth_to_other_extrinsics_publishers_[COLOR]->publish(ex_msg);
}
if (enable_stream_[DEPTH] && enable_stream_[INFRA0] && enable_publish_extrinsic_) {
static const char *frame_id = "depth_to_ir_extrinsics";
OBExtrinsic ex;
try {
ex = base_stream_profile->getExtrinsicTo(stream_profile_[INFRA0]);
} catch (const ob::Error &e) {
RCLCPP_ERROR_STREAM(logger_,
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
}
depth_to_other_extrinsics_[INFRA0] = ex;
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[INFRA0]);
depth_to_other_extrinsics_publishers_[INFRA0]->publish(ex_msg);
}
if (enable_stream_[DEPTH] && enable_stream_[INFRA1] && enable_publish_extrinsic_) {
static const char *frame_id = "depth_to_left_ir_extrinsics";
OBExtrinsic ex;
try {
ex = base_stream_profile->getExtrinsicTo(stream_profile_[INFRA1]);
} catch (const ob::Error &e) {
RCLCPP_ERROR_STREAM(logger_,
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
}
depth_to_other_extrinsics_[INFRA1] = ex;
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[INFRA1]);
depth_to_other_extrinsics_publishers_[INFRA1]->publish(ex_msg);
}
if (enable_stream_[DEPTH] && enable_stream_[INFRA2] && enable_publish_extrinsic_) {
static const char *frame_id = "depth_to_right_ir_extrinsics";
OBExtrinsic ex;
try {
ex = base_stream_profile->getExtrinsicTo(stream_profile_[INFRA2]);
} catch (const ob::Error &e) {
RCLCPP_ERROR_STREAM(logger_,
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
}
ex.trans[0] = -std::abs(ex.trans[0]);
depth_to_other_extrinsics_[INFRA2] = ex;
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[INFRA2]);
depth_to_other_extrinsics_publishers_[INFRA2]->publish(ex_msg);
}
if (enable_stream_[DEPTH] && enable_stream_[ACCEL] && enable_publish_extrinsic_) {
static const char *frame_id = "depth_to_accel_extrinsics";
OBExtrinsic ex;
try {
ex = base_stream_profile->getExtrinsicTo(stream_profile_[ACCEL]);
} catch (const ob::Error &e) {
RCLCPP_ERROR_STREAM(logger_,
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
}
depth_to_other_extrinsics_[ACCEL] = ex;
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[ACCEL]);
depth_to_other_extrinsics_publishers_[ACCEL]->publish(ex_msg);
}
if (enable_stream_[DEPTH] && enable_stream_[GYRO] && enable_publish_extrinsic_) {
static const char *frame_id = "depth_to_gyro_extrinsics";
OBExtrinsic ex;
try {
ex = base_stream_profile->getExtrinsicTo(stream_profile_[GYRO]);
} catch (const ob::Error &e) {
RCLCPP_ERROR_STREAM(logger_,
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
}
depth_to_other_extrinsics_[GYRO] = ex;
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[GYRO]);
depth_to_other_extrinsics_publishers_[GYRO]->publish(ex_msg);
}
if (enable_sync_output_accel_gyro_) {
tf2::Quaternion zero_rot;
zero_rot.setRPY(0.0, 0.0, 0.0);
tf2::Vector3 zero_trans(0, 0, 0);
publishStaticTF(node_->now(), zero_trans, zero_rot, optical_frame_id_[GYRO],
accel_gyro_frame_id_);
}
}
```
#### Function Breakdown
Detailed explanation of the code:
**Quaternion Initialization and Coordinate Transform**
```cpp
tf2::Quaternion quaternion_optical, zero_rot;
zero_rot.setRPY(0.0, 0.0, 0.0);
quaternion_optical.setRPY(-M_PI / 2, 0.0, -M_PI / 2);
```
- `quaternion_optical`: Defines the rotation from optical coordinates to ROS standard (90° rotation)
- Converts camera optical CS (X right, Y down, Z forward) to ROS CS (X forward, Y left, Z up)
**Get Device Info and Base Stream**
```cpp
auto base_stream_profile = stream_profile_[base_stream_];
auto device_info = device_->getDeviceInfo();
// Base stream usually DEPTH
```
- Choose a base stream (usually depth); all other transforms are relative to it
**Iterate Streams and Compute Relative Transforms**
```cpp
for (const auto &item : stream_profile_) {
auto stream_index = item.first;
auto stream_profile = item.second;
OBExtrinsic ex;
ex = stream_profile->getExtrinsicTo(base_stream_profile);
auto Q = rotationMatrixToQuaternion(ex.rot);
Q = quaternion_optical * Q * quaternion_optical.inverse();
tf2::Vector3 trans(ex.trans[0], ex.trans[1], ex.trans[2]);
```
- `OBExtrinsic` holds rotation matrix (`rot`) and translation vector (`trans`)
- Apply optical-to-ROS rotation via quaternion multiplication
**Publish TF Transforms**
```cpp
publishStaticTF(timestamp, trans, Q, frame_id_[base_stream_], frame_id_[stream_index]);
publishStaticTF(timestamp, zero_trans, quaternion_optical, frame_id_[stream_index],
optical_frame_id_[stream_index]);
```
- First: base stream to sensor (translation + rotation)
- Second: sensor frame to optical frame (pure rotation)
**Special Handling for Left/Right IR**
```cpp
if (stream_index.first == OB_STREAM_IR_RIGHT && base_stream_.first == OB_STREAM_DEPTH) {
trans[0] = std::abs(trans[0]);
}
```
- Ensures symmetry consistency between left/right IR cameras
**Publish Depth-to-Other Extrinsics**
```cpp
if (enable_stream_[DEPTH] && enable_stream_[COLOR] && enable_publish_extrinsic_) {
OBExtrinsic ex = base_stream_profile->getExtrinsicTo(stream_profile_[COLOR]);
auto ex_msg = obExtrinsicsToMsg(ex, "depth_to_color_extrinsics");
depth_to_other_extrinsics_publishers_[COLOR]->publish(ex_msg);
}
```
- Publishes raw extrinsics via topic for advanced alignment and registration
@@ -0,0 +1,288 @@
# Launch parameters
> If you are not sure how to set the parameters, you can connect the orbbec camera and open the [OrbbecViewer](https://github.com/orbbec/OrbbecSDK/releases).
The following are the launch parameters available:
### Core & Stream Configuration
* **`camera_name`**
* Start the node namespace.
* **`serial_number`**
* The serial number of the camera. This is required when multiple cameras are used.
* **`usb_port`**
* The USB port of the camera. This is required when multiple cameras are used.
* **`device_num`**
* The number of devices. This must be filled in if multiple cameras are required.
* **`[color|depth|left_ir|right_ir|ir]_[width|height|fps|format]`**
* The resolution and frame rate of the sensor stream.
* **`[color|depth|left_ir|right_ir|ir]_rotation`**
* Set stream image rotation.
* The possible values are `0`, `90`, `180`, `270`.
* **`[color|depth|left_ir|right_ir|ir]_flip`**
* Enable the stream image flip.
* **`[color|depth|left_ir|right_ir|ir]_mirror`**
* Enable the stream image mirror.
* **`enable_point_cloud`**
* Enable the point cloud.
* **`enable_colored_point_cloud`**
* Enable the RGB point cloud.
* **`cloud_frame_id`**
* Modify the `frame_id` name within the ros message.
* **`ordered_pc`**
* Enable filtering of invalid point clouds.
* **`point_cloud_qos`, `[stream]_qos`, `[stream]_camera_info_qos`**
* ROS 2 Message Quality of Service (QoS) settings. The possible values are `SYSTEM_DEFAULT`, `DEFAULT`, `PARAMETER_EVENTS`, `SERVICES_DEFAULT`, `PARAMETERS`, `SENSOR_DATA` and are case-insensitive. These correspond to `rmw_qos_profile_system_default`, `rmw_qos_profile_default`, `rmw_qos_profile_parameter_events`, `rmw_qos_profile_services_default`, `rmw_qos_profile_parameters`, and `SENSOR_DATA`, respectively.
* **`color.image_raw.enable_pub_plugins`**
* Enable Color image transport plugins. Default: `["image_transport/compressed", "image_transport/raw", "image_transport/theora"]`.
* **`depth.image_raw.enable_pub_plugins`**
* Enable Depth image transport plugins. Default: `["image_transport/compressedDepth", "image_transport/raw"]`.
* **`left_ir.image_raw.enable_pub_plugins`**
* Enable Left IR image transport plugins. Default: `["image_transport/compressed", "image_transport/raw", "image_transport/theora"]`.
* **`right_ir.image_raw.enable_pub_plugins`**
* Enable Right IR image transport plugins. Default: `["image_transport/compressed", "image_transport/raw", "image_transport/theora"]`.
* **`point_cloud_decimation_filter_factor`**
* Point cloud downsampling factor. Range: `1–8`. `1` means no downsampling; larger values apply stronger decimation.
* **`preset_resolution_config`**
* Preset resolution configuration for the camera device. Format: "width,height,ir_decimation_factor,depth_decimation_factor". Example: "1280,720,4,4". Only supported on specific devices like Gemini2. Leave empty to disable.
### Sensor Controls
#### Color Stream
* **`enable_color_auto_exposure`**
* Enable the Color auto exposure.
* **`enable_color_auto_exposure_priority`**
* Enable the Color auto exposure priority.
* **`color_exposure`**
* Set the Color exposure.
* **`color_gain`**
* Set the Color gain.
* **`enable_color_auto_white_balance`**
* Enable the Color auto white balance.
* **`color_white_balance`**
* Set the Color white balance.
* **`color_ae_max_exposure`**
* Set the maximum exposure value for Color auto exposure.
* **`color_brightness`**, **`color_sharpness`**, **`color_gamma`**, **`color_saturation`**, **`color_contrast`**, **`color_hue`**
* Set the Color brightness, sharpness, gamma, saturation, contrast, and hue.
* **`color_backlight_compensation`**
* Enables the color camera’s backlight compensation feature. **Range**: `0–6`, **Default**: `3`.
* **`color_powerline_freq`**
* Set the power line freq. The possible values are `disable`, `50hz`, `60hz`, `auto`.
* **`enable_color_decimation_filter`** / **`color_decimation_filter_scale`**
* Enable the Color decimation filter and set its scale.
* **`color_ae_roi_[left|right|top|bottom]`**
* Set Color auto exposure ROI.
* **`color_denoising_level`**
* Enables the ISP denoising feature for Gemini 330 series devices. **Range:** `0–8`, **Default:** `0` (auto).
#### Depth Stream
* **`enable_depth_auto_exposure_priority`**
* Enable the Depth auto exposure priority.
* **`mean_intensity_set_point`**
* Set the target mean intensity of the Depth image. For example: `mean_intensity_set_point:=100`.
> **Note:** This replaces the deprecated `depth_brightness`, which is still supported for backward compatibility.
* **`enable_depth_scale`**
* Enable the depth scale.
* **`depth_precision`**
* The depth precision should be in the format `1mm`. The default value is `1mm`.
* **`depth_ae_roi_[left|right|top|bottom]`**
* Set Depth auto exposure ROI.
#### IR Stream
* **`enable_ir_auto_exposure`**
* Enable the IR auto exposure.
* **`ir_exposure`** / **`ir_gain`**
* Set the IR exposure and gain.
* **`ir_ae_max_exposure`**
* Set the maximum exposure value for IR auto exposure.
* **`ir_brightness`**
* Set the IR brightness.
#### Laser / LDP
* **`enable_laser`**
* Enable the laser. The default value is `true`.
* **`laser_energy_level`**
* Set the laser energy level.
* **`enable_ldp`** / **`ldp_power_level`**
* Enable the LDP and set its power level.
### Device, Sync & Advanced Features
#### Multi-Camera Synchronization
* **`sync_mode`**
* Set sync mode. The default value is `standalone`.
* **`depth_delay_us`** / **`color_delay_us`**
* The delay time (microseconds) of the depth/color image capture after receiving the capture command or trigger signal.
* **`trigger2image_delay_us`**
* The delay time (microseconds) of the image capture after receiving the capture command or trigger signal. Us
* **`trigger_out_delay_us`**
* The delay time (microseconds) of the trigger signal output after receiving the capture command or trigger signal.
* **`trigger_out_enabled`**
* Enable the trigger out signal.
* **`software_trigger_enabled`** / **`software_trigger_period`**
* Enable the software trigger out signal / set the software trigger period in ms.
* **`frames_per_trigger`**
* The frame number of each stream after each trigger in triggering mode.
> Used for [multi camera synced](../5_advanced_guide/multi_camera/multi_camera_synced.md).
#### Network Cameras
* **`enumerate_net_device`**
* Enable automatically enumerate network devices.
* **`net_device_ip`** / **`net_device_port`**
* Set net device's IP address and port (Usually `8090`).
* **`force_ip_enable`**
* Enable the Force IP function. **Default:** `false`
* **`force_ip_mac`**
* Target device MAC address when multiple cameras are connected (e.g., `"54:14:FD:06:07:DA"`). You can use the `list_devices_node` to find the MAC of each device. **Default:** `""`
* **`force_ip_address`**
* Static IP address to assign. **Default:** `192.168.1.10`
* **`force_ip_subnet_mask`**
* Subnet mask for the static IP. **Default:** `255.255.255.0`
* **`force_ip_gateway`**
* Gateway address for the static IP. **Default:** `192.168.1.1`
> Used for [net camera](../5_advanced_guide/configuration/net_camera.md).
#### Device-Specific
* **`device_preset`**
* The default value is `Default`. Only the G330 series is supported. For more information, refer to the [G330 documentation](https://www.orbbec.com/docs/g330-use-depth-presets/). The value should be one of the preset names listed [in the table](../5_advanced_guide/configuration/predefined_presets.md).
* **`enable_gmsl_trigger`** / **`gmsl_trigger_fps`**
* Enable the gmsl trigger out signal / set gmsl trigger fps. Used for [gmsl camera](../5_advanced_guide/multi_camera/gmsl_camera.md).
#### Disparity
* **`disparity_to_depth_mode`**
* `HW`: use hardware disparity to depth conversion. `SW`: use software disparity to depth conversion.
* **`disparity_range_mode`**, **`disparity_search_offset`**, **`disparity_offset_config`**
* Parameters for disparity search offset. Used for [disparity search offset](../5_advanced_guide/configuration/disparity_search_offset.md).
#### Interleave AE Mode
* **`interleave_ae_mode`**
* Set `laser` or `hdr` interleave.
* **`interleave_frame_enable`**, **`interleave_skip_enable`**, **`interleave_skip_index`**
* Parameters to control interleave frame mode.
* **`[hdr|laser]_index[0|1]_[...]`**
* In interleave frame mode, set the 0th and 1st frame parameters of hdr or laser interleaving frames.
* *All interleave parameters are used for [interleave ae mode](../5_advanced_guide/configuration/interleave_ae_mode.md).*
#### Intra-Camera Synchronization
- **`depth_registration`**
* Enable alignment of the depth frame to the color frame. This field is required when the `enable_colored_point_cloud` is set to `true`.
- **`align_mode`**
* The alignment mode to be used. Options are `HW` for hardware alignment and `SW` for software alignment.
- **`align_target_stream`**
* Set align target stream mode.
* The possible values are `COLOR`, `DEPTH`.
* `COLOR`: Align depth to color.
* `DEPTH`: Align color to depth.
- **`intra_camera_sync_reference`**
- Sets the reference point for intra-camera synchronization. Applicable for Gemini 330 series devices when `sync_mode` is set to **software** or **hardware trigger** mode. **Options:** `Start`, `Middle`, `End`. When set to empty, the long baseline device defaults to End, and the short baseline device defaults to Middle.
### Basic & General Parameters
#### Firmware & Backend
* **`upgrade_firmware`**
* The input parameter is the firmware path.
* **`preset_firmware_path`**
* The input parameter is the preset firmware path. If multiple paths are input, each path needs to be separated by `,` and a maximum of 3 firmware paths can be input.
* **`uvc_backend`**
* Optional values: `v4l2`, `libuvc`.
* **`connection_delay`**
* The delay time in milliseconds for reopening the device. Some devices, such as Astra mini, require a longer time to initialize and reopening the device immediately can cause firmware crashes when hot plugging.
* **`retry_on_usb3_detection_failure`**
* If the camera is connected to a USB 2.0 port and is not detected, the system will attempt to reset the camera up to three times. It is recommended to set this parameter to `false` when using a USB 2.0 connection to avoid unnecessary resets.
#### TF, Extrinsics & Calibration
* **`publish_tf`** / **`tf_publish_rate`**
* Enable the TF publish and set its publication rate.
* **`enable_publish_extrinsic`**
* Enable the extrinsics publish.
* **`ir_info_url`** / **`color_info_url`**
* Set URL of the IR/color camera info.
* **`enable_color_undistortion`**
* Enable the Color undistortion.
#### Time Synchronization
* **`enable_sync_host_time`**
* Enable synchronization of the host time with the camera time. The default value is `true`. If using global time, set to `false`.
* **`time_domain`**
* Select timestamp type: `device`, `global`, and `system`.
* **`time_sync_period`**
* Interval (in seconds) for synchronizing the camera time with the host system.
> **Note**: This parameter only needs to be set when **`enable_sync_host_time = true`** and **`time_domain = device`**.
* **`enable_ptp_config`**
* Enable PTP time synchronization. Only for Gemini 335Le. Requires `enable_sync_host_time` to be `false`.
* **`enable_frame_sync`**
* Enable the frame synchronization.
#### Logging & Diagnostics
* **`log_level`**
* SDK log level. Default is `info`. Optional values: `debug`, `info`, `warn`, `error`, `fatal`.
* **`log_file_name`**
* Saved SDK log file name. Effective when `log_level` is `debug`.
* **`diagnostic_period`**
* Diagnostic period in seconds.
* **`enable_heartbeat`**
* Enable the heartbeat function. Default is `false`. If `true`, the camera node will send heartbeat signals to the firmware.
#### Miscellaneous
* **`config_file_path`**
* The path to the YAML configuration file. Default is `""`. If not specified, default parameters from the launch file will be used.
* **`frame_aggregate_mode`**
* Set frame aggregate output mode. Optional values: `full_frame`, `color_frame`, `ANY`, `disable`.
* **`enable_d2c_viewer`**
* Publishes the D2C overlay image (for testing only).
### IMU
* **`enable_accel`** / **`enable_gyro`**
* Enable the Accelerometer/gyroscope and output its info topic data.
* **`enable_sync_output_accel_gyro`**
* Enable the sync `accel_gyro`, and output IMU topic real-time data.
* **`accel_rate`** / **`gyro_rate`**
* The frequency of the accelerometer/gyroscope. Values range from `1.5625hz` to `32khz`.
* **`accel_range`** / **`gyro_range`**
* The range of the accelerometer (`2g`, `4g`, `8g`, `16g`) and gyroscope (`16dps` to `2000dps`).
* **`enable_accel_data_correction`** / **`enable_gyro_data_correction`**
* Enable data correction for the accelerometer/gyroscope.
* **`linear_accel_cov`** / **`angular_vel_cov`**
* Covariance of the linear acceleration and angular velocity.
### Depth Filters
* **`enable_decimation_filter`**
* Enable the Depth decimation filter. Set with `decimation_filter_scale`.
* **`enable_hdr_merge`**
* Enable the Depth hdr merge filter. Set with `hdr_merge_exposure_1`, etc.
* **`enable_sequence_id_filter`**
* Enable the Depth sequence id filter. Set with `sequence_id_filter_id`.
* **`enable_threshold_filter`**
* Enable the Depth threshold filter. Set with `threshold_filter_max`, `threshold_filter_min`.
* **`enable_hardware_noise_removal_filter`**
* Enable the Depth hardware noise removal filter.
* **`enable_noise_removal_filter`**
* Enable the Depth software noise removal filter. Set with `noise_removal_filter_min_diff`, etc.
* **`enable_spatial_filter`**
* Enable the Depth spatial filter. Set with `spatial_filter_alpha`, etc.
* **`enable_temporal_filter`**
* Enable the Depth temporal filter. Set with `temporal_filter_diff_threshold`, etc.
* **`enable_hole_filling_filter`**
* Enable the Depth hole filling filter. Set with `hole_filling_filter_mode`.
* **`enable_spatial_fast_filter`**
* Enable the Depth spatial fast filter. Set with `spatial_fast_filter_radius`.
* **`enable_spatial_moderate_filter`**
* Enable the Depth spatial moderate filter. Set with `spatial_moderate_filter_diff_threshold`, etc.
---
> **_IMPORTANT_**: Please carefully read the instructions regarding software filtering settings at [this link](https://www.orbbec.com/docs/g330-use-depth-post-processing-blocks/). If you are uncertain, do not modify these settings.
@@ -0,0 +1,53 @@
## Enabling and Visualizing Point Cloud in ROS 2
This section demonstrates how to enable point cloud data output from the camera node and visualize it using RViz2.
### Enabling Depth Point Cloud
#### Command to Enable Depth Point Cloud
To activate the point cloud data stream for depth information, use the following command:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py enable_point_cloud:=true
```
#### Visualizing Depth Point Cloud in RViz2
After running the above command, perform the following steps to visualize the depth point cloud:
1. Open RViz2.
2. Add a `PointCloud2` display.
3. Select the `/camera/depth/points` topic for visualization.
4. Set the fixed frame to `camera_link` to properly align the data.
- **Example Visualization**
Here is what the depth point cloud might look like in RViz2:
![Depth Point Cloud Visualization](../image/point_cloud/image5.jpg)
### Enabling Colored Point Cloud
#### Command to Enable Colored Point Cloud
To enable the colored point cloud feature, enter the following command:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py enable_colored_point_cloud:=true
```
#### Visualizing Colored Point Cloud in RViz2
To visualize the colored point cloud data:
1. Launch RViz2 following the command execution.
2. Add a `PointCloud2` display panel.
3. Choose the `/camera/depth_registered/points` topic from the list.
4. Ensure the fixed frame is set to `camera_link`.
- **Example Visualization**
The result of the colored point cloud in RViz2 should look similar to this:
![Colored Point Cloud Visualization](../image/point_cloud/image6.jpg)
@@ -0,0 +1,268 @@
# All available services for camera control
> **Note:** Services related to a specific stream (e.g., `/camera/set_color_*`) are only available if that stream is enabled in the launch file (e.g., `enable_color:=true`).
### Stream Control
#### Color Stream
* `/camera/toggle_color`
```bash
ros2 service call /camera/toggle_color std_srvs/srv/SetBool '{data: true}'
```
* `/camera/get_color_exposure` & `/camera/get_color_gain`
```bash
ros2 service call /camera/get_color_exposure orbbec_camera_msgs/srv/GetInt32 '{}'
ros2 service call /camera/get_color_gain orbbec_camera_msgs/srv/GetInt32 '{}'
```
* `/camera/set_color_auto_exposure`
```bash
ros2 service call /camera/set_color_auto_exposure std_srvs/srv/SetBool '{data: true}'
```
* `/camera/set_color_exposure` & `/camera/set_color_gain`
```bash
ros2 service call /camera/set_color_exposure orbbec_camera_msgs/srv/SetInt32 '{data: 1}'
ros2 service call /camera/set_color_gain orbbec_camera_msgs/srv/SetInt32 '{data: 64}'
```
* `/camera/set_color_mirror`, `/camera/set_color_flip`, `/camera/set_color_rotation`
```bash
ros2 service call /camera/set_color_mirror std_srvs/srv/SetBool '{data: true}'
ros2 service call /camera/set_color_flip std_srvs/srv/SetBool '{data: true}'
ros2 service call /camera/set_color_rotation orbbec_camera_msgs/srv/SetInt32 '{data: 180}'
```
* `/camera/set_color_ae_roi`
```bash
# data_param: [Left, Right, Top, Bottom]
ros2 service call /camera/set_color_ae_roi orbbec_camera_msgs/srv/SetArrays '{data_param: [0,1279,0,719]}'
```
#### Depth Stream
* `/camera/toggle_depth`
```bash
ros2 service call /camera/toggle_depth std_srvs/srv/SetBool '{data: true}'
```
* `/camera/get_depth_exposure` & `/camera/get_depth_gain`
```bash
ros2 service call /camera/get_depth_exposure orbbec_camera_msgs/srv/GetInt32 '{}'
ros2 service call /camera/get_depth_gain orbbec_camera_msgs/srv/GetInt32 '{}'
```
* `/camera/set_depth_auto_exposure`
```bash
ros2 service call /camera/set_depth_auto_exposure std_srvs/srv/SetBool '{data: true}'
```
* `/camera/set_depth_exposure` & `/camera/set_depth_gain`
```bash
ros2 service call /camera/set_depth_exposure orbbec_camera_msgs/srv/SetInt32 '{data: 3000}'
ros2 service call /camera/set_depth_gain orbbec_camera_msgs/srv/SetInt32 '{data: 64}'
```
* `/camera/set_depth_mirror`, `/camera/set_depth_flip`, `/camera/set_depth_rotation`
```bash
ros2 service call /camera/set_depth_mirror std_srvs/srv/SetBool '{data: true}'
ros2 service call /camera/set_depth_flip std_srvs/srv/SetBool '{data: true}'
ros2 service call /camera/set_depth_rotation orbbec_camera_msgs/srv/SetInt32 '{data: 180}'
```
* `/camera/set_depth_ae_roi`
```bash
# data_param: [Left, Right, Top, Bottom]
ros2 service call /camera/set_depth_ae_roi orbbec_camera_msgs/srv/SetArrays '{data_param: [0,847,0,479]}'
```
#### IR Stream
* `/camera/toggle_ir`
```bash
ros2 service call /camera/toggle_ir std_srvs/srv/SetBool '{data: true}'
```
* `/camera/get_ir_exposure` & `/camera/get_ir_gain`
```bash
ros2 service call /camera/get_ir_exposure orbbec_camera_msgs/srv/GetInt32 '{}'
ros2 service call /camera/get_ir_gain orbbec_camera_msgs/srv/GetInt32 '{}'
```
* `/camera/set_ir_long_exposure`
```bash
ros2 service call /camera/set_ir_long_exposure std_srvs/srv/SetBool '{data: true}'
```
* `/camera/set_ir_auto_exposure`
```bash
ros2 service call /camera/set_ir_auto_exposure std_srvs/srv/SetBool '{data: true}'
```
* `/camera/set_ir_exposure` & `/camera/set_ir_gain`
```bash
ros2 service call /camera/set_ir_exposure orbbec_camera_msgs/srv/SetInt32 '{data: 3000}'
ros2 service call /camera/set_ir_gain orbbec_camera_msgs/srv/SetInt32 '{data: 64}'
```
* `/camera/set_ir_mirror`, `/camera/set_ir_flip`, `/camera/set_ir_rotation`
```bash
ros2 service call /camera/set_ir_mirror std_srvs/srv/SetBool '{data: true}'
ros2 service call /camera/set_ir_flip std_srvs/srv/SetBool '{data: true}'
ros2 service call /camera/set_ir_rotation orbbec_camera_msgs/srv/SetInt32 '{data: 180}'
```
* `/camera/switch_ir`
```bash
ros2 service call /camera/switch_ir orbbec_camera_msgs/srv/SetString '{data: left}'
```
#### All Streams
* `/camera/get_streams_enable` & `/camera/set_streams_enable`
```bash
ros2 service call /camera/get_streams_enable orbbec_camera_msgs/srv/GetBool '{}'
ros2 service call /camera/set_streams_enable std_srvs/srv/SetBool '{data: false}'
```
### Sensor & Emitter Control
* `/camera/set_auto_white_balance` & `/camera/get_auto_white_balance`
```bash
ros2 service call /camera/set_auto_white_balance std_srvs/srv/SetBool '{data: true}'
ros2 service call /camera/get_auto_white_balance orbbec_camera_msgs/srv/GetInt32 '{}'
```
* `/camera/set_white_balance` & `/camera/get_white_balance`
```bash
ros2 service call /camera/set_white_balance orbbec_camera_msgs/srv/SetInt32 '{data: 2800}'
ros2 service call /camera/get_white_balance orbbec_camera_msgs/srv/GetInt32 '{}'
```
* `/camera/set_laser_enable`
```bash
ros2 service call /camera/set_laser_enable std_srvs/srv/SetBool '{data: true}'
```
`/camera/get_laser_status`
```bash
ros2 service call /camera/get_laser_status orbbec_camera_msgs/srv/GetBool '{}'
```
* `/camera/set_ldp_enable` & `/camera/get_ldp_status`
```bash
ros2 service call /camera/set_ldp_enable std_srvs/srv/SetBool '{data: true}'
ros2 service call /camera/get_ldp_status orbbec_camera_msgs/srv/GetBool '{}'
```
* `/camera/set_ptp_config` & `/camera/get_ptp_config`
```bash
ros2 service call /camera/set_ptp_config std_srvs/srv/SetBool '{data: true}'
ros2 service call /camera/get_ptp_config orbbec_camera_msgs/srv/GetBool '{}'
```
* `/camera/get_lrm_measure_distance`
```bash
ros2 service call /camera/get_lrm_measure_distance orbbec_camera_msgs/srv/GetInt32 '{}'
```
* `/camera/set_fan_work_mode`
```bash
ros2 service call /camera/set_fan_work_mode orbbec_camera_msgs/srv/SetInt32 '{data: 0}'
```
* `/camera/set_floor_enable`
```bash
ros2 service call /camera/set_floor_enable std_srvs/srv/SetBool '{data: true}'
```
### Device Information & Management
* `/camera/get_device_info`
```bash
ros2 service call /camera/get_device_info orbbec_camera_msgs/srv/GetDeviceInfo
```
* `/camera/get_sdk_version`
```bash
ros2 service call /camera/get_sdk_version orbbec_camera_msgs/srv/GetString
```
* `/camera/reboot_device`
```bash
ros2 service call /camera/reboot_device std_srvs/srv/Empty '{}'
```
### Synchronization & Triggering
* `/camera/send_software_trigger`
```bash
ros2 service call /camera/send_software_trigger std_srvs/srv/SetBool '{data: true}'
```
* `/camera/set_sync_hosttime`
```bash
ros2 service call /camera/set_sync_hosttime std_srvs/srv/SetBool '{data: true}'
```
* `/camera/set_reset_timestamp`
```bash
# Only available when time_domain param is set to device
ros2 service call /camera/set_reset_timestamp std_srvs/srv/SetBool '{data: true}'
```
* `/camera/set_sync_interleaverlaser`
```bash
# Only available if interleave_ae_mode is 'laser' and interleave_frame_enable is true
ros2 service call /camera/set_sync_interleaverlaser orbbec_camera_msgs/srv/SetInt32 '{data: 0}'
```
### Depth Filter Configuration
* `/camera/set_filter`
```bash
# Set DecimationFilter
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: DecimationFilter, filter_enable: false, filter_param: [5]}'
# Set SpatialAdvancedFilter
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: SpatialAdvancedFilter, filter_enable: true, filter_param: [0.5,160,1,8]}'
# Set SequenceIdFilter
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: SequenceIdFilter, filter_enable: true, filter_param: [1]}'
# Set ThresholdFilter
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: ThresholdFilter, filter_enable: true, filter_param: [0,15999]}'
# Set NoiseRemovalFilter
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: NoiseRemovalFilter, filter_enable: true, filter_param: [256,80]}'
# Set HardwareNoiseRemoval
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: HardwareNoiseRemoval, filter_enable: true, filter_param: []}'
# Set SpatialFastFilter
# filter_param: [radius]
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: SpatialFastFilter, filter_enable: true, filter_param: [4]}'
# Set SpatialModerateFilter
# filter_param: [disp_diff, magnitude, radius]
ros2 service call /camera/set_filter orbbec_camera_msgs/srv/SetFilter '{filter_name: SpatialModerateFilter, filter_enable: true, filter_param: [160,1,3]}'
```
### Data Capture & Calibration Management
* `/camera/save_images`
```bash
ros2 service call /camera/save_images std_srvs/srv/Empty '{}'
```
* `/camera/save_point_cloud`
```bash
ros2 service call /camera/save_point_cloud std_srvs/srv/Empty '{}'
```
> **Note**: The following services are currently supported only on the 435Le module. Each service can store only one set of data or string at a time.
* `/camera/write_customer_data` & `/camera/read_customer_data`
```bash
ros2 service call /camera/write_customer_data orbbec_camera_msgs/srv/SetString '{data: "string"}'
ros2 service call /camera/read_customer_data orbbec_camera_msgs/srv/GetString '{}'
```
* `/camera/set_user_calib_params` & `/camera/get_user_calib_params`
```bash
ros2 service call /camera/set_user_calib_params orbbec_camera_msgs/srv/SetUserCalibParams \
'{k: [614.9613647460938, 0.0, 634.91552734375,
0.0, 614.65771484375, 391.407470703125,
0.0, 0.0, 1.0],
d: [-0.03131488710641861,
0.032955970615148544,
9.096559369936585e-05,
-0.0003368517500348389,
-0.01115430984646082,
0.0, 0.0, 0.0],
rotation: [0.9999880790710449, 0.0003024190664291382, -0.004874417092651129,
-0.0002965621242765337, 0.9999992251396179, 0.001202247804030776,
0.004874777048826218, -0.0012007878394797444, 0.9999874234199524],
translation: [-0.023897956848144532,
-9.439220279455185e-05,
-6.804073229432106e-06]}'
ros2 service call /camera/get_user_calib_params orbbec_camera_msgs/srv/GetUserCalibParams '{}'
```
### Point cloud decimation
* `/camera/set_point_cloud_decimation`
```bash
ros2 service call /camera/set_point_cloud_decimation orbbec_camera_msgs/srv/SetInt32 '{data: 8}'
```
* `/camera/get_point_cloud_decimation`
```bash
ros2 service call /camera/get_point_cloud_decimation orbbec_camera_msgs/srv/GetInt32 '{}'
```
@@ -0,0 +1,67 @@
# Available Topics
Topics are organized by stream and function. By default, all topics are published under the `/camera` namespace, which can be changed with the `camera_name` launch parameter.
> **Note:** Topics for a specific stream (e.g., `/camera/color/...`) are only published if their corresponding launch parameter (e.g., `enable_color`) is set to `true`.
### Image Streams
These topics provide the raw image data and corresponding calibration information for each enabled camera stream. The pattern is consistent for `color`, `depth`, `ir`, `left_ir`, and `right_ir` streams.
* `/camera/color/image_raw`
* Raw image data from the color stream.
* `/camera/color/camera_info`
* Camera calibration data and metadata for the color stream.
* `/camera/color/metadata`
* Low-level metadata from the color stream firmware.
* `/camera/depth/image_raw`
* Raw image data from the depth stream.
* `/camera/depth/camera_info`
* Camera calibration data and metadata for the depth stream.
* `/camera/depth/metadata`
* Low-level metadata from the depth stream firmware.
* `/camera/ir/image_raw`
* Raw image data from the infrared (IR) stream.
* `/camera/ir/camera_info`
* Camera calibration data and metadata for the IR stream.
* `/camera/ir/metadata`
* Low-level metadata from the IR stream firmware.
### Point Cloud Topics
* `/camera/depth/points`
* Point cloud data generated from the depth stream.
* **Condition:** Published only when `enable_point_cloud` is `true`.
* `/camera/depth_registered/points`
* Colored point cloud data, where the depth points are registered to the color image frame.
* **Condition:** Published only when `enable_colored_point_cloud` is `true`.
### IMU Topics
The Inertial Measurement Unit (IMU) topics provide accelerometer and gyroscope data. Their behavior depends on the synchronization setting.
* `/camera/accel/sample`
* Individual accelerometer data stream.
* **Condition:** Published when `enable_accel` is `true` AND `enable_sync_output_accel_gyro` is `false`.
* `/camera/gyro/sample`
* Individual gyroscope data stream.
* **Condition:** Published when `enable_gyro` is `true` AND `enable_sync_output_accel_gyro` is `false`.
* `/camera/gyro_accel/sample`
* Synchronized data stream containing both accelerometer and gyroscope data in a single message.
* **Condition:** Published when `enable_sync_output_accel_gyro` is `true`.
### Device Status & Diagnostics
* `/camera/device_status`
* Reports the current status of the camera device.
* `/camera/depth_filter_status`
* Reports the status of the depth sensor's post-processing filters.
* `/diagnostics`
* Publishes diagnostic information about the camera node. Currently, this includes the device temperature.