mirror of
https://github.com/orbbec/OrbbecSDK_ROS2.git
synced 2026-10-04 12:07:46 +08:00
tmp
This commit is contained in:
obyalian
@@ -0,0 +1,70 @@
|
||||
# GMSL_camera
|
||||
|
||||
> This section describes how to use GMSL camera in OrbbecSDK_ROS2.Currently, only Gemini 335Lg GMSL devices are supported, and other GMSL devices will be supported in the near future.
|
||||
|
||||
You can find example usage code in the [example](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples).
|
||||
|
||||
## Single GMSL camera
|
||||
|
||||
The usage of GMSL camera in OrbbecSDK_ROS2 is the same as that of Gemini 330 series camera via USB.
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera gemini_330_gmsl.launch.py
|
||||
```
|
||||
|
||||
## Multi GMSL camera
|
||||
|
||||
To get the `usb_port` of the GMSL camera, plug in the camera and run the following command in the terminal:
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_devices_node
|
||||
```
|
||||
|
||||
For example, the obtained gmsl camera `usb_port`: `gmsl2-1`
|
||||
|
||||
Go to the [multi_gmsl_camera.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/gmsl_camera/multi_gmsl_camera.launch.py) file and change the `usb_port`.
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_gmsl_camera.launch.py
|
||||
```
|
||||
|
||||
> Note: By default, multi_gmsl_camera.launch.py only starts color and left_ir. If you want to start other sensors, please go to [camera_secondary_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/camera_secondary_params.yaml) to modify them.
|
||||
|
||||
## Multi GMSL camera synced
|
||||
|
||||
First, please see how to use [multi_camera_synced](./multi_camera_synced.md).
|
||||
|
||||
In addition, GMSL multi-camera synced does not require Multi-Camera Sync Hub Pro, so there is no need to set the `primary` mode. Each GMSL camera is `secondary`.
|
||||
|
||||
**Additional Parameter Settings**
|
||||
|
||||
* `gmsl_trigger_fps` : set hardware soc trigger source frame rate.
|
||||
* `enable_gmsl_trigger` : enable hardware soc trigger.
|
||||
|
||||
**Run the launch**
|
||||
|
||||
Please refer to the configuration in [multi_gmsl_camera_synced.launch.py.](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/examples/gmsl_camera/multi_gmsl_camera_synced.launch.py)
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_gmsl_camera_synced.launch.py
|
||||
```
|
||||
|
||||
> Note: By default, multi_gmsl_camera_synced.launch.py only starts color and left_ir. If you want to start other sensors, please go to [camera_secondary_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/camera_secondary_params.yaml) and [camera_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/blob/v2-main/orbbec_camera/config/camera_params.yaml) to modify them.
|
||||
|
||||
|
||||
|
||||
## Usage Limitations of GMSL Cameras
|
||||
|
||||
[](https://github.com/orbbec/MIPI_Camera_Platform_Driver/tree/main?tab=readme-ov-file#usage-limitations-of-gmsl-cameras)
|
||||
|
||||
GMSL cameras interface with various deserializer chips such as MAX9296 and MAX92716. Orbbec GMSL cameras support multiple streams including depth, color, IR, and IMU data, but certain usage limitations apply:
|
||||
|
||||
- GMSL only supports V4L2 and YUYV format; MJPG format is not supported. RGB output is derived from YUYV format conversion.
|
||||
- Metadata for Gemini-335Lg is provided via a separate node, while metadata for other models is embedded within video frames, which remains transparent to users.
|
||||
- When using the Max96712 as a deserializer chip, due to the characteristics of the Max96712 chip, a multi - machine synchronous trigger signal must be provided in the secondary_synced mode. Otherwise, data flow interruption will occur when switching the data stream
|
||||
- Two cameras connected on the same MAX9296, MAX96712 LinkA/B, or MAX96712 LinkC/D have the following limitations:
|
||||
- Before driver version v12.02, there was a restriction that the RGB of one camera and the right IR of another camera could not stream simultaneously. After driver version v12.02, the restriction was modified to that the RGB of one camera and the left IR of another camera cannot stream simultaneously.
|
||||
- Before driver version v12.02, there was a restriction that the DEPTH of one camera and the left IR of another camera could not stream simultaneously. After driver version v12.02, the restriction was modified to that the DEPTH of one camera and the right IR of another camera cannot stream simultaneously.
|
||||
- The combined maximum number of active streams from both cameras is limited to four (satisfying the above two conditions ensures compliance).
|
||||
|
||||
For further known limitations, please refer to [Usage Limitations of Orbbec GMSL Cameras](https://github.com/orbbec/MIPI_Camera_Platform_Driver/blob/main/doc/Instructions%20for%20Using%20GMSL%20Camera.md)
|
||||
@@ -0,0 +1,91 @@
|
||||
# Multi-Camera
|
||||
|
||||
- To get the `usb_port` of the camera, plug in the camera and run the following command in the terminal:
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_devices_node
|
||||
```
|
||||
|
||||
- Set the `device_num` parameter to the number of cameras you have.
|
||||
- Go to the `OrbbecSDK_ROS2/launch/multi_xxx.launch.py` file and change the `usb_port`.
|
||||
- Don't forget to put the `include` tag inside the `group` tag.
|
||||
Otherwise, the parameter values of different cameras may become contaminated.
|
||||
|
||||
```python
|
||||
from launch import LaunchDescription
|
||||
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription, GroupAction, ExecuteProcess
|
||||
from launch.launch_description_sources import PythonLaunchDescriptionSource
|
||||
from launch_ros.actions import Node
|
||||
from ament_index_python.packages import get_package_share_directory
|
||||
import os
|
||||
|
||||
|
||||
def generate_launch_description():
|
||||
# Include launch files
|
||||
package_dir = get_package_share_directory('orbbec_camera')
|
||||
launch_file_dir = os.path.join(package_dir, 'launch')
|
||||
launch1_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, 'gemini2L.launch.py')
|
||||
),
|
||||
launch_arguments={
|
||||
'camera_name': 'camera_01',
|
||||
'usb_port': '6-2.4.4.2', # replace your usb port here
|
||||
'device_num': '2'
|
||||
}.items()
|
||||
)
|
||||
|
||||
launch2_include = IncludeLaunchDescription(
|
||||
PythonLaunchDescriptionSource(
|
||||
os.path.join(launch_file_dir, 'gemini2L.launch.py')
|
||||
),
|
||||
launch_arguments={
|
||||
'camera_name': 'camera_02',
|
||||
'usb_port': '6-2.4.1', # replace your usb port here
|
||||
'device_num': '2'
|
||||
}.items()
|
||||
)
|
||||
|
||||
# If you need more cameras, just add more launch_include here, and change the usb_port and device_num
|
||||
|
||||
# Launch description
|
||||
ld = LaunchDescription([
|
||||
GroupAction([launch1_include]),
|
||||
GroupAction([launch2_include]),
|
||||
])
|
||||
|
||||
return ld
|
||||
|
||||
```
|
||||
|
||||
- To launch the cameras, run the following command:
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_camera.launch.py
|
||||
```
|
||||
|
||||
## No Data Stream from Multiple Cameras
|
||||
|
||||
**Insufficient Power Supply**:
|
||||
|
||||
- Ensure that each camera is connected to a separate hub.
|
||||
- Use a powered hub to provide sufficient power to each camera.
|
||||
|
||||
**High Resolution**:
|
||||
|
||||
- Try lowering the resolution to resolve data stream issues.
|
||||
|
||||
**Increase usbfs_memory_mb Value**:
|
||||
|
||||
- Increase the `usbfs_memory_mb` value to 128MB (this is a reference value and can be adjusted based on your system’s needs)
|
||||
by running the following command:
|
||||
|
||||
```bash
|
||||
echo 128 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb
|
||||
```
|
||||
|
||||
- To make this change permanent, check [this link](https://github.com/OpenKinect/libfreenect2/issues/807).
|
||||
|
||||
## Image topic frame rate too low from Multiple Cameras
|
||||
|
||||
Refer to the [Fast DDS Configuration](../performance/fastdds_tuning.md) file.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Multi_camera synced Instructions
|
||||
|
||||
> The purpose of this document is to explain how to use multi-camera synced with OrbbecSDK_ROS2
|
||||
|
||||
### Setup instructions
|
||||
|
||||
* Please read the Multi-Camera Synchronization Setup Guide:[Multi-Camera Synchronization Setup](https://www.orbbec.com/docs/set-up-cameras-for-external-synchronization_v1-2/)
|
||||
* Make sure the camera is correctly connected to the multi-camera synchronizer.
|
||||
|
||||
|
||||
|
||||
### Checking camera port with OrbbecSDK_ROS2
|
||||
|
||||
```bash
|
||||
ros2 run orbbec_camera list_devices_node
|
||||
```
|
||||
|
||||
### OrbbecSDK_ROS2 multi-camera synced configuration
|
||||
|
||||
Open multi_camera_synced.launch.py, and configure the camera settings as shown below:
|
||||
|
||||
|
||||
|
||||
1. `gemini_330_series.launch.py` is the launch file for starting the camera.
|
||||
|
||||
2. Set `camera_name` to `G330_0`. For example, the published color image topic will be `/G330_0/color/image_raw`.
|
||||
|
||||
3. Set `usb_port` to `2-2`, indicating that the camera device on port `2-2` is being used. This value can be found in the output of the `ros2 run orbbec_camera list_devices_node` command.
|
||||
|
||||
4. Set `device_num` to `2`, meaning two cameras will be used.
|
||||
|
||||
5. Set `sync_mode` to `primary` to indicate that the `2-7` camera device is in primary mode. The multi-camera sync mode options can be found in the figure below.
|
||||
|
||||
6. Parameters from the `config_file_path` can override the parameters set in `gemini_330_series.launch.py` (optional).
|
||||
|
||||
7. For slave cameras, set `trigger_out_enabled` to false.
|
||||
|
||||
| **Pattern Nam**e | **Setting effect description** |
|
||||
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| free_run | -Support different frame rate settings<br />The -8-pin synchronization interface does not support external output of synchronization-related signals |
|
||||
| standalone(default) | ● Same as Primary by default<br />● Built-in RGBD frame synchronization<br />● 8-pin synchronous interface does not output signals to the outside by default |
|
||||
| primary | ● Set as primary camera<br />● 8-pin synchronous interface output signal to external device |
|
||||
| secondary | ● Set as secondary (passive synchronization; When there is a hardware continuous trigger signal input from the outside and the continuous trigger signal matches the currently set frame rate, the image is collected according to the external trigger signal; When there is no external trigger signal, the flow is stopped)<br />● 8-pin synchronous interface output signal to external device |
|
||||
| secondary_synced | ● Set to secondary synchronization (passive synchronization; When there is a hardware continuous trigger signal input from the outside and the continuous trigger signal matches the currently set frame rate, the image is collected according to the external trigger signal; When there is no external trigger signal, the image is collected according to the internal trigger signal at the set frame rate)<br />● 8-pin synchronous interface output signal to external device |
|
||||
| hardware_triggering | ● Set as hardware trigger (passive trigger; When there is a hardware trigger signal input from the outside and the trigger signal time interval is not less than the current upper limit, the image is collected according to the external trigger signal; When there is no external trigger signal, the image is not collected)<br />● 8-pin synchronous interface output signal to external device |
|
||||
| software_triggering | ● Set as software trigger (passive trigger; When there is a trigger command input from the host computer and the trigger command time interval is not less than the current upper limit, the image is collected according to the trigger command; When there is no trigger command, the image is not collected)<br />● 8-pin synchronous interface output signal to external device |
|
||||
|
||||
* The master camera should be launched last.
|
||||
* Ideally, there should be a 2-second delay between starting each camera.
|
||||
|
||||
### Run the following command to start the multi-camera synced
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_camera_synced.launch.py
|
||||
```
|
||||
+161
@@ -0,0 +1,161 @@
|
||||
# Multi_camera_synced_verification_tool
|
||||
|
||||
> This article will introduce how to verify the synchronization accuracy of multi-camera synchronization.
|
||||
>
|
||||
> First, please see how to use [multi_camera_synced](./multi_camera_synced.md).
|
||||
|
||||
You can find example usage code in the [example](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples).
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```plaintext
|
||||
.
|
||||
├── multicamera_sync/
|
||||
│ ├── output/
|
||||
│ ├──20250218102900/
|
||||
│ ├──DevicesInfo.txt
|
||||
│ ├──StreamProfileInfo.txt
|
||||
│ ├── Python/
|
||||
│ ├──Config.ini
|
||||
├── gemini_330_series_synced_verify.launch.py
|
||||
├── multi_camera_synced_verify.launch.py
|
||||
```
|
||||
|
||||
**multicamera_sync**
|
||||
|
||||
* `gemini_330_series_synced_verify.launch.py` : Single camera runs launch, which provides the camera running node for multi_camera_synced_verify.launch.py.
|
||||
* `multi_camera_synced_verify.launch.py` : Multi camer synced + launch of save_rgbir tool.
|
||||
|
||||
**output**
|
||||
|
||||
> The `ouput` folder is the folder where the camera pictures are output
|
||||
|
||||
In the output example provided
|
||||
|
||||
* `20250218102900` : Represents the camera image information collected at 10:29:00 on February 18, 2025.
|
||||
* `TotalModeFrames` : Camera image information storage directory.
|
||||
* `results-2025-02-25_163648` : The result after analyzing and matching the image information of TotalModeFrames.
|
||||
* `DevicesInfo.txt` : Camera equipment basic information (need to be modified).
|
||||
* `StreamProfileInfo.txt` : Camera video stream information (no need to modify).
|
||||
|
||||
**Python**
|
||||
|
||||
* `Config.ini` : Configuration file for Python analysis script (need to be modified).
|
||||
|
||||
## Preparation for operation
|
||||
|
||||
**save_rgbir node**
|
||||
|
||||
Edit multi_camera_synced_verify.launch.py and fill in the activated camera device,we can find that save_rgbir is started at the end.
|
||||
|
||||
> IMPORTANT
|
||||
>
|
||||
> If you are using ROS 2 Foxy, you need to launch the save_rgbir node first.
|
||||
|
||||
```python
|
||||
# Launch description
|
||||
ld = LaunchDescription(
|
||||
[
|
||||
shared_orbbec_container,
|
||||
TimerAction(period=0.0, actions=[GroupAction([launch2_include])]),
|
||||
TimerAction(period=2.0, actions=[GroupAction([launch1_include])]),
|
||||
# The primary camera should be launched at last
|
||||
TimerAction(period=6.0, actions=[GroupAction([save_rgbir])]),
|
||||
# save_rgbir is synced verification tool
|
||||
]
|
||||
)
|
||||
|
||||
return ld
|
||||
```
|
||||
|
||||
save_rgbir is a tool for saving images. The configuration file of this tool is in [multi_save_rgbir_params.json](../../config/tools/multisavergbir/multi_save_rgbir_params.json).
|
||||
|
||||
```json
|
||||
{
|
||||
"save_rgbir_params": {
|
||||
"time_domain": "global",
|
||||
"usb_ports": [
|
||||
"2-1",
|
||||
"2-3"
|
||||
],
|
||||
"camera_name": [
|
||||
"camera_01",
|
||||
"camera_02"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
* `time_domain` : Timestamp Type
|
||||
* `usb_ports` : "primary", "secondary 1", "secondary 2", "secondary 3", fill in as many usb_ports as there are cameras
|
||||
* `camera_name` : The name of the camera setting, for example: camera_01
|
||||
|
||||
**DevicesInfo.txt**
|
||||
|
||||
Edit DevicesInfo.txt. Only the `primarySerialNumber`, `index` and `serialNumber` parameters need to be changed. Other parameters do not need to be changed.Refer to the example of [20250218102900](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900).
|
||||
|
||||
* `primarySerialNumber` : The SN serial number of the primary camera
|
||||
* `index` : Camera index
|
||||
* `serialNumber` : The SN serial number of the camera
|
||||
|
||||
**Config.ini**
|
||||
|
||||
Edit Config.ini.Modify `frameRate` and `tspRangeThreshold`.
|
||||
|
||||
## Run this example
|
||||
|
||||
**Run launch and save camera pictures**
|
||||
|
||||
* First terminal
|
||||
|
||||
```bash
|
||||
ros2 launch orbbec_camera multi_camera_synced_verify.launch.py
|
||||
```
|
||||
|
||||
* Second terminal
|
||||
|
||||
```bash
|
||||
ros2 service call /save_rgbir/start_capture orbbec_camera_msgs/srv/SetInt32 '{data: 100}'
|
||||
```
|
||||
|
||||
When the terminal displays "over", the image is saved.A new multicamera_sync folder will be generated under the workspace.
|
||||
|
||||
**Camera pictures naming format**
|
||||
|
||||
Take [color_SNCP1E5420006D_Index0_g1739874543227_f0_s1739874543327_e50_d16_.jpg](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900/TotalModeFrames/SNCP1E5420006D_Index0/color_SNCP1E5420006D_Index0_g1739874543227_f0_s1739874543327_e50_d16_.jpg) as an example
|
||||
|
||||
* `color` : This image is a snapshot of the color stream.
|
||||
* `SNCP1E5420006D` : The camera's SN serial number is CP1E5420006D.
|
||||
* `Index0` : Camera 0 (usually referred to primary camera).
|
||||
* `g1739874543227` : “g” represents the global timestamp, which means the global timestamp of this frame is 1739874543227.
|
||||
* `f0` : The 0th picture of the color stream acquisition of this camera.
|
||||
* `s1739874543327` : "s" represents the timestamp of the current system, which means the current system timestamp of this frame is 1739874543327.
|
||||
* `e50` : The exposure of this frame is 50.
|
||||
* `d16` : The gain of this frame is 16.
|
||||
|
||||
**Analyzing camera image data**
|
||||
|
||||
You need to copy the modified [Python folder](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/Python) to the new multi_camera_synced subdirectory, and copy the modified [DevicesInfo.txt ](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool/multicamera_sync/output/20250218102900/DevicesInfo.txt)and [StreamProfileInfo.txt](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/output/20250218102900/StreamProfileInfo.txt) to the same level directory as the TotalModeFrames folder.
|
||||
|
||||
* Finally everything is ready, run the python script
|
||||
|
||||
```bash
|
||||
cd multicamera_sync/Python
|
||||
python3 SyncFramesMain.py
|
||||
```
|
||||
|
||||
After the operation is successful, you can view the synchronization effect in the `results folder`
|
||||
|
||||
## Files that need to be changed
|
||||
|
||||
**Analysis tools**
|
||||
|
||||
* [multi_save_rgbir_params.json](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/config/tools/multisavergbir/multi_save_rgbir_params.json)
|
||||
* [DevicesInfo.txt ](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/output/20250218102900/DevicesInfo.txt)
|
||||
* [Config.ini](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multicamera_sync/Python/Config.ini)
|
||||
|
||||
**Camera Configuration**
|
||||
|
||||
[camera_params.yaml](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/config/camera_params.yaml)(Camera startup parameter settings)
|
||||
|
||||
[multi_camera_synced_verify.launch.py](https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main/orbbec_camera/examples/multi_camera_synced_verification_tool//multi_camera_synced_verify.launch.py)
|
||||
Reference in New Issue
Block a user