mirror of
https://github.com/orbbec/OrbbecSDK_ROS2.git
synced 2026-10-04 03:57:46 +08:00
deploy: 2c1d661d9c
This commit is contained in:
obyalian
@@ -0,0 +1,261 @@
|
||||
# Introduction
|
||||
|
||||
OrbbecSDK ROS2 Wrapper provides seamless integration of Orbbec cameras with ROS 2 environment. It supports ROS2 Foxy, Humble, and Jazzy distributions.
|
||||
|
||||
By default, we recommend using the **v2-main** branch. For older OpenNI devices not supported by v2-main, please use the **main** branch. Device models that are only supported by the main branch are listed in the table below.
|
||||
|
||||
If you are a user in China, it is recommended to use [gitee Repo](https://gitee.com/orbbecdeveloper/OrbbecSDK_ROS2).
|
||||
|
||||
Here is the device support list of main branch (v1.x) and v2-main branch (v2.x):
|
||||
|
||||
<table border="1" style="border-collapse: collapse; text-align: left; width: 100%;">
|
||||
<thead>
|
||||
<tr style="background-color: #1f4e78; color: white; text-align: center;">
|
||||
<th>Product Series</th>
|
||||
<th>Product</th>
|
||||
<th><a href="https://github.com/orbbec/OrbbecSDK_ROS2/tree/main" style="color: black; text-decoration: none;">Branch main</a></th>
|
||||
<th><a href="https://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main" style="color: black; text-decoration: none;">Branch v2-main</a></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td style="text-align: center; font-weight: bold;">Gemini 435Le</td>
|
||||
<td>Gemini 435Le</td>
|
||||
<td>Not supported</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="8" style="text-align: center; font-weight: bold;">Gemini 330</td>
|
||||
<td>Gemini 335</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 336</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 335L</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 336L</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 335Lg</td>
|
||||
<td>Not supported</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 335Le</td>
|
||||
<td>Not supported</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 330</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 330L</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3" style="text-align: center; font-weight: bold;">Gemini 2</td>
|
||||
<td>Gemini 2</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 2 L</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 2 XL</td>
|
||||
<td>Recommended for new designs</td>
|
||||
<td>To be supported</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3" style="text-align: center; font-weight: bold;">Femto</td>
|
||||
<td>Femto Bolt</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Femto Mega</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Femto Mega I</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>To be supported</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3" style="text-align: center; font-weight: bold;">Astra</td>
|
||||
<td>Astra 2</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Recommended for new designs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Astra+</td>
|
||||
<td>Limited maintenance</td>
|
||||
<td>Not supported</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Astra Pro Plus</td>
|
||||
<td>Limited maintenance</td>
|
||||
<td>Not supported</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align: center; font-weight: bold;">Astra Mini</td>
|
||||
<td>Astra Mini Pro</td>
|
||||
<td>Full maintenance</td>
|
||||
<td>Full maintenance</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
**Note**: If you do not find your device, please contact our FAE or sales representative for help.
|
||||
|
||||
**Definition**:
|
||||
|
||||
1. Recommended for new designs: we will provide full supports with new features, bug fix and performance optimization;
|
||||
2. Full maintenance: we will provide bug fix support;
|
||||
3. Limited maintenance: we will provide critical bug fix support;
|
||||
4. Not supported: we will not support specific device in this version;
|
||||
5. To be supported: we will add support in the near future.
|
||||
|
||||
## Support Hardware Products
|
||||
|
||||
The following devices are supported by the OrbbecSDK ROS2 Wrapper v2-main branch. More devices support will be added in the near future. If you can not find your device in the table below, try the [main](https://github.com/orbbec/OrbbecSDK_ROS2) branch.
|
||||
|
||||
For optimal performance, we strongly recommend updating to the latest firmware version. This ensures that you benefit from the most recent enhancements and bug fixes.
|
||||
|
||||
To learn how to obtain and upgrade the latest firmware, [please click here](../3_quickstarts/orbbecviewer.md).
|
||||
|
||||
| Product List | Minimal Firmware Version | **Launch File** |
|
||||
|:-------------- |:------------------------ |:--------------------------- |
|
||||
| Gemini 435Le | 1.2.04 | gemini435_le.launch.py |
|
||||
| Gemini 335 | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 336 | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 335L | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 336L | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 335Lg | 1.3.46 | gemini_330_series.launch.py |
|
||||
| Gemini 335Le | 1.5.31 | gemini_330_series.launch.py |
|
||||
| Gemini 330 | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 330L | 1.2.20 | gemini_330_series.launch.py |
|
||||
| Gemini 2 | 1.4.92 | gemini2.launch.py |
|
||||
| Gemini 2 L | 1.4.53 | gemini2L.launch.py |
|
||||
| Femto Bolt | 1.1.2 | femto_bolt.launch.py |
|
||||
| Femto Mega | 1.3.0 | femto_mega.launch.py |
|
||||
| Astra 2 | 2.8.20 | astra2.launch.py |
|
||||
| Astra Mini Pro | 2.0.01 | astra.launch.py |
|
||||
|
||||
All launch files are essentially similar, with the primary difference being the default values of the parameters set
|
||||
for different models within the same series. Differences in USB standards, such as USB 2.0 versus USB 3.0, may require adjustments to these parameters. If you encounter a startup failure, please carefully review the specification manual. Pay special attention to the resolution settings in the launch file, as well as other parameters, to ensure compatibility and optimal performance.
|
||||
|
||||
## Orbbec camera datasheet
|
||||
|
||||
Refer to the camera datasheet for more information.
|
||||
|
||||
<style>
|
||||
table {
|
||||
border-collapse: collapse;
|
||||
width: 100%;
|
||||
}
|
||||
th, td {
|
||||
border: 1px solid #ccc;
|
||||
padding: 8px;
|
||||
text-align: left;
|
||||
vertical-align: middle;
|
||||
}
|
||||
thead th {
|
||||
background-color: #1f4e78;
|
||||
color: white;
|
||||
text-align: center;
|
||||
vertical-align: middle;
|
||||
}
|
||||
</style>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Product Series</th>
|
||||
<th>Product</th>
|
||||
<th>Datasheet</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td style="text-align: center;">Gemini 435Le</td>
|
||||
<td>Gemini 435Le</td>
|
||||
<td><a href="https://new-orbbec3d-s3.s3.amazonaws.com/wp-content/uploads/2025/06/04011158/Orbbec-Gemini-435Le-Datasheet-V1.pdf">Orbbec Gemini 435Le Datasheet</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align: center;" rowspan="6">Gemini 330</td>
|
||||
<td>Gemini 335</td>
|
||||
<td rowspan="4"><a href="https://new-orbbec3d-s3.s3.amazonaws.com/wp-content/uploads/2025/04/22062452/Gemini-330-series-Datasheet-V1.6.pdf">Gemini 330 Series Datasheet for USB Devices</a></td>
|
||||
</tr>
|
||||
<tr><td>Gemini 336</td></tr>
|
||||
<tr><td>Gemini 335L</td></tr>
|
||||
<tr><td>Gemini 336L</td></tr>
|
||||
<tr>
|
||||
<td>Gemini 335Lg</td>
|
||||
<td><a href="https://new-orbbec3d-s3.s3.amazonaws.com/wp-content/uploads/2024/10/22030914/Gemini-335Lg-Datasheet-V1.0-241022.pdf">Gemini 330 Series Datasheet for GMSL Devices</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini 335Le</td>
|
||||
<td><a href="https://new-orbbec3d-s3.s3.amazonaws.com/wp-content/uploads/2025/03/24023151/Orbbec-Gemini-335Le-Datasheet-V1-2.pdf">Gemini 330 Series Datasheet for Ethernet Devices</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align: center;" rowspan="3">Gemini 2</td>
|
||||
<td>Gemini 2</td>
|
||||
<td rowspan="2"><a href="https://xm917ch2uk.feishu.cn/file/Khxfb2vdioUghexIMqJcAyL3nXf">Orbbec Gemini 2 Series Datasheet</a></td>
|
||||
</tr>
|
||||
<tr><td>Gemini 2 L</td></tr>
|
||||
<tr>
|
||||
<td>Gemini 2 XL</td>
|
||||
<td><a href="https://xm917ch2uk.feishu.cn/file/QW2vbNvwxoocRIxSL6Zcvut2npS">Orbbec Gemini 2 XL Datasheet</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align: center;" rowspan="3">Femto</td>
|
||||
<td>Femto Bolt</td>
|
||||
<td><a href="https://d1cd332k3pgc17.cloudfront.net/wp-content/uploads/2024/08/ORBBEC_Datasheet_Femto-Bolt-v1.0.pdf">Orbbec Femto Bolt Datasheet</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Femto Mega</td>
|
||||
<td><a href="https://d1cd332k3pgc17.cloudfront.net/wp-content/uploads/2023/04/ORBBEC_Datasheet_Femto-Mega1.pdf">Orbbec Femto Mega Datasheet</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Femto Mega I</td>
|
||||
<td><a href="https://d1cd332k3pgc17.cloudfront.net/wp-content/uploads/2023/08/ORBBEC_Datasheet_Femto-Mega-I.pdf">Orbbec Femto Mega I Datasheet</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align: center;" rowspan="3">Astra</td>
|
||||
<td>Astra 2</td>
|
||||
<td><a href="https://d1cd332k3pgc17.cloudfront.net/wp-content/uploads/2023/04/ORBBEC_Datasheet_Astra-2_V1.2.pdf">Orbbec Astra 2 Datasheet</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Astra+</td>
|
||||
<td><a href="https://xm917ch2uk.feishu.cn/file/Qk0zbx26Doh8XMxw0rIcOgQYnff">Orbbec Astra+ Datasheet</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Astra Mini Pro</td>
|
||||
<td><a href="https://d1cd332k3pgc17.cloudfront.net/wp-content/uploads/2023/04/ORBBEC_Datasheet_Astra-Mini-Pro-1.pdf">Orbbec Astra Mini Pro Datasheet</a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
---
|
||||
|
||||
## Support Platforms
|
||||
|
||||
- Linux x64: tested on Ubuntu 22.04
|
||||
- Linux ARM64: tested on NVIDIA Jetson AGX Orin , NVIDIA Jetson Orin NX , NVIDIA Jetson Orin Nano , NVIDIA Jetson AGX Xavier , NVIDIA Jetson Xavier NX
|
||||
@@ -0,0 +1,116 @@
|
||||
# Orbbec SDK Overview
|
||||
|
||||
This section introduces the Orbbec SDK in C++. Its architecture and concepts are consistent with those of the Python Wrapper.
|
||||
|
||||
## Terms
|
||||
|
||||
| ID | Name | Explain |
|
||||
| --- | --- | --- |
|
||||
| 1 | USB | Universal Serial Bus |
|
||||
| 2 | UVC | USB Video Class |
|
||||
| 3 | Firmware | Firmware of 3D camera |
|
||||
| 4 | Disparity | Disparity is to observe the direction difference of the same target from two points with a certain distance. |
|
||||
| 5 | D2D (Disparity to depth) | Disparity to depth is an image processing technique used to convert disparity information into depth information. |
|
||||
| 6 | Hardware D2D | Disparity to depth is implemented internally in the device, without occupying the computational power of the host computer. |
|
||||
| 7 | Software D2D | Disparity to depth, implemented in Orbbec SDK |
|
||||
| 8 | Depth point cloud | Depth point cloud, the coordinates of points in a three-dimensional world coordinate system, can be transformed into a point cloud using the intrinsic parameters of a Depth camera. |
|
||||
| 9 | RGBD point cloud | Point cloud with overlaid RGB information |
|
||||
| 10 | D2C | The translation of "Depth to Color" is a feature that performs per-pixel geometric transformation on a depth image. Its result is aligning the depth image with its corresponding color image through the D2C transformation, allowing us to locate the depth information of a color pixel by using the same image coordinate position of that pixel in the transformed depth image. After the D2C transformation, we generate a depth image of the same size as the target color image, where the image content represents depth data in the coordinate system of the color camera. In other words, it reconstructs a depth image "captured" using the origin and dimensions of the color camera, where each pixel matches the corresponding pixel coordinates of the color camera. |
|
||||
| 11 | Hardware D2C | Hardware D2C refers to the functionality of performing Depth to Color transformation within the camera itself, with the camera directly outputting the result of the D2C transformation. |
|
||||
| 12 | Software D2C | Performing D2C computation on the host computer side using an SDK. |
|
||||
| 13 | Frame aggregation (FrameSet) | Combining Depth, IR, and Color frames into a Frameset and invoking it through a pipeline. |
|
||||
| 14 | C2D | The translation of "Color to Depth" is a feature that performs per-pixel geometric transformation on a color image. Its result is aligning the color image with its corresponding depth image through the C2D transformation. |
|
||||
| 15 | MetaData | Frame metadata is a set of parameters (or attributes) that provide a snapshot of the sensor configuration and/or system state present during the frame’s generation. |
|
||||
| 16 | HDR | High Dynamic Range (HDR) imaging allows imaging systems to capture images in extremely dark and bright scenes alike. We propose a software solution running on the host CPU to implement this feature. It utilizes data from two consecutive frames and directly synthesizes these two depth images, thereby enhances the dynamic range of 16-bit depth images. |
|
||||
| 17 | LDP | Laser close-range protection |
|
||||
|
||||
## Orbbec SDK v2 Architecture Overview
|
||||
|
||||
|
||||
|
||||
- Application
|
||||
|
||||
OrbbecViewer, Sample, and User Application Implementation.
|
||||
|
||||
- Interfaces and Encapsulation Layer
|
||||
|
||||
OrbbecSDK Interface Encapsulation and Wrapper Encapsulation.
|
||||
|
||||
- High-level Layer
|
||||
|
||||
HighLevel encapsulates the core business components and provides interfaces to the outside using a pipeline.
|
||||
|
||||
- Basic business layer
|
||||
|
||||
The realization of the core business logic framework.
|
||||
|
||||
- Platform abstraction layer
|
||||
|
||||
Cross-platform components abstract operating system differences and provide a unified access interface.
|
||||
|
||||
|
||||
- Platform implementation layer
|
||||
|
||||
The driver implementation of each platform.
|
||||
|
||||
## SDK Concept Overview
|
||||
|
||||
- Context
|
||||
|
||||
|
||||
Context which provides a set of settings includes settings such as device state change callbacks, log levels, and more. The Context can access multiple devices.
|
||||
|
||||
- Device
|
||||
|
||||
|
||||
One actual hardware device corresponds to one Device object, which is used to obtain relevant information of the device and control its attributes.
|
||||
|
||||
- Pipeline
|
||||
|
||||
|
||||
The HighLevel corresponding object encapsulates the interface for quick access to the SDK. It has simple functions that allow users to quickly get started and use the SDK.
|
||||
|
||||
- Config
|
||||
|
||||
|
||||
Provides configuration for enabling data streams, alignment modes, and frame aggregation modes, It is used to control the behavior of the data output.
|
||||
|
||||
- StreamProfile
|
||||
|
||||
|
||||
Stream configuration that defines parameters such as resolution, frame rate, and encoding format, It also provides management of camera parameters.
|
||||
|
||||
- Frame
|
||||
|
||||
|
||||
Represents a frame of data in the Stream, and also contains relevant information about that frame of data, such as timestamp, type, etc.
|
||||
|
||||
- Filter
|
||||
|
||||
|
||||
It mainly refers to some algorithmic processing modules for the composite stream FrameSet, such as point cloud algorithm processing.
|
||||
|
||||
- Record
|
||||
|
||||
|
||||
Recording functionality that captures data streams and saves them as files for later analysis or playback.
|
||||
|
||||
- Playback
|
||||
|
||||
|
||||
Playback functionality that plays recorded files and supports control over playback speed and other related parameters.
|
||||
|
||||
|
||||
## SDK Programming Model
|
||||
|
||||
Here is the C++ programming logic flow chart. Python's programming logic is the same as it.
|
||||
|
||||
- Standard Flowchart:
|
||||
|
||||
|
||||
|
||||
The standard flowchart demonstrates how to create a device from the device list, set and get parameters, and apply post-processing filters.
|
||||
|
||||
- Flowchart using default configuration (stream acquisition based on the default settings in OrbbecSDKConfig.xml):
|
||||
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
Overview
|
||||
======================================================
|
||||
|
||||
This chapter provides an overview of the Orbbec SDK, including supported products, main features, and architecture.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
introduction.md
|
||||
orbbecsdk_overview.md
|
||||
|
||||
Reference in New Issue
Block a user