mirror of
https://github.com/introlab/rtabmap.git
synced 2026-10-06 01:57:45 +08:00
Adding doc and tests (#1492)
* added doc and tests for util2d.h * updated cmake-ros ci * Added util3d.h doc and tests * util3d_transforms.h: Added doc and tests * util3d_filtering.h: started doc and test * util3d_filtering.h: more tests and doc * Added more doc/tests * finished util3d_filtering doc and tests * added test for util2d::depthBleedingFiltering * Added util3d_registration tests * Added util3d_features.h doc/tests * added doc/tests for util3d_correspondences.h * added doc/gtest for util3d_mapping.h (missing hpp functions) * finished testing util3d_mapping.hpp * Added util3d_motion_estimation.h tests (2D->3D done) * finished util3d_motion_estimation.h tests * minimal util3d_surface.h * Added Transform and VisualWord tests * Added doc for CameraModel and StereoCameraModel * Added more logs in ros ci * Passing tests on fical * improved all devcontainer * added devcontainer kilted, fixed source setup.bash, removed ldconfig in ros-cmake workflow * cleanup * source ros * Added utilite tests * Added testing to appveyor, github actions cancellable on re-commit on same branch * appveyor testing without all targets * appveyor: specifying ALL_BUILD target * Fixed Util2dTest.NMSImageBoundsRespected test * Fixing PCL Indices error on old pcl * Added VWDictionary tests and doc. Fixed LSH not working (fix from https://github.com/flann-lib/flann/pull/472 * fixing some appveyor CI errors, added test to check dictionary serialization against all type * Added StereoDense, StereoBM and StereoSGBM doc and tests * Added Stereo tests * Added CameraModel and StereoCameraModel tests * Added doc and test for Statistics * Added doc/tests for Signature * Added doc/test for SensorEvent, added doc for SensorCaptureInfo * Added doc to SensorData * Added SensorData tests * Added SensorCapture and SensorCaptureThread doc and tests * fixed sensordata test * updated SSC test and doc * Added doc and tests for BayesFilter class * Enabled testing on mac, updated windows testing like on linux * added test_link * fixed unresolved on windows * fixed ThreadHandle error on macos ci * Added GPS and GeodeticCoords tests * Added tests for compression * Added Odometry tests (base class only) * Added DBDriver tests * Added coverage report * uniformized test names * fixing concurancy and coverage ci * dont built tools, examples and app for coverage build * fixed report tool rebuilt without qt compilation error * updated coverage option * updated coverage config * added doc CI job * fixing windows and mac ci errors * Added DBDriverSqlite3 tests * Added IMU tests * Added Graph tests * fixing flaky macos test * Added IMUThread and IMUFilter tests * Added Landmarks tests * Added LASWriter tests * fixing seed flaky test * fixing flaky macos timing tests * Added LocalGrid tests * Added LocalGridMaker tests * fixing ci errors * Added GlobalMap tests * Added doc for EnvSensor * Added Features2D tests * Added Registration tests * Added RegistrationVis tests * Added doc for Rtabmap and Memory classes * Added Memory and Rtabmap tests * making some tests less flaky * lcov 1.14 support * updated compatible tool arguments * Added integration tests (RGB-D, Stereo, Lidar2d, Lidar3d) * More octomap checks * Refactored how/when python interpretor is created to simplify library usage * Added python tests * fixed some flaky tests * suppressed some third party related warnings * fixed ceres tests * more flaky fixes * Fixing tests without libpointmatcher * Added RANSAC rejection filter to PCL ICP * fixing multi platform flakiness * Added test to detect regression * Fixing windows pcl link error * fixed some macos flakiness * bigger 2D2D registration error on opencv 4.6.0 * flakiness * fixing flaky tests on windows and mac * flaky thread test on slow mac VM * windows slow test * fixing more ci erros * fxing temp dir on windows * Added Optimizer tests and discovered some bugs (fixed) * fixing flaky tests in mac and windows * Added Optimizer doc * Added GTSAM BA, updated Ceres to use g2o ba parameters. Renamed g2o's ba related parameters to Optimizer group and used by both gtsam and ceres. * fixing build without gtsam * fixing home dir * fixing python ci isssues * Added multicam ba tests * Added Ceres multicam BA support * Aligned BundleAdjustment parameters with Optimizer/Strategy to avoid confusion in the code * Added BA integration test * Added robust graph optimization integration test * Added loop3it test * Added stereo20Hz test * Added smartfactor gtsam * Fixed bugged check and warn if python didn't return any descriptors * Fixing gtsam version build issues * fixing tilt on windows ci * loosing ceres integration test for ci * mac ci flakiness * updating missing param in gui * updating test bound for mac * added appearance-based tests, set min gftt quality to quality level * testing more stuff * improving features2d tests * ci flakiness * fixing flaky ci * ci fixes * flaky fixes * Added RegistrationIcp tests * Added icp integration test with real-worl corridor like env * intermediate nodes * fixing enum * Updated test to catch #1714 * Fixed 2d corridor failing on pcl * flaky pnp test * flaky brisk test * Set rtabmap_integration test as long * updating loop closure test * flaky ci tests * TEsting roundtrip g2o/toro save/load * loosing test bound * fixed cuda capable checks * flaky tests * Debugging test hanging * more debugging stuff * updating limit * windows: disabled cuda on ci to avoid incompatible driver issue. Fixing a bad test mem allocation * trying fixing cuda hanging issue * fixing ci flakyness * flaky tests * Updated BOW flaky tests by checking min precision/recall instead of recall@100precision. Fixed signature test * CameraModel::load() test initRectificationMap param * test dbdriver load dictionary idsOnly * Memory: test keepLinkedInDb param * added dummyDictionary tests * test intermediate nodes count * Added MarkerDetector tests * reverted breaking change of UMutex and USemaphore * Features2d: fixed compiltion warnings with clang about override * clang warnings * fixing test build with pcl 1.8 * g2o and gtsam build errors on android * opencv5 test fixes * disabled testing for ios and android builds * normalized endline characters for easier diff * added LF CRLF rule * bump 0.23.10. fixing doc version * Publish rtabmap website doc from ci * fixing MSCVC build error * macos icp flaky test * fixing ceres macos test bound * ficing more flaky tests * fixing opencv5 related test errors. Also fixed an actual bug in ENU_WGS84ToGeocentric_WGS84() * added comment about mrpt change * removed rosdoc2 (will add it for rtabmap_ros later) * fixing website style * updated download links * locally deployable website with api * sweep doxygen issues * improved/revised doxygen main pages * removed examples empty page * Updated doxygen style * more concise doxygen groups * added api link on main readme * fixing utilite test error * fixing CommonFilteringGroundNormalsUp test * updated precisionRecall test bounds for Freak and brief descriptors * fixing scale check in ba tests * disabled tests on windows cuda build (missing dlls amd runner cannot test cuda anyway) * ceres: missing suitesparse dep in windows ci * adjusting recall thr for fast/freak * ficing more flaky tests * fixing flaky tests * disabled coverage in ros ci * Enable integration tests for ros ci jobs * loosing up some threshold for failing tests * trigger cache * fixing test data in ros ci. Updated flaky test for mac * slaking some test limit * Fixed rtabmap-detectMoreLoopClosures inverted output value * loosing up sift recall on mac * optimizer re-ordered distribution for reproducible results (mac g2o) * macos dump test crash log * combining all tests to save time on shared library reload. Also fixed Logs with missing arguments. * Added ENABLE_FORMAT_ERRORS cmake option * do test only one time * fixed all format warnings * format security android build errors * less verbose tests * updated ImuUThread test * fixed a log * Fixed libpointmatcher 2d normals eigen issue * Fixing libpointmatcher conversion issues * fixing libpointmatcher test on windows ci * cleanup comments, relax some test thr * disabled sequoia-intel ci build (too flaky, would need extensive testing directly on that machine)
This commit is contained in:
@@ -32,10 +32,63 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
namespace rtabmap {
|
||||
|
||||
/**
|
||||
* @class StereoCameraModel
|
||||
* @brief A class representing a calibrated stereo camera system.
|
||||
*
|
||||
* This class encapsulates the calibration data and operations associated with a stereo camera setup,
|
||||
* including intrinsic and extrinsic parameters for both left and right cameras, stereo rectification,
|
||||
* and methods for computing depth or disparity from stereo images.
|
||||
*
|
||||
* It relies internally on two `CameraModel` instances for the left and right cameras.
|
||||
*
|
||||
* Typical uses include:
|
||||
* - Stereo rectification
|
||||
* - Stereo disparity-to-depth conversion
|
||||
* - Saving and loading stereo camera calibration data
|
||||
* - Projecting or reprojecting points
|
||||
*
|
||||
* @see CameraModel
|
||||
*/
|
||||
class RTABMAP_CORE_EXPORT StereoCameraModel
|
||||
{
|
||||
public:
|
||||
/**
|
||||
* @brief Default constructor. Creates an empty stereo model with default suffixes ("left", "right").
|
||||
*/
|
||||
StereoCameraModel() : leftSuffix_("left"), rightSuffix_("right") {}
|
||||
|
||||
/**
|
||||
* @brief Constructs a StereoCameraModel from detailed intrinsic and extrinsic parameters for both cameras.
|
||||
*
|
||||
* Initializes the stereo camera model by specifying the calibration parameters for the left and right cameras,
|
||||
* along with the stereo extrinsic parameters.
|
||||
*
|
||||
* @param name Name identifier for the stereo camera.
|
||||
* @param imageSize1 Image size (width, height) of the left camera.
|
||||
* @param K1 Intrinsic camera matrix (3x3, CV_64FC1) for the left camera.
|
||||
* @param D1 Distortion coefficients for the left camera.
|
||||
* @param R1 Rectification matrix (3x3, CV_64FC1) for the left camera.
|
||||
* @param P1 Projection matrix (3x4, CV_64FC1) for the left camera.
|
||||
* @param imageSize2 Image size (width, height) of the right camera.
|
||||
* @param K2 Intrinsic camera matrix (3x3, CV_64FC1) for the right camera.
|
||||
* @param D2 Distortion coefficients for the right camera.
|
||||
* @param R2 Rectification matrix (3x3, CV_64FC1) for the right camera.
|
||||
* @param P2 Projection matrix (3x4, CV_64FC1) for the right camera.
|
||||
* @param R Rotation matrix (3x3, CV_64FC1) representing the rotation from left to right camera coordinate system.
|
||||
* Can be empty if unknown.
|
||||
* @param T Translation vector (3x1, CV_64FC1) representing the translation from left to right camera coordinate system.
|
||||
* Can be empty if unknown.
|
||||
* @param E Essential matrix (3x3, CV_64FC1) encoding the stereo camera epipolar geometry.
|
||||
* Can be empty if unknown.
|
||||
* @param F Fundamental matrix (3x3, CV_64FC1) encoding the stereo camera epipolar constraints.
|
||||
* Can be empty if unknown.
|
||||
* @param localTransform The local transform associated with the stereo camera model.
|
||||
*
|
||||
* @note All matrices must have correct sizes and types as specified.
|
||||
* The rectification and projection matrices (R1, P1, R2, P2) are used to define the stereo rectification parameters.
|
||||
* The rotation and translation (R, T) define the relative pose between the cameras.
|
||||
*/
|
||||
StereoCameraModel(
|
||||
const std::string & name,
|
||||
const cv::Size & imageSize1,
|
||||
@@ -45,7 +98,29 @@ public:
|
||||
const cv::Mat & R, const cv::Mat & T, const cv::Mat & E, const cv::Mat & F,
|
||||
const Transform & localTransform = Transform(0,0,1,0, -1,0,0,0, 0,-1,0,0));
|
||||
|
||||
// if R and T are not null, left and right camera models should be valid to be rectified.
|
||||
/**
|
||||
* @brief Constructs a StereoCameraModel from two individual camera models and optional stereo extrinsic parameters.
|
||||
*
|
||||
* This constructor initializes the stereo camera model by assigning the provided left and right camera models.
|
||||
* If the stereo extrinsics (`R`, `T`) are provided and valid, stereo rectification will be attempted—provided both
|
||||
* cameras are valid for rectification and their image dimensions match.
|
||||
*
|
||||
* Each camera model will automatically have its name updated using the `name` parameter and default suffixes ("left", "right").
|
||||
*
|
||||
* @param name The base name for the stereo camera model.
|
||||
* @param leftCameraModel The camera model representing the left camera.
|
||||
* @param rightCameraModel The camera model representing the right camera.
|
||||
* @param R (Optional) Rotation matrix of the left camera relative to the right camera coordinate system (3x3, CV_64FC1).
|
||||
* @param T (Optional) Translation vector of the left camera relative to the right camera coordinate system (3x1, CV_64FC1).
|
||||
* @param E (Optional) Essential matrix between the two cameras (3x3, CV_64FC1).
|
||||
* @param F (Optional) Fundamental matrix between the two cameras (3x3, CV_64FC1).
|
||||
*
|
||||
* @throws UException if any of the provided matrices (`R`, `T`, `E`, `F`) are non-empty and not of the expected type/shape.
|
||||
* @throws UException if `R` and `T` are provided but the camera models are not valid for rectification.
|
||||
*
|
||||
* @note Stereo rectification is only attempted if both `R` and `T` are non-empty, the cameras are valid, and their image sizes match.
|
||||
* @see updateStereoRectification()
|
||||
*/
|
||||
StereoCameraModel(
|
||||
const std::string & name,
|
||||
const CameraModel & leftCameraModel,
|
||||
@@ -54,14 +129,54 @@ public:
|
||||
const cv::Mat & T = cv::Mat(),
|
||||
const cv::Mat & E = cv::Mat(),
|
||||
const cv::Mat & F = cv::Mat());
|
||||
// if extrinsics transform is not null, left and right camera models should be valid to be rectified.
|
||||
|
||||
/**
|
||||
* @brief Constructs a StereoCameraModel from two camera models and an extrinsic Transform between them.
|
||||
*
|
||||
* This constructor sets up a stereo camera model using the given left and right camera models along with
|
||||
* an optional 3D transform (`extrinsics`) representing the pose of the left camera relative to the right camera coordinate system.
|
||||
*
|
||||
* If a valid (non-null) transform is provided, the corresponding rotation and translation matrices are extracted
|
||||
* and stored as the stereo extrinsic parameters. Stereo rectification will be attempted if both camera models
|
||||
* are valid for rectification and their image sizes match.
|
||||
*
|
||||
* Each camera model will be renamed using the provided `name` and default suffixes ("left", "right").
|
||||
*
|
||||
* @param name Base name for the stereo camera model.
|
||||
* @param leftCameraModel Camera model for the left camera.
|
||||
* @param rightCameraModel Camera model for the right camera.
|
||||
* @param extrinsics (Optional) Transform of the left camera relative to the right camera coordinate system. If null, no extrinsics are used.
|
||||
*
|
||||
* @throws UException if `extrinsics` is not null and either camera model is not valid for rectification.
|
||||
*
|
||||
* @note Stereo rectification is performed only when `extrinsics` is valid and both camera models are rectifiable
|
||||
* with matching image dimensions.
|
||||
* @see updateStereoRectification()
|
||||
*/
|
||||
StereoCameraModel(
|
||||
const std::string & name,
|
||||
const CameraModel & leftCameraModel,
|
||||
const CameraModel & rightCameraModel,
|
||||
const Transform & extrinsics);
|
||||
|
||||
//minimal
|
||||
/**
|
||||
* @brief Minimal constructor using focal lengths and baseline only.
|
||||
*
|
||||
* Creates a simplified stereo camera model using only the essential intrinsic parameters
|
||||
* and baseline. This constructor assumes the images are already rectified and both cameras
|
||||
* have the same intrinsic parameters.
|
||||
*
|
||||
* @param fx Focal length in x direction (pixels).
|
||||
* @param fy Focal length in y direction (pixels).
|
||||
* @param cx Principal point x coordinate (pixels).
|
||||
* @param cy Principal point y coordinate (pixels).
|
||||
* @param baseline Stereo baseline distance in meters.
|
||||
* @param localTransform Local transform from camera to robot base frame (default: optical rotation).
|
||||
* @param imageSize Image size (width, height). Optional, can be set later.
|
||||
*
|
||||
* @note This constructor creates a simplified model suitable for rectified stereo pairs.
|
||||
* For full calibration with distortion, use the constructors that accept camera matrices.
|
||||
*/
|
||||
StereoCameraModel(
|
||||
double fx,
|
||||
double fy,
|
||||
@@ -70,7 +185,24 @@ public:
|
||||
double baseline,
|
||||
const Transform & localTransform = Transform(0,0,1,0, -1,0,0,0, 0,-1,0,0),
|
||||
const cv::Size & imageSize = cv::Size(0,0));
|
||||
//minimal to be saved
|
||||
|
||||
/**
|
||||
* @brief Minimal constructor that also sets a name, required if we want to save it to a file.
|
||||
*
|
||||
* Same as the minimal constructor but also sets the camera name, which is required
|
||||
* when saving the calibration to disk.
|
||||
*
|
||||
* @param name Camera name identifier (used for saving calibration files).
|
||||
* @param fx Focal length in x direction (pixels).
|
||||
* @param fy Focal length in y direction (pixels).
|
||||
* @param cx Principal point x coordinate (pixels).
|
||||
* @param cy Principal point y coordinate (pixels).
|
||||
* @param baseline Stereo baseline distance in meters.
|
||||
* @param localTransform Local transform from camera to robot base frame (default: optical rotation).
|
||||
* @param imageSize Image size (width, height). Optional, can be set later.
|
||||
*
|
||||
* @note Use this constructor when you plan to save the calibration to a file.
|
||||
*/
|
||||
StereoCameraModel(
|
||||
const std::string & name,
|
||||
double fx,
|
||||
@@ -80,68 +212,347 @@ public:
|
||||
double baseline,
|
||||
const Transform & localTransform = Transform(0,0,1,0, -1,0,0,0, 0,-1,0,0),
|
||||
const cv::Size & imageSize = cv::Size(0,0));
|
||||
|
||||
/**
|
||||
* @brief Destructor.
|
||||
*/
|
||||
virtual ~StereoCameraModel() {}
|
||||
|
||||
/**
|
||||
* @brief Returns true if both left and right models are valid for projection and the baseline is positive.
|
||||
*/
|
||||
bool isValidForProjection() const {return left_.isValidForProjection() && right_.isValidForProjection() && baseline() > 0.0;}
|
||||
|
||||
/**
|
||||
* @brief Returns true if both left and right models are valid for rectification.
|
||||
*/
|
||||
bool isValidForRectification() const {return left_.isValidForRectification() && right_.isValidForRectification();}
|
||||
|
||||
/**
|
||||
* @brief Initializes the rectification maps for both cameras.
|
||||
*/
|
||||
void initRectificationMap() {left_.initRectificationMap(); right_.initRectificationMap();}
|
||||
|
||||
/**
|
||||
* @brief Returns true if rectification maps are initialized.
|
||||
*/
|
||||
bool isRectificationMapInitialized() const {return left_.isRectificationMapInitialized() && right_.isRectificationMapInitialized();}
|
||||
|
||||
/**
|
||||
* @brief Sets the camera name and optional image suffixes for the left and right cameras.
|
||||
*
|
||||
* Updates the stereo camera model name and the suffixes used for identifying left and right
|
||||
* camera calibration files. The suffixes are used when loading/saving calibration data from disk.
|
||||
*
|
||||
* @param name Base name for the stereo camera model.
|
||||
* @param leftSuffix Suffix for the left camera (default: "left"). Used in filenames like "cameraName_left.yaml".
|
||||
* @param rightSuffix Suffix for the right camera (default: "right"). Used in filenames like "cameraName_right.yaml".
|
||||
*
|
||||
* @note The suffixes are used by load() and save() methods to construct filenames for each camera.
|
||||
*/
|
||||
void setName(const std::string & name, const std::string & leftSuffix = "left", const std::string & rightSuffix = "right");
|
||||
|
||||
/**
|
||||
* @brief Gets the camera name.
|
||||
*/
|
||||
const std::string & name() const {return name_;}
|
||||
|
||||
// backward compatibility
|
||||
/**
|
||||
* @brief Sets the image size for both left and right cameras.
|
||||
*/
|
||||
void setImageSize(const cv::Size & size) {left_.setImageSize(size); right_.setImageSize(size);}
|
||||
|
||||
// Set initRectificationMaps=false to skip building the (potentially large) left/right
|
||||
// rectification maps when rectification won't be used (saves time and memory).
|
||||
/**
|
||||
* @brief Loads stereo camera calibration data from disk.
|
||||
*
|
||||
* This method loads the intrinsic parameters for both the left and right cameras from files in the specified directory,
|
||||
* using the provided camera name and internal suffixes. If `ignoreStereoTransform` is false, it also attempts to load
|
||||
* the stereo extrinsic parameters (rotation, translation, essential, and fundamental matrices) from a YAML file.
|
||||
*
|
||||
* The stereo extrinsics are expected in the file:
|
||||
* `directory/cameraName_pose.yaml`, following the ROS calibration format.
|
||||
*
|
||||
* @param directory The directory where the calibration files are located.
|
||||
* @param cameraName The base name of the stereo camera (used to derive filenames).
|
||||
* @param ignoreStereoTransform If true, skips loading stereo extrinsic parameters.
|
||||
* @param initRectificationMaps Set to false to skip building the (potentially large) left/right
|
||||
* rectification maps when rectification won't be used (saves time and memory).
|
||||
* @return true if loading is successful, false otherwise.
|
||||
*
|
||||
* @see save(), saveStereoTransform(), CameraModel::initRectificationMap()
|
||||
*/
|
||||
bool load(const std::string & directory, const std::string & cameraName, bool ignoreStereoTransform = true, bool initRectificationMaps = true);
|
||||
|
||||
/**
|
||||
* @brief Saves stereo camera calibration data to disk.
|
||||
*
|
||||
* This method saves the intrinsic parameters of both left and right cameras to the specified directory.
|
||||
* If `ignoreStereoTransform` is false, it also saves the stereo extrinsic parameters (rotation, translation,
|
||||
* essential, and fundamental matrices) in a ROS-compatible YAML file named `cameraName_pose.yaml`.
|
||||
*
|
||||
* @param directory The directory where calibration files should be saved.
|
||||
* @param ignoreStereoTransform If true, skips saving stereo extrinsic parameters.
|
||||
* @return true if saving was successful, false otherwise.
|
||||
*
|
||||
* @see load(), saveStereoTransform()
|
||||
*/
|
||||
bool save(const std::string & directory, bool ignoreStereoTransform = true) const;
|
||||
|
||||
/**
|
||||
* @brief Saves stereo extrinsic parameters to a YAML file in ROS format.
|
||||
*
|
||||
* This method exports the stereo transform, including rotation, translation, essential, and fundamental matrices,
|
||||
* into a YAML file named `cameraName_pose.yaml` located in the specified directory.
|
||||
* The file format is compatible with ROS camera calibration tools.
|
||||
*
|
||||
* @param directory The target directory for saving the calibration file.
|
||||
* @return true if saving was successful, false if required matrices are missing or invalid.
|
||||
*
|
||||
* @warning If extrinsics (`R_`, `T_`, `E_`, `F_`) are empty or invalid, nothing will be saved and a warning is printed.
|
||||
*
|
||||
* @see load(), save()
|
||||
*/
|
||||
bool saveStereoTransform(const std::string & directory) const;
|
||||
|
||||
/**
|
||||
* @brief Serializes the stereo camera model into a byte vector.
|
||||
*
|
||||
* This method serializes the left and right camera models along with the stereo extrinsic parameters
|
||||
* (rotation matrix R_, translation vector T_, essential matrix E_, and fundamental matrix F_) into a
|
||||
* contiguous byte array. The serialization format starts with a fixed-size integer header containing
|
||||
* version info, stereo type, matrix sizes, and serialized data sizes, followed by the actual matrices and
|
||||
* serialized camera data.
|
||||
*
|
||||
* The serialized data can later be restored using the corresponding `deserialize()` method.
|
||||
*
|
||||
* @return A vector of unsigned char containing the serialized stereo camera data.
|
||||
*/
|
||||
std::vector<unsigned char> serialize() const;
|
||||
|
||||
/**
|
||||
* @brief Deserializes stereo camera model data from a byte vector.
|
||||
*
|
||||
* This method wraps the pointer-based `deserialize()` and attempts to restore the stereo camera
|
||||
* model from the given serialized byte vector.
|
||||
*
|
||||
* @param data The vector of bytes containing previously serialized stereo camera model data.
|
||||
* @return The number of bytes read from the data if successful, 0 otherwise.
|
||||
*
|
||||
* @see deserialize(const unsigned char*, unsigned int)
|
||||
*/
|
||||
unsigned int deserialize(const std::vector<unsigned char>& data);
|
||||
|
||||
/**
|
||||
* @brief Deserializes stereo camera model data from a raw byte array.
|
||||
*
|
||||
* This method reconstructs the stereo camera model from the provided serialized data buffer.
|
||||
* It expects the data format to match the one produced by `serialize()`, including a header with
|
||||
* version info, matrix sizes, and data sizes, followed by the serialized extrinsic matrices and
|
||||
* serialized left and right camera data.
|
||||
*
|
||||
* The method performs various sanity checks on data sizes and matrix dimensions and will fail if
|
||||
* the data format or sizes are inconsistent.
|
||||
*
|
||||
* @param data Pointer to the raw serialized data buffer.
|
||||
* @param dataSize Size in bytes of the data buffer.
|
||||
* @return The number of bytes consumed during deserialization if successful, or 0 on failure.
|
||||
*
|
||||
* @warning The stereo camera model is reset to a default empty state before deserialization.
|
||||
* @warning If the serialized data type is not stereo (type != 1), deserialization will fail.
|
||||
*
|
||||
* @see serialize()
|
||||
*/
|
||||
unsigned int deserialize(const unsigned char * data, unsigned int dataSize);
|
||||
|
||||
/**
|
||||
* @brief Returns the stereo baseline in meters.
|
||||
*
|
||||
* Computes the baseline distance between the left and right cameras using the projection
|
||||
* matrices. The baseline is calculated as the difference in x-translation (Tx) normalized
|
||||
* by the focal length.
|
||||
*
|
||||
* @return The baseline distance in meters. Returns 0.0 if focal lengths are invalid or zero.
|
||||
*
|
||||
* @note The baseline is a physical distance and is essential for depth computation from disparity.
|
||||
*/
|
||||
double baseline() const {return right_.fx()!=0.0 && left_.fx() != 0.0 ? left_.Tx() / left_.fx() - right_.Tx()/right_.fx():0.0;}
|
||||
|
||||
/**
|
||||
* @brief Computes the depth (Z coordinate) from a given disparity value.
|
||||
*
|
||||
* Uses the stereo camera model parameters to convert disparity to depth using the formula:
|
||||
* \f[
|
||||
* \text{depth} = \frac{\text{baseline} \times f_x}{\text{disparity} + (c_{x_{right}} - c_{x_{left}})}
|
||||
* \f]
|
||||
* where \( f_x \) is the focal length of the left camera and \( c_x \) are principal points.
|
||||
*
|
||||
* @param disparity The disparity value (difference in pixel coordinates between left and right images).
|
||||
* @return The computed depth in the same unit as the baseline (typically meters).
|
||||
* Returns 0 if disparity is zero or if the model is not valid for projection.
|
||||
*
|
||||
* @note This function requires the stereo camera to be valid for projection (i.e., calibrated and rectified).
|
||||
*/
|
||||
float computeDepth(float disparity) const;
|
||||
|
||||
/**
|
||||
* @brief Computes the disparity value from a given depth.
|
||||
*
|
||||
* Converts depth back to disparity using the inverse formula:
|
||||
* \f[
|
||||
* \text{disparity} = \frac{\text{baseline} \times f_x}{\text{depth}} - (c_{x_{right}} - c_{x_{left}})
|
||||
* \f]
|
||||
*
|
||||
* @param depth Depth value in the same unit as the baseline (typically meters).
|
||||
* @return The computed disparity in pixels.
|
||||
* Returns 0 if depth is zero or if the model is not valid for projection.
|
||||
*
|
||||
* @note This function requires the stereo camera to be valid for projection (i.e., calibrated and rectified).
|
||||
*/
|
||||
float computeDisparity(float depth) const; // m
|
||||
|
||||
/**
|
||||
* @brief Computes the disparity value from a depth given in unsigned short format (millimeters).
|
||||
*
|
||||
* Converts depth expressed as an unsigned short (in millimeters) to disparity.
|
||||
* The depth is first converted to meters before computing disparity using the formula:
|
||||
* \f[
|
||||
* \text{disparity} = \frac{\text{baseline} \times f_x}{\text{depth (meters)}} - (c_{x_{right}} - c_{x_{left}})
|
||||
* \f]
|
||||
*
|
||||
* @param depth Depth value in millimeters as an unsigned short.
|
||||
* @return The computed disparity in pixels.
|
||||
* Returns 0 if depth is zero or if the model is not valid for projection.
|
||||
*
|
||||
* @note This function requires the stereo camera to be valid for projection (i.e., calibrated and rectified).
|
||||
*/
|
||||
float computeDisparity(unsigned short depth) const; // mm
|
||||
|
||||
const cv::Mat & R() const {return R_;} //extrinsic rotation matrix
|
||||
const cv::Mat & T() const {return T_;} //extrinsic translation matrix
|
||||
const cv::Mat & E() const {return E_;} //extrinsic essential matrix
|
||||
const cv::Mat & F() const {return F_;} //extrinsic fundamental matrix
|
||||
const cv::Mat & R() const {return R_;} ///< Stereo extrinsic rotation matrix.
|
||||
const cv::Mat & T() const {return T_;} ///< Stereo extrinsic translation vector.
|
||||
const cv::Mat & E() const {return E_;} ///< Essential matrix.
|
||||
const cv::Mat & F() const {return F_;} ///< Fundamental matrix
|
||||
|
||||
/**
|
||||
* @brief Scales both cameras' calibration by a factor.
|
||||
*
|
||||
* Scales the intrinsic parameters (focal lengths, principal points) and image sizes
|
||||
* of both left and right cameras by the given scale factor. This is useful when working
|
||||
* with downscaled or upscaled images.
|
||||
*
|
||||
* @param scale Scaling factor (> 0). For example, use 0.5 to downscale or 2.0 to upscale.
|
||||
*
|
||||
* @note The baseline is not scaled, as it represents a physical distance between cameras.
|
||||
* @note Only valid camera models are scaled. Invalid models are left unchanged.
|
||||
*/
|
||||
void scale(double scale);
|
||||
|
||||
/**
|
||||
* @brief Applies region-of-interest (ROI) cropping to both cameras.
|
||||
*
|
||||
* Adjusts both camera models for a region of interest by shifting the principal points
|
||||
* and updating the image sizes. This is useful when working with cropped or subwindowed images.
|
||||
*
|
||||
* @param roi Region of interest rectangle. The top-left corner defines the offset for principal points.
|
||||
*
|
||||
* @note The principal points (cx, cy) are adjusted by subtracting the ROI's top-left coordinates.
|
||||
* @note The image size is set to the ROI size.
|
||||
* @note Only valid camera models are adjusted. Invalid models are left unchanged.
|
||||
*/
|
||||
void roi(const cv::Rect & roi);
|
||||
|
||||
/**
|
||||
* @brief Sets the local transform from left camera to robot base.
|
||||
*/
|
||||
void setLocalTransform(const Transform & transform) {left_.setLocalTransform(transform);}
|
||||
|
||||
/**
|
||||
* @brief Gets the local transform from left camera to robot base.
|
||||
*/
|
||||
const Transform & localTransform() const {return left_.localTransform();}
|
||||
|
||||
/**
|
||||
* @brief Returns the stereo transform (left camera relative to right camera coordinate system).
|
||||
*
|
||||
* The stereo transform brings points given in the
|
||||
* first (left) camera's coordinate system to points in the second (right) camera's coordinate
|
||||
* system. In more technical terms, it performs a change of basis from the
|
||||
* first camera's coordinate system to the second camera's coordinate system. Due to its duality,
|
||||
* it is equivalent to the position of the first camera with respect to the second
|
||||
* camera coordinate system.
|
||||
*
|
||||
* @return Transform from left camera to right camera coordinate system. Returns identity if R_ or T_ are empty.
|
||||
*
|
||||
* @note The transform is constructed from the stereo extrinsic parameters R_ and T_.
|
||||
*
|
||||
* @par Example:
|
||||
* For a stereo camera with a baseline of 15 cm, where the right camera is positioned to the
|
||||
* right of the left camera, the x value of the returned Transform would be -0.15 (negative
|
||||
* because it represents the position of the left camera in the right camera's coordinate system).
|
||||
* @code
|
||||
* StereoCameraModel stereo(...);
|
||||
* Transform transform = stereo.stereoTransform();
|
||||
* // If baseline is 0.15 m, transform.x() would be approximately -0.15
|
||||
* @endcode
|
||||
*/
|
||||
Transform stereoTransform() const;
|
||||
|
||||
/**
|
||||
* @brief Returns the left camera model.
|
||||
*/
|
||||
const CameraModel & left() const {return left_;}
|
||||
/**
|
||||
* @brief Returns the right camera model.
|
||||
*/
|
||||
const CameraModel & right() const {return right_;}
|
||||
|
||||
/**
|
||||
* @brief Gets the suffix used for the left camera calibration file.
|
||||
*/
|
||||
const std::string & getLeftSuffix() const {return leftSuffix_;}
|
||||
/**
|
||||
* @brief Gets the suffix used for the right camera calibration file.
|
||||
*/
|
||||
const std::string & getRightSuffix() const {return rightSuffix_;}
|
||||
|
||||
private:
|
||||
/**
|
||||
* @brief Updates stereo rectification parameters for both cameras.
|
||||
*
|
||||
* This private method computes the rectification and projection matrices for both left and right
|
||||
* cameras based on the stereo extrinsic parameters (R_, T_). It is called automatically when
|
||||
* constructing a StereoCameraModel with valid extrinsics.
|
||||
*
|
||||
* @note Requires both R_ and T_ to be non-empty and valid.
|
||||
* @note Both camera models must be valid for rectification.
|
||||
*/
|
||||
void updateStereoRectification();
|
||||
|
||||
private:
|
||||
std::string leftSuffix_;
|
||||
std::string rightSuffix_;
|
||||
CameraModel left_;
|
||||
CameraModel right_;
|
||||
std::string name_;
|
||||
cv::Mat R_;
|
||||
cv::Mat T_;
|
||||
cv::Mat E_;
|
||||
cv::Mat F_;
|
||||
std::string leftSuffix_; ///< Suffix for the left calibration file.
|
||||
std::string rightSuffix_; ///< Suffix for the right calibration file.
|
||||
CameraModel left_; ///< Left camera model.
|
||||
CameraModel right_; ///< Right camera model.
|
||||
std::string name_; ///< Model name or ID.
|
||||
|
||||
cv::Mat R_; ///< Rotation matrix between cameras.
|
||||
cv::Mat T_; ///< Translation vector between cameras.
|
||||
cv::Mat E_; ///< Essential matrix.
|
||||
cv::Mat F_; ///< Fundamental matrix.
|
||||
};
|
||||
|
||||
/**
|
||||
* @brief Outputs a textual representation of the StereoCameraModel to the given output stream.
|
||||
*
|
||||
* This operator prints the details of the stereo camera model including:
|
||||
* - The left camera parameters.
|
||||
* - The right camera parameters.
|
||||
* - The stereo extrinsic matrices: Rotation (R), Translation (T), Essential (E), and Fundamental (F).
|
||||
* - The baseline distance between the two cameras.
|
||||
*
|
||||
* @param os The output stream to write to.
|
||||
* @param model The StereoCameraModel instance to output.
|
||||
* @return A reference to the output stream after writing the model information.
|
||||
*/
|
||||
RTABMAP_CORE_EXPORT std::ostream& operator<<(std::ostream& os, const StereoCameraModel& model);
|
||||
|
||||
} // rtabmap
|
||||
|
||||
Reference in New Issue
Block a user