Adding doc and tests (#1492)

* added doc and tests for util2d.h

* updated cmake-ros ci

* Added util3d.h doc and tests

* util3d_transforms.h: Added doc and tests

* util3d_filtering.h: started doc and test

* util3d_filtering.h: more tests and doc

* Added more doc/tests

* finished util3d_filtering doc and tests

* added test for util2d::depthBleedingFiltering

* Added util3d_registration tests

* Added util3d_features.h doc/tests

* added doc/tests for util3d_correspondences.h

* added doc/gtest for util3d_mapping.h (missing hpp functions)

* finished testing util3d_mapping.hpp

* Added util3d_motion_estimation.h tests (2D->3D done)

* finished util3d_motion_estimation.h tests

* minimal util3d_surface.h

* Added Transform and VisualWord tests

* Added doc for CameraModel and StereoCameraModel

* Added more logs in ros ci

* Passing tests on fical

* improved all devcontainer

* added devcontainer kilted, fixed source setup.bash, removed ldconfig in ros-cmake workflow

* cleanup

* source ros

* Added utilite tests

* Added testing to appveyor, github actions cancellable on re-commit on same branch

* appveyor testing without all targets

* appveyor: specifying ALL_BUILD target

* Fixed Util2dTest.NMSImageBoundsRespected test

* Fixing PCL Indices error on old pcl

* Added VWDictionary tests and doc. Fixed LSH not working (fix from https://github.com/flann-lib/flann/pull/472

* fixing some appveyor CI errors, added test to check dictionary serialization against all type

* Added StereoDense, StereoBM and StereoSGBM doc and tests

* Added Stereo tests

* Added CameraModel and StereoCameraModel tests

* Added doc and test for Statistics

* Added doc/tests for Signature

* Added doc/test for SensorEvent, added doc for SensorCaptureInfo

* Added doc to SensorData

* Added SensorData tests

* Added SensorCapture and SensorCaptureThread doc and tests

* fixed sensordata test

* updated SSC test and doc

* Added doc and tests for BayesFilter class

* Enabled testing on mac, updated windows testing like on linux

* added test_link

* fixed unresolved on windows

* fixed ThreadHandle error on macos ci

* Added GPS and GeodeticCoords tests

* Added tests for compression

* Added Odometry tests (base class only)

* Added DBDriver tests

* Added coverage report

* uniformized test names

* fixing concurancy and coverage ci

* dont built tools, examples and app for coverage build

* fixed report tool rebuilt without qt compilation error

* updated coverage option

* updated coverage config

* added doc CI job

* fixing windows and mac ci errors

* Added DBDriverSqlite3 tests

* Added IMU tests

* Added Graph tests

* fixing flaky macos test

* Added IMUThread and IMUFilter tests

* Added Landmarks tests

* Added LASWriter tests

* fixing seed flaky test

* fixing flaky macos timing tests

* Added LocalGrid tests

* Added LocalGridMaker tests

* fixing ci errors

* Added GlobalMap tests

* Added doc for EnvSensor

* Added Features2D tests

* Added Registration tests

* Added RegistrationVis tests

* Added doc for Rtabmap and Memory classes

* Added Memory and Rtabmap tests

* making some tests less flaky

* lcov 1.14 support

* updated compatible tool arguments

* Added integration tests (RGB-D, Stereo, Lidar2d, Lidar3d)

* More octomap checks

* Refactored how/when python interpretor is created to simplify library usage

* Added python tests

* fixed some flaky tests

* suppressed some third party related warnings

* fixed ceres tests

* more flaky fixes

* Fixing tests without libpointmatcher

* Added RANSAC rejection filter to PCL ICP

* fixing multi platform flakiness

* Added test to detect regression

* Fixing windows pcl link error

* fixed some macos flakiness

* bigger 2D2D registration error on opencv 4.6.0

* flakiness

* fixing flaky tests on windows and mac

* flaky thread test on slow mac VM

* windows slow test

* fixing more ci erros

* fxing temp dir on windows

* Added Optimizer tests and discovered some bugs (fixed)

* fixing flaky tests in mac and windows

* Added Optimizer doc

* Added GTSAM BA, updated Ceres to use g2o ba parameters. Renamed g2o's ba related parameters to Optimizer group and used by both gtsam and ceres.

* fixing build without gtsam

* fixing home dir

* fixing python ci isssues

* Added multicam ba tests

* Added Ceres multicam BA support

* Aligned BundleAdjustment parameters with Optimizer/Strategy to avoid confusion in the code

* Added BA integration test

* Added robust graph optimization integration test

* Added loop3it test

* Added stereo20Hz test

* Added smartfactor gtsam

* Fixed bugged check and warn if python didn't return any descriptors

* Fixing gtsam version build issues

* fixing tilt on windows ci

* loosing ceres integration test for ci

* mac ci flakiness

* updating missing param in gui

* updating test bound for mac

* added appearance-based tests, set min gftt quality to quality level

* testing more stuff

* improving features2d tests

* ci flakiness

* fixing flaky ci

* ci fixes

* flaky fixes

* Added RegistrationIcp tests

* Added icp integration test with real-worl corridor like env

* intermediate nodes

* fixing enum

* Updated test to catch #1714

* Fixed 2d corridor failing on pcl

* flaky pnp test

* flaky brisk test

* Set rtabmap_integration test as long

* updating loop closure test

* flaky ci tests

* TEsting roundtrip g2o/toro save/load

* loosing test bound

* fixed cuda capable checks

* flaky tests

* Debugging test hanging

* more debugging stuff

* updating limit

* windows: disabled cuda on ci to avoid incompatible driver issue. Fixing a bad test mem allocation

* trying fixing cuda hanging issue

* fixing ci flakyness

* flaky tests

* Updated BOW flaky tests by checking min precision/recall instead of recall@100precision. Fixed signature test

* CameraModel::load() test initRectificationMap param

* test dbdriver load dictionary idsOnly

* Memory: test keepLinkedInDb param

* added dummyDictionary tests

* test intermediate nodes count

* Added MarkerDetector tests

* reverted breaking change of UMutex and USemaphore

* Features2d: fixed compiltion warnings with clang about override

* clang warnings

* fixing test build with pcl 1.8

* g2o and gtsam build errors on android

* opencv5 test fixes

* disabled testing for ios and android builds

* normalized endline characters for easier diff

* added LF CRLF rule

* bump 0.23.10. fixing doc version

* Publish rtabmap website doc from ci

* fixing MSCVC build error

* macos icp flaky test

* fixing ceres macos test bound

* ficing more flaky tests

* fixing opencv5 related test errors. Also fixed an actual bug in ENU_WGS84ToGeocentric_WGS84()

* added comment about mrpt change

* removed rosdoc2 (will add it for rtabmap_ros later)

* fixing website style

* updated download links

* locally deployable website with api

* sweep doxygen issues

* improved/revised doxygen main pages

* removed examples empty page

* Updated doxygen style

* more concise doxygen groups

* added api link on main readme

* fixing utilite test error

* fixing CommonFilteringGroundNormalsUp test

* updated precisionRecall test bounds for Freak and brief descriptors

* fixing scale check in ba tests

* disabled tests on windows cuda build (missing dlls amd runner cannot test cuda anyway)

* ceres: missing suitesparse dep in windows ci

* adjusting recall thr for fast/freak

* ficing more flaky tests

* fixing flaky tests

* disabled coverage in ros ci

* Enable integration tests for ros ci jobs

* loosing up some threshold for failing tests

* trigger cache

* fixing test data in ros ci. Updated flaky test for mac

* slaking some test limit

* Fixed rtabmap-detectMoreLoopClosures inverted output value

* loosing up sift recall on mac

* optimizer re-ordered distribution for reproducible results (mac g2o)

* macos dump test crash log

* combining all tests to save time on shared library reload. Also fixed Logs with missing arguments.

* Added ENABLE_FORMAT_ERRORS cmake option

* do test only one time

* fixed all format warnings

* format security android build errors

* less verbose tests

* updated ImuUThread test

* fixed a log

* Fixed libpointmatcher 2d normals eigen issue

* Fixing libpointmatcher conversion issues

* fixing libpointmatcher test on windows ci

* cleanup comments, relax some test thr

* disabled sequoia-intel ci build (too flaky, would need extensive testing directly on that machine)
This commit is contained in:
matlabbe
2026-08-06 13:32:20 -07:00
committed by GitHub
parent bcdb4b4546
commit ee49beaf4f
309 changed files with 67468 additions and 3069 deletions
+457 -31
View File
@@ -35,21 +35,53 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
namespace rtabmap {
/**
* @class CameraModel
* @brief Represents a pinhole camera model containing intrinsic and extrinsic
* parameters, used for projection, rectification, and transformation.
*
* This class encapsulates camera calibration data, including intrinsic parameters (fx, fy, cx, cy),
* distortion coefficients, rectification and projection matrices. It provides utility functions
* for image rectification, projection from 2D to 3D, and vice versa.
*
* This class supports the 4 to 14 parameters Radial Tangential distortion model (also called Plumb Bob or
* Brown-Conrady model) and 4 parameters Fish Eye model (also known as Equidistant model).
*
* @see OpenCV's calib3d module for all supported camera models.
*/
class RTABMAP_CORE_EXPORT CameraModel
{
public:
/**
* Optical rotation used to transform image coordinate frame (x->right, y->down, z->forward)
* to robot coordinate frame (x->forward, y->left, z->up).
* @brief Returns the default optical rotation to convert image coordinates to robot coordinates.
*
* Image frame: x -> right, y -> down, z -> forward
* Robot frame: x -> forward, y -> left, z -> up
*
* @return Transform rotation matrix.
*/
static Transform opticalRotation() {return Transform(0,0,1,0, -1,0,0,0, 0,-1,0,0);}
public:
/// Default constructor.
CameraModel();
// K is the camera intrinsic 3x3 CV_64FC1
// D is the distortion coefficients 1x5 CV_64FC1
// R is the rectification matrix 3x3 CV_64FC1 (computed from stereo or Identity)
// P is the projection matrix 3x4 CV_64FC1 (computed from stereo or equal to [K [0 0 1]'])
/**
* @brief Constructor using full camera parameters.
*
* @param name Camera name or ID.
* @param imageSize Size of the image (width x height).
* @param K Intrinsic matrix (3x3).
* @param D Distortion coefficients, 1xN matrix where N is between 4 and 14 parameters:
* k1,k2,p1,p2[,k3[,k4,k5,k6[,s1,s2,s3,s4[,tx,ty]]]]).
* To set Fish Eye / Equidistant model, it is implicitly used if we
* provide 6 values like this: [k1,k2,0,0,k3,k4], where you only need to
* fill "k" parameters.
* @param R Rectification matrix (3x3).
* @param P Projection matrix (3x4).
* @param localTransform Local transform to apply to the camera frame.
*/
CameraModel(
const std::string & name,
const cv::Size & imageSize,
@@ -59,7 +91,17 @@ public:
const cv::Mat & P,
const Transform & localTransform = opticalRotation());
// minimal
/**
* @brief Minimal constructor using intrinsic parameters. This assumes the images are already rectified.
*
* @param fx Focal length x.
* @param fy Focal length y.
* @param cx Principal point x.
* @param cy Principal point y.
* @param localTransform Local transform to apply to the camera frame.
* @param Tx Baseline * fx (optional). Mainly used in case of stereo pair.
* @param imageSize Image size (optional).
*/
CameraModel(
double fx,
double fy,
@@ -68,7 +110,10 @@ public:
const Transform & localTransform = opticalRotation(),
double Tx = 0.0f,
const cv::Size & imageSize = cv::Size(0,0));
// minimal to be saved
/**
* @brief Minimal constructor with name for saving.
*/
CameraModel(
const std::string & name,
double fx,
@@ -79,13 +124,51 @@ public:
double Tx = 0.0f,
const cv::Size & imageSize = cv::Size(0,0));
/// Destructor.
virtual ~CameraModel() {}
/**
* @brief Initializes the rectification maps used to undistort and rectify images.
*
* This function prepares the `mapX_` and `mapY_` lookup tables used for image rectification.
* It supports both standard radial-tangential distortion and fisheye/equidistant distortion models.
*
* - If the distortion model is **fisheye** (indicated by `D_.cols == 6`), it uses
* `cv::fisheye::initUndistortRectifyMap()` to create the rectification maps. This requires OpenCV ≥ 2.4.10.
* - Otherwise, it uses the standard `cv::initUndistortRectifyMap()` for plumb bob or rational polynomial models.
*
* @pre The camera model must be valid for rectification:
* - `imageSize_` must be non-zero.
* - `D_` must be a 1-row matrix with an accepted number of columns (4, 5, 6, 8, 12, or 14).
* - `R_` must be a 3x3 rectification matrix.
* - `P_` must be a 3x4 projection matrix.
*
* @return `true` if the rectification maps were successfully initialized (`mapX_` and `mapY_` are not empty),
* `false` otherwise.
*
* @see isRectificationMapInitialized(), rectifyImage(), rectifyDepth()
*
* @warning Requires OpenCV 2.4.10 or newer for fisheye support. If the version is older, fisheye rectification will not work.
*/
bool initRectificationMap();
/**
* @brief Checks if the rectification map has been initialized.
* @return True if both mapX_ and mapY_ are initialized.
*/
bool isRectificationMapInitialized() const {return !mapX_.empty() && !mapY_.empty();}
/**
* @brief Checks if the model is valid for 2D->3D projection.
*/
bool isValidForProjection() const {return fx()>0.0 && fy()>0.0 && cx()>0.0 && cy()>0.0;}
/**
* @brief Checks if the model is valid for 3D->2D reprojection.
*/
bool isValidForReprojection() const {return fx()>0.0 && fy()>0.0 && cx()>0.0 && cy()>0.0 && imageWidth()>0 && imageHeight()>0;}
/**
* @brief Checks if the model has sufficient data for image rectification.
*/
bool isValidForRectification() const
{
return imageSize_.width>0 &&
@@ -96,71 +179,414 @@ public:
!P_.empty();
}
/// Sets the camera name, used to set a camera name when saving to a file.
void setName(const std::string & name) {name_=name;}
/// Returns the camera name.
const std::string & name() const {return name_;}
/// Returns focal length in x.
double fx() const {return P_.empty()?K_.empty()?0.0:K_.at<double>(0,0):P_.at<double>(0,0);}
/// Returns focal length in y.
double fy() const {return P_.empty()?K_.empty()?0.0:K_.at<double>(1,1):P_.at<double>(1,1);}
/// Returns principal point x.
double cx() const {return P_.empty()?K_.empty()?0.0:K_.at<double>(0,2):P_.at<double>(0,2);}
/// Returns principal point y.
double cy() const {return P_.empty()?K_.empty()?0.0:K_.at<double>(1,2):P_.at<double>(1,2);}
/// Returns the x translation (usually fx * baseline in case of stereo, otherwise would be 0).
double Tx() const {return P_.empty()?0.0:P_.at<double>(0,3);}
cv::Mat K_raw() const {return K_;} //intrinsic camera matrix (before rectification)
cv::Mat D_raw() const {return D_;} //intrinsic distorsion matrix (before rectification)
cv::Mat K() const {return !P_.empty()?P_.colRange(0,3):K_;} // if P exists, return rectified version
/// Returns the raw intrinsic matrix (before rectification).
cv::Mat K_raw() const {return K_;}
/// Returns the raw distortion coefficients (before rectification).
cv::Mat D_raw() const {return D_;}
/// Returns the rectified camera intrinsic matrix if the projection matrix P exists, otherwise returns the raw intrinsic matrix.
cv::Mat K() const {return !P_.empty()?P_.colRange(0,3):K_;}
/// Returns the rectified distortion coefficients (1x5 filled with zeros) if the projection matrix P exists, otherwise returns the raw distortion coefficients.
cv::Mat D() const {return P_.empty()&&!D_.empty()?D_:cv::Mat::zeros(1,5,CV_64FC1);} // if P exists, return rectified version
cv::Mat R() const {return R_;} //rectification matrix
cv::Mat P() const {return P_;} //projection matrix
/// Returns the rectification matrix.
cv::Mat R() const {return R_;}
/// Returns the projection matrix.
cv::Mat P() const {return P_;}
/// Sets the local transform of the camera (base frame to optical frame).
void setLocalTransform(const Transform & transform) {localTransform_ = transform;}
/// Returns the local transform (base frame to optical frame).
const Transform & localTransform() const {return localTransform_;}
/**
* @brief Sets the image size of the camera model and updates the principal point if undefined.
*
* This function updates the internal image size (`imageSize_`) with the provided size.
* If the intrinsic matrices (`K_` or `P_`) are present and the principal point coordinates
* (`cx`, `cy`) are zero, they are set to the image center (`width/2 - 0.5`, `height/2 - 0.5`).
*
* This ensures the camera model remains valid and useful even when the calibration file
* has no principal point set or the image size is updated manually.
*
* @param size The new image size. It must be either both dimensions zero (clearing) or both positive.
*
* @pre `size.width > 0 && size.height > 0` or `size.width == 0 && size.height == 0`
* @post Updates the `imageSize_`, and adjusts `cx` and `cy` in `K_` and `P_` if they were initially zero.
*
* @warning If `K_` or `P_` are not initialized (`empty()`), no updates will be applied to them.
*
* @see imageSize(), imageWidth(), imageHeight()
*/
void setImageSize(const cv::Size & size);
const cv::Size & imageSize() const {return imageSize_;}
int imageWidth() const {return imageSize_.width;}
int imageHeight() const {return imageSize_.height;}
double fovX() const; // in radians
double fovY() const; // in radians
double horizontalFOV() const; // in degrees
double verticalFOV() const; // in degrees
/**
* @brief Returns the horizontal field of view (FoV) in radians.
*
* The FoV is computed using the pinhole camera model as:
* \f[
* \text{FoV}_x = 2 \cdot \tan^{-1}\left(\frac{\text{image width}}{2 \cdot f_x}\right)
* \f]
*
* @return Horizontal field of view in radians. Returns 0.0 if image width or focal length is invalid.
*/
double fovX() const;
/**
* @brief Returns the vertical field of view (FoV) in radians.
*
* The FoV is computed using the pinhole camera model as:
* \f[
* \text{FoV}_y = 2 \cdot \tan^{-1}\left(\frac{\text{image height}}{2 \cdot f_y}\right)
* \f]
*
* @return Vertical field of view in radians. Returns 0.0 if image height or focal length is invalid.
*/
double fovY() const;
/**
* @brief Returns the horizontal field of view in degrees.
*
* Converts the result of `fovX()` from radians to degrees.
*
* @return Horizontal field of view in degrees. Returns 0.0 if the result is invalid.
*/
double horizontalFOV() const;
/**
* @brief Returns the vertical field of view in degrees.
*
* Converts the result of `fovY()` from radians to degrees.
*
* @return Vertical field of view in degrees. Returns 0.0 if the result is invalid.
*/
double verticalFOV() const;
/// Checks if the distortion model is fisheye (6 coefficients: k1,k2,0,0,k3,k4).
bool isFisheye() const {return D_.cols == 6;}
// Set initRectificationMaps=false to skip building the (potentially large)
// rectification maps when rectification won't be used (saves time and memory).
/**
* @brief Loads the camera model parameters from a YAML calibration file.
*
* This method attempts to read camera intrinsic/extrinsic parameters and image size from a YAML file,
* typically in the ROS calibration format. If the distortion model is "fisheye" or "equidistant", we expect
* 4 coefficients, which are converted to a 6-coefficient format for internal representation.
*
* Fields loaded (if present):
* - `camera_name`
* - `image_width` (pixels)
* - `image_height` (pixels)
* - `camera_matrix` (K, 3x3 double matrix)
* - `distortion_coefficients` (D, 1xN double matrix)
* - `distortion_model` (string: name of the model)
* - `rectification_matrix` (R, 3x3 double matrix)
* - `projection_matrix` (P, 3x4 double matrix)
* - `local_transform` (camera pose w.r.t robot frame)
*
* On success, the internal matrices and settings of the camera model are updated. If the model
* is valid for rectification, the rectification map is initialized.
*
* @param filePath Absolute or relative path to the YAML file.
* @param initRectificationMaps Set to false to skip building the (potentially large) rectification
* maps when rectification won't be used (saves time and memory).
* @return True if the file was successfully loaded and parsed, false otherwise.
*
* @warning Logs warnings if any fields are missing. If file does not exist or parsing fails, returns false.
*
* @see initRectificationMap()
*/
bool load(const std::string & filePath, bool initRectificationMaps = true);
/**
* @brief Loads the camera model by constructing a file path from a directory and camera name.
*
* This is a convenience wrapper around `load(filePath)` that constructs the file path as:
* `directory + "/" + cameraName + ".yaml"`.
*
* @param directory Path to the folder containing the camera YAML file.
* @param cameraName Base name of the camera file (without extension).
* @param initRectificationMaps Set to false to skip building the (potentially large) rectification
* maps when rectification won't be used (saves time and memory).
* @return True if loading from the constructed path succeeds, false otherwise.
*/
bool load(const std::string & directory, const std::string & cameraName, bool initRectificationMaps = true);
/**
* @brief Saves the camera model parameters to a YAML calibration file in ROS format.
*
* The file will include the following fields if they are not empty:
* - `camera_name`
* - `image_width`
* - `image_height`
* - `camera_matrix` (K)
* - `distortion_coefficients` (D)
* - `distortion_model` (auto-detected based on number of distortion coefficients)
* - `rectification_matrix` (R)
* - `projection_matrix` (P)
* - `local_transform` (camera pose w.r.t robot frame)
*
* If the distortion matrix contains 6 coefficients (used for fisheye), it is converted
* to a standard 4-coefficient format for ROS compatibility.
*
* @param directory Path to the folder where the YAML file will be saved.
* @return True if saving was successful, false otherwise.
*
* @note If `name_` is empty, "camera.yaml" is used as the default filename.
* @warning Returns false and logs an error if none of the matrices are set.
*/
bool save(const std::string & directory) const;
/**
* @brief Serializes the camera model to a binary format.
*
* The serialization includes the camera intrinsics (`K_`, `D_`), rectification matrix (`R_`),
* projection matrix (`P_`), image size, and the local transform. The format is compact and suitable
* for file storage or transmission over a network.
*
* Data layout:
* - Header (11 integers):
* - [0-2] RTAB-Map version (major, minor, patch)
* - [3] Camera type (0 = mono, 1=stereo)
* - [4-5] Image width, height
* - [6-9] Element counts for K, D, R, P matrices
* - [10] Size of localTransform (0 if null)
* - Data section (in order): raw memory blocks for K, D, R, P (`double` values), followed by `float` values for localTransform
*
* @return A byte vector containing the serialized data. The format is compatible with `deserialize()`.
*
* @note This is a custom binary format, not meant to be human-readable.
* @see deserialize(), StereoCameraModel
*/
std::vector<unsigned char> serialize() const;
/**
* @brief Deserializes a camera model from a byte vector.
*
* This is a convenience wrapper around `deserialize(const unsigned char*, unsigned int)`
* that takes a `std::vector<unsigned char>` instead of a raw buffer.
*
* @param data Byte vector containing data serialized by `serialize()`.
* @return The number of bytes successfully read and parsed. Returns 0 on failure.
*
* @see serialize(), deserialize(const unsigned char*, unsigned int)
*/
unsigned int deserialize(const std::vector<unsigned char>& data);
/**
* @brief Deserializes a camera model from a raw byte buffer.
*
* Reads the camera intrinsics, distortion, rectification, projection matrices, image size,
* and local transform from a serialized binary format previously created with `serialize()`.
*
* @param data Pointer to the binary data buffer.
* @param dataSize Size of the data buffer in bytes.
* @return The number of bytes successfully read. Returns 0 on error or if the format is invalid.
*
* @warning If the buffer format does not match the expected layout or version, an error is logged
* and the camera model remains in a default-initialized state.
*
* @note Assumes little-endian architecture and strict size/type matching. The serialized format
* must be created by `CameraModel::serialize()`. Non-mono camera types are not supported.
* See `StereoCameraModel` to serialize/deserialize stereo models.
*
* @see serialize()
*/
unsigned int deserialize(const unsigned char * data, unsigned int dataSize);
/**
* @brief Returns a new camera model with all intrinsic parameters scaled by a given factor.
*
* This method scales the camera's intrinsic matrix (`K_`) and projection matrix (`P_`), as well as
* the image size, by the given `scale` factor. The distortion and rectification matrices are left unchanged.
*
* Only valid camera models (i.e., those for which `isValidForProjection()` returns true) are scaled.
* If the model is invalid, a warning is issued and the original model is returned unchanged.
*
* @param scale Scaling factor (> 0). For example, use 0.5 to downscale or 2.0 to upscale.
* @return A scaled copy of the camera model with updated intrinsics and image size.
*
* @warning If the camera model is not valid for projection, the scale operation is ignored.
*/
CameraModel scaled(double scale) const;
/**
* @brief Returns a new camera model adjusted for a given region of interest (ROI).
*
* This method shifts the principal point (`cx`, `cy`) in the intrinsic matrix (`K_`) and projection matrix (`P_`)
* by subtracting the ROI’s top-left `(x, y)` offset. The image size is also set to the ROI size.
*
* Only valid camera models (i.e., those for which `isValidForProjection()` returns true) can be adjusted.
* If the model is invalid, a warning is issued and the original model is returned unchanged.
*
* @param roi Region of interest defined as a rectangle (typically a subwindow of the full image).
* @return A new camera model adapted to the ROI with adjusted intrinsics and image size.
*
* @warning If the camera model is not valid for projection, the ROI operation is ignored.
*/
CameraModel roi(const cv::Rect & roi) const;
// For depth images, your should use cv::INTER_NEAREST
/**
* @brief Rectifies a raw image using the precomputed rectification maps.
*
* This function applies geometric correction (rectification) to an image using the camera model's
* `mapX_` and `mapY_` rectification maps. It is typically used to correct lens distortion in images
* based on the calibration parameters.
*
* @param raw Input raw image (e.g., from camera). Must be a valid `cv::Mat`.
* @param interpolation Interpolation method to use. Typically `cv::INTER_LINEAR` or `cv::INTER_NEAREST`.
*
* @return Rectified image. If the rectification maps are not initialized, the function logs an error
* and returns a clone of the original image.
*
* @pre `mapX_` and `mapY_` must be initialized using `initRectificationMap()`.
*
* @note Works for color and grayscale images of any valid type.
*
* @see initRectificationMap(), rectifyDepth()
*/
cv::Mat rectifyImage(const cv::Mat & raw, int interpolation = cv::INTER_LINEAR) const;
/**
* @brief Rectifies a raw depth image using the precomputed rectification maps.
*
* This function applies geometric correction (rectification) to a 16-bit unsigned depth image.
* It performs a pixel-by-pixel bilinear interpolation, only if all neighboring pixels have valid
* (non-zero) depth values, and the variation among them is within 1% of their average.
*
* The method is optimized to avoid introducing noise in regions of high depth variance.
*
* @param raw Input raw depth image (`CV_16UC1`). Must contain 16-bit unsigned depth values.
*
* @return Rectified depth image. If the rectification maps are not initialized or the input
* image is not of type `CV_16UC1`, the function logs an error and returns a clone of the input.
*
* @pre Input image must be of type `CV_16UC1`. `mapX_` and `mapY_` must be initialized.
*
* @note Inspired by the Kinect2 CPU depth registration implementation from:
* https://github.com/code-iai/iai_kinect2
*
* @warning Invalid or noisy regions are skipped in interpolation to maintain depth consistency.
*
* @see initRectificationMap(), rectifyImage()
*/
cv::Mat rectifyDepth(const cv::Mat & raw) const;
// Project 2D pixel to 3D (in /camera_link frame)
/**
* @brief Projects a 2D pixel and depth value into a 3D point in the camera coordinate frame (/camera_link).
*
* This function uses the camera's intrinsic parameters to compute the 3D point corresponding to the given
* 2D image coordinates and depth value.
*
* @param u Horizontal image coordinate (in pixels).
* @param v Vertical image coordinate (in pixels).
* @param depth Depth value at (u, v) in meters.
* @param[out] x Output X coordinate in 3D space.
* @param[out] y Output Y coordinate in 3D space.
* @param[out] z Output Z coordinate in 3D space (equals `depth`).
*
* @note If `depth <= 0`, the output (x, y, z) will be set to `NaN`.
*
* @see reproject()
*/
void project(float u, float v, float depth, float & x, float & y, float & z) const;
// Reproject 3D point (in /camera_link frame) to pixel
/**
* @brief Reprojects a 3D point in the camera frame (/camera_link) into 2D image coordinates (floating-point).
*
* This function computes the image plane coordinates for a given 3D point using the camera's
* intrinsic parameters.
*
* @param x X coordinate in camera space.
* @param y Y coordinate in camera space.
* @param z Z coordinate in camera space (must be non-zero).
* @param[out] u Output horizontal image coordinate (float).
* @param[out] v Output vertical image coordinate (float).
*
* @pre `z != 0`
*
* @see project(), reproject(int&, int&)
*/
void reproject(float x, float y, float z, float & u, float & v) const;
/**
* @brief Reprojects a 3D point in the camera frame (/camera_link) into 2D image coordinates (rounded to int).
*
* This version of `reproject()` returns integer pixel indices, computed from the 3D position.
*
* @param x X coordinate in camera space.
* @param y Y coordinate in camera space.
* @param z Z coordinate in camera space (must be non-zero).
* @param[out] u Output horizontal image coordinate (integer pixel).
* @param[out] v Output vertical image coordinate (integer pixel).
*
* @pre `z != 0`
*
* @see project(), reproject(float&, float&)
*/
void reproject(float x, float y, float z, int & u, int & v) const;
/**
* @brief Checks if a given pixel coordinate lies within the image bounds.
*
* @param u Horizontal image coordinate (in pixels).
* @param v Vertical image coordinate (in pixels).
* @return `true` if the pixel is within the image dimensions, `false` otherwise.
*
* @note Inclusive lower bound, exclusive upper bound: `[0, width)`, `[0, height)`
*/
bool inFrame(int u, int v) const;
private:
std::string name_;
cv::Size imageSize_;
cv::Mat K_;
cv::Mat D_;
cv::Mat R_;
cv::Mat P_;
cv::Mat mapX_;
cv::Mat mapY_;
Transform localTransform_;
std::string name_; ///< Camera name.
cv::Size imageSize_; ///< Image size.
cv::Mat K_; ///< Intrinsic matrix.
cv::Mat D_; ///< Distortion coefficients.
cv::Mat R_; ///< Rectification matrix.
cv::Mat P_; ///< Projection matrix.
cv::Mat mapX_; ///< Rectification map X.
cv::Mat mapY_; ///< Rectification map Y.
Transform localTransform_; ///< Transform from camera to base link.
};
/**
* @brief Stream operator for printing a camera model to an output stream.
*
* This function outputs the name, image size, and camera matrices (K, D, R, P)
* along with the local transformation.
*
* Example output:
* ```
* Name: camera1
* Size: 640x480
* K= [fx, 0, cx;
* 0, fy, cy;
* 0, 0, 1]
* D= [...]
* R= [...]
* P= [...]
* LocalTransform= [...]
* ```
*
* @param os Output stream.
* @param model Camera model to print.
* @return The modified output stream.
*/
RTABMAP_CORE_EXPORT std::ostream& operator<<(std::ostream& os, const CameraModel& model);
} /* namespace rtabmap */