RTAB-Map C++ API {#mainpage} ================ RTAB-Map (Real-Time Appearance-Based Mapping) is a RGB-D, stereo and lidar graph-based SLAM library built around an incremental appearance-based loop closure detector, with memory management that keeps the online constraints satisfiable on large-scale, long-term maps. These pages document the public C++ API of the `rtabmap_core` and `rtabmap_utilite` libraries. For installation, tutorials and the ROS packages, see the [project website](https://introlab.github.io/rtabmap/) and the [wiki](https://github.com/introlab/rtabmap/wiki). Start here ---------- rtabmap::Rtabmap is the entry point: it owns the map and runs one full SLAM iteration per call to rtabmap::Rtabmap::process(). A minimal loop feeds it a rtabmap::SensorData and the odometry pose that goes with it: ~~~{.cpp} #include #include rtabmap::Odometry * odometry = rtabmap::Odometry::create(); rtabmap::Rtabmap rtabmap; rtabmap.init(); // optionally: init(parameters, databasePath) const double mapUpdateRate = 1.0; // Hz, i.e. Rtabmap/DetectionRate double lastProcessStamp = -1.0; rtabmap::Transform mapToOdom = rtabmap::Transform::getIdentity(); while(/* frames available */) { rtabmap::SensorData data = camera.takeImage(); // Odometry sees every frame: dropping any would break the motion tracking. rtabmap::Transform odomPose = odometry->process(data); // The map is updated at a lower rate. The frames skipped here are not lost // work: the motion they carry is already integrated in the pose above, so // the next accepted frame arrives with an up-to-date odometry pose. if(lastProcessStamp < 0.0 || data.stamp() - lastProcessStamp >= 1.0/mapUpdateRate) { lastProcessStamp = data.stamp(); if(rtabmap.process(data, odomPose)) // true when a new node was added { const rtabmap::Statistics & stats = rtabmap.getStatistics(); // A loop closure or a proximity detection re-optimizes the graph, // which shifts the map frame under the odometry frame. if(!stats.mapCorrection().isNull()) { mapToOdom = stats.mapCorrection(); } if(rtabmap.getLoopClosureId() > 0) { // a loop closure was accepted on this iteration } } } // Robot pose in the map frame, on every frame and always with the matching // correction: composed after the block above, so an iteration that just // re-optimized the graph uses its new correction rather than the previous // one. In ROS terms: (/map -> /odom) * (/odom -> /base_link). rtabmap::Transform mapPose = mapToOdom * odomPose; } ~~~ Throttling is the caller's job here: rtabmap::Rtabmap::process() maps every frame it is given. rtabmap::RtabmapThread does this same stamp comparison internally, from @ref rtabmap::Parameters::kRtabmapDetectionRate() "Rtabmap/DetectionRate", so the threaded pipeline only needs the parameter to be set. Odometry drifts and the graph gets re-optimized, so the odometry pose is not a map pose. rtabmap::Statistics::mapCorrection() is what reconciles the two, and since it only changes on a map update it can be applied to every incoming frame -- which is how the pose stays available at full rate while the map is built at 1 Hz. This is the transform published as `/map` → `/odom` by the ROS wrapper, and what the `MapBuilder` of each example composes with the live odometry pose to place the clouds. It is also readable outside the statistics, as rtabmap::Rtabmap::getMapCorrection(). Complete programs live under [`examples/`](https://github.com/introlab/rtabmap/tree/master/examples) in the source tree: | Example | What it shows | | ------- | ------------- | | [BOWMapping](https://github.com/introlab/rtabmap/blob/master/examples/BOWMapping/main.cpp) | The smallest useful loop: images from disk into rtabmap::Rtabmap, appearance-only loop closure detection (no odometry, no GUI) | | [NoEventsExample](https://github.com/introlab/rtabmap/blob/master/examples/NoEventsExample/main.cpp) | Driving the pipeline by direct calls -- camera, rtabmap::Odometry and rtabmap::Rtabmap in one explicit loop, without the event system | | [RGBDMapping](https://github.com/introlab/rtabmap/blob/master/examples/RGBDMapping/main.cpp) | The threaded event-based pipeline (rtabmap::SensorCaptureThread → rtabmap::OdometryThread → rtabmap::RtabmapThread) with any supported RGB-D or stereo camera | | [LidarMapping](https://github.com/introlab/rtabmap/blob/master/examples/LidarMapping/main.cpp) | The same threaded pipeline driven by a 3D lidar (rtabmap::LidarVLP16) instead of a camera | What each iteration does, and which parameters influence it, is documented on rtabmap::Rtabmap itself -- memory update, loop-closure hypothesis, hypothesis selection, retrieval, proximity detection and transfer to long-term memory. Occupancy grid -------------- With @ref rtabmap::Parameters::kRGBDCreateOccupancyGrid() "RGBD/CreateOccupancyGrid" enabled, every node carries a local occupancy grid computed from its depth images or laser scan (rtabmap::LocalGridMaker). Assembling those into a global grid is left to the caller, so that the result always follows the optimized poses: ~~~{.cpp} #include rtabmap::LocalGridCache localGrids; rtabmap::OccupancyGrid grid(&localGrids, parameters); // reads the Grid/... parameters // ... inside the "if(rtabmap.process(data, odomPose))" block of the loop above: const rtabmap::Signature & node = stats.getLastSignatureData(); if(node.sensorData().gridCellSize() > 0.0f && grid.addedNodes().find(node.id()) == grid.addedNodes().end()) { // Local grid of the new node, as stored in the database (compressed). cv::Mat ground, obstacles, empty; node.sensorData().uncompressDataConst(0, 0, 0, 0, &ground, &obstacles, &empty); localGrids.add(node.id(), ground, obstacles, empty, node.sensorData().gridCellSize(), node.sensorData().gridViewPoint()); } // Draws the nodes that are not assembled yet. If the last optimization moved // poses by more than GridGlobal/UpdateError, the grid is cleared first and // redrawn entirely from the cache -- which is why the cache is kept around. grid.update(stats.poses()); float xMin, yMin; // grid origin (m), in the map frame cv::Mat map = grid.getMap(xMin, yMin); // CV_8S: -1 unknown, 0 free, 100 occupied ~~~ For the node that was just added, the cells are already there uncompressed, and rtabmap::SensorData::uncompressDataConst() returns them as they are -- it only decompresses what comes back empty, which is what makes the same code work for a node retrieved from the database. The occupancy grid is kept on the published copy even with @ref rtabmap::Parameters::kRtabmapPublishLastSignature() "Rtabmap/PublishLastSignature" disabled (only images, scans and user data are dropped), precisely so that the global grid can still be assembled; statistics themselves must be published (@ref rtabmap::Parameters::kRtabmapPublishStats() "Rtabmap/PublishStats", on by default). rtabmap::OccupancyGrid is one of the rtabmap::GlobalMap back-ends: rtabmap::OctoMap, rtabmap::CloudMap and rtabmap::GridMap consume the same cache the same way. Each example's `MapBuilder` does exactly this, then hands the result to the viewer. For a 3D map, rtabmap::CloudMap assembles the very same cells into PCL clouds instead of a 2D grid. It shares the cache, so both can be kept up to date from one set of local grids: ~~~{.cpp} #include #include rtabmap::CloudMap cloudMap(&localGrids, parameters); // same cache as above // ... right after the localGrids.add() of the block above: cloudMap.update(stats.poses()); pcl::PointCloud::Ptr ground = cloudMap.getMapGround(); pcl::PointCloud::Ptr obstacles = cloudMap.getMapObstacles(); pcl::PointCloud::Ptr emptySpace = cloudMap.getMapEmptyCells(); pcl::io::savePCDFileBinary("obstacles.pcd", *obstacles); ~~~ The clouds are in the map frame and voxelized at @ref rtabmap::Parameters::kGridCellSize() "Grid/CellSize". Points keep the colour of the local grid when it has one, otherwise ground is green and obstacles red. Note that this assembles the *cells*, not the raw sensor clouds: with @ref rtabmap::Parameters::kGrid3D() "Grid/3D" disabled they are flattened onto the xy plane, so it must stay enabled for a 3D result. For a full-resolution cloud, assemble the nodes themselves rather than their cells. Ask rtabmap::Rtabmap::getGraph() for the optimized poses along with the node data, then rebuild a cloud per node and transform it to its pose: ~~~{.cpp} #include #include #include std::map poses; std::multimap links; std::map nodes; rtabmap.getGraph(poses, links, true, true, &nodes, true); // optimized, global, with images pcl::PointCloud::Ptr assembled(new pcl::PointCloud); for(std::map::const_iterator iter=poses.begin(); iter!=poses.end(); ++iter) { rtabmap::SensorData data = nodes.at(iter->first).sensorData(); data.uncompressData(); pcl::IndicesPtr indices(new std::vector); pcl::PointCloud::Ptr cloud = rtabmap::util3d::cloudRGBFromSensorData( data, 4, // image decimation 4.0f, // max depth (m), 0 = no limit 0.0f, // min depth (m) indices.get()); cloud = rtabmap::util3d::voxelize(cloud, indices, 0.01f); // 1 cm *assembled += *rtabmap::util3d::transformPointCloud(cloud, iter->second); } assembled = rtabmap::util3d::voxelize(assembled, 0.01f); // one last pass over the overlaps pcl::io::savePCDFileBinary("cloud.pcd", *assembled); ~~~ @note Do this once the session is over, not on every iteration. It decompresses and re-projects every node, so the cost grows with the whole map, and the poses are only worth exporting once the graph has been optimized -- the same cloud assembled mid-session would carry the drift that later loop closures correct. This is what @ref tool_export "rtabmap-export" does, with more filtering options. Memory management ----------------- RTAB-Map keeps the map in three tiers (rtabmap::Memory): a **short-term memory** of the last @ref rtabmap::Parameters::kMemSTMSize() "Mem/STMSize" nodes, where neighbours are too similar to be loop closure candidates; a **working memory** holding everything loop closure detection compares against; and a **long-term memory**, the part of the map that stays in the database and is not searched. By default nothing leaves the working memory, so the iteration time grows with the map. Memory management caps it, and is enabled by setting a budget -- either one, or both: ~~~{.cpp} rtabmap::ParametersMap parameters; // Keep each update under 700 ms... parameters.insert(rtabmap::ParametersPair(rtabmap::Parameters::kRtabmapTimeThr(), "700")); // ... and/or keep at most 500 nodes in the working memory. parameters.insert(rtabmap::ParametersPair(rtabmap::Parameters::kRtabmapMemoryThr(), "500")); rtabmap.init(parameters, "map.db"); ~~~ When an iteration goes over budget, the nodes of lowest weight are moved to the long-term memory at the end of it -- age only breaks ties between equal weights, so it is not simply the oldest that go. Weight is how often a place has been seen: while a node is still in the short-term memory, a new node similar enough to it (@ref rtabmap::Parameters::kMemRehearsalSimilarity() "Mem/RehearsalSimilarity") is merged into it and raises its weight -- the rehearsal mechanism. Places the robot dwells on or revisits therefore stay in the working memory, while views seen once leave first. They are not lost: when a loop closure is found, their neighbours are brought back into the working memory for the next iterations, up to @ref rtabmap::Parameters::kRtabmapMaxRetrieved() "Rtabmap/MaxRetrieved" nodes (plus @ref rtabmap::Parameters::kRGBDMaxLocalRetrieved() "RGBD/MaxLocalRetrieved" around the current pose and along a planned path). This is what makes long-term mapping practical: the robot keeps a bounded, relevant working set and pulls the rest back as it recognizes where it is. Retrieval and node immunization only run when memory management is on. Which nodes go first is controlled by three parameters: @ref rtabmap::Parameters::kMemRecentWmRatio() "Mem/RecentWmRatio" protects the most recent part of the working memory, @ref rtabmap::Parameters::kRGBDLocalImmunizationRatio() "RGBD/LocalImmunizationRatio" protects the nodes around the current pose, and @ref rtabmap::Parameters::kMemTransferSortingByWeightId() "Mem/TransferSortingByWeightId" selects the ordering. The step-by-step behaviour is documented on rtabmap::Rtabmap (steps 4 and 6), and the `Memory/Working_memory_size` and `Memory/Signatures_retrieved` entries of rtabmap::Statistics report what happens at runtime. Configuration ------------- Every parameter is a string key/value pair in a rtabmap::ParametersMap, declared with its default and description in `Parameters.h` (for example `Parameters::kMemSTMSize()`, `Parameters::kRGBDLinearUpdate()`). The same keys are used by the applications, the ROS wrappers and the `--Param value` command-line arguments of the tools, so a setting found here applies everywhere. The @ref parameters "Parameter reference" lists all of them, grouped, with their type, default value and description. ~~~{.cpp} rtabmap::ParametersMap parameters; parameters.insert(rtabmap::ParametersPair(rtabmap::Parameters::kMemSTMSize(), "20")); rtabmap.init(parameters, "map.db"); ~~~ The main classes ---------------- Doxygen lists the classes alphabetically; this is the same set arranged by the role they play, as a starting point into the API. ### The map structure | Class | Role | | ----- | ---- | | rtabmap::Rtabmap | The entry point: one SLAM iteration per call, owning everything below | | rtabmap::Memory | Three-tiered memory (STM / WM / LTM) holding the map and deciding what stays online | | rtabmap::Signature | One node: sensor data, visual words, pose and links | | rtabmap::Link | One edge: neighbour, loop closure, landmark or prior constraint | | rtabmap::DBDriver | Persistence of the map to the database (see rtabmap::DBDriverSqlite3) | | rtabmap::Statistics | Everything the pipeline reports about an iteration | ### Inputs | Class | Role | | ----- | ---- | | rtabmap::SensorData | An observation: images, depth, laser scan, IMU, GPS, landmarks | | rtabmap::CameraModel, rtabmap::StereoCameraModel | Intrinsics, extrinsics and rectification | | rtabmap::LaserScan | Point cloud / laser scan container and its formats | | rtabmap::Transform | The 3D rigid transform used everywhere in the API | | rtabmap::SensorCapture, rtabmap::SensorCaptureThread | Drivers and the thread that pumps them | ### Building blocks | Class | Role | | ----- | ---- | | rtabmap::Odometry | Visual / lidar odometry front-ends | | rtabmap::Registration, rtabmap::RegistrationVis, rtabmap::RegistrationIcp | Relative transform between two nodes | | rtabmap::Optimizer | Graph optimization back-ends (g2o, GTSAM, Ceres, TORO) | | rtabmap::Feature2D, rtabmap::VWDictionary | Keypoint detectors/descriptors and the bag-of-words dictionary | | rtabmap::BayesFilter | Loop-closure hypothesis estimation | | rtabmap::LocalGridMaker, rtabmap::GlobalMap | Occupancy grid generation and assembly | Free functions for point cloud, image and geometry processing are grouped in `util2d.h`, `util3d.h`, `util3d_filtering.h`, `util3d_registration.h`, `util3d_surface.h`, `util3d_transforms.h` and `util3d_mapping.h`.