Files
rtabmap/corelib/test/test_optimizer.cpp

2890 lines
122 KiB
C++
Raw Normal View History

Adding doc and tests (#1492) * added doc and tests for util2d.h * updated cmake-ros ci * Added util3d.h doc and tests * util3d_transforms.h: Added doc and tests * util3d_filtering.h: started doc and test * util3d_filtering.h: more tests and doc * Added more doc/tests * finished util3d_filtering doc and tests * added test for util2d::depthBleedingFiltering * Added util3d_registration tests * Added util3d_features.h doc/tests * added doc/tests for util3d_correspondences.h * added doc/gtest for util3d_mapping.h (missing hpp functions) * finished testing util3d_mapping.hpp * Added util3d_motion_estimation.h tests (2D->3D done) * finished util3d_motion_estimation.h tests * minimal util3d_surface.h * Added Transform and VisualWord tests * Added doc for CameraModel and StereoCameraModel * Added more logs in ros ci * Passing tests on fical * improved all devcontainer * added devcontainer kilted, fixed source setup.bash, removed ldconfig in ros-cmake workflow * cleanup * source ros * Added utilite tests * Added testing to appveyor, github actions cancellable on re-commit on same branch * appveyor testing without all targets * appveyor: specifying ALL_BUILD target * Fixed Util2dTest.NMSImageBoundsRespected test * Fixing PCL Indices error on old pcl * Added VWDictionary tests and doc. Fixed LSH not working (fix from https://github.com/flann-lib/flann/pull/472 * fixing some appveyor CI errors, added test to check dictionary serialization against all type * Added StereoDense, StereoBM and StereoSGBM doc and tests * Added Stereo tests * Added CameraModel and StereoCameraModel tests * Added doc and test for Statistics * Added doc/tests for Signature * Added doc/test for SensorEvent, added doc for SensorCaptureInfo * Added doc to SensorData * Added SensorData tests * Added SensorCapture and SensorCaptureThread doc and tests * fixed sensordata test * updated SSC test and doc * Added doc and tests for BayesFilter class * Enabled testing on mac, updated windows testing like on linux * added test_link * fixed unresolved on windows * fixed ThreadHandle error on macos ci * Added GPS and GeodeticCoords tests * Added tests for compression * Added Odometry tests (base class only) * Added DBDriver tests * Added coverage report * uniformized test names * fixing concurancy and coverage ci * dont built tools, examples and app for coverage build * fixed report tool rebuilt without qt compilation error * updated coverage option * updated coverage config * added doc CI job * fixing windows and mac ci errors * Added DBDriverSqlite3 tests * Added IMU tests * Added Graph tests * fixing flaky macos test * Added IMUThread and IMUFilter tests * Added Landmarks tests * Added LASWriter tests * fixing seed flaky test * fixing flaky macos timing tests * Added LocalGrid tests * Added LocalGridMaker tests * fixing ci errors * Added GlobalMap tests * Added doc for EnvSensor * Added Features2D tests * Added Registration tests * Added RegistrationVis tests * Added doc for Rtabmap and Memory classes * Added Memory and Rtabmap tests * making some tests less flaky * lcov 1.14 support * updated compatible tool arguments * Added integration tests (RGB-D, Stereo, Lidar2d, Lidar3d) * More octomap checks * Refactored how/when python interpretor is created to simplify library usage * Added python tests * fixed some flaky tests * suppressed some third party related warnings * fixed ceres tests * more flaky fixes * Fixing tests without libpointmatcher * Added RANSAC rejection filter to PCL ICP * fixing multi platform flakiness * Added test to detect regression * Fixing windows pcl link error * fixed some macos flakiness * bigger 2D2D registration error on opencv 4.6.0 * flakiness * fixing flaky tests on windows and mac * flaky thread test on slow mac VM * windows slow test * fixing more ci erros * fxing temp dir on windows * Added Optimizer tests and discovered some bugs (fixed) * fixing flaky tests in mac and windows * Added Optimizer doc * Added GTSAM BA, updated Ceres to use g2o ba parameters. Renamed g2o's ba related parameters to Optimizer group and used by both gtsam and ceres. * fixing build without gtsam * fixing home dir * fixing python ci isssues * Added multicam ba tests * Added Ceres multicam BA support * Aligned BundleAdjustment parameters with Optimizer/Strategy to avoid confusion in the code * Added BA integration test * Added robust graph optimization integration test * Added loop3it test * Added stereo20Hz test * Added smartfactor gtsam * Fixed bugged check and warn if python didn't return any descriptors * Fixing gtsam version build issues * fixing tilt on windows ci * loosing ceres integration test for ci * mac ci flakiness * updating missing param in gui * updating test bound for mac * added appearance-based tests, set min gftt quality to quality level * testing more stuff * improving features2d tests * ci flakiness * fixing flaky ci * ci fixes * flaky fixes * Added RegistrationIcp tests * Added icp integration test with real-worl corridor like env * intermediate nodes * fixing enum * Updated test to catch #1714 * Fixed 2d corridor failing on pcl * flaky pnp test * flaky brisk test * Set rtabmap_integration test as long * updating loop closure test * flaky ci tests * TEsting roundtrip g2o/toro save/load * loosing test bound * fixed cuda capable checks * flaky tests * Debugging test hanging * more debugging stuff * updating limit * windows: disabled cuda on ci to avoid incompatible driver issue. Fixing a bad test mem allocation * trying fixing cuda hanging issue * fixing ci flakyness * flaky tests * Updated BOW flaky tests by checking min precision/recall instead of recall@100precision. Fixed signature test * CameraModel::load() test initRectificationMap param * test dbdriver load dictionary idsOnly * Memory: test keepLinkedInDb param * added dummyDictionary tests * test intermediate nodes count * Added MarkerDetector tests * reverted breaking change of UMutex and USemaphore * Features2d: fixed compiltion warnings with clang about override * clang warnings * fixing test build with pcl 1.8 * g2o and gtsam build errors on android * opencv5 test fixes * disabled testing for ios and android builds * normalized endline characters for easier diff * added LF CRLF rule * bump 0.23.10. fixing doc version * Publish rtabmap website doc from ci * fixing MSCVC build error * macos icp flaky test * fixing ceres macos test bound * ficing more flaky tests * fixing opencv5 related test errors. Also fixed an actual bug in ENU_WGS84ToGeocentric_WGS84() * added comment about mrpt change * removed rosdoc2 (will add it for rtabmap_ros later) * fixing website style * updated download links * locally deployable website with api * sweep doxygen issues * improved/revised doxygen main pages * removed examples empty page * Updated doxygen style * more concise doxygen groups * added api link on main readme * fixing utilite test error * fixing CommonFilteringGroundNormalsUp test * updated precisionRecall test bounds for Freak and brief descriptors * fixing scale check in ba tests * disabled tests on windows cuda build (missing dlls amd runner cannot test cuda anyway) * ceres: missing suitesparse dep in windows ci * adjusting recall thr for fast/freak * ficing more flaky tests * fixing flaky tests * disabled coverage in ros ci * Enable integration tests for ros ci jobs * loosing up some threshold for failing tests * trigger cache * fixing test data in ros ci. Updated flaky test for mac * slaking some test limit * Fixed rtabmap-detectMoreLoopClosures inverted output value * loosing up sift recall on mac * optimizer re-ordered distribution for reproducible results (mac g2o) * macos dump test crash log * combining all tests to save time on shared library reload. Also fixed Logs with missing arguments. * Added ENABLE_FORMAT_ERRORS cmake option * do test only one time * fixed all format warnings * format security android build errors * less verbose tests * updated ImuUThread test * fixed a log * Fixed libpointmatcher 2d normals eigen issue * Fixing libpointmatcher conversion issues * fixing libpointmatcher test on windows ci * cleanup comments, relax some test thr * disabled sequoia-intel ci build (too flaky, would need extensive testing directly on that machine)
2026-08-06 13:32:20 -07:00
// Tests for rtabmap::Optimizer -- the base class. We exercise:
// - factory + isAvailable()
// - getters/setters and parseParameters()
// - getConnectedGraph(): a non-virtual graph utility that drives every
// concrete optimizer's input prep (filters disconnected nodes, applies
// priorsIgnored/landmarksIgnored, normalizes link direction).
// - optimize() / optimizeIncremental() smoke tests on a deterministic
// 2/3-node chain -- whichever concrete optimizer is built in.
//
// The Optimizer base class itself is abstract; concrete behavior comes from
// TORO/g2o/GTSAM/Ceres backends. We require *some* backend to be available
// (the factory throws otherwise), and stress what's portable across all of
// them rather than backend-specific quirks.
#include <gtest/gtest.h>
#include <rtabmap/core/Optimizer.h>
#include <rtabmap/core/Link.h>
#include <rtabmap/core/Parameters.h>
#include <rtabmap/core/Transform.h>
#include <rtabmap/core/CameraModel.h>
#include <rtabmap/core/util3d_transforms.h>
#include <rtabmap/utilite/UConversion.h>
#include <rtabmap/utilite/UTimer.h>
#include <memory>
#include <random>
using namespace rtabmap;
namespace {
// One unit-trace covariance per edge. Real callers populate a 6x6 inverse;
// for these tests we just need something invertible/non-zero.
cv::Mat unitCov()
{
return cv::Mat::eye(6, 6, CV_64FC1);
}
// Identity-translation link between two consecutive nodes a -> b along +X.
Link neighborLink(int from, int to, float dx = 1.0f)
{
return Link(from, to, Link::kNeighbor,
Transform(dx, 0, 0, 0, 0, 0),
unitCov());
}
// Build a straight 3-node chain (N1 -> N2 -> N3) at +X intervals.
void makeChain3(std::map<int, Transform> & poses, std::multimap<int, Link> & links)
{
poses[1] = Transform(0.0f, 0, 0, 0, 0, 0);
poses[2] = Transform(1.0f, 0, 0, 0, 0, 0);
poses[3] = Transform(2.0f, 0, 0, 0, 0, 0);
links.insert({1, neighborLink(1, 2)});
links.insert({2, neighborLink(2, 3)});
}
} // namespace
// ---------------------------------------------------------------------------
// Type / factory
// ---------------------------------------------------------------------------
TEST(OptimizerTest, TypeEnumValuesAreStable)
{
// Persisted in databases via Parameters/RGBD::OptimizerStrategy and in
// other DBs; reordering would break backward compat.
EXPECT_EQ(static_cast<int>(Optimizer::kTypeUndef), -1);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeTORO), 0);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeG2O), 1);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeGTSAM), 2);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeCeres), 3);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeCVSBA), 4);
}
TEST(OptimizerTest, IsAvailableAtLeastOneOptimizer)
{
// The factory requires at least one backend; if all return false, no test
// using create() can ever pass. Catching it here is more diagnostic.
const bool anyAvailable =
Optimizer::isAvailable(Optimizer::kTypeTORO) ||
Optimizer::isAvailable(Optimizer::kTypeG2O) ||
Optimizer::isAvailable(Optimizer::kTypeGTSAM) ||
Optimizer::isAvailable(Optimizer::kTypeCeres);
EXPECT_TRUE(anyAvailable) << "No graph optimizer was compiled in";
}
TEST(OptimizerTest, IsAvailableReturnsFalseForUndef)
{
EXPECT_FALSE(Optimizer::isAvailable(Optimizer::kTypeUndef));
}
TEST(OptimizerTest, CreateReturnsNonNullForDefaultStrategy)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
ASSERT_NE(opt.get(), nullptr);
EXPECT_NE(opt->type(), Optimizer::kTypeUndef);
}
TEST(OptimizerTest, CreateFallsBackWhenTypeNotAvailable)
{
// Pick whatever IS available and ask for it explicitly.
for(int t : {Optimizer::kTypeTORO, Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM, Optimizer::kTypeCeres,
Optimizer::kTypeCVSBA})
{
if(Optimizer::isAvailable(static_cast<Optimizer::Type>(t)))
{
std::unique_ptr<Optimizer> opt(
Optimizer::create(static_cast<Optimizer::Type>(t)));
ASSERT_NE(opt.get(), nullptr);
EXPECT_EQ(opt->type(), static_cast<Optimizer::Type>(t));
return;
}
}
GTEST_SKIP() << "No optimizer backend available";
}
TEST(OptimizerTest, CreateViaParametersMapHonorsOptimizerStrategy)
{
// Find any available backend and request it through the parameter map
// (the path used by Rtabmap::init when reading user config).
for(int t : {Optimizer::kTypeTORO, Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM, Optimizer::kTypeCeres,
Optimizer::kTypeCVSBA})
{
if(Optimizer::isAvailable(static_cast<Optimizer::Type>(t)))
{
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(t);
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
EXPECT_EQ(opt->type(), static_cast<Optimizer::Type>(t));
return;
}
}
GTEST_SKIP() << "No optimizer backend available";
}
// ---------------------------------------------------------------------------
// Getters / setters / parseParameters
// ---------------------------------------------------------------------------
TEST(OptimizerTest, DefaultsMatchParameterDefaults)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
ASSERT_NE(opt.get(), nullptr);
EXPECT_EQ(opt->iterations(), Parameters::defaultOptimizerIterations());
EXPECT_EQ(opt->isSlam2d(), Parameters::defaultRegForce3DoF());
EXPECT_EQ(opt->isCovarianceIgnored(),Parameters::defaultOptimizerVarianceIgnored());
EXPECT_DOUBLE_EQ(opt->epsilon(), Parameters::defaultOptimizerEpsilon());
EXPECT_EQ(opt->isRobust(), Parameters::defaultOptimizerRobust());
EXPECT_EQ(opt->priorsIgnored(), Parameters::defaultOptimizerPriorsIgnored());
EXPECT_EQ(opt->landmarksIgnored(), Parameters::defaultOptimizerLandmarksIgnored());
EXPECT_FLOAT_EQ(opt->gravitySigma(), Parameters::defaultOptimizerGravitySigma());
}
TEST(OptimizerTest, SettersUpdateGettersIndependently)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
ASSERT_NE(opt.get(), nullptr);
opt->setIterations(123);
opt->setSlam2d(true);
opt->setCovarianceIgnored(true);
opt->setEpsilon(1e-4);
opt->setRobust(true);
opt->setPriorsIgnored(true);
opt->setLandmarksIgnored(true);
opt->setGravitySigma(0.42f);
EXPECT_EQ(opt->iterations(), 123);
EXPECT_TRUE(opt->isSlam2d());
EXPECT_TRUE(opt->isCovarianceIgnored());
EXPECT_DOUBLE_EQ(opt->epsilon(), 1e-4);
EXPECT_TRUE(opt->isRobust());
EXPECT_TRUE(opt->priorsIgnored());
EXPECT_TRUE(opt->landmarksIgnored());
EXPECT_FLOAT_EQ(opt->gravitySigma(), 0.42f);
}
TEST(OptimizerTest, ParseParametersFromMap)
{
ParametersMap params;
params[Parameters::kOptimizerIterations()] = "77";
params[Parameters::kRegForce3DoF()] = "true";
params[Parameters::kOptimizerVarianceIgnored()] = "true";
params[Parameters::kOptimizerEpsilon()] = "0.001";
params[Parameters::kOptimizerRobust()] = "true";
params[Parameters::kOptimizerPriorsIgnored()] = "false";
params[Parameters::kOptimizerLandmarksIgnored()]= "true";
params[Parameters::kOptimizerGravitySigma()] = "0.5";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
EXPECT_EQ(opt->iterations(), 77);
EXPECT_TRUE(opt->isSlam2d());
EXPECT_TRUE(opt->isCovarianceIgnored());
EXPECT_DOUBLE_EQ(opt->epsilon(), 0.001);
EXPECT_TRUE(opt->isRobust());
EXPECT_FALSE(opt->priorsIgnored());
EXPECT_TRUE(opt->landmarksIgnored());
EXPECT_FLOAT_EQ(opt->gravitySigma(), 0.5f);
// parseParameters() can also be called post-construction to update fields
// in-place (Rtabmap::parseParameters does this on parameter change).
ParametersMap update;
update[Parameters::kOptimizerIterations()] = "5";
opt->parseParameters(update);
EXPECT_EQ(opt->iterations(), 5);
// Unspecified fields are preserved.
EXPECT_TRUE(opt->isRobust());
}
// ---------------------------------------------------------------------------
// getConnectedGraph
// ---------------------------------------------------------------------------
TEST(OptimizerTest, GetConnectedGraphReturnsSingleNodeWhenNoLinks)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
in[1] = Transform(0, 0, 0, 0, 0, 0);
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, std::multimap<int, Link>(), out, outLinks);
ASSERT_EQ(out.size(), 1u);
EXPECT_TRUE(out.count(1));
EXPECT_TRUE(outLinks.empty());
}
TEST(OptimizerTest, GetConnectedGraphKeepsFullChain)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(out.size(), 3u);
EXPECT_EQ(outLinks.size(), 2u);
}
TEST(OptimizerTest, GetConnectedGraphDropsDisconnectedNodes)
{
// N1 -> N2 -> N3 is the connected component starting from N1.
// N10 has no link to the others -> must NOT appear in the output.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
in[10] = Transform(99.0f, 0, 0, 0, 0, 0);
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(out.size(), 3u);
EXPECT_FALSE(out.count(10));
}
TEST(OptimizerTest, GetConnectedGraphRecomputesPosesAlongEdges)
{
// The input pose at N3 is "wrong" (10m away) but the link chain says
// 0 -> 1 -> 2. getConnectedGraph must walk the links, not trust the
// caller's poses, so the output should reflect the link-implied chain.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
in[3] = Transform(10.0f, 0, 0, 0, 0, 0); // wrong on purpose
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
ASSERT_TRUE(out.count(3));
EXPECT_NEAR(out.at(3).x(), 2.0f, 1e-5);
}
TEST(OptimizerTest, GetConnectedGraphFollowsLoopClosureLinks)
{
// Loop closure between N3 and N1 should pull N3 into the connected
// component even though the chain (N1 N2 N3) already does so. We add
// loop-closure types to make sure they're respected.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
in[1] = Transform(0, 0, 0, 0, 0, 0);
in[2] = Transform(1, 0, 0, 0, 0, 0);
in[3] = Transform(0, 1, 0, 0, 0, 0);
inLinks.insert({1, neighborLink(1, 2)});
inLinks.insert({1, Link(1, 3, Link::kGlobalClosure,
Transform(0, 1, 0, 0, 0, 0), unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(out.size(), 3u);
EXPECT_EQ(outLinks.size(), 2u);
}
TEST(OptimizerTest, GetConnectedGraphPriorLinksIncludedWhenAllowed)
{
// kOptimizerPriorsIgnored defaults to true, so flip it off to exercise
// the inclusion path.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
opt->setPriorsIgnored(false);
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
// Pose prior on N1 (self-link with kPosePrior).
inLinks.insert({1, Link(1, 1, Link::kPosePrior,
in[1], unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
// 2 chain links + 1 prior.
EXPECT_EQ(outLinks.size(), 3u);
}
TEST(OptimizerTest, GetConnectedGraphPriorLinksDroppedWhenIgnored)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
opt->setPriorsIgnored(true);
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
inLinks.insert({1, Link(1, 1, Link::kPosePrior, in[1], unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(outLinks.size(), 2u); // prior dropped
}
TEST(OptimizerTest, GetConnectedGraphLandmarkLinksDroppedWhenIgnored)
{
// Landmarks use negative IDs by rtabmap convention. landmarksIgnored
// keeps the landmark out of the output graph entirely.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
opt->setLandmarksIgnored(true);
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
in[-1] = Transform(0.5f, 0.5f, 0, 0, 0, 0);
inLinks.insert({2, Link(2, -1, Link::kLandmark,
Transform(0, 0.5f, 0, 0, 0, 0), unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(out.size(), 3u);
EXPECT_FALSE(out.count(-1));
}
TEST(OptimizerTest, GetConnectedGraphLandmarkLinksKeptByDefault)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
ASSERT_FALSE(opt->landmarksIgnored());
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
in[-1] = Transform(0.5f, 0.5f, 0, 0, 0, 0);
inLinks.insert({2, Link(2, -1, Link::kLandmark,
Transform(0, 0.5f, 0, 0, 0, 0), unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_TRUE(out.count(-1));
}
// ---------------------------------------------------------------------------
// optimize() / optimizeIncremental() smoke tests
//
// Parameterized over the four pose-graph backends. CVSBA is excluded -- it's
// bundle-adjustment-only and doesn't implement plain pose-graph optimize().
// Each test skips at runtime when the backend isn't built in.
// ---------------------------------------------------------------------------
namespace {
std::string optimizerTypeName(Optimizer::Type type)
{
switch(type)
{
case Optimizer::kTypeTORO: return "TORO";
case Optimizer::kTypeG2O: return "G2O";
case Optimizer::kTypeGTSAM: return "GTSAM";
case Optimizer::kTypeCeres: return "Ceres";
case Optimizer::kTypeCVSBA: return "CVSBA";
default: return "Undef";
}
}
class OptimizerBackendTest : public ::testing::TestWithParam<Optimizer::Type>
{
protected:
void SetUp() override
{
if(!Optimizer::isAvailable(GetParam()))
{
GTEST_SKIP() << optimizerTypeName(GetParam()) << " not built in";
}
opt_.reset(Optimizer::create(GetParam()));
ASSERT_NE(opt_.get(), nullptr);
ASSERT_EQ(opt_->type(), GetParam());
}
std::unique_ptr<Optimizer> opt_;
};
} // namespace
TEST_P(OptimizerBackendTest, OptimizeConsistentChainReturnsSamePoses)
{
// A chain whose link transforms match the poses exactly has zero residual,
// so any optimizer should return ~the same poses.
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
std::map<int, Transform> out = opt_->optimize(1, in, inLinks);
ASSERT_EQ(out.size(), 3u);
for(const auto & kv : in)
{
ASSERT_TRUE(out.count(kv.first));
EXPECT_NEAR(out.at(kv.first).x(), kv.second.x(), 1e-2)
<< optimizerTypeName(GetParam()) << " node " << kv.first;
}
}
TEST_P(OptimizerBackendTest, OptimizeAnchorsRootPose)
{
// The root pose should remain fixed at the input value -- it's the gauge.
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
std::map<int, Transform> out = opt_->optimize(/*rootId=*/1, in, inLinks);
ASSERT_TRUE(out.count(1));
EXPECT_NEAR(out.at(1).x(), 0.0f, 1e-4) << optimizerTypeName(GetParam());
EXPECT_NEAR(out.at(1).y(), 0.0f, 1e-4) << optimizerTypeName(GetParam());
EXPECT_NEAR(out.at(1).z(), 0.0f, 1e-4) << optimizerTypeName(GetParam());
}
TEST_P(OptimizerBackendTest, OptimizeIncrementalConsistentChain)
{
// Same as the smoke test above but driven through the incremental path.
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
// Incremental needs at least one loop closure to "fire"; without one it
// short-circuits to the regular optimize() call at the end.
inLinks.insert({1, Link(1, 3, Link::kGlobalClosure,
Transform(2.0f, 0, 0, 0, 0, 0), unitCov())});
std::map<int, Transform> out = opt_->optimizeIncremental(1, in, inLinks);
ASSERT_EQ(out.size(), 3u);
for(const auto & kv : in)
{
ASSERT_TRUE(out.count(kv.first));
EXPECT_NEAR(out.at(kv.first).x(), kv.second.x(), 1e-2)
<< optimizerTypeName(GetParam()) << " node " << kv.first;
}
}
INSTANTIATE_TEST_SUITE_P(
AllPoseGraphBackends,
OptimizerBackendTest,
::testing::Values(
Optimizer::kTypeTORO,
Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM,
Optimizer::kTypeCeres),
[](const ::testing::TestParamInfo<Optimizer::Type> & info) {
return optimizerTypeName(info.param);
});
// ---------------------------------------------------------------------------
// Closed-loop circular trajectory with landmarks + gravity + pose priors.
//
// Setup: 30 evenly-spaced poses on a 10 m radius circle in the XY plane. Each
// pose's yaw points tangentially (i.e. toward the next pose -- a differential
// robot orbiting the center). Constraints:
// * 29 sequential neighbor links (1->2, 2->3, ..., 29->30) carrying the
// exact pairwise transform.
// * 1 loop-closure (30->1) carrying the exact transform.
// * 1 landmark at the origin (id = -1) with identity rotation. Even-numbered
// poses (2, 4, ..., 30) observe it via kLandmark links.
// * A kPosePrior self-link on every pose, transform = the pose itself.
// * A kGravity self-link on every pose, transform = the pose itself
// (roll/pitch are what the optimizer reads).
// All link covariances are unit 6x6, including the landmark links.
//
// This is a "perfect world" -- every constraint is exactly satisfied by the
// input poses, so the optimizer's first residual is zero and the output
// should equal the input (modulo the root being the gauge). We exercise the
// 16 combinations of {Force3DoF, GravitySigma, PriorsIgnored, LandmarksIgnored,
// useLastAsRoot} on every available pose-graph backend.
// ---------------------------------------------------------------------------
namespace {
constexpr int kCircleN = 30;
constexpr float kCircleRadius = 10.0f;
constexpr int kLandmarkId = -1;
constexpr int kLandmarkId2 = -2; // second landmark, used when numLandmarks=2
// Per-axis 1-sigma values used both for the information matrix and (when
// noise is requested) for the initial-guess perturbation.
constexpr double kLinSigma = 0.05; // 5 cm
const double kAngSigma = 1.0 * CV_PI / 180.0; // 1 deg in rad
struct CircleGraph
{
std::map<int, Transform> truePoses; // ground truth (link transforms came from these)
std::map<int, Transform> poses; // optimizer input -- equals truePoses
// except for noised non-anchor nodes
std::multimap<int, Link> links;
};
// Build the perfectly-consistent circle graph described above. When noisy=true,
// every pose except 1, kCircleN, and the landmark is perturbed by Gaussian
// noise (sigma = kLinSigma, kAngSigma) -- giving the optimizer a noisy
// initial guess while the constraints still describe the true graph.
CircleGraph buildCircleGraph(bool noisy = false, int numLandmarks = 1, bool force3DoF = false)
{
CircleGraph g;
// Realistic information matrix (diag(1/sigma^2)): 5 cm 1-sigma on each
// linear axis, 1 deg 1-sigma on each angular axis.
const double linInfo = 1.0 / (kLinSigma * kLinSigma);
const double angInfo = 1.0 / (kAngSigma * kAngSigma);
cv::Mat info = cv::Mat::zeros(6, 6, CV_64FC1);
info.at<double>(0, 0) = linInfo;
info.at<double>(1, 1) = linInfo;
info.at<double>(2, 2) = linInfo;
info.at<double>(3, 3) = angInfo;
info.at<double>(4, 4) = angInfo;
info.at<double>(5, 5) = angInfo;
// Sample poses around the circle. Yaw aims at the next pose so a
// differential robot walking the chain only translates forward in its
// own frame between consecutive nodes.
//
// In 6DoF mode, z and pitch vary sinusoidally (three hills around the
// loop, ~0.5 m crest, ~8.6 deg max pitch) to simulate a robot driving
// over rolling terrain. The pitch matches the slope so the chain is
// physically consistent. In 3DoF mode the loop stays flat -- the 3DoF
// optimizer doesn't model z/pitch anyway.
const float hillAmp = force3DoF ? 0.0f : 0.5f;
const float hillFreq = 3.0f;
for(int i = 1; i <= kCircleN; ++i)
{
const float theta = 2.0f * static_cast<float>(CV_PI) * static_cast<float>(i - 1) / static_cast<float>(kCircleN);
const float x = kCircleRadius * std::cos(theta);
const float y = kCircleRadius * std::sin(theta);
const float z = hillAmp * std::sin(hillFreq * theta);
const float yaw = theta + static_cast<float>(CV_PI) / 2.0f; // tangent to circle
// Slope along the body's forward direction. dz/ds = (1/R) * dz/dtheta.
// Nose up when going uphill -> negative pitch in ZYX Euler.
const float pitch = -hillAmp * hillFreq / kCircleRadius * std::cos(hillFreq * theta);
g.truePoses[i] = Transform(x, y, z, 0.0f, pitch, yaw);
}
// Landmark at the origin with identity rotation. Optional 2nd landmark
// horizontally offset along +X so the relative bearing between the two
// (in body frame) is a function of each pose's position (not orientation),
// breaking the bearing-only ambiguity of a single-landmark setup.
g.truePoses[kLandmarkId] = Transform::getIdentity();
if(numLandmarks >= 2)
{
g.truePoses[kLandmarkId2] = Transform(5.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f);
}
// Neighbor chain 1 -> 2 -> ... -> 30.
for(int i = 1; i < kCircleN; ++i)
{
const Transform t = g.truePoses[i].inverse() * g.truePoses[i + 1];
g.links.insert({i, Link(i, i + 1, Link::kNeighbor, t, info)});
}
// Loop closure 30 -> 1.
{
const Transform t = g.truePoses[kCircleN].inverse() * g.truePoses[1];
g.links.insert({kCircleN, Link(kCircleN, 1, Link::kGlobalClosure, t, info)});
}
// Landmark observations on even poses. Built in the post-getConnectedGraph
// normalized format: from=landmark_id (negative), to=observer, transform =
// landmark -> observer. This is the format the optimizer backends expect
// and lets us bypass getConnectedGraph (which would erase pose noise by
// reconstructing poses from links via a tree walk).
for(int i = 2; i <= kCircleN; i += 2)
{
const Transform t = g.truePoses[kLandmarkId].inverse() * g.truePoses[i];
g.links.insert({kLandmarkId, Link(kLandmarkId, i, Link::kLandmark, t, info)});
if(numLandmarks >= 2)
{
const Transform t2 = g.truePoses[kLandmarkId2].inverse() * g.truePoses[i];
g.links.insert({kLandmarkId2, Link(kLandmarkId2, i, Link::kLandmark, t2, info)});
}
}
// Per-pose pose prior and gravity self-link, both equal to the pose itself.
for(int i = 1; i <= kCircleN; ++i)
{
g.links.insert({i, Link(i, i, Link::kPosePrior, g.truePoses[i], info)});
g.links.insert({i, Link(i, i, Link::kGravity, g.truePoses[i], info)});
}
// Optimizer input starts at ground truth. With noisy=true, every pose
// EXCEPT id 1 and id kCircleN (those may be used as the gauge anchor) is
// perturbed by Gaussian noise drawn from the same (kLinSigma, kAngSigma)
// the links were weighted against. The landmark is also perturbed -- it
// isn't a gauge anchor; the optimizer should pull it back via its
// observation links. The constraints still describe the true graph --
// only the initial guess is wrong, so the optimizer should pull back to
// the truth.
g.poses = g.truePoses;
if(noisy)
{
// Fixed seed: identical noise sequence on every platform / run.
std::mt19937 rng(42);
std::normal_distribution<double> linDist(0.0, kLinSigma);
std::normal_distribution<double> angDist(0.0, kAngSigma);
auto perturb = [&](int id) {
const Transform & t = g.truePoses.at(id);
float r, p, y;
t.getEulerAngles(r, p, y);
const float dx = static_cast<float>(linDist(rng));
const float dy = static_cast<float>(linDist(rng));
const float dz = static_cast<float>(linDist(rng));
const float dr = static_cast<float>(angDist(rng));
const float dp = static_cast<float>(angDist(rng));
const float dyaw = static_cast<float>(angDist(rng));
g.poses[id] = Transform(
t.x() + dx, t.y() + dy, t.z() + dz,
r + dr, p + dp, y + dyaw);
};
for(int i = 2; i <= kCircleN - 1; ++i)
{
perturb(i);
}
perturb(kLandmarkId);
if(numLandmarks >= 2)
{
perturb(kLandmarkId2);
}
}
return g;
}
// Tuple ordering: optimizer, force3DoF, gravitySigma, priorsIgnored,
// landmarksIgnored, useLastAsRoot. Bool params are wrapped in int for
// readability in gtest's default printer; the lambda below pretty-prints.
using CircleParam = std::tuple<Optimizer::Type, bool, float, bool, bool, bool>;
class CircleGraphTest : public ::testing::TestWithParam<CircleParam>
{
protected:
void SetUp() override
{
if(!Optimizer::isAvailable(std::get<0>(GetParam())))
{
GTEST_SKIP() << optimizerTypeName(std::get<0>(GetParam())) << " not built in";
}
}
};
} // namespace
TEST_P(CircleGraphTest, NoisyInitialGuessConvergesToTruth)
{
const auto & p = GetParam();
const Optimizer::Type backend = std::get<0>(p);
const bool force3DoF = std::get<1>(p);
const float gravitySigma = std::get<2>(p);
const bool priorsIgnored = std::get<3>(p);
const bool landmarksIgnored = std::get<4>(p);
const bool useLastAsRoot = std::get<5>(p);
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kRegForce3DoF()] = force3DoF ? "true" : "false";
params[Parameters::kOptimizerGravitySigma()] = uNumber2Str(gravitySigma);
params[Parameters::kOptimizerPriorsIgnored()] = priorsIgnored ? "true" : "false";
params[Parameters::kOptimizerLandmarksIgnored()]= landmarksIgnored ? "true" : "false";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
// Build with noise: every non-anchor pose (i.e. not 1, not kCircleN, not
// the landmark) is perturbed. The links still encode the truth, so the
// optimizer should pull the poses back to truePoses.
CircleGraph g = buildCircleGraph(/*noisy=*/true, /*numLandmarks=*/1, force3DoF);
const int rootId = useLastAsRoot ? kCircleN : 1;
// Optional orientation perturbation on top of the per-axis noise:
// pre-multiply each input pose's rotation (positions left untouched) by
// a small world-frame roll/pitch/(yaw) rotation. The relative link
// constraints still describe the truth, so the optimizer needs an
// *absolute* orientation reference to recover. The two interesting
// cases are:
//
// priors-only (gravitySigma == 0, priorsIgnored == false):
// Per-pose pose priors anchor every pose's full orientation, so
// every pose -- including the roots -- gets roll/pitch/yaw
// perturbed. The priors pull each pose back to truth.
//
// gravity-only (gravitySigma > 0, priorsIgnored == true):
// Gravity ties only roll/pitch to world up; yaw remains gauge-free.
// So the roots (id 1, kCircleN) get roll/pitch perturbed but their
// yaw is left at truth -- the optimizer's gauge fix in yaw depends
// on the root staying at truth. Non-roots also get yaw perturbed;
// the chain of links propagates the root's true yaw and the relative
// yaw constraints pull the non-roots back.
//
// Skipped for TORO (its tree-based SGD doesn't converge tightly enough)
// and for 3DoF mode (roll/pitch are not optimized there). The "neither"
// and "both" configs are also skipped: "neither" is gauge-free in yaw,
// and "both" is already covered by the unperturbed noisy test.
const bool priorsOnly = (gravitySigma == 0.0f && !priorsIgnored);
const bool gravityOnly = (gravitySigma > 0.0f && priorsIgnored);
if((opt->type() == Optimizer::kTypeG2O || opt->type() == Optimizer::kTypeGTSAM) && !force3DoF && (priorsOnly || gravityOnly))
{
const float deg = static_cast<float>(CV_PI) / 180.0f;
const float dRoll = 3.0f * deg;
const float dPitch = 4.0f * deg;
const float dYawNonRoot = 5.0f * deg;
const float dYawRoot = priorsOnly ? 5.0f * deg : 0.0f;
// On gravity-only roots, perturb only ONE of roll/pitch (drop pitch).
// Perturbing both induces a small geometric yaw drift on the root that
// gravity can't recover (yaw is gauge-free in that mode): after
// gravity rotates the tilted local Z back to world Z, the local X-axis
// settles at a slightly different XY direction, and that yaw twist
// propagates around the chain. Pure roll alone (or pure pitch alone)
// keeps the local X-axis in the XY plane, so no drift.
const float dPitchRoot = gravityOnly ? 0.0f : dPitch;
// World-frame perturbation on non-roots: pre-multiplied onto each
// pose's transform so roll/pitch deltas act about world axes (what
// gravity observes). On roots we post-multiply (body-frame) so the
// root's truth position is preserved -- in gravity-only mode the
// optimizer holds root setFixed in position, and a world-frame
// pre-multiply would shift it about the world origin and drag the
// whole chain along.
for(int i = 1; i <= kCircleN; ++i)
{
const bool isRoot = (i == 1 || i == kCircleN);
const float dPi = isRoot ? dPitchRoot : dPitch;
const float dYaw = isRoot ? dYawRoot : dYawNonRoot;
const Transform R(0.0f, 0.0f, 0.0f, dRoll, dPi, dYaw);
g.poses[i] = isRoot ? g.poses.at(i) * R : R * g.poses.at(i);
}
}
// Use getConnectedGraph() only to get the filtered link set (it drops
// kPosePrior self-links when priorsIgnored and landmarks when
// landmarksIgnored). Its pose output is the result of a tree-walk from
// the root that would overwrite every pose -- erasing the noise we just
// added -- so we discard it and feed the optimizer our noisy g.poses
// instead.
std::map<int, Transform> connectedPoses;
std::multimap<int, Link> inLinks;
opt->getConnectedGraph(rootId, g.poses, g.links, connectedPoses, inLinks);
ASSERT_EQ(connectedPoses.size(), g.poses.size() - (landmarksIgnored ? 1u : 0u));
std::map<int, Transform> inPoses = g.poses;
if(landmarksIgnored)
{
inPoses.erase(kLandmarkId);
}
// In 3DoF mode the optimizer doesn't touch z/roll/pitch, so any noise we
// added on those axes would survive into the output verbatim and fail the
// truth comparison for no real reason. Strip it from the input poses up
// front so what we feed in matches the DoFs the optimizer actually solves.
if(force3DoF)
{
// 3DoF optimizer doesn't touch z/roll/pitch on poses OR on the
// landmark, so strip any z/roll/pitch noise from every input pose
// (including the landmark) to match what the optimizer can actually
// solve.
for(auto & kv : inPoses)
{
kv.second = kv.second.to3DoF();
}
}
std::map<int, Transform> out = opt->optimize(rootId, inPoses, inLinks);
ASSERT_FALSE(out.empty()) << "Optimizer returned no poses";
// Only g2o and GTSAM actually optimize landmarks; TORO and Ceres log a
// warning and drop the landmark silently from the output. So the landmark
// appears in the output when (a) the backend supports it AND
// (b) landmarksIgnored is false.
const bool backendSupportsLandmarks =
backend == Optimizer::kTypeG2O || backend == Optimizer::kTypeGTSAM;
const bool expectLandmarkInOutput = backendSupportsLandmarks && !landmarksIgnored;
EXPECT_EQ(out.size(), kCircleN + (expectLandmarkInOutput ? 1u : 0u));
// Constraints describe the truth, so the optimizer should pull the noisy
// initial guess back to truePoses. Tolerances are loose enough to absorb
// each backend's residual-at-convergence, tight enough that a "didn't
// move" regression would trip immediately (noise is up to ~3*sigma_lin
// = 15 cm per axis from truth).
for(const auto & kv : g.truePoses)
{
const int id = kv.first;
if(id == kLandmarkId && !expectLandmarkInOutput)
{
continue;
}
ASSERT_TRUE(out.count(id)) << "missing id " << id;
// TORO is a tree-based SGD optimizer with a fixed iteration count;
// the others (g2o/GTSAM/Ceres) use Gauss-Newton / Levenberg-Marquardt
// and converge to numerical zero, so we hold them to much tighter
// bounds.
const float distTol = (backend == Optimizer::kTypeTORO) ? 0.015f : 0.002f;
const float angTolDeg = (backend == Optimizer::kTypeTORO) ? 0.5f : 0.01f;
EXPECT_LT(out.at(id).getDistance(kv.second), distTol)
<< optimizerTypeName(backend) << " id=" << id
<< " expected=" << kv.second.prettyPrint()
<< " got=" << out.at(id).prettyPrint();
const float angDeg = out.at(id).getAngle(kv.second) * 180.0f / static_cast<float>(M_PI);
EXPECT_LT(angDeg, angTolDeg)
<< optimizerTypeName(backend) << " id=" << id
<< " angle=" << angDeg << " deg";
}
}
INSTANTIATE_TEST_SUITE_P(
AllBackendsAndConfigs,
CircleGraphTest,
::testing::Combine(
::testing::Values(Optimizer::kTypeTORO,
Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM,
Optimizer::kTypeCeres),
::testing::Bool(), // Reg/Force3DoF
::testing::Values(0.0f, 0.01f), // Optimizer/GravitySigma
::testing::Bool(), // Optimizer/PriorsIgnored
::testing::Bool(), // Optimizer/LandmarksIgnored
::testing::Bool()), // useLastAsRoot
[](const ::testing::TestParamInfo<CircleParam> & info) {
std::string name = optimizerTypeName(std::get<0>(info.param));
name += std::get<1>(info.param) ? "_3DoF" : "_6DoF";
name += std::get<2>(info.param) > 0.0f ? "_grav" : "_nograv";
name += std::get<3>(info.param) ? "_noprior" : "_prior";
name += std::get<4>(info.param) ? "_nolm" : "_lm";
name += std::get<5>(info.param) ? "_rootLast" : "_rootFirst";
return name;
});
// ---------------------------------------------------------------------------
// CircleGraphBadLoopClosureTest -- corrupt the loop-closure link with a large
// translation offset and verify that the Optimizer/Robust kernel rejects the
// outlier loop closure (g2o + GTSAM only; relies on Vertigo switchable
// factors which are conditional on WITH_VERTIGO build flag).
// ---------------------------------------------------------------------------
namespace {
using CircleBadLcParam = std::tuple<Optimizer::Type, bool /*useRobust*/>;
class CircleGraphBadLoopClosureTest : public ::testing::TestWithParam<CircleBadLcParam>
{
protected:
void SetUp() override
{
const Optimizer::Type t = std::get<0>(GetParam());
if(!Optimizer::isAvailable(t))
{
GTEST_SKIP() << optimizerTypeName(t) << " not built in";
}
}
};
} // namespace
TEST_P(CircleGraphBadLoopClosureTest, RobustRejectsCorruptedLoopClosure)
{
const Optimizer::Type backend = std::get<0>(GetParam());
const bool useRobust = std::get<1>(GetParam());
// Use the noisy CircleGraph (5 cm pose + point noise) so the test has a
// realistic baseline. Force 6DoF, no gravity, ignore priors+landmarks so
// the only relevant constraints are the neighbor chain + the (corrupted)
// loop closure.
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kRegForce3DoF()] = "false";
params[Parameters::kOptimizerGravitySigma()] = "0";
params[Parameters::kOptimizerPriorsIgnored()] = "true";
params[Parameters::kOptimizerLandmarksIgnored()] = "true";
params[Parameters::kOptimizerRobust()] = useRobust ? "true" : "false";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
CircleGraph g = buildCircleGraph(/*noisy=*/true);
// Corrupt the loop closure: take the truth kGlobalClosure transform and
// add a 5 m translation offset on its X axis. We need at least a few
// meters for the robust kernel to clearly prefer "disable the LC" over
// "absorb the offset into the chain": 11 neighbor links with
// sigma=5 cm (info=400/axis) can absorb ~1 m of LC offset with chain
// chi^2 ~ 11 * (1m/11)^2 * 400 ~ 36, comparable to the cost of
// dropping the switch (~1) + the chain-at-truth noise residual (~30).
// At 5 m the chain absorption costs ~890 chi^2, which is decisively
// worse than disabling the LC -- so the Vertigo switch reliably drives
// to 0.
std::multimap<int, Link> corruptedLinks;
for(const auto & kv : g.links)
{
if(kv.second.type() == Link::kGlobalClosure)
{
const Transform & t = kv.second.transform();
float r, p, y;
t.getEulerAngles(r, p, y);
const Transform bad(t.x() + 5.0f, t.y(), t.z(), r, p, y);
corruptedLinks.insert({kv.first,
Link(kv.second.from(), kv.second.to(), kv.second.type(),
bad, kv.second.infMatrix())});
}
else
{
corruptedLinks.insert(kv);
}
}
g.links = corruptedLinks;
const int rootId = 1;
std::map<int, Transform> connectedPoses;
std::multimap<int, Link> inLinks;
opt->getConnectedGraph(rootId, g.poses, g.links, connectedPoses, inLinks);
std::map<int, Transform> out = opt->optimize(rootId, g.poses, inLinks);
ASSERT_FALSE(out.empty()) << "Optimizer returned no poses";
// Worst pose displacement from truth.
float worstPoseD = 0.0f;
for(const auto & kv : g.truePoses)
{
if(kv.first < 0) continue; // skip landmarks (we ignored them anyway)
ASSERT_TRUE(out.count(kv.first)) << "missing id " << kv.first;
worstPoseD = std::max(worstPoseD, out.at(kv.first).getDistance(kv.second));
}
if(useRobust)
{
// Vertigo switchable factor wraps the loop closure with a switch
// variable initialized to 1 (LC fully active) and a prior keeping
// it near 1; the optimizer is free to drive the switch toward 0
// to disable the LC. With the Jacobian fix in
// betweenFactorSwitchable.h, both backends drive the switch fully
// to 0 and recover to ~mm/sub-mm pose error:
// * G2O ~1 mm.
// * GTSAM ~30 microns (slightly tighter than g2o on this
// particular setup because GaussNewton is used by default).
EXPECT_LT(worstPoseD, 0.005f)
<< optimizerTypeName(backend) << " robust=on worstPoseD=" << worstPoseD;
}
else
{
// Without robust, the bad loop closure is a hard constraint -- the
// chain stretches to satisfy it and pose 30 ends up ~5 m off truth.
EXPECT_GT(worstPoseD, 4.5f)
<< optimizerTypeName(backend) << " robust=off worstPoseD=" << worstPoseD;
}
}
INSTANTIATE_TEST_SUITE_P(
Backends,
CircleGraphBadLoopClosureTest,
::testing::Values(
std::make_tuple(Optimizer::kTypeG2O, false),
std::make_tuple(Optimizer::kTypeG2O, true),
std::make_tuple(Optimizer::kTypeGTSAM, false),
std::make_tuple(Optimizer::kTypeGTSAM, true)),
[](const ::testing::TestParamInfo<CircleBadLcParam> & info)
{
std::string name = optimizerTypeName(std::get<0>(info.param));
name += std::get<1>(info.param) ? "_Robust" : "_NoRobust";
return name;
});
// ---------------------------------------------------------------------------
// LandmarkCovarianceTest -- effect of per-link landmark covariance on g2o/GTSAM
// ---------------------------------------------------------------------------
// Reuses the CircleGraph (closed loop + landmark observed by even poses), but
// rewrites the information matrix on every Link::kLandmark edge to one of
// several test cases before optimizing. Limited to g2o and GTSAM -- TORO and
// Ceres drop landmarks silently (see backendSupportsLandmarks above).
namespace {
// Mirrors the Marker/Variance{Linear,Angular,OrientationIgnored} triplet from
// Parameters.h. Memory.cpp uses these to build the per-landmark-link 6x6
// covariance fed to the optimizer; we re-create that same construction here
// so our tests exercise the exact code path real marker detections take.
struct LandmarkCovCase
{
const char * name;
double linVar; // Marker/VarianceLinear (1.0 in the 3x3 translation block)
double angVar; // Marker/VarianceAngular (1.0 in the 3x3 rotation block; 9999 = sentinel)
bool orientationIgnored; // Marker/VarianceOrientationIgnored
double linkLinNoise; // Sigma (m) of Gaussian noise added to every kLandmark link's translation.
// 0 means no extra noise (default; equivalent to a "perfect" marker detector).
bool linkRangeOnly; // If true, noise is applied ALONG the bearing direction only -- preserves the
// direction (perfect bearing) but corrupts the magnitude (noisy range). If
// false (default), noise is isotropic on x/y/z and corrupts both bearing and range.
bool openLoopWithDrift; // If true: drop the loop-closure link, perturb every kNeighbor link's transform
// (Gaussian translation/rotation noise plus a constant -1 deg yaw bias), and
// seed the optimizer with poses derived from this perturbed chain (via
// getConnectedGraph's tree walk) instead of g.poses. Simulates open-loop
// odometry drift -- the optimizer has to correct it from the landmark
// observations alone.
int numLandmarks; // 1 (default) or 2. Adds a second landmark at (5, 0, 0) so each observer pose has
// two horizontally-separated bearings -- the relative bearing between them
// constrains pose position independently of orientation (breaks bearing-only
// ambiguity).
};
const LandmarkCovCase kLandmarkCovCases[] = {
// Full-pose landmark observations (orientation estimated). Identical
// encoding for g2o and GTSAM; force3DoF doesn't change the matrix.
{ "default", 0.001, 0.01, false, 0.0, false, false, 1 }, // Marker defaults
{ "tightLin", 0.0001, 0.01, false, 0.0, false, false, 1 },
{ "looseLin", 0.01, 0.01, false, 0.0, false, false, 1 },
{ "looseAng", 0.001, 1.0, false, 0.0, false, false, 1 },
// orientationIgnored=true: rotation block is set to 9999 (no orientation
// estimation). For g2o only linVar matters. For GTSAM the encoding
// switches to bearing/range -- angVar drives bearing, linVar drives range.
// linVar=9999 is the documented sentinel that asks GTSAM to skip the
// range factor (bearing-only).
{ "noOrient", 0.001, 0.01, true, 0.0, false, false, 1 },
{ "noOrientBO", 9999, 0.01, true, 0.0, false, false, 1 }, // bearing-only (GTSAM)
// Same as noOrient but with sigma=0.3 m injected into every landmark
// observation's translation. linVar is bumped to 0.1 (sigma~0.32 m) so
// the info matrix matches the actual noise level. GTSAM's bearing/range
// factorization is expected to handle this gracefully (multiple noisy
// observations triangulate); g2o's Cartesian EdgeSE3PointXYZ is expected
// to do worse on the landmark.
{ "noOrientNoisy", 0.1, 0.001, true, 0.3, false, false, 1 },
// Range-only noise: sigma=0.3 m applied along the bearing direction
// (perfect bearing, noisy range). This is the cleanest stress test for
// GTSAM's BearingRangeFactor: multiple noise-free bearings still cross
// at the true landmark, so the landmark should triangulate to truth and
// pose rotations should stay essentially untouched. g2o's Cartesian
// EdgeSE3PointXYZ sees the same noisy translations and can't exploit
// the bearing-purity, so it should still struggle.
{ "noOrientRangeNoisy", 0.1, 0.0001, true, 0.3, true, false, 1 },
// Open-loop odometry with drift + perfect-bearing landmark observations:
// loop closure dropped, neighbor links translation/rotation-perturbed
// with a constant -1 deg yaw bias accumulating around the chain
// (~30 deg total drift at the open end), input poses seeded from that
// drifted chain. Only the landmark observations can pull the trajectory
// back to truth -- a stress test of the bearing/range factor's
// triangulating capability vs g2o's Cartesian factor.
{ "openLoopRangeNoisy", 2.25, 0.0001, true, 1.5, true, true, 1 },
// Same as openLoopRangeNoisy but with 2 horizontally-separated landmarks.
// The added relative-bearing position constraint plus the range factor
// (even with 1.5 m noise) should both tighten the chain further than
// either alone.
{ "openLoopRangeNoisy2Lm", 2.25, 0.0001, true, 1.5, true, true, 2 },
// Same setup as openLoopRangeNoisy but with linVar=9999 -- the GTSAM
// sentinel that disables the range factor entirely (BearingFactor
// instead of BearingRangeFactor). With 1.5 m range noise but perfect
// bearings, GTSAM should triangulate the landmark from bearings alone
// and never get fooled by the (huge) range noise. For g2o this isn't
// meaningful -- linVar=9999 sets its EdgeSE3PointXYZ info to ~1e-4 on
// translation, the landmark becomes unobservable, and with the loop
// closure also dropped the whole chain has no global correction; we
// skip g2o's assertions for this case.
{ "openLoopBearingOnly", 9999, 0.0001, true, 1.5, true, true, 1 },
// Same setup as openLoopBearingOnly but with a SECOND landmark at
// (5, 0, 0). The two landmarks are horizontally separated, so each
// observer pose's relative-bearing-between-the-two-landmarks is a
// function of the pose's POSITION only (orientation cancels). This
// breaks the bearing-only ambiguity (rotate-in-place + slide-along-ray)
// of a single-landmark setup. Expected: GTSAM 6DoF bearing-only should
// converge dramatically tighter than with one landmark.
{ "openLoopBearingOnly2Lm", 9999, 0.0001, true, 1.5, true, true, 2 },
};
using LandmarkCovParam = std::tuple<Optimizer::Type, int /* index into kLandmarkCovCases */, bool /* force3DoF */>;
class LandmarkCovarianceTest : public ::testing::TestWithParam<LandmarkCovParam>
{
protected:
void SetUp() override
{
const Optimizer::Type t = std::get<0>(GetParam());
if(!Optimizer::isAvailable(t))
{
GTEST_SKIP() << optimizerTypeName(t) << " not built in";
}
}
};
// Build the per-landmark-link 6x6 covariance the same way Memory.cpp does
// when ingesting a marker detection (see _markerOrientationIgnored branches
// at Memory.cpp:6076-6101). The optimizer backends interpret the diagonal
// blocks specially when orientationIgnored=true (g2o uses linVar on the
// translation block; GTSAM repacks it as bearing(angVar)/range(linVar)).
// Returns the COVARIANCE -- the caller must invert to get the info matrix
// the Link stores.
cv::Mat buildMarkerCov(const LandmarkCovCase & c, Optimizer::Type backend, bool force3DoF)
{
cv::Mat cov = cv::Mat::eye(6, 6, CV_64FC1);
if(c.orientationIgnored)
{
cov(cv::Range(3, 6), cv::Range(3, 6)) *= 9999; // disable orientation estimation
const bool isGTSAM = (backend == Optimizer::kTypeGTSAM);
if(!isGTSAM)
{
cov(cv::Range(0, 3), cv::Range(0, 3)) *= c.linVar;
}
else if(force3DoF)
{
// GTSAM 2D bearing/range: X=bearing, Y=range. (Z is unused by
// the Pose2/Point2 BearingRange factor.)
cov(cv::Range(0, 1), cv::Range(0, 1)) *= c.angVar;
cov(cv::Range(1, 2), cv::Range(1, 2)) *= c.linVar;
}
else
{
// GTSAM 3D bearing/range: X,Y=bearing, Z=range.
cov(cv::Range(0, 2), cv::Range(0, 2)) *= c.angVar;
cov(cv::Range(2, 3), cv::Range(2, 3)) *= c.linVar;
}
}
else
{
cov(cv::Range(0, 3), cv::Range(0, 3)) *= c.linVar;
cov(cv::Range(3, 6), cv::Range(3, 6)) *= c.angVar;
}
return cov;
}
} // namespace
TEST_P(LandmarkCovarianceTest, LandmarkCovarianceAffectsConvergence)
{
const auto & p = GetParam();
const Optimizer::Type backend = std::get<0>(p);
const LandmarkCovCase & cov = kLandmarkCovCases[std::get<1>(p)];
const bool force3DoF = std::get<2>(p);
// Skip g2o + linVar>=9999: it's the GTSAM "no range factor" sentinel,
// not meaningful for g2o (which has only one landmark factor type --
// EdgeSE3PointXYZ -- and would just see a near-zero-info Cartesian
// observation). The test cases are GTSAM-only.
if(backend == Optimizer::kTypeG2O && cov.orientationIgnored && cov.linVar >= 9999)
{
GTEST_SKIP() << "linVar=9999 (bearing-only sentinel) is GTSAM-only; "
"g2o would just see a near-zero-info Cartesian observation";
}
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kRegForce3DoF()] = force3DoF ? "true" : "false";
// Gravity is meaningful only in 6DoF -- in 3DoF the optimizer doesn't
// touch z/roll/pitch, so a gravity prior would be redundant.
params[Parameters::kOptimizerGravitySigma()] = force3DoF ? "0" : "0.01";
// For open-loop variants, enable priors and add an identity prior on the
// landmark to anchor it -- the optimizer can no longer satisfy bearings
// by sliding the landmark and must instead deform/rotate the pose chain.
const bool experimentLandmarkPrior = cov.openLoopWithDrift;
params[Parameters::kOptimizerPriorsIgnored()] = experimentLandmarkPrior ? "false" : "true";
params[Parameters::kOptimizerLandmarksIgnored()] = "false";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
// Reuse the noisy CircleGraph baseline.
CircleGraph g = buildCircleGraph(/*noisy=*/true, cov.numLandmarks, force3DoF);
// Rewrite landmark-link information matrices to the test case, using the
// same Memory.cpp construction marker detections go through. When the
// case requests it, also inject Gaussian translation noise into every
// kLandmark link's transform to simulate a noisy marker detector.
const cv::Mat lmInfo = buildMarkerCov(cov, backend, force3DoF).inv();
std::mt19937 linkRng(43);
std::normal_distribution<double> linkLinDist(0.0, cov.linkLinNoise);
// Open-loop drift on neighbor links: 1 cm linear sigma, 0.1 deg angular
// sigma, plus a constant -1 deg yaw bias on every neighbor edge.
std::mt19937 neighRng(44);
std::normal_distribution<double> neighLinDist(0.0, 0.01);
std::normal_distribution<double> neighAngDist(0.0, 0.1 * CV_PI / 180.0);
const float neighYawBias = -1.0f * static_cast<float>(CV_PI) / 180.0f;
std::multimap<int, Link> patchedLinks;
for(const auto & kv : g.links)
{
const Link & link = kv.second;
if(cov.openLoopWithDrift && link.type() == Link::kGlobalClosure)
{
// Drop the loop closure: simulates pure odometry, no place
// recognition.
continue;
}
if(cov.openLoopWithDrift && link.type() == Link::kNeighbor)
{
// Apply the -1 deg yaw bias as a left-multiplying rotation so it
// rotates the link's translation as well as its rotation -- models
// a yaw calibration bias (body frame rotated 1 deg from the true
// robot frame), where each forward step picks up a small lateral
// component proportional to the bias.
Transform t = Transform(0.0f, 0.0f, 0.0f, 0.0f, 0.0f, neighYawBias) * link.transform();
float r, p, y;
t.getEulerAngles(r, p, y);
const float nx = t.x() + static_cast<float>(neighLinDist(neighRng));
const float ny = t.y() + static_cast<float>(neighLinDist(neighRng));
const float nz = t.z() + static_cast<float>(neighLinDist(neighRng));
const float nr = r + static_cast<float>(neighAngDist(neighRng));
const float np_ = p + static_cast<float>(neighAngDist(neighRng));
const float nyaw = y + static_cast<float>(neighAngDist(neighRng));
t = Transform(
nx, ny, nz, nr, np_, nyaw);
patchedLinks.insert({kv.first, Link(link.from(), link.to(), link.type(), t, link.infMatrix())});
continue;
}
if(link.type() == Link::kLandmark)
{
Transform t = link.transform();
if(cov.linkLinNoise > 0.0)
{
float r, p, y;
t.getEulerAngles(r, p, y);
if(cov.linkRangeOnly)
{
// Scale t along its own direction: noisy magnitude
// (range), exact direction (bearing). With p(observer)
// at identity in the BearingRangeFactor construction,
// bearing = normalize(t) and range = |t|, so this
// perturbs the range only.
const float range = std::sqrt(t.x()*t.x() + t.y()*t.y() + t.z()*t.z());
if(range > 1e-6f)
{
const float scale = 1.0f + static_cast<float>(linkLinDist(linkRng)) / range;
t = Transform(
t.x() * scale,
t.y() * scale,
t.z() * scale,
r, p, y);
}
}
else
{
// Isotropic Gaussian on x/y/z (corrupts both bearing
// and range).
const float lx = t.x() + static_cast<float>(linkLinDist(linkRng));
const float ly = t.y() + static_cast<float>(linkLinDist(linkRng));
const float lz = t.z() + static_cast<float>(linkLinDist(linkRng));
t = Transform(lx, ly, lz, r, p, y);
}
}
patchedLinks.insert({kv.first, Link(link.from(), link.to(), link.type(), t, lmInfo)});
}
else
{
// In the landmark-prior experiment we flip priorsIgnored to
// false, which would otherwise activate the per-pose kPosePrior
// at truth and trivialize the test. Filter them out here -- the
// landmark prior we add below is the only prior we want active.
if(experimentLandmarkPrior && link.type() == Link::kPosePrior)
{
continue;
}
patchedLinks.insert(kv);
}
}
if(experimentLandmarkPrior)
{
// Position-only ("GPS-like") prior on the landmark at truth.
// Rotation-block info is set just below the 1/9999 "ignored"
// threshold so the GTSAM prior-detection logic treats this as a
// GPS prior (keeps the root setFixed). The translation block is
// tight (info ~10000 per axis -> sigma ~1 cm) to pin the landmark.
cv::Mat priorInfo = cv::Mat::zeros(6, 6, CV_64FC1);
for(int i = 0; i < 3; ++i) priorInfo.at<double>(i, i) = 10000.0;
for(int i = 3; i < 6; ++i) priorInfo.at<double>(i, i) = 1.0 / 10000.0;
patchedLinks.insert({kLandmarkId,
Link(kLandmarkId, kLandmarkId, Link::kPosePrior,
g.truePoses.at(kLandmarkId), priorInfo)});
if(cov.numLandmarks >= 2)
{
patchedLinks.insert({kLandmarkId2,
Link(kLandmarkId2, kLandmarkId2, Link::kPosePrior,
g.truePoses.at(kLandmarkId2), priorInfo)});
}
}
const int rootId = 1;
std::map<int, Transform> connectedPoses;
std::multimap<int, Link> inLinks;
opt->getConnectedGraph(rootId, g.poses, patchedLinks, connectedPoses, inLinks);
// In the open-loop variant, seed the optimizer with poses derived from
// the perturbed neighbor chain (a tree walk from root that accumulates
// the -1 deg/segment yaw bias) rather than the noisy ground-truth-ish
// g.poses. Everywhere else, keep g.poses as the input.
std::map<int, Transform> inPoses = cov.openLoopWithDrift ? connectedPoses : g.poses;
if(force3DoF)
{
// 3DoF optimizer doesn't touch z/roll/pitch on poses, so strip the
// noise we added on those axes. The landmark is left alone -- in
// production it's observed in full 3D and the optimizer keeps it on
// the SE(3) manifold even in 2D SLAM mode.
for(auto & kv : inPoses)
{
if(kv.first < 0)
{
continue;
}
kv.second = kv.second.to3DoF();
}
}
std::map<int, Transform> out = opt->optimize(rootId, inPoses, inLinks);
ASSERT_FALSE(out.empty()) << "Optimizer returned no poses";
EXPECT_EQ(out.size(), kCircleN + static_cast<size_t>(cov.numLandmarks)); // 30 poses + N landmarks
// TODO: per-case assertions. Sketch:
// - tight : output should hew very close to truth (landmark dominates).
// - default : matches the CircleGraphTest baseline tolerance.
// - loose : landmark contributes little; poses still converge via
// links + priors but with looser bounds on the landmark id.
// With perfect landmark observations every case converges to ~numerical
// zero on all DoFs the optimizer touches; with sigma=0.3 m noise injected
// into the landmark link transforms (kLandmarkCovCases::linkLinNoise),
// expect roughly the noise magnitude on the landmark and propagating to
// the poses, with g2o's Cartesian EdgeSE3PointXYZ struggling more than
// GTSAM's BearingRange in 6DoF (in 3DoF the two are comparable).
// Convergence bound selection:
// noise-free -> tight 1 mm / 0.01 deg / 1 mm.
// linkRangeOnly (perfect bearing, sigma=0.3m on range only):
// GTSAM exploits bearing-purity (BearingRangeFactor) and collapses
// to near-truth (poseD<5cm, ang<0.3deg, lmD<5cm); g2o 6DoF's
// EdgeSE3PointXYZ can't decompose so it does ~2x worse.
// isotropic noise (sigma=0.3m on x/y/z):
// Both bearings and ranges are corrupted; g2o 6DoF struggles
// particularly because its Cartesian factor weighs every axis equally.
const bool noisy = (cov.linkLinNoise > 0.0);
const bool g2o6DoF = (backend == Optimizer::kTypeG2O && !force3DoF);
// Non-noisy default bounds. Bumped from 1 mm to 2 mm because gravity (now
// enabled for 6DoF) adds extra Z-axis constraints that nudge G2O's
// noOrientBO solution to ~1.4 mm (still tight, just over the original
// 1 mm). GTSAM at ~4 microns either way.
float landmarkDistMax = 0.002f;
float poseDistMax = 0.002f;
float poseAngMaxDeg = 0.01f;
if(noisy && cov.openLoopWithDrift)
{
// Open-loop chain with -1 deg/segment yaw bias (~30 deg cumulative
// drift at the open end). A tight identity prior pins the landmark
// at truth (lmD essentially zero across all variants); the chain
// then converges from the prior's pull through the landmark
// observations. With gravity on (6DoF), roll/pitch are extra-tight.
//
// Behavior summary:
// * GTSAM 3DoF (both BearingRange and BearingOnly) recovers the
// chain best -- 2D bearing factor's Jacobian is well-conditioned
// and the pinned landmark provides a strong anchor (~3.7-3.9 m
// / 20-21 deg, vs ~5.8 m without prior).
// * GTSAM 6DoF: bearing-factor linearization on SO(3) hits a local
// minimum even with the landmark pinned (~4.7-5.5 m / 25-30 deg).
// * g2o 6DoF RangeNoisy: pinning the landmark actually HURTS
// convergence (~5.1 m, vs ~3.2 m without prior) -- the
// Cartesian observation edges can no longer balance landmark
// and chain residuals, and the solver settles in a worse
// local minimum.
// * g2o BearingOnly: chain stays fully uncorrected (~5.75 m /
// 29.5 deg) regardless of prior -- the EdgeSE3PointXYZ info
// matrix is ~1e-4 (linVar=9999), so the landmark is connected
// to the chain via near-zero-information edges.
landmarkDistMax = 0.01f; // pinned by prior, ~0 to 1 mm in practice
const bool isG2O = (backend == Optimizer::kTypeG2O);
const bool multiLm = (cov.numLandmarks >= 2);
if(multiLm && !isG2O)
{
// GTSAM with 2+ horizontally-separated landmarks: relative
// bearing between landmarks is a pure-position constraint
// (orientation-independent), resecting the pose. With or
// without the range factor, GTSAM converges to ~0.5 m / 3 deg
// in BOTH 3DoF and 6DoF.
poseDistMax = 0.70f;
poseAngMaxDeg = 5.0f;
}
else if(multiLm && isG2O)
{
// G2O EdgeSE3PointXYZ with FINITE landmark-edge info (linVar~2)
// triangulates well with 2 landmarks: chain converges to ~2 m
// / 15-18 deg. (The bearing-only g2o variant is skipped above
// since linVar=9999 is a GTSAM-only sentinel.)
poseDistMax = force3DoF ? 3.00f : 2.50f;
poseAngMaxDeg = force3DoF ? 20.0f : 17.0f;
}
else if(isG2O)
{
// G2O + 1 landmark + finite linVar: BearingRange-equivalent
// EdgeSE3PointXYZ. Chain partially recovers (~5 m).
poseDistMax = 6.00f;
poseAngMaxDeg = 30.0f;
}
else if(force3DoF)
{
// GTSAM 3DoF + 1 landmark + pinned: 2D bearing factor benefits
// from the prior but still has one degenerate DoF.
poseDistMax = 4.50f;
poseAngMaxDeg = 22.0f;
}
else
{
// GTSAM 6DoF + 1 landmark: bearing-factor SO(3) linearization
// stuck in local minimum at large initial yaw error.
poseDistMax = 5.70f;
poseAngMaxDeg = 30.0f;
}
}
else if(noisy && cov.linkRangeOnly)
{
const bool isG2O = (backend == Optimizer::kTypeG2O);
landmarkDistMax = isG2O ? 0.15f : 0.05f; // g2o has no bearing-purity advantage
poseDistMax = 0.20f;
poseAngMaxDeg = 1.0f;
}
else if(noisy)
{
landmarkDistMax = 0.30f;
poseDistMax = g2o6DoF ? 0.40f : 0.30f;
poseAngMaxDeg = g2o6DoF ? 3.0f : 2.0f;
}
// Open-loop "loop-closure delta" check: pose 1 (root, fixed at truth)
// and pose kCircleN are the would-be loop-closure pair (link dropped).
// The recovered distance/angle between them should match the truth
// delta (~2.09 m / 12 deg = one chord/segment), within the same chain
// bounds since the worst pose deviation is at the open end.
if(cov.openLoopWithDrift)
{
const float expectedD = g.truePoses.at(1).getDistance(g.truePoses.at(kCircleN));
const float expectedADeg = g.truePoses.at(1).getAngle(g.truePoses.at(kCircleN))
* 180.0f / static_cast<float>(M_PI);
const float actualD = out.at(1).getDistance(out.at(kCircleN));
const float actualADeg = out.at(1).getAngle(out.at(kCircleN))
* 180.0f / static_cast<float>(M_PI);
EXPECT_LT(std::abs(actualD - expectedD), poseDistMax)
<< optimizerTypeName(backend) << " case=" << cov.name
<< " loop-closure distance expected=" << expectedD
<< " actual=" << actualD;
EXPECT_LT(std::abs(actualADeg - expectedADeg), poseAngMaxDeg)
<< optimizerTypeName(backend) << " case=" << cov.name
<< " loop-closure angle expected=" << expectedADeg << " deg"
<< " actual=" << actualADeg << " deg";
}
for(const auto & kv : g.truePoses)
{
const int id = kv.first;
ASSERT_TRUE(out.count(id)) << optimizerTypeName(backend)
<< " case=" << cov.name << " missing id=" << id;
// Landmark converges very tightly in position regardless of whether
// orientation is tracked:
// 6DoF, orient tracked: G2O sub-mm + ~0.001 deg, GTSAM ~numerical zero.
// 6DoF, orient ignored: G2O ~0.14 mm, GTSAM sub-um (no angle to check
// -- the landmark vertex is Point3 and the output
// rpy is just the noisy input verbatim).
// 3DoF: x/y/yaw match truth to ~numerical zero. The z/roll/pitch we
// noised onto the landmark survive verbatim (the optimizer
// treats it as Pose2/Point2 and pastes the input z/roll/pitch
// into the output), so project both sides to3DoF() to drop
// those untouched axes from the comparison.
if(id < 0)
{
const Transform expected = force3DoF ? kv.second.to3DoF() : kv.second;
const Transform got = force3DoF ? out.at(id).to3DoF() : out.at(id);
EXPECT_LT(got.getDistance(expected), landmarkDistMax)
<< optimizerTypeName(backend) << " case=" << cov.name
<< " landmark got=" << got.prettyPrint();
// Angle is only meaningful when orientation is tracked and the
// landmark observations are noise-free; with linkLinNoise>0 the
// noisy translations leak into the recovered orientation chain
// (~0.9 deg on the landmark in our experiments), which is fine
// for the noisy case so we skip the angle assert there.
if(!cov.orientationIgnored && !noisy)
{
const float angDeg = got.getAngle(expected) * 180.0f / static_cast<float>(M_PI);
EXPECT_LT(angDeg, 0.01f)
<< optimizerTypeName(backend) << " case=" << cov.name
<< " landmark angle=" << angDeg << " deg";
}
continue;
}
// Non-landmark ids: noise-free cases converge to truth at well under
// 1 mm / 0.01 deg (worst observed is G2O+tightLin in 6DoF at
// ~0.44 mm / 0.002 deg; GTSAM is at numerical zero across the board).
// Noisy cases stay within poseDistMax/poseAngMaxDeg set above; in 6DoF
// g2o needs a looser envelope than GTSAM because its Cartesian
// EdgeSE3PointXYZ propagates landmark-link translation noise into the
// chain more strongly than GTSAM's BearingRange factorization.
EXPECT_LT(out.at(id).getDistance(kv.second), poseDistMax)
<< optimizerTypeName(backend) << " case=" << cov.name << " id=" << id
<< " got=" << out.at(id).prettyPrint();
const float poseAngDeg = out.at(id).getAngle(kv.second) * 180.0f / static_cast<float>(M_PI);
EXPECT_LT(poseAngDeg, poseAngMaxDeg)
<< optimizerTypeName(backend) << " case=" << cov.name << " id=" << id
<< " angle=" << poseAngDeg << " deg";
}
}
INSTANTIATE_TEST_SUITE_P(
G2OAndGTSAM,
LandmarkCovarianceTest,
::testing::Combine(
::testing::Values(Optimizer::kTypeG2O, Optimizer::kTypeGTSAM),
::testing::Range(0, static_cast<int>(sizeof(kLandmarkCovCases) / sizeof(kLandmarkCovCases[0]))),
::testing::Bool()), // Reg/Force3DoF
[](const ::testing::TestParamInfo<LandmarkCovParam> & info)
{
std::string name = optimizerTypeName(std::get<0>(info.param));
name += "_";
name += kLandmarkCovCases[std::get<1>(info.param)].name;
name += std::get<2>(info.param) ? "_3DoF" : "_6DoF";
return name;
});
// ---------------------------------------------------------------------------
// BundleAdjustmentTest -- cameras on a circle looking at the center, observing
// a small point cloud near the origin. Verifies that BA-capable backends
// (g2o / Ceres / CVSBA) recover both the frame poses AND the 3D point cloud
// from noisy initial guesses + clean 2D observations.
// ---------------------------------------------------------------------------
namespace {
constexpr int kBaNumCameras = 12;
constexpr int kBaNumPoints = 200;
constexpr float kBaCircleRadius = 5.0f;
// 752 x 480 camera @ 100 deg horizontal FOV (typical of fisheye / wide-angle
// imagers on small robots).
// fx = cx / tan(half_HFOV) = 376 / tan(50 deg) ~= 315.5
// vertical FOV = 2 * atan(cy/fy) = 2 * atan(240/315.5) ~= 75 deg
constexpr int kBaImageWidth = 752;
constexpr int kBaImageHeight = 480;
constexpr double kBaFx = 315.5;
constexpr double kBaFy = 315.5;
constexpr double kBaCx = 376.0;
constexpr double kBaCy = 240.0;
// 3D point cloud half-extents. Sized so that a "worst-case" point at the
// corner (Lxy, Lxy, Lz) projects to ~95% of each image dimension from any
// camera position. Derivation for u (image width):
// u_offset_max = Lxy * fx / (R - Lxy)
// Solving for 95% coverage (u_offset = 0.475 * 752 = 357):
// Lxy = 357 * R / (357 + fx) = 357 * 5 / (357 + 315.5) ~= 2.65 m
// Same form for v gives Lz ~= 1.65 m.
constexpr float kBaPointXY = 3.5f;
constexpr float kBaPointZ = 2.4f;
struct BundleGraph
{
std::map<int, Transform> truePoses;
std::map<int, cv::Point3f> truePoints3D;
std::map<int, std::vector<CameraModel>> models;
std::map<int, Transform> initialPoses; // possibly noisy
std::map<int, cv::Point3f> initialPoints3D; // possibly noisy
std::multimap<int, Link> links; // pose-graph constraints
std::map<int, std::map<int, FeatureBA>> wordReferences; // word_id -> camera_id -> FeatureBA
};
// Build the BA scenario:
// * kBaNumCameras cameras on a circle of radius kBaCircleRadius, all looking
// toward the origin (body +X points at origin -> yaw = theta + pi).
// * kBaNumPoints 3D points scattered in a [-1,1]^3 box around the origin.
// * Every camera observes every point (well within ~63 deg FOV).
// * Neighbor links 1->2->...->N between consecutive frames.
// When noisy=true:
// * Every non-anchor pose (id 2..N) gets 5 cm linear / 1 deg angular noise.
// * Every 3D point gets 5 cm noise.
// * Every neighbor link's transform gets the same noise on top of truth.
BundleGraph buildBundleGraph(bool noisy = false, bool roundPixels = false, int numCameras = kBaNumCameras, int numPoints = kBaNumPoints)
{
BundleGraph g;
const CameraModel model(kBaFx, kBaFy, kBaCx, kBaCy,
CameraModel::opticalRotation(),
/*Tx=*/0.0,
cv::Size(kBaImageWidth, kBaImageHeight));
// Cameras on the circle, body +X aimed at the origin.
for(int i = 1; i <= numCameras; ++i)
{
const float theta = 2.0f * static_cast<float>(CV_PI) * static_cast<float>(i - 1) / static_cast<float>(numCameras);
const float x = kBaCircleRadius * std::cos(theta);
const float y = kBaCircleRadius * std::sin(theta);
// body +X = (cos yaw, sin yaw, 0); to point at origin, +X = -position direction.
const float yaw = theta + static_cast<float>(CV_PI);
g.truePoses[i] = Transform(x, y, 0.0f, 0.0f, 0.0f, yaw);
g.models[i] = std::vector<CameraModel>{model};
}
// 3D point cloud: uniform in a box around the origin sized so that every
// camera around the circle sees the points spread across most of its
// image frame. Z extent is smaller than X,Y to match the narrower
// vertical FOV (67 deg vs 100 deg horizontal at 1080p).
{
std::mt19937 rng(7);
std::uniform_real_distribution<float> distXY(-kBaPointXY, kBaPointXY);
std::uniform_real_distribution<float> distZ (-kBaPointZ, kBaPointZ);
for(int p = 1; p <= numPoints; ++p)
{
// Draws are pulled into locals first: argument evaluation order
// is unspecified in C++, so passing three dist(rng) calls
// directly makes the generated scene compiler-dependent (GCC
// evaluates right-to-left, Clang left-to-right).
const float px = distXY(rng);
const float py = distXY(rng);
const float pz = distZ(rng);
g.truePoints3D[p] = cv::Point3f(px, py, pz);
}
}
// Project every point into every camera. Skip projections that fall behind
// the camera or outside the image (defensive -- with our geometry every
// point should be visible in every camera).
for(const auto & ptkv : g.truePoints3D)
{
const int pointId = ptkv.first;
for(const auto & posekv : g.truePoses)
{
const int camId = posekv.first;
const Transform world_to_optical = (posekv.second * model.localTransform()).inverse();
const cv::Point3f pc = util3d::transformPoint(ptkv.second, world_to_optical);
if(pc.z <= 0.1f)
{
continue;
}
float u, v;
model.reproject(pc.x, pc.y, pc.z, u, v);
if(u < 0.0f || u >= kBaImageWidth || v < 0.0f || v >= kBaImageHeight)
{
continue;
}
if(roundPixels)
{
// Simulate the discrete-pixel detection that real feature
// extractors produce -- adds +-0.5 px quantization noise to
// every reprojection residual.
u = std::round(u);
v = std::round(v);
}
g.wordReferences[pointId].insert({camId, FeatureBA(cv::KeyPoint(u, v, 1.0f))});
}
}
// Neighbor chain 1 -> 2 -> ... -> N.
cv::Mat info = cv::Mat::eye(6, 6, CV_64FC1);
for(int i = 0; i < 3; ++i) info.at<double>(i, i) = 1.0 / (0.05 * 0.05);
for(int i = 3; i < 6; ++i) info.at<double>(i, i) = 1.0 / (1.0 * CV_PI / 180.0 * 1.0 * CV_PI / 180.0);
for(int i = 1; i < numCameras; ++i)
{
const Transform t = g.truePoses[i].inverse() * g.truePoses[i + 1];
g.links.insert({i, Link(i, i + 1, Link::kNeighbor, t, info)});
}
// Initial guess: truth by default, noisy on request.
g.initialPoses = g.truePoses;
g.initialPoints3D = g.truePoints3D;
if(noisy)
{
std::mt19937 rng(11);
std::normal_distribution<double> linNoise(0.0, 0.05); // 5 cm on poses
std::normal_distribution<double> angNoise(0.0, 1.0 * CV_PI / 180.0); // 1 deg
std::normal_distribution<double> ptNoise(0.0, 0.05); // 5 cm on point positions
std::normal_distribution<double> linkLinNoise(0.0, 0.10); // 10 cm on neighbor link translations
// Perturb every pose except the root (id 1 = anchor for optimizeBA).
for(int i = 2; i <= numCameras; ++i)
{
const Transform & t = g.truePoses[i];
float r, p, y;
t.getEulerAngles(r, p, y);
const float dx = static_cast<float>(linNoise(rng));
const float dy = static_cast<float>(linNoise(rng));
const float dz = static_cast<float>(linNoise(rng));
const float dr = static_cast<float>(angNoise(rng));
const float dp = static_cast<float>(angNoise(rng));
const float dyaw = static_cast<float>(angNoise(rng));
g.initialPoses[i] = Transform(
t.x() + dx, t.y() + dy, t.z() + dz,
r + dr, p + dp, y + dyaw);
}
for(auto & kv : g.initialPoints3D)
{
kv.second.x += static_cast<float>(ptNoise(rng));
kv.second.y += static_cast<float>(ptNoise(rng));
kv.second.z += static_cast<float>(ptNoise(rng));
}
std::multimap<int, Link> noisyLinks;
for(const auto & kv : g.links)
{
const Link & link = kv.second;
Transform t = link.transform();
float r, p, y;
t.getEulerAngles(r, p, y);
const float dx = static_cast<float>(linkLinNoise(rng));
const float dy = static_cast<float>(linkLinNoise(rng));
const float dz = static_cast<float>(linkLinNoise(rng));
const float dr = static_cast<float>(angNoise(rng));
const float dp = static_cast<float>(angNoise(rng));
const float dyaw = static_cast<float>(angNoise(rng));
t = Transform(
t.x() + dx, t.y() + dy, t.z() + dz,
r + dr, p + dp, y + dyaw);
noisyLinks.insert({kv.first, Link(link.from(), link.to(), link.type(), t, link.infMatrix())});
}
g.links = noisyLinks;
}
return g;
}
// Per-test variant. Default exercises the full setup (poses + points +
// noisy neighbor links, mono reprojection). The other two are g2o-specific
// extensions:
// * G2ONoLinks: drop the kNeighbor edges before calling optimizeBA, so
// g2o BA is reduced to reprojection-only -- mimics how Ceres/CVSBA
// handle BA, and isolates the noisy-link contribution.
// * G2OWithDepth: provide a valid depth in every FeatureBA (range from
// camera to 3D point + Gaussian noise) and a Tx on the CameraModel
// so g2o creates EdgeStereoSE3ProjectXYZ "stereo" edges (u, v,
// u-disparity) instead of mono EdgeSE3ProjectXYZ. Tests g2o's RGB-D /
// stereo BA path.
enum class BaVariant {
kDefault,
kNoLinks,
kWithDepth,
kWithDepthNoLinks,
kWithDepthNoLinksTuned, // WithDepth + per-axis info calibrated to actual noise
kWithLidarDepthNoLinksTuned, // Accurate depth source (LiDAR-fused, 1 cm sigma) + tight DisparityVariance
};
// (backend, variant, roundPixels). roundPixels=true simulates discrete
// pixel-coordinate detection (every keypoint rounded to nearest integer)
// instead of the continuous reprojection -- adds +-0.5 px quantization
// noise to every observation.
using BaParam = std::tuple<Optimizer::Type, BaVariant, bool>;
const char * baVariantName(BaVariant v)
{
switch(v)
{
case BaVariant::kDefault: return "Default";
case BaVariant::kNoLinks: return "NoLinks";
case BaVariant::kWithDepth: return "WithDepth";
case BaVariant::kWithDepthNoLinks: return "WithDepthNoLinks";
case BaVariant::kWithDepthNoLinksTuned: return "WithDepthNoLinksTuned";
case BaVariant::kWithLidarDepthNoLinksTuned: return "WithLidarDepthNoLinksTuned";
}
return "?";
}
class BundleAdjustmentTest : public ::testing::TestWithParam<BaParam>
{
protected:
void SetUp() override
{
const Optimizer::Type t = std::get<0>(GetParam());
if(!Optimizer::isAvailable(t))
{
GTEST_SKIP() << optimizerTypeName(t) << " not built in";
}
}
};
} // namespace
TEST_P(BundleAdjustmentTest, CircleCamerasRecoverPosesAndPoints)
{
const Optimizer::Type backend = std::get<0>(GetParam());
const BaVariant variant = std::get<1>(GetParam());
const bool roundPixels = std::get<2>(GetParam());
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kOptimizerIterations()] = "200"; // probe
// For the stereo BA variants we set Tx on the CameraModel below, which
// g2o picks up directly. Set g2o/Baseline to the same 0.15 m for
// consistency (used as a fallback when Tx isn't set on the model).
params[Parameters::kOptimizerBaseline()] = "0.15";
if(variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
// Accurate-depth scenario (LiDAR-fused / structured-light): the
// depth measurement is *more* precise than a typical feature
// detector's u/v, so the optimizer should trust depth more.
// PixelVariance=1 (sigma_uv ~ 1 px, typical detector),
// DisparityVariance=0.1 (sigma_disp ~ 0.3 px, reflecting the
// fused depth's tighter precision). 1:10 ratio puts depth ~10x
// tighter than each pixel. Tighter still works empirically
// (e.g. 0.01 gives same residuals) but g2o's Hessian becomes
// ill-conditioned around info=10000, so we stay comfortably
// within the stable range.
params[Parameters::kOptimizerPixelVariance()] = "1.0";
params[Parameters::kOptimizerDisparityVariance()] = "0.1";
}
else if(variant == BaVariant::kWithDepthNoLinksTuned)
{
// Sub-pixel feature detector + standard stereo block matcher tuning:
// PixelVariance=0.1 (sigma_uv ~ 0.3 px), DisparityVariance=1
// (sigma_disp ~ 1 px). The 10:1 ratio gives u/v slightly more weight
// than disparity, which breaks the info-matrix asymmetry trap seen
// in WithDepth (default pv=1, dispVar=1 -> 12.8 cm worst point error
// because the optimizer over-trusts noisy disparity and pushes points
// along the depth axis to fit it). With this tuning the points
// converge to ~4 mm.
params[Parameters::kOptimizerPixelVariance()] = "0.1";
params[Parameters::kOptimizerDisparityVariance()] = "1.0";
}
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
BundleGraph g = buildBundleGraph(/*noisy=*/true, /*roundPixels=*/roundPixels);
if(variant == BaVariant::kNoLinks
|| variant == BaVariant::kWithDepthNoLinks
|| variant == BaVariant::kWithDepthNoLinksTuned
|| variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
// Drop the noisy neighbor links so g2o BA is reduced to a pure
// reprojection problem (like Ceres/CVSBA).
g.links.clear();
}
if(variant == BaVariant::kWithDepth
|| variant == BaVariant::kWithDepthNoLinks
|| variant == BaVariant::kWithDepthNoLinksTuned
|| variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
// Switch to a stereo camera model: baseline = 0.15 m (a common
// medium-baseline value e.g. ZED Mini / RealSense D435i). Tx =
// -baseline*fx so g2o detects it and uses the
// EdgeStereoSE3ProjectXYZ path (3-axis observation: u, v,
// u-disparity).
const double baselineMeters = 0.15;
const double Tx = -baselineMeters * kBaFx;
const CameraModel stereoModel(kBaFx, kBaFy, kBaCx, kBaCy,
CameraModel::opticalRotation(), Tx,
cv::Size(kBaImageWidth, kBaImageHeight));
for(auto & kv : g.models) kv.second = std::vector<CameraModel>{stereoModel};
// Realistic stereo noise: 1 px sigma on DISPARITY, constant across
// range (this is what a block matcher / SGM produces). Depth is
// then derived from the noisy disparity --
// sigma_depth = depth^2 / (baseline*fx) * sigma_disp
// -- so depth precision degrades quadratically with range. With our
// 0.15 m baseline and fx=315.5: sigma_depth ranges from ~13 cm at
// 2.5 m to ~1.2 m at 7.5 m. The optimizer's stereo edge measures
// the disparity directly via (u, v, u-disparity), so the
// quadratic range degradation is handled automatically by the
// projection geometry; the calibrated info matrix matches the
// constant ~1 px disparity noise (g2o's default PixelVariance=1
// is well-suited).
// In the LiDAR-fused variant we replace the stereo noise model with
// a small constant sigma on depth (range-independent, as a depth
// sensor / fused range source would produce). DisparityVariance is
// set tight in the params block to reflect this higher precision.
const bool lidarDepth = (variant == BaVariant::kWithLidarDepthNoLinksTuned);
std::mt19937 dispRng(13);
std::normal_distribution<double> dispNoise (0.0, 1.0); // 1 px on disparity (stereo / RGB-D)
std::normal_distribution<double> depthLidarNoise(0.0, 0.01); // 1 cm on depth (LiDAR / structured-light)
for(auto & wkv : g.wordReferences)
{
const cv::Point3f & worldPt = g.truePoints3D.at(wkv.first);
for(auto & ckv : wkv.second)
{
const Transform world_to_optical =
(g.truePoses.at(ckv.first) * stereoModel.localTransform()).inverse();
const cv::Point3f pc = util3d::transformPoint(worldPt, world_to_optical);
float depthNoisy;
if(lidarDepth)
{
depthNoisy = pc.z + static_cast<float>(depthLidarNoise(dispRng));
}
else
{
const double dispTrue = baselineMeters * kBaFx / pc.z;
const double dispNoisy = dispTrue + dispNoise(dispRng);
depthNoisy = static_cast<float>(
baselineMeters * kBaFx / std::max(dispNoisy, 0.1));
}
ckv.second = FeatureBA(ckv.second.kpt, depthNoisy);
}
}
}
// Sanity: every point should be observed from at least 6 cameras. With
// the spread point cloud (kBaPointXY=3.5, kBaPointZ=2.4) giving ~95%
// image coverage from each frame, points near the box corners get
// dropped by the most-oblique-angle cameras (depth too shallow or
// projection outside FOV). 6 observations is still plenty for BA.
for(const auto & ptkv : g.truePoints3D)
{
ASSERT_GE(g.wordReferences.at(ptkv.first).size(), 6u)
<< "point " << ptkv.first << " only observed from "
<< g.wordReferences.at(ptkv.first).size() << " cameras";
}
std::map<int, cv::Point3f> outPoints = g.initialPoints3D;
UTimer baTimer;
std::map<int, Transform> outPoses = opt->optimizeBA(
/*rootId=*/1, g.initialPoses, g.links, g.models, outPoints, g.wordReferences);
const double baSeconds = baTimer.getElapsedTime();
std::cerr << "[BA-timing] backend=" << optimizerTypeName(backend)
<< " variant=" << baVariantName(variant)
<< " roundPixels=" << (roundPixels?1:0)
<< " seconds=" << baSeconds << "\n";
ASSERT_FALSE(outPoses.empty()) << "optimizeBA returned no poses";
ASSERT_EQ(outPoses.size(), g.truePoses.size());
// BA recovers the trajectory and point-cloud SHAPE, but the gauge
// (global rigid transform) depends on the backend: g2o fixes the root
// pose via setFixed(rootId); Ceres and CVSBA don't fix any pose. To
// compare gauge-independently, express every pose and point in the
// optimizer's root frame and compare to truth in truth's root frame --
// the rigid transform between the two solutions cancels out.
const Transform truthRootInv = g.truePoses.at(1).inverse();
const Transform outRootInv = outPoses.at(1).inverse();
// Per-backend bounds. Setup: 752x480 / 100 deg HFOV / 200 features
// uniformly in [-3.5,3.5]x[-3.5,3.5]x[-2.4,2.4] box / 12 cameras on a
// 5 m circle / 5 cm pose+point noise / 10 cm noisy-link translation
// noise / 5 cm depth noise (for WithDepth variants) / 200 iterations.
// Chain rotation recovers to ~0 deg across all backends; position
// residual depends on how each backend handles the noisy chain:
// * Ceres / CVSBA: BA only uses reprojection; chain ignored. ~1.5 cm.
// * g2o + chain (Default / WithDepth): BA includes the kNeighbor
// edges with their full info matrix. The 10 cm link noise pulls
// poses 3-6 cm off truth.
// * g2o without chain (NoLinks / WithDepthNoLinks): tightest case.
// Bounds cover both continuous and rounded-pixel observations. Rounding
// adds +-0.5 px quantization noise per observation; with 200 obs per
// pose the pose residual is barely affected, but individual point
// residuals can grow ~80% (one point still depends on its few specific
// observations).
float poseDistMax = 0.03f;
float poseAngMaxDeg = 1.0f;
float pointDistMax = 0.025f;
if(backend == Optimizer::kTypeG2O)
{
if(variant == BaVariant::kNoLinks)
{
poseDistMax = 0.015f;
pointDistMax = 0.015f;
}
else if(variant == BaVariant::kWithDepth)
{
// With realistic stereo noise (1 px on disparity), point recovery
// at long range is dominated by the depth uncertainty. The
// continuous-pixel variant ALSO suffers from the info-matrix
// asymmetry: u/v residual = 0 at truth so the optimizer
// over-trusts them vs the 1 px disparity, pushing points along
// the depth axis to better fit the noisy disparity. Rounded u/v
// adds ~0.5 px noise that *can* balance the info matrix and
// recover ~8x tighter points (observed on Linux/OpenBLAS), but
// the pattern doesn't reproduce on macOS/Accelerate, where
// recovery sits around the continuous-pixel spread regardless
// of rounding -- so both the pose bound and the rounded point
// bound are widened to cover that. The continuous-pixel point
// bound was already loose (0.15) for the info-matrix reason.
poseDistMax = 0.06f;
pointDistMax = roundPixels ? 0.06f : 0.15f;
}
else if(variant == BaVariant::kWithDepthNoLinks)
{
poseDistMax = 0.015f;
pointDistMax = roundPixels ? 0.035f : 0.15f;
}
else if(variant == BaVariant::kWithDepthNoLinksTuned)
{
// Calibrated per-axis info matrix + no noisy chain. Tightest
// case of all the WithDepth variants: both pose AND point
// converge to ~5 mm.
poseDistMax = 0.015f;
pointDistMax = 0.015f;
}
else if(variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
// Accurate depth (1 cm sigma) + tight DisparityVariance: the
// tightest of all WithDepth variants on both pose and point.
poseDistMax = 0.005f;
pointDistMax = 0.005f;
}
else
{
// Default: chain pulled by noisy links.
poseDistMax = 0.10f;
pointDistMax = 0.09f;
}
}
else if(backend == Optimizer::kTypeGTSAM)
{
// GTSAM soft-fixes the root via a tight PriorFactor (no native
// "setFixed" like g2o), so the gauge is slightly looser; the LM
// solver also stops at a relativeErrorTol that leaves a tiny bit of
// residual on the longest-range points. Both effects are sub-cm on
// the pose, ~few-cm on point cloud points that are 7-8 m from the
// root. Stereo variants converge to the same bounds as g2o because
// the depth constraint pins the gauge.
if(variant == BaVariant::kWithDepthNoLinksTuned)
{
poseDistMax = 0.015f;
pointDistMax = 0.015f;
}
else if(variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
poseDistMax = 0.005f;
pointDistMax = 0.005f;
}
else if(variant == BaVariant::kWithDepth || variant == BaVariant::kWithDepthNoLinks)
{
// These two variants keep the DEFAULT noise weighting
// (PixelVariance=1, DisparityVariance=1), which the Tuned variants
// above exist to fix: weighting u/v and disparity equally makes the
// optimizer over-trust noisy disparity and push points along the
// depth axis, the "info-matrix asymmetry trap" described at the
// kWithDepthNoLinksTuned params. That is why these are the variants
// with 12.8 cm point error on g2o while the tuned ones reach 4 mm.
// The point bound is wide because of that depth-axis distortion,
// which is the behaviour these variants exist to pin down.
//
// These bounds now measure SHAPE only -- the global scale is solved
// and divided out below, and checked separately there. That is what
// makes them portable. Both GTSAM and Ceres used to fail these on
// macOS with a uniform 0.3-0.5% scale shrink and an otherwise
// near-perfect shape, which an absolute position bound turns into
// 5 cm of "error" on the poses 10 m from the root and nothing on the
// near ones. Once scale is factored out, the linux pose residual on
// this variant drops from 0.0058 to 0.0030 and the macOS ratios were
// uniform to 4 decimals, so both platforms land in the same place.
//
// The scale error itself was not a convergence failure: on linux the
// result is bit-identical across Optimizer/Epsilon from 1e-5 down to
// 0, Optimizer/Iterations from 200 to 2000, and PCG vs direct
// multifrontal Cholesky. It is the converged optimum of an
// ill-conditioned problem, and two different toolchains land on
// slightly different optima. Nothing to tighten, which is why it is
// measured rather than chased.
poseDistMax = 0.04f;
pointDistMax = roundPixels ? 0.04f : 0.15f;
}
else if(variant == BaVariant::kNoLinks)
{
// Pure mono reprojection. Mono BA has a 7-DOF gauge -- the
// recovered geometry is only correct up to scale -- but we
// solve for that scale against truth below before checking
// bounds, so the bounds match the clean-converged case.
poseDistMax = 0.015f;
pointDistMax = 0.02f;
}
else
{
// kDefault: mono BA + noisy chain. Scale handled below; the
// chain noise still pulls poses ~few cm.
poseDistMax = 0.04f;
pointDistMax = 0.04f;
}
}
else if(backend == Optimizer::kTypeCeres)
{
if(roundPixels &&
(variant == BaVariant::kWithDepth || variant == BaVariant::kWithDepthNoLinks))
{
// Ceres uses the generic 0.025 default everywhere else; these two
// variants get a little more room for the +-0.5 px u/v quantization
// stacked on the 1 px disparity noise. Both variants share the noise
// model, so they share the bound.
//
// This bound used to be doing double duty, absorbing a macOS scale
// bias as well: the recovered points came out a uniform 0.29% short
// (ratio 0.9971 on every failing point -- one global factor, not
// per-point noise), which is 2.6 cm at 9 m from the root against
// 0.0018 on linux. That is now handled where it belongs, by the
// scale solve below, so this only has to cover quantization again.
pointDistMax = 0.04f;
}
}
else if(backend == Optimizer::kTypeCVSBA)
{
poseDistMax = 0.06f;
pointDistMax = 0.04f;
}
if(roundPixels)
{
// +-0.5 px quantization noise hits the point residuals harder than
// the pose residuals (each point depends on its few observations,
// each pose averages over ~200). Loosen the point bound a bit.
// Note this only lifts bounds that are still below the floor: the
// branches above already set rounded-specific bounds where needed.
pointDistMax = std::max(pointDistMax, 0.02f);
}
// Gauge handling. Every variant is compared to truth after dividing out one
// global scale factor, solved in closed form from the camera positions in
// the root-relative frame:
// s = Σ(d_out · d_truth) / Σ(d_out · d_out)
// then applied to BOTH poses and points. This is the test-side counterpart
// to the brute-force scale scan in tools/Report/main.cpp; here we know the
// relationship is quadratic so the closed form is exact.
//
// For mono BA this is the only way to compare at all: fixing one pose still
// leaves a 1-DOF scale ambiguity the optimizer is free to land anywhere on.
// That covers the pure-mono variants, and also every variant on CVSBA --
// rtabmap's CVSBA path uses cvsba's Sba::run(), which takes 2D image points
// only, so it runs mono BA even when handed stereo-style observations.
//
// The depth variants are a different case: g2o, GTSAM and Ceres switch to a
// stereo cost function when depth+baseline are available, so scale IS
// observable there and a scale error is a real error rather than a gauge
// choice. They get the same treatment anyway, because it separates two
// failure modes that a raw position comparison conflates. A uniform scale
// bias and a distorted shape are very different defects, but an absolute
// position bound charges for both at once -- and since the error from a scale
// bias grows with distance from the root, it charges the far poses several
// times more than the near ones for the identical relative mistake. Dividing
// scale out means the position bounds below measure shape only, and scale
// gets its own flat check where a 0.5% bias reads as 0.5% wherever the pose
// sits.
//
// Splitting them is what makes these bounds portable. On macOS both GTSAM and
// Ceres recover this graph with a 0.3-0.5% scale shrink and an otherwise
// near-perfect shape -- the per-pose and per-point ratios agree to 4-5
// decimals -- which used to trip the absolute bounds only on the poses and
// points 8-10 m out, and only on the ill-conditioned default-weighting
// variants. Nothing about the shape was wrong.
const bool monoVariant = (variant == BaVariant::kDefault || variant == BaVariant::kNoLinks);
const bool monoOnlyBackend = (backend == Optimizer::kTypeCVSBA);
const bool monoBA = monoVariant || monoOnlyBackend;
float scale = 1.0f;
{
double num = 0.0;
double den = 0.0;
for(const auto & kv : g.truePoses)
{
if(kv.first == 1) continue;
if(!outPoses.count(kv.first)) continue;
const Transform truthRel = truthRootInv * kv.second;
const Transform outRel = outRootInv * outPoses.at(kv.first);
num += outRel.x()*truthRel.x() + outRel.y()*truthRel.y() + outRel.z()*truthRel.z();
den += outRel.x()*outRel.x() + outRel.y()*outRel.y() + outRel.z()*outRel.z();
}
if(den > 1e-12)
{
scale = static_cast<float>(num / den);
}
}
// Now that scale is factored out of the position checks, assert on it
// directly wherever it is observable -- otherwise dividing it out would
// silently discard a real failure mode (a broken baseline, a wrong Tx sign,
// or a bad disparity-to-depth conversion all show up as a scale error and
// nothing else). Mono BA has no scale to be wrong about, so it is exempt.
//
// 2% is loose next to the 0.5% worst case observed across platforms, but it
// is a flat bound on a quantity that should be exactly 1, and the failures
// it guards against are order-of-percent-to-2x, not fractions of a percent.
if(!monoBA)
{
EXPECT_NEAR(scale, 1.0f, 0.02f)
<< optimizerTypeName(backend) << " variant=" << (int)variant
<< " rounded=" << roundPixels
<< ": stereo/depth BA should pin scale, but the recovered geometry"
" is off by a global factor of " << scale;
}
// Recovered poses (relative to root). maxPoseDist and the radius of the
// pose that produced it are reported together below: these bounds are
// absolute metres applied to poses up to 10 m from the root, so the same
// bound is a much tighter relative constraint on the far poses than on the
// near ones, and the ratio is what tells a scale bias apart from noise.
float maxPoseDist = 0.0f;
float maxPoseDistRadius = 0.0f;
for(const auto & kv : g.truePoses)
{
const int id = kv.first;
if(id == 1) continue; // root is its own reference (delta = identity in both frames)
ASSERT_TRUE(outPoses.count(id));
const Transform truthRel = truthRootInv * kv.second;
Transform outRel = outRootInv * outPoses.at(id);
outRel.x() *= scale;
outRel.y() *= scale;
outRel.z() *= scale;
if(outRel.getDistance(truthRel) > maxPoseDist)
{
maxPoseDist = outRel.getDistance(truthRel);
maxPoseDistRadius = truthRel.getNorm();
}
EXPECT_LT(outRel.getDistance(truthRel), poseDistMax)
<< optimizerTypeName(backend) << " pose " << id
<< " got(rel)=" << outRel.prettyPrint()
<< " truth(rel)=" << truthRel.prettyPrint();
const float angDeg = outRel.getAngle(truthRel) * 180.0f / static_cast<float>(CV_PI);
EXPECT_LT(angDeg, poseAngMaxDeg)
<< optimizerTypeName(backend) << " pose " << id << " angle=" << angDeg << " deg";
}
// Recovered points (relative to root).
float maxPointDist = 0.0f;
for(const auto & kv : g.truePoints3D)
{
const int id = kv.first;
ASSERT_TRUE(outPoints.count(id));
const cv::Point3f truthRel = util3d::transformPoint(kv.second, truthRootInv);
cv::Point3f outRel = util3d::transformPoint(outPoints.at(id), outRootInv);
outRel.x *= scale;
outRel.y *= scale;
outRel.z *= scale;
const cv::Point3f diff = outRel - truthRel;
const float d = std::sqrt(diff.x*diff.x + diff.y*diff.y + diff.z*diff.z);
maxPointDist = std::max(maxPointDist, d);
EXPECT_LT(d, pointDistMax)
<< optimizerTypeName(backend) << " point " << id
<< " got(rel)=(" << outRel.x << "," << outRel.y << "," << outRel.z << ")"
<< " truth(rel)=(" << truthRel.x << "," << truthRel.y << "," << truthRel.z << ")";
}
std::cerr << "[bound] " << optimizerTypeName(backend) << " variant=" << (int)variant
<< " rounded=" << roundPixels << " maxPointDist=" << maxPointDist
<< " bound=" << pointDistMax
<< " | maxPoseDist=" << maxPoseDist << " bound=" << poseDistMax
<< " (worst pose is " << maxPoseDistRadius << " m from root, so "
<< (maxPoseDistRadius > 0.0f ? 100.0f*maxPoseDist/maxPoseDistRadius : 0.0f)
<< "% of its range)"
<< " | scale=" << scale << (monoBA ? " (gauge, unchecked)" : "")
<< "\n";
}
INSTANTIATE_TEST_SUITE_P(
BABackends,
BundleAdjustmentTest,
::testing::Values(
// Continuous-pixel observations (the test's original setup).
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kDefault, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kNoLinks, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepth, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepthNoLinks, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithLidarDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kDefault, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kNoLinks, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepth, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepthNoLinks, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithLidarDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kDefault, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kNoLinks, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepth, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepthNoLinks, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithLidarDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeCVSBA, BaVariant::kDefault, false),
// Same setups but with keypoints rounded to integer pixel
// coordinates (simulates a real detector).
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kDefault, true),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kNoLinks, true),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepth, true),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepthNoLinks, true),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kDefault, true),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kNoLinks, true),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepth, true),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepthNoLinks, true),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kDefault, true),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kNoLinks, true),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepth, true),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepthNoLinks, true),
std::make_tuple(Optimizer::kTypeCVSBA, BaVariant::kDefault, true)),
[](const ::testing::TestParamInfo<BaParam> & info)
{
std::string name = optimizerTypeName(std::get<0>(info.param));
name += "_";
name += baVariantName(std::get<1>(info.param));
if(std::get<2>(info.param)) name += "_Rounded";
return name;
});
// ---------------------------------------------------------------------------
// Regression test: wordReferences with negative ids. Memory.cpp assigns
// sequential negative ids (-1, -2, -3, ...) to features that aren't
// quantized into the visual vocabulary, and OdometryF2M forwards them
// straight into optimizeBA(). gtsam::Symbol packs the index into 56
// unsigned bits and used to overflow on these ("Symbol index is too
// large"); g2o remaps them via negVertexOffset; Ceres handles them too.
// Build the standard BA scenario, then negate every point id so the
// optimizer sees what F2M's wordReferences actually looks like in
// production.
// ---------------------------------------------------------------------------
class NegativeWordIdBaTest : public ::testing::TestWithParam<Optimizer::Type>
{
protected:
void SetUp() override
{
const Optimizer::Type t = GetParam();
if(!Optimizer::isAvailable(t))
{
GTEST_SKIP() << optimizerTypeName(t) << " not built in";
}
}
};
TEST_P(NegativeWordIdBaTest, OptimizeBaHandlesNegativeIds)
{
const Optimizer::Type backend = GetParam();
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kOptimizerIterations()] = "200";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
BundleGraph g = buildBundleGraph(/*noisy=*/true, /*roundPixels=*/false);
// Remap every point id p -> -p in truePoints3D, initialPoints3D, and
// wordReferences. Pose ids stay positive — the bug only triggers on
// the landmark/feature side of the BA graph.
std::map<int, cv::Point3f> truePoints3D;
std::map<int, cv::Point3f> initialPoints3D;
std::map<int, std::map<int, FeatureBA>> wordReferences;
for(const auto & kv : g.truePoints3D) truePoints3D[-kv.first] = kv.second;
for(const auto & kv : g.initialPoints3D) initialPoints3D[-kv.first] = kv.second;
for(const auto & kv : g.wordReferences) wordReferences[-kv.first] = kv.second;
std::map<int, cv::Point3f> outPoints = initialPoints3D;
std::map<int, Transform> outPoses = opt->optimizeBA(
/*rootId=*/1, g.initialPoses, g.links, g.models, outPoints,
wordReferences);
ASSERT_FALSE(outPoses.empty())
<< optimizerTypeName(backend) << " optimizeBA returned no poses "
<< "with negative word ids";
ASSERT_EQ(outPoses.size(), g.truePoses.size());
// Sanity: optimized points should still match the (negated) ids in
// outPoints — i.e. the optimizer round-tripped the ids without
// dropping or aliasing them.
for(const auto & kv : truePoints3D)
{
EXPECT_TRUE(outPoints.count(kv.first))
<< optimizerTypeName(backend)
<< " dropped point id " << kv.first;
}
}
INSTANTIATE_TEST_SUITE_P(
BABackends,
NegativeWordIdBaTest,
::testing::Values(
Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM,
Optimizer::kTypeCeres),
[](const ::testing::TestParamInfo<Optimizer::Type> & info)
{
return optimizerTypeName(info.param);
});
// ---------------------------------------------------------------------------
// PlanarBundleAdjustmentTest -- verifies that isSlam2d() in BA locks the
// recovered trajectory to its initial Z plane. g2o has supported this since
// forever via EdgeSBACamPrior; GTSAM and Ceres got matching planar
// constraints in this rev.
// ---------------------------------------------------------------------------
class PlanarBundleAdjustmentTest : public ::testing::TestWithParam<Optimizer::Type>
{
protected:
void SetUp() override
{
if(!Optimizer::isAvailable(GetParam()))
{
GTEST_SKIP() << optimizerTypeName(GetParam()) << " not built in";
}
}
};
TEST_P(PlanarBundleAdjustmentTest, RecoveredTrajectoryStaysOnInitialPlane)
{
const Optimizer::Type backend = GetParam();
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kOptimizerIterations()] = "200";
params[Parameters::kRegForce3DoF()] = "true"; // → isSlam2d() == true
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_TRUE(opt->isSlam2d());
BundleGraph g = buildBundleGraph(/*noisy=*/true, /*roundPixels=*/false);
// Planar test: initial poses reflect a real 2D-SLAM gauge -- the robot
// lives on a single horizontal plane at some non-zero height (z=0.5 m
// here, e.g. a camera mounted on a robot half a meter off the floor)
// with no roll/pitch. Strip the z/roll/pitch noise from the noisy
// initial poses and keep only the x/y/yaw noise. Using a non-zero
// reference z verifies the constraint locks to the INITIAL plane,
// not to z=0 by accident.
const float planeZ = 0.5f;
for(auto & kv : g.initialPoses)
{
float x, y, z, roll, pitch, yaw;
kv.second.getTranslationAndEulerAngles(x, y, z, roll, pitch, yaw);
kv.second = Transform(x, y, planeZ, 0.0f, 0.0f, yaw);
}
std::map<int, cv::Point3f> outPoints = g.initialPoints3D;
std::map<int, Transform> outPoses = opt->optimizeBA(
/*rootId=*/1, g.initialPoses, g.links, g.models, outPoints, g.wordReferences);
ASSERT_FALSE(outPoses.empty()) << "optimizeBA returned no poses";
ASSERT_EQ(outPoses.size(), g.truePoses.size());
// With the planar constraint locking every camera to its initial
// z = planeZ (weight 1e9 = sub-mm tolerance), no recovered pose
// should drift off the plane. Without the constraint, a 3D BA on
// this dataset can drift up to a few mm in z due to reprojection /
// chain residuals.
for(const auto & kv : outPoses)
{
EXPECT_NEAR(kv.second.z(), planeZ, 0.0005f) // 0.5 mm
<< optimizerTypeName(backend) << " pose " << kv.first
<< " drifted off the z=" << planeZ << " plane: z=" << kv.second.z();
}
}
INSTANTIATE_TEST_SUITE_P(
Backends,
PlanarBundleAdjustmentTest,
::testing::Values(Optimizer::kTypeG2O, Optimizer::kTypeGTSAM, Optimizer::kTypeCeres),
[](const ::testing::TestParamInfo<Optimizer::Type> & info)
{
return optimizerTypeName(info.param);
});
// ---------------------------------------------------------------------------
// MultiCamBundleAdjustmentTest -- 4 cameras at body origin pointing
// forward/left/backward/right, 30 poses on the same hilly 10 m circle as
// CircleGraphTest, random 3D points in a 20 m x 20 m x 1.5 m box. Each
// camera only "sees" points within 10 m of the body (avoids projecting
// across-trajectory points). Verifies that g2o and GTSAM multi-camera BA
// recovers the chain + point cloud. Ceres / CVSBA explicitly reject
// multi-cam (`models.size() > 1` in their optimizeBA) so they're not
// instantiated here.
// ---------------------------------------------------------------------------
namespace {
constexpr int kMultiCamNumPoses = 30;
constexpr float kMultiCamCircleRadius = 10.0f;
constexpr float kMultiCamHillAmp = 0.5f;
constexpr float kMultiCamHillFreq = 3.0f;
constexpr int kMultiCamNumPoints = 750;
constexpr float kMultiCamPointBoxXY = 20.0f;
constexpr float kMultiCamPointBoxZ = 1.5f;
constexpr float kMultiCamVisibilityRange = 10.0f;
constexpr int kMultiCamNumCameras = 4;
// Realistic robot body: vacuum-sized round chassis. Each side-facing
// camera sits on the rim at ±17.5 cm from the body center, on a small
// stalk 5 cm above the chassis plane. Opposite cameras are 35 cm apart
// (a real translational baseline that adds triangulation strength
// independent of the body's motion along the chain).
constexpr float kMultiCamRobotRadius = 0.175f;
constexpr float kMultiCamCameraZ = 0.05f;
// Narrow-FOV intrinsics: fx = 376 / tan(30°) ≈ 651 gives ~60° HFOV. With
// 4 cameras 90° apart this leaves ~15° gaps each side between neighbours
// -- no FOV overlap, so points are only ever observed by one camera per
// frame. Scale ambiguity returns (mono BA behavior) and the test has to
// fall back on scale-recovery + looser bounds. Used by the kNarrowFov
// variant below.
constexpr double kMultiCamNarrowFx = 651.0;
constexpr double kMultiCamNarrowFy = 651.0;
enum class MultiCamFov { kWide, kNarrow };
enum class MultiCamMode { kMono, kStereo };
enum class MultiCamLinks { kWithLinks, kNoLinks };
// Stereo baseline used when MultiCamMode::kStereo is set: 15 cm (e.g.
// ZED Mini / RealSense D435i tier). Each of the 4 cameras becomes its
// own stereo pair, so every observation carries a noisy disparity-derived
// depth in its FeatureBA (1 px sigma on the disparity channel).
constexpr double kMultiCamStereoBaseline = 0.15;
const char * multiCamFovName(MultiCamFov f)
{
return f == MultiCamFov::kWide ? "WideFov" : "NarrowFov";
}
const char * multiCamModeName(MultiCamMode m)
{
return m == MultiCamMode::kMono ? "Mono" : "Stereo";
}
const char * multiCamLinksName(MultiCamLinks l)
{
return l == MultiCamLinks::kWithLinks ? "WithLinks" : "NoLinks";
}
struct MultiCamBundleGraph
{
std::map<int, Transform> truePoses;
std::map<int, Transform> initialPoses;
std::map<int, cv::Point3f> truePoints3D;
std::map<int, cv::Point3f> initialPoints3D;
std::map<int, std::vector<CameraModel>> models;
std::multimap<int, Link> links;
std::map<int, std::map<int, FeatureBA>> wordReferences;
};
MultiCamBundleGraph buildMultiCamBundleGraph(MultiCamFov fov = MultiCamFov::kWide,
MultiCamMode mode = MultiCamMode::kMono)
{
MultiCamBundleGraph g;
const double fx = fov == MultiCamFov::kNarrow ? kMultiCamNarrowFx : kBaFx;
const double fy = fov == MultiCamFov::kNarrow ? kMultiCamNarrowFy : kBaFy;
// Negative Tx triggers the stereo-edge path in g2o / GTSAM BA; mono
// keeps it at 0 so depth observations are ignored.
const double Tx = mode == MultiCamMode::kStereo ? -kMultiCamStereoBaseline * fx : 0.0;
// 1) Hilly circular trajectory (matches buildCircleGraph's hill profile).
for(int i = 1; i <= kMultiCamNumPoses; ++i)
{
const float theta = 2.0f * static_cast<float>(CV_PI) * static_cast<float>(i - 1) / static_cast<float>(kMultiCamNumPoses);
const float x = kMultiCamCircleRadius * std::cos(theta);
const float y = kMultiCamCircleRadius * std::sin(theta);
const float z = kMultiCamHillAmp * std::sin(kMultiCamHillFreq * theta);
const float yaw = theta + static_cast<float>(CV_PI) / 2.0f; // tangent CCW
const float pitch = -kMultiCamHillAmp * kMultiCamHillFreq / kMultiCamCircleRadius
* std::cos(kMultiCamHillFreq * theta);
g.truePoses[i] = Transform(x, y, z, 0.0f, pitch, yaw);
}
// 2) Four cameras mounted on the round-robot rim pointing forward /
// left / backward / right. localTransform per camera =
// (rim offset in body frame) * (body-frame yaw) * opticalRotation
// so each camera's optical +Z aligns with the indicated body
// direction while sitting ~17.5 cm out from the body center.
const float R = kMultiCamRobotRadius;
const float Z = kMultiCamCameraZ;
const float multicamYaws[kMultiCamNumCameras] = {
0.0f, // 0: forward (body +X)
static_cast<float>(CV_PI) / 2.0f, // 1: left (body +Y)
static_cast<float>(CV_PI), // 2: back (body -X)
-static_cast<float>(CV_PI) / 2.0f // 3: right (body -Y)
};
const float multicamOffsetX[kMultiCamNumCameras] = { R, 0, -R, 0 };
const float multicamOffsetY[kMultiCamNumCameras] = { 0, R, 0, -R };
for(auto & kv : g.truePoses)
{
std::vector<CameraModel> rig;
for(int c = 0; c < kMultiCamNumCameras; ++c)
{
const Transform localTransform =
Transform(multicamOffsetX[c], multicamOffsetY[c], Z,
0.0f, 0.0f, multicamYaws[c])
* CameraModel::opticalRotation();
rig.emplace_back(fx, fy, kBaCx, kBaCy,
localTransform, Tx,
cv::Size(kBaImageWidth, kBaImageHeight));
}
g.models[kv.first] = std::move(rig);
}
// 3) Random points uniformly in a 20 m x 20 m x 1.5 m box. The box is
// wider than the 10 m trajectory radius, so points sit both inside
// the circle (mid-frame visibility) and in the outer annulus
// (back/side visibility from the nearer poses).
std::mt19937 rng(7);
std::uniform_real_distribution<float> distXY(-kMultiCamPointBoxXY, kMultiCamPointBoxXY);
std::uniform_real_distribution<float> distZ (-kMultiCamPointBoxZ, kMultiCamPointBoxZ);
for(int p = 1; p <= kMultiCamNumPoints; ++p)
{
const float px = distXY(rng);
const float py = distXY(rng);
const float pz = distZ(rng);
g.truePoints3D[p] = cv::Point3f(px, py, pz);
}
// 4) Project. Visibility filter: body-to-point distance < 10 m
// (suppresses projecting points from the far side of the circle
// through the robot), point in front of camera (z > 0.1 m), pixel
// in image bounds. In stereo mode each observation also gets a
// depth derived from a noisy disparity (1 px sigma) -- so depth
// error grows quadratically with range, matching a real stereo
// block matcher / SGM at long range.
std::mt19937 dispRng(13);
std::normal_distribution<double> dispNoise(0.0, 1.0);
for(const auto & ptkv : g.truePoints3D)
{
const int pointId = ptkv.first;
for(const auto & posekv : g.truePoses)
{
const int frameId = posekv.first;
const float dx = ptkv.second.x - posekv.second.x();
const float dy = ptkv.second.y - posekv.second.y();
const float dz = ptkv.second.z - posekv.second.z();
if(std::sqrt(dx*dx + dy*dy + dz*dz) > kMultiCamVisibilityRange)
{
continue;
}
for(int c = 0; c < kMultiCamNumCameras; ++c)
{
const CameraModel & m = g.models.at(frameId)[c];
const Transform world_to_optical = (posekv.second * m.localTransform()).inverse();
const cv::Point3f pc = util3d::transformPoint(ptkv.second, world_to_optical);
if(pc.z <= 0.1f)
{
continue;
}
float u, v;
m.reproject(pc.x, pc.y, pc.z, u, v);
if(u < 0.0f || u >= kBaImageWidth || v < 0.0f || v >= kBaImageHeight)
{
continue;
}
float depthForObs = 0.0f;
if(mode == MultiCamMode::kStereo)
{
// Realistic stereo depth: noisy disparity → depth via
// d = b*fx / disparity. 1 px sigma on disparity gives
// σ_depth ≈ z² / (b*fx) * σ_disp, so depth error
// grows quadratically with range.
const double dispTrue = kMultiCamStereoBaseline * fx / pc.z;
const double dispNoisy = dispTrue + dispNoise(dispRng);
depthForObs = static_cast<float>(
kMultiCamStereoBaseline * fx / std::max(dispNoisy, 0.1));
}
g.wordReferences[pointId].insert(
{frameId, FeatureBA(cv::KeyPoint(u, v, 1.0f), depthForObs, cv::Mat(), c)});
}
}
}
// 5) Neighbor links 1->2->...->N (no loop closure; the existing BA
// test scenes are open chains).
const cv::Mat info = cv::Mat::eye(6, 6, CV_64FC1);
for(int i = 1; i < kMultiCamNumPoses; ++i)
{
const Transform t = g.truePoses[i].inverse() * g.truePoses[i + 1];
g.links.insert({i, Link(i, i + 1, Link::kNeighbor, t, info)});
}
// 6) Noisy initial guess: 5 cm linear / 1 deg angular per pose (root
// held at truth), 5 cm linear per point.
std::mt19937 nRng(42);
std::normal_distribution<float> linNoise(0.0f, 0.05f);
std::normal_distribution<float> angNoise(0.0f, 1.0f * static_cast<float>(CV_PI) / 180.0f);
for(const auto & kv : g.truePoses)
{
if(kv.first == 1)
{
g.initialPoses[kv.first] = kv.second;
continue;
}
float x, y, z, roll, pitch, yaw;
kv.second.getTranslationAndEulerAngles(x, y, z, roll, pitch, yaw);
const float nx = x + linNoise(nRng);
const float ny = y + linNoise(nRng);
const float nz = z + linNoise(nRng);
const float nroll = roll + angNoise(nRng);
const float npitch = pitch + angNoise(nRng);
const float nyaw = yaw + angNoise(nRng);
g.initialPoses[kv.first] = Transform(nx, ny, nz, nroll, npitch, nyaw);
}
for(const auto & kv : g.truePoints3D)
{
const float ptx = kv.second.x + linNoise(nRng);
const float pty = kv.second.y + linNoise(nRng);
const float ptz = kv.second.z + linNoise(nRng);
g.initialPoints3D[kv.first] = cv::Point3f(ptx, pty, ptz);
}
return g;
}
} // namespace
using MultiCamParam = std::tuple<Optimizer::Type, MultiCamFov, MultiCamMode, MultiCamLinks>;
class MultiCamBundleAdjustmentTest : public ::testing::TestWithParam<MultiCamParam>
{
protected:
void SetUp() override
{
if(!Optimizer::isAvailable(std::get<0>(GetParam())))
{
GTEST_SKIP() << optimizerTypeName(std::get<0>(GetParam())) << " not built in";
}
}
};
TEST_P(MultiCamBundleAdjustmentTest, FourCameraRigRecoversPosesAndPoints)
{
const Optimizer::Type backend = std::get<0>(GetParam());
const MultiCamFov fov = std::get<1>(GetParam());
const MultiCamMode mode = std::get<2>(GetParam());
const MultiCamLinks links = std::get<3>(GetParam());
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kOptimizerIterations()] = "200";
if(mode == MultiCamMode::kStereo)
{
// Sub-pixel u/v + ~1 px disparity tuning, same recipe as the
// kWithDepthNoLinksTuned single-cam BA variant. Default
// pv = dispVar = 1 falls into the info-matrix asymmetry trap at
// long range (10 m): observed u/v residuals are ~0 px but
// disparity has 1 px sigma; the optimizer over-trusts the
// noisier disparity channel and pushes points along the depth
// axis (multi-meter residuals). pv = 0.1 calibrates u/v as
// sub-pixel so both channels carry similar relative weight.
params[Parameters::kOptimizerPixelVariance()] = "0.1";
params[Parameters::kOptimizerDisparityVariance()] = "1.0";
}
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
MultiCamBundleGraph g = buildMultiCamBundleGraph(fov, mode);
if(links == MultiCamLinks::kNoLinks)
{
// Drop the pose-graph chain so BA has to recover the trajectory
// purely from reprojection (mono) or reprojection+disparity
// (stereo). For multi-cam mono this still converges to truth
// because the cross-camera, cross-frame triangulation pins
// scale; for stereo it's the disparity that anchors the gauge.
g.links.clear();
}
// Sanity: with 360° coverage + 10 m visibility, every point should be
// observed by many camera-frame pairs across the chain.
int totalObs = 0;
for(const auto & wkv : g.wordReferences) totalObs += static_cast<int>(wkv.second.size());
// Narrow FOV sees ~half the area of wide FOV per camera, so we relax
// the sanity floor accordingly.
const int minObs = fov == MultiCamFov::kWide ? 3000 : 1500;
ASSERT_GT(totalObs, minObs) << "Too few observations: " << totalObs;
std::map<int, cv::Point3f> outPoints = g.initialPoints3D;
std::map<int, Transform> outPoses = opt->optimizeBA(
/*rootId=*/1, g.initialPoses, g.links, g.models, outPoints, g.wordReferences);
ASSERT_FALSE(outPoses.empty()) << "optimizeBA returned no poses";
ASSERT_EQ(outPoses.size(), g.truePoses.size());
// Multi-camera BA is fully observable in BOTH FOV regimes -- even when
// adjacent cameras' fields of view don't overlap (narrow mode), the
// chain of 30 poses with rigidly-attached 4-camera rigs is enough to
// pin scale: a point first observed by the forward camera at pose K
// is observed by the left camera at some later pose K+n once the body
// has yawed, providing cross-camera triangulation across frames. No
// scale recovery needed -- both backends converge to µm-scale pose
// residuals against truth in MONO mode. STEREO loosens to ~2 cm
// because 1-px disparity noise on the 15 cm baseline propagates
// quadratically with range (σ_depth ≈ z²/(b·fx)·σ_disp = ~2 m at
// 10 m range per observation) and biases the pose estimates.
const Transform truthRootInv = g.truePoses.at(1).inverse();
const Transform outRootInv = outPoses.at(1).inverse();
// Stereo loosens to ~2 cm regardless of chain. Mono converges to µm
// with the chain anchor; without links the BA has to lock the gauge
// purely from reprojection, which costs a few mm of pose precision.
float poseBound;
if(mode == MultiCamMode::kStereo) poseBound = 0.03f;
else if(links == MultiCamLinks::kNoLinks) poseBound = 0.005f;
else poseBound = 0.001f;
const float angBound = mode == MultiCamMode::kStereo ? 0.05f : 0.01f;
for(const auto & kv : g.truePoses)
{
if(kv.first == 1) continue;
ASSERT_TRUE(outPoses.count(kv.first));
const Transform truthRel = truthRootInv * kv.second;
const Transform outRel = outRootInv * outPoses.at(kv.first);
EXPECT_LT(outRel.getDistance(truthRel), poseBound)
<< optimizerTypeName(backend) << " pose " << kv.first
<< " got(rel)=" << outRel.prettyPrint()
<< " truth(rel)=" << truthRel.prettyPrint();
const float angDeg = outRel.getAngle(truthRel) * 180.0f / static_cast<float>(CV_PI);
EXPECT_LT(angDeg, angBound)
<< optimizerTypeName(backend) << " pose " << kv.first << " angle=" << angDeg << " deg";
}
int verifiedPoints = 0;
for(const auto & kv : g.truePoints3D)
{
// Only check points that are well-triangulated -- the random
// scatter includes far-corner points that no camera sees within
// the 10 m visibility cap (NaN-filled by g2o / left at noisy
// initial by GTSAM), plus single-observation points that BA
// can't constrain in any meaningful way (mono: stays at noisy
// initial; stereo: ~σ_depth ≈ z²/(b·fx)·σ_disp ≈ 8 m at 20 m
// range from one disparity sample). Require ≥3 obs.
const auto wkv = g.wordReferences.find(kv.first);
if(wkv == g.wordReferences.end() || wkv->second.size() < 3) continue;
if(!outPoints.count(kv.first)) continue;
const cv::Point3f outRaw = outPoints.at(kv.first);
if(!std::isfinite(outRaw.x) || !std::isfinite(outRaw.y) || !std::isfinite(outRaw.z)) continue;
const cv::Point3f truthRel = util3d::transformPoint(kv.second, truthRootInv);
const cv::Point3f outRel = util3d::transformPoint(outRaw, outRootInv);
const cv::Point3f diff = outRel - truthRel;
const float d = std::sqrt(diff.x*diff.x + diff.y*diff.y + diff.z*diff.z);
// Point residuals are dominated by triangulation noise at the
// edge of the visibility cone (~10 m from the observing pose).
// Mono Wide: ~9 cm worst; Mono Narrow: ~11 cm worst (slightly
// weaker triangulation -- no simultaneous overlap means each
// point is effectively reconstructed from inter-frame baselines
// only). Stereo direct depth tightens both.
float pointBound;
if(mode == MultiCamMode::kStereo)
{
pointBound = fov == MultiCamFov::kWide ? 0.08f : 0.10f;
}
else
{
pointBound = fov == MultiCamFov::kWide ? 0.10f : 0.12f;
}
EXPECT_LT(d, pointBound)
<< optimizerTypeName(backend) << " point " << kv.first
<< " got(rel)=(" << outRel.x << "," << outRel.y << "," << outRel.z << ")"
<< " truth(rel)=(" << truthRel.x << "," << truthRel.y << "," << truthRel.z << ")";
++verifiedPoints;
}
// Sanity that we're actually verifying something -- if the visibility
// filter is too aggressive, we'd silently pass with no point checks.
EXPECT_GT(verifiedPoints, 200);
}
INSTANTIATE_TEST_SUITE_P(
Backends,
MultiCamBundleAdjustmentTest,
// g2o, GTSAM, and Ceres all support multi-camera BA. CVSBA
// doesn't (its underlying Sba::run() API is single-camera).
// Cross every FOV regime (wide 100° / narrow 60°) with mono /
// stereo (15 cm baseline + noisy disparity-derived depth) with
// or without the pose-graph chain. 24 variants total.
::testing::Combine(
::testing::Values(Optimizer::kTypeG2O, Optimizer::kTypeGTSAM, Optimizer::kTypeCeres),
::testing::Values(MultiCamFov::kWide, MultiCamFov::kNarrow),
::testing::Values(MultiCamMode::kMono, MultiCamMode::kStereo),
::testing::Values(MultiCamLinks::kWithLinks, MultiCamLinks::kNoLinks)),
[](const ::testing::TestParamInfo<MultiCamParam> & info)
{
return std::string(optimizerTypeName(std::get<0>(info.param))) + "_"
+ multiCamFovName(std::get<1>(info.param)) + "_"
+ multiCamModeName(std::get<2>(info.param)) + "_"
+ multiCamLinksName(std::get<3>(info.param));
});
TEST(OptimizerTest, CvsbaPoseGraphOptimizeReturnsEmpty)
{
// CVSBA only overrides optimizeBA() -- it doesn't implement pose-graph
// optimize(). Calling optimize() on a CVSBA instance hits the base
// Optimizer::optimize() which logs an error and returns an empty map.
// Pin that contract so callers can safely fall back.
if(!Optimizer::isAvailable(Optimizer::kTypeCVSBA))
{
GTEST_SKIP() << "CVSBA not built in";
}
std::unique_ptr<Optimizer> opt(Optimizer::create(Optimizer::kTypeCVSBA));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), Optimizer::kTypeCVSBA);
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
std::map<int, Transform> out = opt->optimize(1, in, inLinks);
EXPECT_TRUE(out.empty()) << "CVSBA should not handle pose-graph optimize()";
}