Files
rtabmap/corelib/test/test_optimizer.cpp
matlabbe ee49beaf4f Adding doc and tests (#1492)
* added doc and tests for util2d.h

* updated cmake-ros ci

* Added util3d.h doc and tests

* util3d_transforms.h: Added doc and tests

* util3d_filtering.h: started doc and test

* util3d_filtering.h: more tests and doc

* Added more doc/tests

* finished util3d_filtering doc and tests

* added test for util2d::depthBleedingFiltering

* Added util3d_registration tests

* Added util3d_features.h doc/tests

* added doc/tests for util3d_correspondences.h

* added doc/gtest for util3d_mapping.h (missing hpp functions)

* finished testing util3d_mapping.hpp

* Added util3d_motion_estimation.h tests (2D->3D done)

* finished util3d_motion_estimation.h tests

* minimal util3d_surface.h

* Added Transform and VisualWord tests

* Added doc for CameraModel and StereoCameraModel

* Added more logs in ros ci

* Passing tests on fical

* improved all devcontainer

* added devcontainer kilted, fixed source setup.bash, removed ldconfig in ros-cmake workflow

* cleanup

* source ros

* Added utilite tests

* Added testing to appveyor, github actions cancellable on re-commit on same branch

* appveyor testing without all targets

* appveyor: specifying ALL_BUILD target

* Fixed Util2dTest.NMSImageBoundsRespected test

* Fixing PCL Indices error on old pcl

* Added VWDictionary tests and doc. Fixed LSH not working (fix from https://github.com/flann-lib/flann/pull/472

* fixing some appveyor CI errors, added test to check dictionary serialization against all type

* Added StereoDense, StereoBM and StereoSGBM doc and tests

* Added Stereo tests

* Added CameraModel and StereoCameraModel tests

* Added doc and test for Statistics

* Added doc/tests for Signature

* Added doc/test for SensorEvent, added doc for SensorCaptureInfo

* Added doc to SensorData

* Added SensorData tests

* Added SensorCapture and SensorCaptureThread doc and tests

* fixed sensordata test

* updated SSC test and doc

* Added doc and tests for BayesFilter class

* Enabled testing on mac, updated windows testing like on linux

* added test_link

* fixed unresolved on windows

* fixed ThreadHandle error on macos ci

* Added GPS and GeodeticCoords tests

* Added tests for compression

* Added Odometry tests (base class only)

* Added DBDriver tests

* Added coverage report

* uniformized test names

* fixing concurancy and coverage ci

* dont built tools, examples and app for coverage build

* fixed report tool rebuilt without qt compilation error

* updated coverage option

* updated coverage config

* added doc CI job

* fixing windows and mac ci errors

* Added DBDriverSqlite3 tests

* Added IMU tests

* Added Graph tests

* fixing flaky macos test

* Added IMUThread and IMUFilter tests

* Added Landmarks tests

* Added LASWriter tests

* fixing seed flaky test

* fixing flaky macos timing tests

* Added LocalGrid tests

* Added LocalGridMaker tests

* fixing ci errors

* Added GlobalMap tests

* Added doc for EnvSensor

* Added Features2D tests

* Added Registration tests

* Added RegistrationVis tests

* Added doc for Rtabmap and Memory classes

* Added Memory and Rtabmap tests

* making some tests less flaky

* lcov 1.14 support

* updated compatible tool arguments

* Added integration tests (RGB-D, Stereo, Lidar2d, Lidar3d)

* More octomap checks

* Refactored how/when python interpretor is created to simplify library usage

* Added python tests

* fixed some flaky tests

* suppressed some third party related warnings

* fixed ceres tests

* more flaky fixes

* Fixing tests without libpointmatcher

* Added RANSAC rejection filter to PCL ICP

* fixing multi platform flakiness

* Added test to detect regression

* Fixing windows pcl link error

* fixed some macos flakiness

* bigger 2D2D registration error on opencv 4.6.0

* flakiness

* fixing flaky tests on windows and mac

* flaky thread test on slow mac VM

* windows slow test

* fixing more ci erros

* fxing temp dir on windows

* Added Optimizer tests and discovered some bugs (fixed)

* fixing flaky tests in mac and windows

* Added Optimizer doc

* Added GTSAM BA, updated Ceres to use g2o ba parameters. Renamed g2o's ba related parameters to Optimizer group and used by both gtsam and ceres.

* fixing build without gtsam

* fixing home dir

* fixing python ci isssues

* Added multicam ba tests

* Added Ceres multicam BA support

* Aligned BundleAdjustment parameters with Optimizer/Strategy to avoid confusion in the code

* Added BA integration test

* Added robust graph optimization integration test

* Added loop3it test

* Added stereo20Hz test

* Added smartfactor gtsam

* Fixed bugged check and warn if python didn't return any descriptors

* Fixing gtsam version build issues

* fixing tilt on windows ci

* loosing ceres integration test for ci

* mac ci flakiness

* updating missing param in gui

* updating test bound for mac

* added appearance-based tests, set min gftt quality to quality level

* testing more stuff

* improving features2d tests

* ci flakiness

* fixing flaky ci

* ci fixes

* flaky fixes

* Added RegistrationIcp tests

* Added icp integration test with real-worl corridor like env

* intermediate nodes

* fixing enum

* Updated test to catch #1714

* Fixed 2d corridor failing on pcl

* flaky pnp test

* flaky brisk test

* Set rtabmap_integration test as long

* updating loop closure test

* flaky ci tests

* TEsting roundtrip g2o/toro save/load

* loosing test bound

* fixed cuda capable checks

* flaky tests

* Debugging test hanging

* more debugging stuff

* updating limit

* windows: disabled cuda on ci to avoid incompatible driver issue. Fixing a bad test mem allocation

* trying fixing cuda hanging issue

* fixing ci flakyness

* flaky tests

* Updated BOW flaky tests by checking min precision/recall instead of recall@100precision. Fixed signature test

* CameraModel::load() test initRectificationMap param

* test dbdriver load dictionary idsOnly

* Memory: test keepLinkedInDb param

* added dummyDictionary tests

* test intermediate nodes count

* Added MarkerDetector tests

* reverted breaking change of UMutex and USemaphore

* Features2d: fixed compiltion warnings with clang about override

* clang warnings

* fixing test build with pcl 1.8

* g2o and gtsam build errors on android

* opencv5 test fixes

* disabled testing for ios and android builds

* normalized endline characters for easier diff

* added LF CRLF rule

* bump 0.23.10. fixing doc version

* Publish rtabmap website doc from ci

* fixing MSCVC build error

* macos icp flaky test

* fixing ceres macos test bound

* ficing more flaky tests

* fixing opencv5 related test errors. Also fixed an actual bug in ENU_WGS84ToGeocentric_WGS84()

* added comment about mrpt change

* removed rosdoc2 (will add it for rtabmap_ros later)

* fixing website style

* updated download links

* locally deployable website with api

* sweep doxygen issues

* improved/revised doxygen main pages

* removed examples empty page

* Updated doxygen style

* more concise doxygen groups

* added api link on main readme

* fixing utilite test error

* fixing CommonFilteringGroundNormalsUp test

* updated precisionRecall test bounds for Freak and brief descriptors

* fixing scale check in ba tests

* disabled tests on windows cuda build (missing dlls amd runner cannot test cuda anyway)

* ceres: missing suitesparse dep in windows ci

* adjusting recall thr for fast/freak

* ficing more flaky tests

* fixing flaky tests

* disabled coverage in ros ci

* Enable integration tests for ros ci jobs

* loosing up some threshold for failing tests

* trigger cache

* fixing test data in ros ci. Updated flaky test for mac

* slaking some test limit

* Fixed rtabmap-detectMoreLoopClosures inverted output value

* loosing up sift recall on mac

* optimizer re-ordered distribution for reproducible results (mac g2o)

* macos dump test crash log

* combining all tests to save time on shared library reload. Also fixed Logs with missing arguments.

* Added ENABLE_FORMAT_ERRORS cmake option

* do test only one time

* fixed all format warnings

* format security android build errors

* less verbose tests

* updated ImuUThread test

* fixed a log

* Fixed libpointmatcher 2d normals eigen issue

* Fixing libpointmatcher conversion issues

* fixing libpointmatcher test on windows ci

* cleanup comments, relax some test thr

* disabled sequoia-intel ci build (too flaky, would need extensive testing directly on that machine)
2026-08-06 13:32:20 -07:00

2890 lines
122 KiB
C++
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// Tests for rtabmap::Optimizer -- the base class. We exercise:
// - factory + isAvailable()
// - getters/setters and parseParameters()
// - getConnectedGraph(): a non-virtual graph utility that drives every
// concrete optimizer's input prep (filters disconnected nodes, applies
// priorsIgnored/landmarksIgnored, normalizes link direction).
// - optimize() / optimizeIncremental() smoke tests on a deterministic
// 2/3-node chain -- whichever concrete optimizer is built in.
//
// The Optimizer base class itself is abstract; concrete behavior comes from
// TORO/g2o/GTSAM/Ceres backends. We require *some* backend to be available
// (the factory throws otherwise), and stress what's portable across all of
// them rather than backend-specific quirks.
#include <gtest/gtest.h>
#include <rtabmap/core/Optimizer.h>
#include <rtabmap/core/Link.h>
#include <rtabmap/core/Parameters.h>
#include <rtabmap/core/Transform.h>
#include <rtabmap/core/CameraModel.h>
#include <rtabmap/core/util3d_transforms.h>
#include <rtabmap/utilite/UConversion.h>
#include <rtabmap/utilite/UTimer.h>
#include <memory>
#include <random>
using namespace rtabmap;
namespace {
// One unit-trace covariance per edge. Real callers populate a 6x6 inverse;
// for these tests we just need something invertible/non-zero.
cv::Mat unitCov()
{
return cv::Mat::eye(6, 6, CV_64FC1);
}
// Identity-translation link between two consecutive nodes a -> b along +X.
Link neighborLink(int from, int to, float dx = 1.0f)
{
return Link(from, to, Link::kNeighbor,
Transform(dx, 0, 0, 0, 0, 0),
unitCov());
}
// Build a straight 3-node chain (N1 -> N2 -> N3) at +X intervals.
void makeChain3(std::map<int, Transform> & poses, std::multimap<int, Link> & links)
{
poses[1] = Transform(0.0f, 0, 0, 0, 0, 0);
poses[2] = Transform(1.0f, 0, 0, 0, 0, 0);
poses[3] = Transform(2.0f, 0, 0, 0, 0, 0);
links.insert({1, neighborLink(1, 2)});
links.insert({2, neighborLink(2, 3)});
}
} // namespace
// ---------------------------------------------------------------------------
// Type / factory
// ---------------------------------------------------------------------------
TEST(OptimizerTest, TypeEnumValuesAreStable)
{
// Persisted in databases via Parameters/RGBD::OptimizerStrategy and in
// other DBs; reordering would break backward compat.
EXPECT_EQ(static_cast<int>(Optimizer::kTypeUndef), -1);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeTORO), 0);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeG2O), 1);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeGTSAM), 2);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeCeres), 3);
EXPECT_EQ(static_cast<int>(Optimizer::kTypeCVSBA), 4);
}
TEST(OptimizerTest, IsAvailableAtLeastOneOptimizer)
{
// The factory requires at least one backend; if all return false, no test
// using create() can ever pass. Catching it here is more diagnostic.
const bool anyAvailable =
Optimizer::isAvailable(Optimizer::kTypeTORO) ||
Optimizer::isAvailable(Optimizer::kTypeG2O) ||
Optimizer::isAvailable(Optimizer::kTypeGTSAM) ||
Optimizer::isAvailable(Optimizer::kTypeCeres);
EXPECT_TRUE(anyAvailable) << "No graph optimizer was compiled in";
}
TEST(OptimizerTest, IsAvailableReturnsFalseForUndef)
{
EXPECT_FALSE(Optimizer::isAvailable(Optimizer::kTypeUndef));
}
TEST(OptimizerTest, CreateReturnsNonNullForDefaultStrategy)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
ASSERT_NE(opt.get(), nullptr);
EXPECT_NE(opt->type(), Optimizer::kTypeUndef);
}
TEST(OptimizerTest, CreateFallsBackWhenTypeNotAvailable)
{
// Pick whatever IS available and ask for it explicitly.
for(int t : {Optimizer::kTypeTORO, Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM, Optimizer::kTypeCeres,
Optimizer::kTypeCVSBA})
{
if(Optimizer::isAvailable(static_cast<Optimizer::Type>(t)))
{
std::unique_ptr<Optimizer> opt(
Optimizer::create(static_cast<Optimizer::Type>(t)));
ASSERT_NE(opt.get(), nullptr);
EXPECT_EQ(opt->type(), static_cast<Optimizer::Type>(t));
return;
}
}
GTEST_SKIP() << "No optimizer backend available";
}
TEST(OptimizerTest, CreateViaParametersMapHonorsOptimizerStrategy)
{
// Find any available backend and request it through the parameter map
// (the path used by Rtabmap::init when reading user config).
for(int t : {Optimizer::kTypeTORO, Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM, Optimizer::kTypeCeres,
Optimizer::kTypeCVSBA})
{
if(Optimizer::isAvailable(static_cast<Optimizer::Type>(t)))
{
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(t);
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
EXPECT_EQ(opt->type(), static_cast<Optimizer::Type>(t));
return;
}
}
GTEST_SKIP() << "No optimizer backend available";
}
// ---------------------------------------------------------------------------
// Getters / setters / parseParameters
// ---------------------------------------------------------------------------
TEST(OptimizerTest, DefaultsMatchParameterDefaults)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
ASSERT_NE(opt.get(), nullptr);
EXPECT_EQ(opt->iterations(), Parameters::defaultOptimizerIterations());
EXPECT_EQ(opt->isSlam2d(), Parameters::defaultRegForce3DoF());
EXPECT_EQ(opt->isCovarianceIgnored(),Parameters::defaultOptimizerVarianceIgnored());
EXPECT_DOUBLE_EQ(opt->epsilon(), Parameters::defaultOptimizerEpsilon());
EXPECT_EQ(opt->isRobust(), Parameters::defaultOptimizerRobust());
EXPECT_EQ(opt->priorsIgnored(), Parameters::defaultOptimizerPriorsIgnored());
EXPECT_EQ(opt->landmarksIgnored(), Parameters::defaultOptimizerLandmarksIgnored());
EXPECT_FLOAT_EQ(opt->gravitySigma(), Parameters::defaultOptimizerGravitySigma());
}
TEST(OptimizerTest, SettersUpdateGettersIndependently)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
ASSERT_NE(opt.get(), nullptr);
opt->setIterations(123);
opt->setSlam2d(true);
opt->setCovarianceIgnored(true);
opt->setEpsilon(1e-4);
opt->setRobust(true);
opt->setPriorsIgnored(true);
opt->setLandmarksIgnored(true);
opt->setGravitySigma(0.42f);
EXPECT_EQ(opt->iterations(), 123);
EXPECT_TRUE(opt->isSlam2d());
EXPECT_TRUE(opt->isCovarianceIgnored());
EXPECT_DOUBLE_EQ(opt->epsilon(), 1e-4);
EXPECT_TRUE(opt->isRobust());
EXPECT_TRUE(opt->priorsIgnored());
EXPECT_TRUE(opt->landmarksIgnored());
EXPECT_FLOAT_EQ(opt->gravitySigma(), 0.42f);
}
TEST(OptimizerTest, ParseParametersFromMap)
{
ParametersMap params;
params[Parameters::kOptimizerIterations()] = "77";
params[Parameters::kRegForce3DoF()] = "true";
params[Parameters::kOptimizerVarianceIgnored()] = "true";
params[Parameters::kOptimizerEpsilon()] = "0.001";
params[Parameters::kOptimizerRobust()] = "true";
params[Parameters::kOptimizerPriorsIgnored()] = "false";
params[Parameters::kOptimizerLandmarksIgnored()]= "true";
params[Parameters::kOptimizerGravitySigma()] = "0.5";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
EXPECT_EQ(opt->iterations(), 77);
EXPECT_TRUE(opt->isSlam2d());
EXPECT_TRUE(opt->isCovarianceIgnored());
EXPECT_DOUBLE_EQ(opt->epsilon(), 0.001);
EXPECT_TRUE(opt->isRobust());
EXPECT_FALSE(opt->priorsIgnored());
EXPECT_TRUE(opt->landmarksIgnored());
EXPECT_FLOAT_EQ(opt->gravitySigma(), 0.5f);
// parseParameters() can also be called post-construction to update fields
// in-place (Rtabmap::parseParameters does this on parameter change).
ParametersMap update;
update[Parameters::kOptimizerIterations()] = "5";
opt->parseParameters(update);
EXPECT_EQ(opt->iterations(), 5);
// Unspecified fields are preserved.
EXPECT_TRUE(opt->isRobust());
}
// ---------------------------------------------------------------------------
// getConnectedGraph
// ---------------------------------------------------------------------------
TEST(OptimizerTest, GetConnectedGraphReturnsSingleNodeWhenNoLinks)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
in[1] = Transform(0, 0, 0, 0, 0, 0);
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, std::multimap<int, Link>(), out, outLinks);
ASSERT_EQ(out.size(), 1u);
EXPECT_TRUE(out.count(1));
EXPECT_TRUE(outLinks.empty());
}
TEST(OptimizerTest, GetConnectedGraphKeepsFullChain)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(out.size(), 3u);
EXPECT_EQ(outLinks.size(), 2u);
}
TEST(OptimizerTest, GetConnectedGraphDropsDisconnectedNodes)
{
// N1 -> N2 -> N3 is the connected component starting from N1.
// N10 has no link to the others -> must NOT appear in the output.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
in[10] = Transform(99.0f, 0, 0, 0, 0, 0);
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(out.size(), 3u);
EXPECT_FALSE(out.count(10));
}
TEST(OptimizerTest, GetConnectedGraphRecomputesPosesAlongEdges)
{
// The input pose at N3 is "wrong" (10m away) but the link chain says
// 0 -> 1 -> 2. getConnectedGraph must walk the links, not trust the
// caller's poses, so the output should reflect the link-implied chain.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
in[3] = Transform(10.0f, 0, 0, 0, 0, 0); // wrong on purpose
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
ASSERT_TRUE(out.count(3));
EXPECT_NEAR(out.at(3).x(), 2.0f, 1e-5);
}
TEST(OptimizerTest, GetConnectedGraphFollowsLoopClosureLinks)
{
// Loop closure between N3 and N1 should pull N3 into the connected
// component even though the chain (N1 N2 N3) already does so. We add
// loop-closure types to make sure they're respected.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
in[1] = Transform(0, 0, 0, 0, 0, 0);
in[2] = Transform(1, 0, 0, 0, 0, 0);
in[3] = Transform(0, 1, 0, 0, 0, 0);
inLinks.insert({1, neighborLink(1, 2)});
inLinks.insert({1, Link(1, 3, Link::kGlobalClosure,
Transform(0, 1, 0, 0, 0, 0), unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(out.size(), 3u);
EXPECT_EQ(outLinks.size(), 2u);
}
TEST(OptimizerTest, GetConnectedGraphPriorLinksIncludedWhenAllowed)
{
// kOptimizerPriorsIgnored defaults to true, so flip it off to exercise
// the inclusion path.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
opt->setPriorsIgnored(false);
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
// Pose prior on N1 (self-link with kPosePrior).
inLinks.insert({1, Link(1, 1, Link::kPosePrior,
in[1], unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
// 2 chain links + 1 prior.
EXPECT_EQ(outLinks.size(), 3u);
}
TEST(OptimizerTest, GetConnectedGraphPriorLinksDroppedWhenIgnored)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
opt->setPriorsIgnored(true);
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
inLinks.insert({1, Link(1, 1, Link::kPosePrior, in[1], unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(outLinks.size(), 2u); // prior dropped
}
TEST(OptimizerTest, GetConnectedGraphLandmarkLinksDroppedWhenIgnored)
{
// Landmarks use negative IDs by rtabmap convention. landmarksIgnored
// keeps the landmark out of the output graph entirely.
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
opt->setLandmarksIgnored(true);
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
in[-1] = Transform(0.5f, 0.5f, 0, 0, 0, 0);
inLinks.insert({2, Link(2, -1, Link::kLandmark,
Transform(0, 0.5f, 0, 0, 0, 0), unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_EQ(out.size(), 3u);
EXPECT_FALSE(out.count(-1));
}
TEST(OptimizerTest, GetConnectedGraphLandmarkLinksKeptByDefault)
{
std::unique_ptr<Optimizer> opt(Optimizer::create(ParametersMap()));
ASSERT_FALSE(opt->landmarksIgnored());
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
in[-1] = Transform(0.5f, 0.5f, 0, 0, 0, 0);
inLinks.insert({2, Link(2, -1, Link::kLandmark,
Transform(0, 0.5f, 0, 0, 0, 0), unitCov())});
std::map<int, Transform> out;
std::multimap<int, Link> outLinks;
opt->getConnectedGraph(1, in, inLinks, out, outLinks);
EXPECT_TRUE(out.count(-1));
}
// ---------------------------------------------------------------------------
// optimize() / optimizeIncremental() smoke tests
//
// Parameterized over the four pose-graph backends. CVSBA is excluded -- it's
// bundle-adjustment-only and doesn't implement plain pose-graph optimize().
// Each test skips at runtime when the backend isn't built in.
// ---------------------------------------------------------------------------
namespace {
std::string optimizerTypeName(Optimizer::Type type)
{
switch(type)
{
case Optimizer::kTypeTORO: return "TORO";
case Optimizer::kTypeG2O: return "G2O";
case Optimizer::kTypeGTSAM: return "GTSAM";
case Optimizer::kTypeCeres: return "Ceres";
case Optimizer::kTypeCVSBA: return "CVSBA";
default: return "Undef";
}
}
class OptimizerBackendTest : public ::testing::TestWithParam<Optimizer::Type>
{
protected:
void SetUp() override
{
if(!Optimizer::isAvailable(GetParam()))
{
GTEST_SKIP() << optimizerTypeName(GetParam()) << " not built in";
}
opt_.reset(Optimizer::create(GetParam()));
ASSERT_NE(opt_.get(), nullptr);
ASSERT_EQ(opt_->type(), GetParam());
}
std::unique_ptr<Optimizer> opt_;
};
} // namespace
TEST_P(OptimizerBackendTest, OptimizeConsistentChainReturnsSamePoses)
{
// A chain whose link transforms match the poses exactly has zero residual,
// so any optimizer should return ~the same poses.
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
std::map<int, Transform> out = opt_->optimize(1, in, inLinks);
ASSERT_EQ(out.size(), 3u);
for(const auto & kv : in)
{
ASSERT_TRUE(out.count(kv.first));
EXPECT_NEAR(out.at(kv.first).x(), kv.second.x(), 1e-2)
<< optimizerTypeName(GetParam()) << " node " << kv.first;
}
}
TEST_P(OptimizerBackendTest, OptimizeAnchorsRootPose)
{
// The root pose should remain fixed at the input value -- it's the gauge.
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
std::map<int, Transform> out = opt_->optimize(/*rootId=*/1, in, inLinks);
ASSERT_TRUE(out.count(1));
EXPECT_NEAR(out.at(1).x(), 0.0f, 1e-4) << optimizerTypeName(GetParam());
EXPECT_NEAR(out.at(1).y(), 0.0f, 1e-4) << optimizerTypeName(GetParam());
EXPECT_NEAR(out.at(1).z(), 0.0f, 1e-4) << optimizerTypeName(GetParam());
}
TEST_P(OptimizerBackendTest, OptimizeIncrementalConsistentChain)
{
// Same as the smoke test above but driven through the incremental path.
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
// Incremental needs at least one loop closure to "fire"; without one it
// short-circuits to the regular optimize() call at the end.
inLinks.insert({1, Link(1, 3, Link::kGlobalClosure,
Transform(2.0f, 0, 0, 0, 0, 0), unitCov())});
std::map<int, Transform> out = opt_->optimizeIncremental(1, in, inLinks);
ASSERT_EQ(out.size(), 3u);
for(const auto & kv : in)
{
ASSERT_TRUE(out.count(kv.first));
EXPECT_NEAR(out.at(kv.first).x(), kv.second.x(), 1e-2)
<< optimizerTypeName(GetParam()) << " node " << kv.first;
}
}
INSTANTIATE_TEST_SUITE_P(
AllPoseGraphBackends,
OptimizerBackendTest,
::testing::Values(
Optimizer::kTypeTORO,
Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM,
Optimizer::kTypeCeres),
[](const ::testing::TestParamInfo<Optimizer::Type> & info) {
return optimizerTypeName(info.param);
});
// ---------------------------------------------------------------------------
// Closed-loop circular trajectory with landmarks + gravity + pose priors.
//
// Setup: 30 evenly-spaced poses on a 10 m radius circle in the XY plane. Each
// pose's yaw points tangentially (i.e. toward the next pose -- a differential
// robot orbiting the center). Constraints:
// * 29 sequential neighbor links (1->2, 2->3, ..., 29->30) carrying the
// exact pairwise transform.
// * 1 loop-closure (30->1) carrying the exact transform.
// * 1 landmark at the origin (id = -1) with identity rotation. Even-numbered
// poses (2, 4, ..., 30) observe it via kLandmark links.
// * A kPosePrior self-link on every pose, transform = the pose itself.
// * A kGravity self-link on every pose, transform = the pose itself
// (roll/pitch are what the optimizer reads).
// All link covariances are unit 6x6, including the landmark links.
//
// This is a "perfect world" -- every constraint is exactly satisfied by the
// input poses, so the optimizer's first residual is zero and the output
// should equal the input (modulo the root being the gauge). We exercise the
// 16 combinations of {Force3DoF, GravitySigma, PriorsIgnored, LandmarksIgnored,
// useLastAsRoot} on every available pose-graph backend.
// ---------------------------------------------------------------------------
namespace {
constexpr int kCircleN = 30;
constexpr float kCircleRadius = 10.0f;
constexpr int kLandmarkId = -1;
constexpr int kLandmarkId2 = -2; // second landmark, used when numLandmarks=2
// Per-axis 1-sigma values used both for the information matrix and (when
// noise is requested) for the initial-guess perturbation.
constexpr double kLinSigma = 0.05; // 5 cm
const double kAngSigma = 1.0 * CV_PI / 180.0; // 1 deg in rad
struct CircleGraph
{
std::map<int, Transform> truePoses; // ground truth (link transforms came from these)
std::map<int, Transform> poses; // optimizer input -- equals truePoses
// except for noised non-anchor nodes
std::multimap<int, Link> links;
};
// Build the perfectly-consistent circle graph described above. When noisy=true,
// every pose except 1, kCircleN, and the landmark is perturbed by Gaussian
// noise (sigma = kLinSigma, kAngSigma) -- giving the optimizer a noisy
// initial guess while the constraints still describe the true graph.
CircleGraph buildCircleGraph(bool noisy = false, int numLandmarks = 1, bool force3DoF = false)
{
CircleGraph g;
// Realistic information matrix (diag(1/sigma^2)): 5 cm 1-sigma on each
// linear axis, 1 deg 1-sigma on each angular axis.
const double linInfo = 1.0 / (kLinSigma * kLinSigma);
const double angInfo = 1.0 / (kAngSigma * kAngSigma);
cv::Mat info = cv::Mat::zeros(6, 6, CV_64FC1);
info.at<double>(0, 0) = linInfo;
info.at<double>(1, 1) = linInfo;
info.at<double>(2, 2) = linInfo;
info.at<double>(3, 3) = angInfo;
info.at<double>(4, 4) = angInfo;
info.at<double>(5, 5) = angInfo;
// Sample poses around the circle. Yaw aims at the next pose so a
// differential robot walking the chain only translates forward in its
// own frame between consecutive nodes.
//
// In 6DoF mode, z and pitch vary sinusoidally (three hills around the
// loop, ~0.5 m crest, ~8.6 deg max pitch) to simulate a robot driving
// over rolling terrain. The pitch matches the slope so the chain is
// physically consistent. In 3DoF mode the loop stays flat -- the 3DoF
// optimizer doesn't model z/pitch anyway.
const float hillAmp = force3DoF ? 0.0f : 0.5f;
const float hillFreq = 3.0f;
for(int i = 1; i <= kCircleN; ++i)
{
const float theta = 2.0f * static_cast<float>(CV_PI) * static_cast<float>(i - 1) / static_cast<float>(kCircleN);
const float x = kCircleRadius * std::cos(theta);
const float y = kCircleRadius * std::sin(theta);
const float z = hillAmp * std::sin(hillFreq * theta);
const float yaw = theta + static_cast<float>(CV_PI) / 2.0f; // tangent to circle
// Slope along the body's forward direction. dz/ds = (1/R) * dz/dtheta.
// Nose up when going uphill -> negative pitch in ZYX Euler.
const float pitch = -hillAmp * hillFreq / kCircleRadius * std::cos(hillFreq * theta);
g.truePoses[i] = Transform(x, y, z, 0.0f, pitch, yaw);
}
// Landmark at the origin with identity rotation. Optional 2nd landmark
// horizontally offset along +X so the relative bearing between the two
// (in body frame) is a function of each pose's position (not orientation),
// breaking the bearing-only ambiguity of a single-landmark setup.
g.truePoses[kLandmarkId] = Transform::getIdentity();
if(numLandmarks >= 2)
{
g.truePoses[kLandmarkId2] = Transform(5.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f);
}
// Neighbor chain 1 -> 2 -> ... -> 30.
for(int i = 1; i < kCircleN; ++i)
{
const Transform t = g.truePoses[i].inverse() * g.truePoses[i + 1];
g.links.insert({i, Link(i, i + 1, Link::kNeighbor, t, info)});
}
// Loop closure 30 -> 1.
{
const Transform t = g.truePoses[kCircleN].inverse() * g.truePoses[1];
g.links.insert({kCircleN, Link(kCircleN, 1, Link::kGlobalClosure, t, info)});
}
// Landmark observations on even poses. Built in the post-getConnectedGraph
// normalized format: from=landmark_id (negative), to=observer, transform =
// landmark -> observer. This is the format the optimizer backends expect
// and lets us bypass getConnectedGraph (which would erase pose noise by
// reconstructing poses from links via a tree walk).
for(int i = 2; i <= kCircleN; i += 2)
{
const Transform t = g.truePoses[kLandmarkId].inverse() * g.truePoses[i];
g.links.insert({kLandmarkId, Link(kLandmarkId, i, Link::kLandmark, t, info)});
if(numLandmarks >= 2)
{
const Transform t2 = g.truePoses[kLandmarkId2].inverse() * g.truePoses[i];
g.links.insert({kLandmarkId2, Link(kLandmarkId2, i, Link::kLandmark, t2, info)});
}
}
// Per-pose pose prior and gravity self-link, both equal to the pose itself.
for(int i = 1; i <= kCircleN; ++i)
{
g.links.insert({i, Link(i, i, Link::kPosePrior, g.truePoses[i], info)});
g.links.insert({i, Link(i, i, Link::kGravity, g.truePoses[i], info)});
}
// Optimizer input starts at ground truth. With noisy=true, every pose
// EXCEPT id 1 and id kCircleN (those may be used as the gauge anchor) is
// perturbed by Gaussian noise drawn from the same (kLinSigma, kAngSigma)
// the links were weighted against. The landmark is also perturbed -- it
// isn't a gauge anchor; the optimizer should pull it back via its
// observation links. The constraints still describe the true graph --
// only the initial guess is wrong, so the optimizer should pull back to
// the truth.
g.poses = g.truePoses;
if(noisy)
{
// Fixed seed: identical noise sequence on every platform / run.
std::mt19937 rng(42);
std::normal_distribution<double> linDist(0.0, kLinSigma);
std::normal_distribution<double> angDist(0.0, kAngSigma);
auto perturb = [&](int id) {
const Transform & t = g.truePoses.at(id);
float r, p, y;
t.getEulerAngles(r, p, y);
const float dx = static_cast<float>(linDist(rng));
const float dy = static_cast<float>(linDist(rng));
const float dz = static_cast<float>(linDist(rng));
const float dr = static_cast<float>(angDist(rng));
const float dp = static_cast<float>(angDist(rng));
const float dyaw = static_cast<float>(angDist(rng));
g.poses[id] = Transform(
t.x() + dx, t.y() + dy, t.z() + dz,
r + dr, p + dp, y + dyaw);
};
for(int i = 2; i <= kCircleN - 1; ++i)
{
perturb(i);
}
perturb(kLandmarkId);
if(numLandmarks >= 2)
{
perturb(kLandmarkId2);
}
}
return g;
}
// Tuple ordering: optimizer, force3DoF, gravitySigma, priorsIgnored,
// landmarksIgnored, useLastAsRoot. Bool params are wrapped in int for
// readability in gtest's default printer; the lambda below pretty-prints.
using CircleParam = std::tuple<Optimizer::Type, bool, float, bool, bool, bool>;
class CircleGraphTest : public ::testing::TestWithParam<CircleParam>
{
protected:
void SetUp() override
{
if(!Optimizer::isAvailable(std::get<0>(GetParam())))
{
GTEST_SKIP() << optimizerTypeName(std::get<0>(GetParam())) << " not built in";
}
}
};
} // namespace
TEST_P(CircleGraphTest, NoisyInitialGuessConvergesToTruth)
{
const auto & p = GetParam();
const Optimizer::Type backend = std::get<0>(p);
const bool force3DoF = std::get<1>(p);
const float gravitySigma = std::get<2>(p);
const bool priorsIgnored = std::get<3>(p);
const bool landmarksIgnored = std::get<4>(p);
const bool useLastAsRoot = std::get<5>(p);
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kRegForce3DoF()] = force3DoF ? "true" : "false";
params[Parameters::kOptimizerGravitySigma()] = uNumber2Str(gravitySigma);
params[Parameters::kOptimizerPriorsIgnored()] = priorsIgnored ? "true" : "false";
params[Parameters::kOptimizerLandmarksIgnored()]= landmarksIgnored ? "true" : "false";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
// Build with noise: every non-anchor pose (i.e. not 1, not kCircleN, not
// the landmark) is perturbed. The links still encode the truth, so the
// optimizer should pull the poses back to truePoses.
CircleGraph g = buildCircleGraph(/*noisy=*/true, /*numLandmarks=*/1, force3DoF);
const int rootId = useLastAsRoot ? kCircleN : 1;
// Optional orientation perturbation on top of the per-axis noise:
// pre-multiply each input pose's rotation (positions left untouched) by
// a small world-frame roll/pitch/(yaw) rotation. The relative link
// constraints still describe the truth, so the optimizer needs an
// *absolute* orientation reference to recover. The two interesting
// cases are:
//
// priors-only (gravitySigma == 0, priorsIgnored == false):
// Per-pose pose priors anchor every pose's full orientation, so
// every pose -- including the roots -- gets roll/pitch/yaw
// perturbed. The priors pull each pose back to truth.
//
// gravity-only (gravitySigma > 0, priorsIgnored == true):
// Gravity ties only roll/pitch to world up; yaw remains gauge-free.
// So the roots (id 1, kCircleN) get roll/pitch perturbed but their
// yaw is left at truth -- the optimizer's gauge fix in yaw depends
// on the root staying at truth. Non-roots also get yaw perturbed;
// the chain of links propagates the root's true yaw and the relative
// yaw constraints pull the non-roots back.
//
// Skipped for TORO (its tree-based SGD doesn't converge tightly enough)
// and for 3DoF mode (roll/pitch are not optimized there). The "neither"
// and "both" configs are also skipped: "neither" is gauge-free in yaw,
// and "both" is already covered by the unperturbed noisy test.
const bool priorsOnly = (gravitySigma == 0.0f && !priorsIgnored);
const bool gravityOnly = (gravitySigma > 0.0f && priorsIgnored);
if((opt->type() == Optimizer::kTypeG2O || opt->type() == Optimizer::kTypeGTSAM) && !force3DoF && (priorsOnly || gravityOnly))
{
const float deg = static_cast<float>(CV_PI) / 180.0f;
const float dRoll = 3.0f * deg;
const float dPitch = 4.0f * deg;
const float dYawNonRoot = 5.0f * deg;
const float dYawRoot = priorsOnly ? 5.0f * deg : 0.0f;
// On gravity-only roots, perturb only ONE of roll/pitch (drop pitch).
// Perturbing both induces a small geometric yaw drift on the root that
// gravity can't recover (yaw is gauge-free in that mode): after
// gravity rotates the tilted local Z back to world Z, the local X-axis
// settles at a slightly different XY direction, and that yaw twist
// propagates around the chain. Pure roll alone (or pure pitch alone)
// keeps the local X-axis in the XY plane, so no drift.
const float dPitchRoot = gravityOnly ? 0.0f : dPitch;
// World-frame perturbation on non-roots: pre-multiplied onto each
// pose's transform so roll/pitch deltas act about world axes (what
// gravity observes). On roots we post-multiply (body-frame) so the
// root's truth position is preserved -- in gravity-only mode the
// optimizer holds root setFixed in position, and a world-frame
// pre-multiply would shift it about the world origin and drag the
// whole chain along.
for(int i = 1; i <= kCircleN; ++i)
{
const bool isRoot = (i == 1 || i == kCircleN);
const float dPi = isRoot ? dPitchRoot : dPitch;
const float dYaw = isRoot ? dYawRoot : dYawNonRoot;
const Transform R(0.0f, 0.0f, 0.0f, dRoll, dPi, dYaw);
g.poses[i] = isRoot ? g.poses.at(i) * R : R * g.poses.at(i);
}
}
// Use getConnectedGraph() only to get the filtered link set (it drops
// kPosePrior self-links when priorsIgnored and landmarks when
// landmarksIgnored). Its pose output is the result of a tree-walk from
// the root that would overwrite every pose -- erasing the noise we just
// added -- so we discard it and feed the optimizer our noisy g.poses
// instead.
std::map<int, Transform> connectedPoses;
std::multimap<int, Link> inLinks;
opt->getConnectedGraph(rootId, g.poses, g.links, connectedPoses, inLinks);
ASSERT_EQ(connectedPoses.size(), g.poses.size() - (landmarksIgnored ? 1u : 0u));
std::map<int, Transform> inPoses = g.poses;
if(landmarksIgnored)
{
inPoses.erase(kLandmarkId);
}
// In 3DoF mode the optimizer doesn't touch z/roll/pitch, so any noise we
// added on those axes would survive into the output verbatim and fail the
// truth comparison for no real reason. Strip it from the input poses up
// front so what we feed in matches the DoFs the optimizer actually solves.
if(force3DoF)
{
// 3DoF optimizer doesn't touch z/roll/pitch on poses OR on the
// landmark, so strip any z/roll/pitch noise from every input pose
// (including the landmark) to match what the optimizer can actually
// solve.
for(auto & kv : inPoses)
{
kv.second = kv.second.to3DoF();
}
}
std::map<int, Transform> out = opt->optimize(rootId, inPoses, inLinks);
ASSERT_FALSE(out.empty()) << "Optimizer returned no poses";
// Only g2o and GTSAM actually optimize landmarks; TORO and Ceres log a
// warning and drop the landmark silently from the output. So the landmark
// appears in the output when (a) the backend supports it AND
// (b) landmarksIgnored is false.
const bool backendSupportsLandmarks =
backend == Optimizer::kTypeG2O || backend == Optimizer::kTypeGTSAM;
const bool expectLandmarkInOutput = backendSupportsLandmarks && !landmarksIgnored;
EXPECT_EQ(out.size(), kCircleN + (expectLandmarkInOutput ? 1u : 0u));
// Constraints describe the truth, so the optimizer should pull the noisy
// initial guess back to truePoses. Tolerances are loose enough to absorb
// each backend's residual-at-convergence, tight enough that a "didn't
// move" regression would trip immediately (noise is up to ~3*sigma_lin
// = 15 cm per axis from truth).
for(const auto & kv : g.truePoses)
{
const int id = kv.first;
if(id == kLandmarkId && !expectLandmarkInOutput)
{
continue;
}
ASSERT_TRUE(out.count(id)) << "missing id " << id;
// TORO is a tree-based SGD optimizer with a fixed iteration count;
// the others (g2o/GTSAM/Ceres) use Gauss-Newton / Levenberg-Marquardt
// and converge to numerical zero, so we hold them to much tighter
// bounds.
const float distTol = (backend == Optimizer::kTypeTORO) ? 0.015f : 0.002f;
const float angTolDeg = (backend == Optimizer::kTypeTORO) ? 0.5f : 0.01f;
EXPECT_LT(out.at(id).getDistance(kv.second), distTol)
<< optimizerTypeName(backend) << " id=" << id
<< " expected=" << kv.second.prettyPrint()
<< " got=" << out.at(id).prettyPrint();
const float angDeg = out.at(id).getAngle(kv.second) * 180.0f / static_cast<float>(M_PI);
EXPECT_LT(angDeg, angTolDeg)
<< optimizerTypeName(backend) << " id=" << id
<< " angle=" << angDeg << " deg";
}
}
INSTANTIATE_TEST_SUITE_P(
AllBackendsAndConfigs,
CircleGraphTest,
::testing::Combine(
::testing::Values(Optimizer::kTypeTORO,
Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM,
Optimizer::kTypeCeres),
::testing::Bool(), // Reg/Force3DoF
::testing::Values(0.0f, 0.01f), // Optimizer/GravitySigma
::testing::Bool(), // Optimizer/PriorsIgnored
::testing::Bool(), // Optimizer/LandmarksIgnored
::testing::Bool()), // useLastAsRoot
[](const ::testing::TestParamInfo<CircleParam> & info) {
std::string name = optimizerTypeName(std::get<0>(info.param));
name += std::get<1>(info.param) ? "_3DoF" : "_6DoF";
name += std::get<2>(info.param) > 0.0f ? "_grav" : "_nograv";
name += std::get<3>(info.param) ? "_noprior" : "_prior";
name += std::get<4>(info.param) ? "_nolm" : "_lm";
name += std::get<5>(info.param) ? "_rootLast" : "_rootFirst";
return name;
});
// ---------------------------------------------------------------------------
// CircleGraphBadLoopClosureTest -- corrupt the loop-closure link with a large
// translation offset and verify that the Optimizer/Robust kernel rejects the
// outlier loop closure (g2o + GTSAM only; relies on Vertigo switchable
// factors which are conditional on WITH_VERTIGO build flag).
// ---------------------------------------------------------------------------
namespace {
using CircleBadLcParam = std::tuple<Optimizer::Type, bool /*useRobust*/>;
class CircleGraphBadLoopClosureTest : public ::testing::TestWithParam<CircleBadLcParam>
{
protected:
void SetUp() override
{
const Optimizer::Type t = std::get<0>(GetParam());
if(!Optimizer::isAvailable(t))
{
GTEST_SKIP() << optimizerTypeName(t) << " not built in";
}
}
};
} // namespace
TEST_P(CircleGraphBadLoopClosureTest, RobustRejectsCorruptedLoopClosure)
{
const Optimizer::Type backend = std::get<0>(GetParam());
const bool useRobust = std::get<1>(GetParam());
// Use the noisy CircleGraph (5 cm pose + point noise) so the test has a
// realistic baseline. Force 6DoF, no gravity, ignore priors+landmarks so
// the only relevant constraints are the neighbor chain + the (corrupted)
// loop closure.
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kRegForce3DoF()] = "false";
params[Parameters::kOptimizerGravitySigma()] = "0";
params[Parameters::kOptimizerPriorsIgnored()] = "true";
params[Parameters::kOptimizerLandmarksIgnored()] = "true";
params[Parameters::kOptimizerRobust()] = useRobust ? "true" : "false";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
CircleGraph g = buildCircleGraph(/*noisy=*/true);
// Corrupt the loop closure: take the truth kGlobalClosure transform and
// add a 5 m translation offset on its X axis. We need at least a few
// meters for the robust kernel to clearly prefer "disable the LC" over
// "absorb the offset into the chain": 11 neighbor links with
// sigma=5 cm (info=400/axis) can absorb ~1 m of LC offset with chain
// chi^2 ~ 11 * (1m/11)^2 * 400 ~ 36, comparable to the cost of
// dropping the switch (~1) + the chain-at-truth noise residual (~30).
// At 5 m the chain absorption costs ~890 chi^2, which is decisively
// worse than disabling the LC -- so the Vertigo switch reliably drives
// to 0.
std::multimap<int, Link> corruptedLinks;
for(const auto & kv : g.links)
{
if(kv.second.type() == Link::kGlobalClosure)
{
const Transform & t = kv.second.transform();
float r, p, y;
t.getEulerAngles(r, p, y);
const Transform bad(t.x() + 5.0f, t.y(), t.z(), r, p, y);
corruptedLinks.insert({kv.first,
Link(kv.second.from(), kv.second.to(), kv.second.type(),
bad, kv.second.infMatrix())});
}
else
{
corruptedLinks.insert(kv);
}
}
g.links = corruptedLinks;
const int rootId = 1;
std::map<int, Transform> connectedPoses;
std::multimap<int, Link> inLinks;
opt->getConnectedGraph(rootId, g.poses, g.links, connectedPoses, inLinks);
std::map<int, Transform> out = opt->optimize(rootId, g.poses, inLinks);
ASSERT_FALSE(out.empty()) << "Optimizer returned no poses";
// Worst pose displacement from truth.
float worstPoseD = 0.0f;
for(const auto & kv : g.truePoses)
{
if(kv.first < 0) continue; // skip landmarks (we ignored them anyway)
ASSERT_TRUE(out.count(kv.first)) << "missing id " << kv.first;
worstPoseD = std::max(worstPoseD, out.at(kv.first).getDistance(kv.second));
}
if(useRobust)
{
// Vertigo switchable factor wraps the loop closure with a switch
// variable initialized to 1 (LC fully active) and a prior keeping
// it near 1; the optimizer is free to drive the switch toward 0
// to disable the LC. With the Jacobian fix in
// betweenFactorSwitchable.h, both backends drive the switch fully
// to 0 and recover to ~mm/sub-mm pose error:
// * G2O ~1 mm.
// * GTSAM ~30 microns (slightly tighter than g2o on this
// particular setup because GaussNewton is used by default).
EXPECT_LT(worstPoseD, 0.005f)
<< optimizerTypeName(backend) << " robust=on worstPoseD=" << worstPoseD;
}
else
{
// Without robust, the bad loop closure is a hard constraint -- the
// chain stretches to satisfy it and pose 30 ends up ~5 m off truth.
EXPECT_GT(worstPoseD, 4.5f)
<< optimizerTypeName(backend) << " robust=off worstPoseD=" << worstPoseD;
}
}
INSTANTIATE_TEST_SUITE_P(
Backends,
CircleGraphBadLoopClosureTest,
::testing::Values(
std::make_tuple(Optimizer::kTypeG2O, false),
std::make_tuple(Optimizer::kTypeG2O, true),
std::make_tuple(Optimizer::kTypeGTSAM, false),
std::make_tuple(Optimizer::kTypeGTSAM, true)),
[](const ::testing::TestParamInfo<CircleBadLcParam> & info)
{
std::string name = optimizerTypeName(std::get<0>(info.param));
name += std::get<1>(info.param) ? "_Robust" : "_NoRobust";
return name;
});
// ---------------------------------------------------------------------------
// LandmarkCovarianceTest -- effect of per-link landmark covariance on g2o/GTSAM
// ---------------------------------------------------------------------------
// Reuses the CircleGraph (closed loop + landmark observed by even poses), but
// rewrites the information matrix on every Link::kLandmark edge to one of
// several test cases before optimizing. Limited to g2o and GTSAM -- TORO and
// Ceres drop landmarks silently (see backendSupportsLandmarks above).
namespace {
// Mirrors the Marker/Variance{Linear,Angular,OrientationIgnored} triplet from
// Parameters.h. Memory.cpp uses these to build the per-landmark-link 6x6
// covariance fed to the optimizer; we re-create that same construction here
// so our tests exercise the exact code path real marker detections take.
struct LandmarkCovCase
{
const char * name;
double linVar; // Marker/VarianceLinear (1.0 in the 3x3 translation block)
double angVar; // Marker/VarianceAngular (1.0 in the 3x3 rotation block; 9999 = sentinel)
bool orientationIgnored; // Marker/VarianceOrientationIgnored
double linkLinNoise; // Sigma (m) of Gaussian noise added to every kLandmark link's translation.
// 0 means no extra noise (default; equivalent to a "perfect" marker detector).
bool linkRangeOnly; // If true, noise is applied ALONG the bearing direction only -- preserves the
// direction (perfect bearing) but corrupts the magnitude (noisy range). If
// false (default), noise is isotropic on x/y/z and corrupts both bearing and range.
bool openLoopWithDrift; // If true: drop the loop-closure link, perturb every kNeighbor link's transform
// (Gaussian translation/rotation noise plus a constant -1 deg yaw bias), and
// seed the optimizer with poses derived from this perturbed chain (via
// getConnectedGraph's tree walk) instead of g.poses. Simulates open-loop
// odometry drift -- the optimizer has to correct it from the landmark
// observations alone.
int numLandmarks; // 1 (default) or 2. Adds a second landmark at (5, 0, 0) so each observer pose has
// two horizontally-separated bearings -- the relative bearing between them
// constrains pose position independently of orientation (breaks bearing-only
// ambiguity).
};
const LandmarkCovCase kLandmarkCovCases[] = {
// Full-pose landmark observations (orientation estimated). Identical
// encoding for g2o and GTSAM; force3DoF doesn't change the matrix.
{ "default", 0.001, 0.01, false, 0.0, false, false, 1 }, // Marker defaults
{ "tightLin", 0.0001, 0.01, false, 0.0, false, false, 1 },
{ "looseLin", 0.01, 0.01, false, 0.0, false, false, 1 },
{ "looseAng", 0.001, 1.0, false, 0.0, false, false, 1 },
// orientationIgnored=true: rotation block is set to 9999 (no orientation
// estimation). For g2o only linVar matters. For GTSAM the encoding
// switches to bearing/range -- angVar drives bearing, linVar drives range.
// linVar=9999 is the documented sentinel that asks GTSAM to skip the
// range factor (bearing-only).
{ "noOrient", 0.001, 0.01, true, 0.0, false, false, 1 },
{ "noOrientBO", 9999, 0.01, true, 0.0, false, false, 1 }, // bearing-only (GTSAM)
// Same as noOrient but with sigma=0.3 m injected into every landmark
// observation's translation. linVar is bumped to 0.1 (sigma~0.32 m) so
// the info matrix matches the actual noise level. GTSAM's bearing/range
// factorization is expected to handle this gracefully (multiple noisy
// observations triangulate); g2o's Cartesian EdgeSE3PointXYZ is expected
// to do worse on the landmark.
{ "noOrientNoisy", 0.1, 0.001, true, 0.3, false, false, 1 },
// Range-only noise: sigma=0.3 m applied along the bearing direction
// (perfect bearing, noisy range). This is the cleanest stress test for
// GTSAM's BearingRangeFactor: multiple noise-free bearings still cross
// at the true landmark, so the landmark should triangulate to truth and
// pose rotations should stay essentially untouched. g2o's Cartesian
// EdgeSE3PointXYZ sees the same noisy translations and can't exploit
// the bearing-purity, so it should still struggle.
{ "noOrientRangeNoisy", 0.1, 0.0001, true, 0.3, true, false, 1 },
// Open-loop odometry with drift + perfect-bearing landmark observations:
// loop closure dropped, neighbor links translation/rotation-perturbed
// with a constant -1 deg yaw bias accumulating around the chain
// (~30 deg total drift at the open end), input poses seeded from that
// drifted chain. Only the landmark observations can pull the trajectory
// back to truth -- a stress test of the bearing/range factor's
// triangulating capability vs g2o's Cartesian factor.
{ "openLoopRangeNoisy", 2.25, 0.0001, true, 1.5, true, true, 1 },
// Same as openLoopRangeNoisy but with 2 horizontally-separated landmarks.
// The added relative-bearing position constraint plus the range factor
// (even with 1.5 m noise) should both tighten the chain further than
// either alone.
{ "openLoopRangeNoisy2Lm", 2.25, 0.0001, true, 1.5, true, true, 2 },
// Same setup as openLoopRangeNoisy but with linVar=9999 -- the GTSAM
// sentinel that disables the range factor entirely (BearingFactor
// instead of BearingRangeFactor). With 1.5 m range noise but perfect
// bearings, GTSAM should triangulate the landmark from bearings alone
// and never get fooled by the (huge) range noise. For g2o this isn't
// meaningful -- linVar=9999 sets its EdgeSE3PointXYZ info to ~1e-4 on
// translation, the landmark becomes unobservable, and with the loop
// closure also dropped the whole chain has no global correction; we
// skip g2o's assertions for this case.
{ "openLoopBearingOnly", 9999, 0.0001, true, 1.5, true, true, 1 },
// Same setup as openLoopBearingOnly but with a SECOND landmark at
// (5, 0, 0). The two landmarks are horizontally separated, so each
// observer pose's relative-bearing-between-the-two-landmarks is a
// function of the pose's POSITION only (orientation cancels). This
// breaks the bearing-only ambiguity (rotate-in-place + slide-along-ray)
// of a single-landmark setup. Expected: GTSAM 6DoF bearing-only should
// converge dramatically tighter than with one landmark.
{ "openLoopBearingOnly2Lm", 9999, 0.0001, true, 1.5, true, true, 2 },
};
using LandmarkCovParam = std::tuple<Optimizer::Type, int /* index into kLandmarkCovCases */, bool /* force3DoF */>;
class LandmarkCovarianceTest : public ::testing::TestWithParam<LandmarkCovParam>
{
protected:
void SetUp() override
{
const Optimizer::Type t = std::get<0>(GetParam());
if(!Optimizer::isAvailable(t))
{
GTEST_SKIP() << optimizerTypeName(t) << " not built in";
}
}
};
// Build the per-landmark-link 6x6 covariance the same way Memory.cpp does
// when ingesting a marker detection (see _markerOrientationIgnored branches
// at Memory.cpp:6076-6101). The optimizer backends interpret the diagonal
// blocks specially when orientationIgnored=true (g2o uses linVar on the
// translation block; GTSAM repacks it as bearing(angVar)/range(linVar)).
// Returns the COVARIANCE -- the caller must invert to get the info matrix
// the Link stores.
cv::Mat buildMarkerCov(const LandmarkCovCase & c, Optimizer::Type backend, bool force3DoF)
{
cv::Mat cov = cv::Mat::eye(6, 6, CV_64FC1);
if(c.orientationIgnored)
{
cov(cv::Range(3, 6), cv::Range(3, 6)) *= 9999; // disable orientation estimation
const bool isGTSAM = (backend == Optimizer::kTypeGTSAM);
if(!isGTSAM)
{
cov(cv::Range(0, 3), cv::Range(0, 3)) *= c.linVar;
}
else if(force3DoF)
{
// GTSAM 2D bearing/range: X=bearing, Y=range. (Z is unused by
// the Pose2/Point2 BearingRange factor.)
cov(cv::Range(0, 1), cv::Range(0, 1)) *= c.angVar;
cov(cv::Range(1, 2), cv::Range(1, 2)) *= c.linVar;
}
else
{
// GTSAM 3D bearing/range: X,Y=bearing, Z=range.
cov(cv::Range(0, 2), cv::Range(0, 2)) *= c.angVar;
cov(cv::Range(2, 3), cv::Range(2, 3)) *= c.linVar;
}
}
else
{
cov(cv::Range(0, 3), cv::Range(0, 3)) *= c.linVar;
cov(cv::Range(3, 6), cv::Range(3, 6)) *= c.angVar;
}
return cov;
}
} // namespace
TEST_P(LandmarkCovarianceTest, LandmarkCovarianceAffectsConvergence)
{
const auto & p = GetParam();
const Optimizer::Type backend = std::get<0>(p);
const LandmarkCovCase & cov = kLandmarkCovCases[std::get<1>(p)];
const bool force3DoF = std::get<2>(p);
// Skip g2o + linVar>=9999: it's the GTSAM "no range factor" sentinel,
// not meaningful for g2o (which has only one landmark factor type --
// EdgeSE3PointXYZ -- and would just see a near-zero-info Cartesian
// observation). The test cases are GTSAM-only.
if(backend == Optimizer::kTypeG2O && cov.orientationIgnored && cov.linVar >= 9999)
{
GTEST_SKIP() << "linVar=9999 (bearing-only sentinel) is GTSAM-only; "
"g2o would just see a near-zero-info Cartesian observation";
}
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kRegForce3DoF()] = force3DoF ? "true" : "false";
// Gravity is meaningful only in 6DoF -- in 3DoF the optimizer doesn't
// touch z/roll/pitch, so a gravity prior would be redundant.
params[Parameters::kOptimizerGravitySigma()] = force3DoF ? "0" : "0.01";
// For open-loop variants, enable priors and add an identity prior on the
// landmark to anchor it -- the optimizer can no longer satisfy bearings
// by sliding the landmark and must instead deform/rotate the pose chain.
const bool experimentLandmarkPrior = cov.openLoopWithDrift;
params[Parameters::kOptimizerPriorsIgnored()] = experimentLandmarkPrior ? "false" : "true";
params[Parameters::kOptimizerLandmarksIgnored()] = "false";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
// Reuse the noisy CircleGraph baseline.
CircleGraph g = buildCircleGraph(/*noisy=*/true, cov.numLandmarks, force3DoF);
// Rewrite landmark-link information matrices to the test case, using the
// same Memory.cpp construction marker detections go through. When the
// case requests it, also inject Gaussian translation noise into every
// kLandmark link's transform to simulate a noisy marker detector.
const cv::Mat lmInfo = buildMarkerCov(cov, backend, force3DoF).inv();
std::mt19937 linkRng(43);
std::normal_distribution<double> linkLinDist(0.0, cov.linkLinNoise);
// Open-loop drift on neighbor links: 1 cm linear sigma, 0.1 deg angular
// sigma, plus a constant -1 deg yaw bias on every neighbor edge.
std::mt19937 neighRng(44);
std::normal_distribution<double> neighLinDist(0.0, 0.01);
std::normal_distribution<double> neighAngDist(0.0, 0.1 * CV_PI / 180.0);
const float neighYawBias = -1.0f * static_cast<float>(CV_PI) / 180.0f;
std::multimap<int, Link> patchedLinks;
for(const auto & kv : g.links)
{
const Link & link = kv.second;
if(cov.openLoopWithDrift && link.type() == Link::kGlobalClosure)
{
// Drop the loop closure: simulates pure odometry, no place
// recognition.
continue;
}
if(cov.openLoopWithDrift && link.type() == Link::kNeighbor)
{
// Apply the -1 deg yaw bias as a left-multiplying rotation so it
// rotates the link's translation as well as its rotation -- models
// a yaw calibration bias (body frame rotated 1 deg from the true
// robot frame), where each forward step picks up a small lateral
// component proportional to the bias.
Transform t = Transform(0.0f, 0.0f, 0.0f, 0.0f, 0.0f, neighYawBias) * link.transform();
float r, p, y;
t.getEulerAngles(r, p, y);
const float nx = t.x() + static_cast<float>(neighLinDist(neighRng));
const float ny = t.y() + static_cast<float>(neighLinDist(neighRng));
const float nz = t.z() + static_cast<float>(neighLinDist(neighRng));
const float nr = r + static_cast<float>(neighAngDist(neighRng));
const float np_ = p + static_cast<float>(neighAngDist(neighRng));
const float nyaw = y + static_cast<float>(neighAngDist(neighRng));
t = Transform(
nx, ny, nz, nr, np_, nyaw);
patchedLinks.insert({kv.first, Link(link.from(), link.to(), link.type(), t, link.infMatrix())});
continue;
}
if(link.type() == Link::kLandmark)
{
Transform t = link.transform();
if(cov.linkLinNoise > 0.0)
{
float r, p, y;
t.getEulerAngles(r, p, y);
if(cov.linkRangeOnly)
{
// Scale t along its own direction: noisy magnitude
// (range), exact direction (bearing). With p(observer)
// at identity in the BearingRangeFactor construction,
// bearing = normalize(t) and range = |t|, so this
// perturbs the range only.
const float range = std::sqrt(t.x()*t.x() + t.y()*t.y() + t.z()*t.z());
if(range > 1e-6f)
{
const float scale = 1.0f + static_cast<float>(linkLinDist(linkRng)) / range;
t = Transform(
t.x() * scale,
t.y() * scale,
t.z() * scale,
r, p, y);
}
}
else
{
// Isotropic Gaussian on x/y/z (corrupts both bearing
// and range).
const float lx = t.x() + static_cast<float>(linkLinDist(linkRng));
const float ly = t.y() + static_cast<float>(linkLinDist(linkRng));
const float lz = t.z() + static_cast<float>(linkLinDist(linkRng));
t = Transform(lx, ly, lz, r, p, y);
}
}
patchedLinks.insert({kv.first, Link(link.from(), link.to(), link.type(), t, lmInfo)});
}
else
{
// In the landmark-prior experiment we flip priorsIgnored to
// false, which would otherwise activate the per-pose kPosePrior
// at truth and trivialize the test. Filter them out here -- the
// landmark prior we add below is the only prior we want active.
if(experimentLandmarkPrior && link.type() == Link::kPosePrior)
{
continue;
}
patchedLinks.insert(kv);
}
}
if(experimentLandmarkPrior)
{
// Position-only ("GPS-like") prior on the landmark at truth.
// Rotation-block info is set just below the 1/9999 "ignored"
// threshold so the GTSAM prior-detection logic treats this as a
// GPS prior (keeps the root setFixed). The translation block is
// tight (info ~10000 per axis -> sigma ~1 cm) to pin the landmark.
cv::Mat priorInfo = cv::Mat::zeros(6, 6, CV_64FC1);
for(int i = 0; i < 3; ++i) priorInfo.at<double>(i, i) = 10000.0;
for(int i = 3; i < 6; ++i) priorInfo.at<double>(i, i) = 1.0 / 10000.0;
patchedLinks.insert({kLandmarkId,
Link(kLandmarkId, kLandmarkId, Link::kPosePrior,
g.truePoses.at(kLandmarkId), priorInfo)});
if(cov.numLandmarks >= 2)
{
patchedLinks.insert({kLandmarkId2,
Link(kLandmarkId2, kLandmarkId2, Link::kPosePrior,
g.truePoses.at(kLandmarkId2), priorInfo)});
}
}
const int rootId = 1;
std::map<int, Transform> connectedPoses;
std::multimap<int, Link> inLinks;
opt->getConnectedGraph(rootId, g.poses, patchedLinks, connectedPoses, inLinks);
// In the open-loop variant, seed the optimizer with poses derived from
// the perturbed neighbor chain (a tree walk from root that accumulates
// the -1 deg/segment yaw bias) rather than the noisy ground-truth-ish
// g.poses. Everywhere else, keep g.poses as the input.
std::map<int, Transform> inPoses = cov.openLoopWithDrift ? connectedPoses : g.poses;
if(force3DoF)
{
// 3DoF optimizer doesn't touch z/roll/pitch on poses, so strip the
// noise we added on those axes. The landmark is left alone -- in
// production it's observed in full 3D and the optimizer keeps it on
// the SE(3) manifold even in 2D SLAM mode.
for(auto & kv : inPoses)
{
if(kv.first < 0)
{
continue;
}
kv.second = kv.second.to3DoF();
}
}
std::map<int, Transform> out = opt->optimize(rootId, inPoses, inLinks);
ASSERT_FALSE(out.empty()) << "Optimizer returned no poses";
EXPECT_EQ(out.size(), kCircleN + static_cast<size_t>(cov.numLandmarks)); // 30 poses + N landmarks
// TODO: per-case assertions. Sketch:
// - tight : output should hew very close to truth (landmark dominates).
// - default : matches the CircleGraphTest baseline tolerance.
// - loose : landmark contributes little; poses still converge via
// links + priors but with looser bounds on the landmark id.
// With perfect landmark observations every case converges to ~numerical
// zero on all DoFs the optimizer touches; with sigma=0.3 m noise injected
// into the landmark link transforms (kLandmarkCovCases::linkLinNoise),
// expect roughly the noise magnitude on the landmark and propagating to
// the poses, with g2o's Cartesian EdgeSE3PointXYZ struggling more than
// GTSAM's BearingRange in 6DoF (in 3DoF the two are comparable).
// Convergence bound selection:
// noise-free -> tight 1 mm / 0.01 deg / 1 mm.
// linkRangeOnly (perfect bearing, sigma=0.3m on range only):
// GTSAM exploits bearing-purity (BearingRangeFactor) and collapses
// to near-truth (poseD<5cm, ang<0.3deg, lmD<5cm); g2o 6DoF's
// EdgeSE3PointXYZ can't decompose so it does ~2x worse.
// isotropic noise (sigma=0.3m on x/y/z):
// Both bearings and ranges are corrupted; g2o 6DoF struggles
// particularly because its Cartesian factor weighs every axis equally.
const bool noisy = (cov.linkLinNoise > 0.0);
const bool g2o6DoF = (backend == Optimizer::kTypeG2O && !force3DoF);
// Non-noisy default bounds. Bumped from 1 mm to 2 mm because gravity (now
// enabled for 6DoF) adds extra Z-axis constraints that nudge G2O's
// noOrientBO solution to ~1.4 mm (still tight, just over the original
// 1 mm). GTSAM at ~4 microns either way.
float landmarkDistMax = 0.002f;
float poseDistMax = 0.002f;
float poseAngMaxDeg = 0.01f;
if(noisy && cov.openLoopWithDrift)
{
// Open-loop chain with -1 deg/segment yaw bias (~30 deg cumulative
// drift at the open end). A tight identity prior pins the landmark
// at truth (lmD essentially zero across all variants); the chain
// then converges from the prior's pull through the landmark
// observations. With gravity on (6DoF), roll/pitch are extra-tight.
//
// Behavior summary:
// * GTSAM 3DoF (both BearingRange and BearingOnly) recovers the
// chain best -- 2D bearing factor's Jacobian is well-conditioned
// and the pinned landmark provides a strong anchor (~3.7-3.9 m
// / 20-21 deg, vs ~5.8 m without prior).
// * GTSAM 6DoF: bearing-factor linearization on SO(3) hits a local
// minimum even with the landmark pinned (~4.7-5.5 m / 25-30 deg).
// * g2o 6DoF RangeNoisy: pinning the landmark actually HURTS
// convergence (~5.1 m, vs ~3.2 m without prior) -- the
// Cartesian observation edges can no longer balance landmark
// and chain residuals, and the solver settles in a worse
// local minimum.
// * g2o BearingOnly: chain stays fully uncorrected (~5.75 m /
// 29.5 deg) regardless of prior -- the EdgeSE3PointXYZ info
// matrix is ~1e-4 (linVar=9999), so the landmark is connected
// to the chain via near-zero-information edges.
landmarkDistMax = 0.01f; // pinned by prior, ~0 to 1 mm in practice
const bool isG2O = (backend == Optimizer::kTypeG2O);
const bool multiLm = (cov.numLandmarks >= 2);
if(multiLm && !isG2O)
{
// GTSAM with 2+ horizontally-separated landmarks: relative
// bearing between landmarks is a pure-position constraint
// (orientation-independent), resecting the pose. With or
// without the range factor, GTSAM converges to ~0.5 m / 3 deg
// in BOTH 3DoF and 6DoF.
poseDistMax = 0.70f;
poseAngMaxDeg = 5.0f;
}
else if(multiLm && isG2O)
{
// G2O EdgeSE3PointXYZ with FINITE landmark-edge info (linVar~2)
// triangulates well with 2 landmarks: chain converges to ~2 m
// / 15-18 deg. (The bearing-only g2o variant is skipped above
// since linVar=9999 is a GTSAM-only sentinel.)
poseDistMax = force3DoF ? 3.00f : 2.50f;
poseAngMaxDeg = force3DoF ? 20.0f : 17.0f;
}
else if(isG2O)
{
// G2O + 1 landmark + finite linVar: BearingRange-equivalent
// EdgeSE3PointXYZ. Chain partially recovers (~5 m).
poseDistMax = 6.00f;
poseAngMaxDeg = 30.0f;
}
else if(force3DoF)
{
// GTSAM 3DoF + 1 landmark + pinned: 2D bearing factor benefits
// from the prior but still has one degenerate DoF.
poseDistMax = 4.50f;
poseAngMaxDeg = 22.0f;
}
else
{
// GTSAM 6DoF + 1 landmark: bearing-factor SO(3) linearization
// stuck in local minimum at large initial yaw error.
poseDistMax = 5.70f;
poseAngMaxDeg = 30.0f;
}
}
else if(noisy && cov.linkRangeOnly)
{
const bool isG2O = (backend == Optimizer::kTypeG2O);
landmarkDistMax = isG2O ? 0.15f : 0.05f; // g2o has no bearing-purity advantage
poseDistMax = 0.20f;
poseAngMaxDeg = 1.0f;
}
else if(noisy)
{
landmarkDistMax = 0.30f;
poseDistMax = g2o6DoF ? 0.40f : 0.30f;
poseAngMaxDeg = g2o6DoF ? 3.0f : 2.0f;
}
// Open-loop "loop-closure delta" check: pose 1 (root, fixed at truth)
// and pose kCircleN are the would-be loop-closure pair (link dropped).
// The recovered distance/angle between them should match the truth
// delta (~2.09 m / 12 deg = one chord/segment), within the same chain
// bounds since the worst pose deviation is at the open end.
if(cov.openLoopWithDrift)
{
const float expectedD = g.truePoses.at(1).getDistance(g.truePoses.at(kCircleN));
const float expectedADeg = g.truePoses.at(1).getAngle(g.truePoses.at(kCircleN))
* 180.0f / static_cast<float>(M_PI);
const float actualD = out.at(1).getDistance(out.at(kCircleN));
const float actualADeg = out.at(1).getAngle(out.at(kCircleN))
* 180.0f / static_cast<float>(M_PI);
EXPECT_LT(std::abs(actualD - expectedD), poseDistMax)
<< optimizerTypeName(backend) << " case=" << cov.name
<< " loop-closure distance expected=" << expectedD
<< " actual=" << actualD;
EXPECT_LT(std::abs(actualADeg - expectedADeg), poseAngMaxDeg)
<< optimizerTypeName(backend) << " case=" << cov.name
<< " loop-closure angle expected=" << expectedADeg << " deg"
<< " actual=" << actualADeg << " deg";
}
for(const auto & kv : g.truePoses)
{
const int id = kv.first;
ASSERT_TRUE(out.count(id)) << optimizerTypeName(backend)
<< " case=" << cov.name << " missing id=" << id;
// Landmark converges very tightly in position regardless of whether
// orientation is tracked:
// 6DoF, orient tracked: G2O sub-mm + ~0.001 deg, GTSAM ~numerical zero.
// 6DoF, orient ignored: G2O ~0.14 mm, GTSAM sub-um (no angle to check
// -- the landmark vertex is Point3 and the output
// rpy is just the noisy input verbatim).
// 3DoF: x/y/yaw match truth to ~numerical zero. The z/roll/pitch we
// noised onto the landmark survive verbatim (the optimizer
// treats it as Pose2/Point2 and pastes the input z/roll/pitch
// into the output), so project both sides to3DoF() to drop
// those untouched axes from the comparison.
if(id < 0)
{
const Transform expected = force3DoF ? kv.second.to3DoF() : kv.second;
const Transform got = force3DoF ? out.at(id).to3DoF() : out.at(id);
EXPECT_LT(got.getDistance(expected), landmarkDistMax)
<< optimizerTypeName(backend) << " case=" << cov.name
<< " landmark got=" << got.prettyPrint();
// Angle is only meaningful when orientation is tracked and the
// landmark observations are noise-free; with linkLinNoise>0 the
// noisy translations leak into the recovered orientation chain
// (~0.9 deg on the landmark in our experiments), which is fine
// for the noisy case so we skip the angle assert there.
if(!cov.orientationIgnored && !noisy)
{
const float angDeg = got.getAngle(expected) * 180.0f / static_cast<float>(M_PI);
EXPECT_LT(angDeg, 0.01f)
<< optimizerTypeName(backend) << " case=" << cov.name
<< " landmark angle=" << angDeg << " deg";
}
continue;
}
// Non-landmark ids: noise-free cases converge to truth at well under
// 1 mm / 0.01 deg (worst observed is G2O+tightLin in 6DoF at
// ~0.44 mm / 0.002 deg; GTSAM is at numerical zero across the board).
// Noisy cases stay within poseDistMax/poseAngMaxDeg set above; in 6DoF
// g2o needs a looser envelope than GTSAM because its Cartesian
// EdgeSE3PointXYZ propagates landmark-link translation noise into the
// chain more strongly than GTSAM's BearingRange factorization.
EXPECT_LT(out.at(id).getDistance(kv.second), poseDistMax)
<< optimizerTypeName(backend) << " case=" << cov.name << " id=" << id
<< " got=" << out.at(id).prettyPrint();
const float poseAngDeg = out.at(id).getAngle(kv.second) * 180.0f / static_cast<float>(M_PI);
EXPECT_LT(poseAngDeg, poseAngMaxDeg)
<< optimizerTypeName(backend) << " case=" << cov.name << " id=" << id
<< " angle=" << poseAngDeg << " deg";
}
}
INSTANTIATE_TEST_SUITE_P(
G2OAndGTSAM,
LandmarkCovarianceTest,
::testing::Combine(
::testing::Values(Optimizer::kTypeG2O, Optimizer::kTypeGTSAM),
::testing::Range(0, static_cast<int>(sizeof(kLandmarkCovCases) / sizeof(kLandmarkCovCases[0]))),
::testing::Bool()), // Reg/Force3DoF
[](const ::testing::TestParamInfo<LandmarkCovParam> & info)
{
std::string name = optimizerTypeName(std::get<0>(info.param));
name += "_";
name += kLandmarkCovCases[std::get<1>(info.param)].name;
name += std::get<2>(info.param) ? "_3DoF" : "_6DoF";
return name;
});
// ---------------------------------------------------------------------------
// BundleAdjustmentTest -- cameras on a circle looking at the center, observing
// a small point cloud near the origin. Verifies that BA-capable backends
// (g2o / Ceres / CVSBA) recover both the frame poses AND the 3D point cloud
// from noisy initial guesses + clean 2D observations.
// ---------------------------------------------------------------------------
namespace {
constexpr int kBaNumCameras = 12;
constexpr int kBaNumPoints = 200;
constexpr float kBaCircleRadius = 5.0f;
// 752 x 480 camera @ 100 deg horizontal FOV (typical of fisheye / wide-angle
// imagers on small robots).
// fx = cx / tan(half_HFOV) = 376 / tan(50 deg) ~= 315.5
// vertical FOV = 2 * atan(cy/fy) = 2 * atan(240/315.5) ~= 75 deg
constexpr int kBaImageWidth = 752;
constexpr int kBaImageHeight = 480;
constexpr double kBaFx = 315.5;
constexpr double kBaFy = 315.5;
constexpr double kBaCx = 376.0;
constexpr double kBaCy = 240.0;
// 3D point cloud half-extents. Sized so that a "worst-case" point at the
// corner (Lxy, Lxy, Lz) projects to ~95% of each image dimension from any
// camera position. Derivation for u (image width):
// u_offset_max = Lxy * fx / (R - Lxy)
// Solving for 95% coverage (u_offset = 0.475 * 752 = 357):
// Lxy = 357 * R / (357 + fx) = 357 * 5 / (357 + 315.5) ~= 2.65 m
// Same form for v gives Lz ~= 1.65 m.
constexpr float kBaPointXY = 3.5f;
constexpr float kBaPointZ = 2.4f;
struct BundleGraph
{
std::map<int, Transform> truePoses;
std::map<int, cv::Point3f> truePoints3D;
std::map<int, std::vector<CameraModel>> models;
std::map<int, Transform> initialPoses; // possibly noisy
std::map<int, cv::Point3f> initialPoints3D; // possibly noisy
std::multimap<int, Link> links; // pose-graph constraints
std::map<int, std::map<int, FeatureBA>> wordReferences; // word_id -> camera_id -> FeatureBA
};
// Build the BA scenario:
// * kBaNumCameras cameras on a circle of radius kBaCircleRadius, all looking
// toward the origin (body +X points at origin -> yaw = theta + pi).
// * kBaNumPoints 3D points scattered in a [-1,1]^3 box around the origin.
// * Every camera observes every point (well within ~63 deg FOV).
// * Neighbor links 1->2->...->N between consecutive frames.
// When noisy=true:
// * Every non-anchor pose (id 2..N) gets 5 cm linear / 1 deg angular noise.
// * Every 3D point gets 5 cm noise.
// * Every neighbor link's transform gets the same noise on top of truth.
BundleGraph buildBundleGraph(bool noisy = false, bool roundPixels = false, int numCameras = kBaNumCameras, int numPoints = kBaNumPoints)
{
BundleGraph g;
const CameraModel model(kBaFx, kBaFy, kBaCx, kBaCy,
CameraModel::opticalRotation(),
/*Tx=*/0.0,
cv::Size(kBaImageWidth, kBaImageHeight));
// Cameras on the circle, body +X aimed at the origin.
for(int i = 1; i <= numCameras; ++i)
{
const float theta = 2.0f * static_cast<float>(CV_PI) * static_cast<float>(i - 1) / static_cast<float>(numCameras);
const float x = kBaCircleRadius * std::cos(theta);
const float y = kBaCircleRadius * std::sin(theta);
// body +X = (cos yaw, sin yaw, 0); to point at origin, +X = -position direction.
const float yaw = theta + static_cast<float>(CV_PI);
g.truePoses[i] = Transform(x, y, 0.0f, 0.0f, 0.0f, yaw);
g.models[i] = std::vector<CameraModel>{model};
}
// 3D point cloud: uniform in a box around the origin sized so that every
// camera around the circle sees the points spread across most of its
// image frame. Z extent is smaller than X,Y to match the narrower
// vertical FOV (67 deg vs 100 deg horizontal at 1080p).
{
std::mt19937 rng(7);
std::uniform_real_distribution<float> distXY(-kBaPointXY, kBaPointXY);
std::uniform_real_distribution<float> distZ (-kBaPointZ, kBaPointZ);
for(int p = 1; p <= numPoints; ++p)
{
// Draws are pulled into locals first: argument evaluation order
// is unspecified in C++, so passing three dist(rng) calls
// directly makes the generated scene compiler-dependent (GCC
// evaluates right-to-left, Clang left-to-right).
const float px = distXY(rng);
const float py = distXY(rng);
const float pz = distZ(rng);
g.truePoints3D[p] = cv::Point3f(px, py, pz);
}
}
// Project every point into every camera. Skip projections that fall behind
// the camera or outside the image (defensive -- with our geometry every
// point should be visible in every camera).
for(const auto & ptkv : g.truePoints3D)
{
const int pointId = ptkv.first;
for(const auto & posekv : g.truePoses)
{
const int camId = posekv.first;
const Transform world_to_optical = (posekv.second * model.localTransform()).inverse();
const cv::Point3f pc = util3d::transformPoint(ptkv.second, world_to_optical);
if(pc.z <= 0.1f)
{
continue;
}
float u, v;
model.reproject(pc.x, pc.y, pc.z, u, v);
if(u < 0.0f || u >= kBaImageWidth || v < 0.0f || v >= kBaImageHeight)
{
continue;
}
if(roundPixels)
{
// Simulate the discrete-pixel detection that real feature
// extractors produce -- adds +-0.5 px quantization noise to
// every reprojection residual.
u = std::round(u);
v = std::round(v);
}
g.wordReferences[pointId].insert({camId, FeatureBA(cv::KeyPoint(u, v, 1.0f))});
}
}
// Neighbor chain 1 -> 2 -> ... -> N.
cv::Mat info = cv::Mat::eye(6, 6, CV_64FC1);
for(int i = 0; i < 3; ++i) info.at<double>(i, i) = 1.0 / (0.05 * 0.05);
for(int i = 3; i < 6; ++i) info.at<double>(i, i) = 1.0 / (1.0 * CV_PI / 180.0 * 1.0 * CV_PI / 180.0);
for(int i = 1; i < numCameras; ++i)
{
const Transform t = g.truePoses[i].inverse() * g.truePoses[i + 1];
g.links.insert({i, Link(i, i + 1, Link::kNeighbor, t, info)});
}
// Initial guess: truth by default, noisy on request.
g.initialPoses = g.truePoses;
g.initialPoints3D = g.truePoints3D;
if(noisy)
{
std::mt19937 rng(11);
std::normal_distribution<double> linNoise(0.0, 0.05); // 5 cm on poses
std::normal_distribution<double> angNoise(0.0, 1.0 * CV_PI / 180.0); // 1 deg
std::normal_distribution<double> ptNoise(0.0, 0.05); // 5 cm on point positions
std::normal_distribution<double> linkLinNoise(0.0, 0.10); // 10 cm on neighbor link translations
// Perturb every pose except the root (id 1 = anchor for optimizeBA).
for(int i = 2; i <= numCameras; ++i)
{
const Transform & t = g.truePoses[i];
float r, p, y;
t.getEulerAngles(r, p, y);
const float dx = static_cast<float>(linNoise(rng));
const float dy = static_cast<float>(linNoise(rng));
const float dz = static_cast<float>(linNoise(rng));
const float dr = static_cast<float>(angNoise(rng));
const float dp = static_cast<float>(angNoise(rng));
const float dyaw = static_cast<float>(angNoise(rng));
g.initialPoses[i] = Transform(
t.x() + dx, t.y() + dy, t.z() + dz,
r + dr, p + dp, y + dyaw);
}
for(auto & kv : g.initialPoints3D)
{
kv.second.x += static_cast<float>(ptNoise(rng));
kv.second.y += static_cast<float>(ptNoise(rng));
kv.second.z += static_cast<float>(ptNoise(rng));
}
std::multimap<int, Link> noisyLinks;
for(const auto & kv : g.links)
{
const Link & link = kv.second;
Transform t = link.transform();
float r, p, y;
t.getEulerAngles(r, p, y);
const float dx = static_cast<float>(linkLinNoise(rng));
const float dy = static_cast<float>(linkLinNoise(rng));
const float dz = static_cast<float>(linkLinNoise(rng));
const float dr = static_cast<float>(angNoise(rng));
const float dp = static_cast<float>(angNoise(rng));
const float dyaw = static_cast<float>(angNoise(rng));
t = Transform(
t.x() + dx, t.y() + dy, t.z() + dz,
r + dr, p + dp, y + dyaw);
noisyLinks.insert({kv.first, Link(link.from(), link.to(), link.type(), t, link.infMatrix())});
}
g.links = noisyLinks;
}
return g;
}
// Per-test variant. Default exercises the full setup (poses + points +
// noisy neighbor links, mono reprojection). The other two are g2o-specific
// extensions:
// * G2ONoLinks: drop the kNeighbor edges before calling optimizeBA, so
// g2o BA is reduced to reprojection-only -- mimics how Ceres/CVSBA
// handle BA, and isolates the noisy-link contribution.
// * G2OWithDepth: provide a valid depth in every FeatureBA (range from
// camera to 3D point + Gaussian noise) and a Tx on the CameraModel
// so g2o creates EdgeStereoSE3ProjectXYZ "stereo" edges (u, v,
// u-disparity) instead of mono EdgeSE3ProjectXYZ. Tests g2o's RGB-D /
// stereo BA path.
enum class BaVariant {
kDefault,
kNoLinks,
kWithDepth,
kWithDepthNoLinks,
kWithDepthNoLinksTuned, // WithDepth + per-axis info calibrated to actual noise
kWithLidarDepthNoLinksTuned, // Accurate depth source (LiDAR-fused, 1 cm sigma) + tight DisparityVariance
};
// (backend, variant, roundPixels). roundPixels=true simulates discrete
// pixel-coordinate detection (every keypoint rounded to nearest integer)
// instead of the continuous reprojection -- adds +-0.5 px quantization
// noise to every observation.
using BaParam = std::tuple<Optimizer::Type, BaVariant, bool>;
const char * baVariantName(BaVariant v)
{
switch(v)
{
case BaVariant::kDefault: return "Default";
case BaVariant::kNoLinks: return "NoLinks";
case BaVariant::kWithDepth: return "WithDepth";
case BaVariant::kWithDepthNoLinks: return "WithDepthNoLinks";
case BaVariant::kWithDepthNoLinksTuned: return "WithDepthNoLinksTuned";
case BaVariant::kWithLidarDepthNoLinksTuned: return "WithLidarDepthNoLinksTuned";
}
return "?";
}
class BundleAdjustmentTest : public ::testing::TestWithParam<BaParam>
{
protected:
void SetUp() override
{
const Optimizer::Type t = std::get<0>(GetParam());
if(!Optimizer::isAvailable(t))
{
GTEST_SKIP() << optimizerTypeName(t) << " not built in";
}
}
};
} // namespace
TEST_P(BundleAdjustmentTest, CircleCamerasRecoverPosesAndPoints)
{
const Optimizer::Type backend = std::get<0>(GetParam());
const BaVariant variant = std::get<1>(GetParam());
const bool roundPixels = std::get<2>(GetParam());
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kOptimizerIterations()] = "200"; // probe
// For the stereo BA variants we set Tx on the CameraModel below, which
// g2o picks up directly. Set g2o/Baseline to the same 0.15 m for
// consistency (used as a fallback when Tx isn't set on the model).
params[Parameters::kOptimizerBaseline()] = "0.15";
if(variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
// Accurate-depth scenario (LiDAR-fused / structured-light): the
// depth measurement is *more* precise than a typical feature
// detector's u/v, so the optimizer should trust depth more.
// PixelVariance=1 (sigma_uv ~ 1 px, typical detector),
// DisparityVariance=0.1 (sigma_disp ~ 0.3 px, reflecting the
// fused depth's tighter precision). 1:10 ratio puts depth ~10x
// tighter than each pixel. Tighter still works empirically
// (e.g. 0.01 gives same residuals) but g2o's Hessian becomes
// ill-conditioned around info=10000, so we stay comfortably
// within the stable range.
params[Parameters::kOptimizerPixelVariance()] = "1.0";
params[Parameters::kOptimizerDisparityVariance()] = "0.1";
}
else if(variant == BaVariant::kWithDepthNoLinksTuned)
{
// Sub-pixel feature detector + standard stereo block matcher tuning:
// PixelVariance=0.1 (sigma_uv ~ 0.3 px), DisparityVariance=1
// (sigma_disp ~ 1 px). The 10:1 ratio gives u/v slightly more weight
// than disparity, which breaks the info-matrix asymmetry trap seen
// in WithDepth (default pv=1, dispVar=1 -> 12.8 cm worst point error
// because the optimizer over-trusts noisy disparity and pushes points
// along the depth axis to fit it). With this tuning the points
// converge to ~4 mm.
params[Parameters::kOptimizerPixelVariance()] = "0.1";
params[Parameters::kOptimizerDisparityVariance()] = "1.0";
}
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
BundleGraph g = buildBundleGraph(/*noisy=*/true, /*roundPixels=*/roundPixels);
if(variant == BaVariant::kNoLinks
|| variant == BaVariant::kWithDepthNoLinks
|| variant == BaVariant::kWithDepthNoLinksTuned
|| variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
// Drop the noisy neighbor links so g2o BA is reduced to a pure
// reprojection problem (like Ceres/CVSBA).
g.links.clear();
}
if(variant == BaVariant::kWithDepth
|| variant == BaVariant::kWithDepthNoLinks
|| variant == BaVariant::kWithDepthNoLinksTuned
|| variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
// Switch to a stereo camera model: baseline = 0.15 m (a common
// medium-baseline value e.g. ZED Mini / RealSense D435i). Tx =
// -baseline*fx so g2o detects it and uses the
// EdgeStereoSE3ProjectXYZ path (3-axis observation: u, v,
// u-disparity).
const double baselineMeters = 0.15;
const double Tx = -baselineMeters * kBaFx;
const CameraModel stereoModel(kBaFx, kBaFy, kBaCx, kBaCy,
CameraModel::opticalRotation(), Tx,
cv::Size(kBaImageWidth, kBaImageHeight));
for(auto & kv : g.models) kv.second = std::vector<CameraModel>{stereoModel};
// Realistic stereo noise: 1 px sigma on DISPARITY, constant across
// range (this is what a block matcher / SGM produces). Depth is
// then derived from the noisy disparity --
// sigma_depth = depth^2 / (baseline*fx) * sigma_disp
// -- so depth precision degrades quadratically with range. With our
// 0.15 m baseline and fx=315.5: sigma_depth ranges from ~13 cm at
// 2.5 m to ~1.2 m at 7.5 m. The optimizer's stereo edge measures
// the disparity directly via (u, v, u-disparity), so the
// quadratic range degradation is handled automatically by the
// projection geometry; the calibrated info matrix matches the
// constant ~1 px disparity noise (g2o's default PixelVariance=1
// is well-suited).
// In the LiDAR-fused variant we replace the stereo noise model with
// a small constant sigma on depth (range-independent, as a depth
// sensor / fused range source would produce). DisparityVariance is
// set tight in the params block to reflect this higher precision.
const bool lidarDepth = (variant == BaVariant::kWithLidarDepthNoLinksTuned);
std::mt19937 dispRng(13);
std::normal_distribution<double> dispNoise (0.0, 1.0); // 1 px on disparity (stereo / RGB-D)
std::normal_distribution<double> depthLidarNoise(0.0, 0.01); // 1 cm on depth (LiDAR / structured-light)
for(auto & wkv : g.wordReferences)
{
const cv::Point3f & worldPt = g.truePoints3D.at(wkv.first);
for(auto & ckv : wkv.second)
{
const Transform world_to_optical =
(g.truePoses.at(ckv.first) * stereoModel.localTransform()).inverse();
const cv::Point3f pc = util3d::transformPoint(worldPt, world_to_optical);
float depthNoisy;
if(lidarDepth)
{
depthNoisy = pc.z + static_cast<float>(depthLidarNoise(dispRng));
}
else
{
const double dispTrue = baselineMeters * kBaFx / pc.z;
const double dispNoisy = dispTrue + dispNoise(dispRng);
depthNoisy = static_cast<float>(
baselineMeters * kBaFx / std::max(dispNoisy, 0.1));
}
ckv.second = FeatureBA(ckv.second.kpt, depthNoisy);
}
}
}
// Sanity: every point should be observed from at least 6 cameras. With
// the spread point cloud (kBaPointXY=3.5, kBaPointZ=2.4) giving ~95%
// image coverage from each frame, points near the box corners get
// dropped by the most-oblique-angle cameras (depth too shallow or
// projection outside FOV). 6 observations is still plenty for BA.
for(const auto & ptkv : g.truePoints3D)
{
ASSERT_GE(g.wordReferences.at(ptkv.first).size(), 6u)
<< "point " << ptkv.first << " only observed from "
<< g.wordReferences.at(ptkv.first).size() << " cameras";
}
std::map<int, cv::Point3f> outPoints = g.initialPoints3D;
UTimer baTimer;
std::map<int, Transform> outPoses = opt->optimizeBA(
/*rootId=*/1, g.initialPoses, g.links, g.models, outPoints, g.wordReferences);
const double baSeconds = baTimer.getElapsedTime();
std::cerr << "[BA-timing] backend=" << optimizerTypeName(backend)
<< " variant=" << baVariantName(variant)
<< " roundPixels=" << (roundPixels?1:0)
<< " seconds=" << baSeconds << "\n";
ASSERT_FALSE(outPoses.empty()) << "optimizeBA returned no poses";
ASSERT_EQ(outPoses.size(), g.truePoses.size());
// BA recovers the trajectory and point-cloud SHAPE, but the gauge
// (global rigid transform) depends on the backend: g2o fixes the root
// pose via setFixed(rootId); Ceres and CVSBA don't fix any pose. To
// compare gauge-independently, express every pose and point in the
// optimizer's root frame and compare to truth in truth's root frame --
// the rigid transform between the two solutions cancels out.
const Transform truthRootInv = g.truePoses.at(1).inverse();
const Transform outRootInv = outPoses.at(1).inverse();
// Per-backend bounds. Setup: 752x480 / 100 deg HFOV / 200 features
// uniformly in [-3.5,3.5]x[-3.5,3.5]x[-2.4,2.4] box / 12 cameras on a
// 5 m circle / 5 cm pose+point noise / 10 cm noisy-link translation
// noise / 5 cm depth noise (for WithDepth variants) / 200 iterations.
// Chain rotation recovers to ~0 deg across all backends; position
// residual depends on how each backend handles the noisy chain:
// * Ceres / CVSBA: BA only uses reprojection; chain ignored. ~1.5 cm.
// * g2o + chain (Default / WithDepth): BA includes the kNeighbor
// edges with their full info matrix. The 10 cm link noise pulls
// poses 3-6 cm off truth.
// * g2o without chain (NoLinks / WithDepthNoLinks): tightest case.
// Bounds cover both continuous and rounded-pixel observations. Rounding
// adds +-0.5 px quantization noise per observation; with 200 obs per
// pose the pose residual is barely affected, but individual point
// residuals can grow ~80% (one point still depends on its few specific
// observations).
float poseDistMax = 0.03f;
float poseAngMaxDeg = 1.0f;
float pointDistMax = 0.025f;
if(backend == Optimizer::kTypeG2O)
{
if(variant == BaVariant::kNoLinks)
{
poseDistMax = 0.015f;
pointDistMax = 0.015f;
}
else if(variant == BaVariant::kWithDepth)
{
// With realistic stereo noise (1 px on disparity), point recovery
// at long range is dominated by the depth uncertainty. The
// continuous-pixel variant ALSO suffers from the info-matrix
// asymmetry: u/v residual = 0 at truth so the optimizer
// over-trusts them vs the 1 px disparity, pushing points along
// the depth axis to better fit the noisy disparity. Rounded u/v
// adds ~0.5 px noise that *can* balance the info matrix and
// recover ~8x tighter points (observed on Linux/OpenBLAS), but
// the pattern doesn't reproduce on macOS/Accelerate, where
// recovery sits around the continuous-pixel spread regardless
// of rounding -- so both the pose bound and the rounded point
// bound are widened to cover that. The continuous-pixel point
// bound was already loose (0.15) for the info-matrix reason.
poseDistMax = 0.06f;
pointDistMax = roundPixels ? 0.06f : 0.15f;
}
else if(variant == BaVariant::kWithDepthNoLinks)
{
poseDistMax = 0.015f;
pointDistMax = roundPixels ? 0.035f : 0.15f;
}
else if(variant == BaVariant::kWithDepthNoLinksTuned)
{
// Calibrated per-axis info matrix + no noisy chain. Tightest
// case of all the WithDepth variants: both pose AND point
// converge to ~5 mm.
poseDistMax = 0.015f;
pointDistMax = 0.015f;
}
else if(variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
// Accurate depth (1 cm sigma) + tight DisparityVariance: the
// tightest of all WithDepth variants on both pose and point.
poseDistMax = 0.005f;
pointDistMax = 0.005f;
}
else
{
// Default: chain pulled by noisy links.
poseDistMax = 0.10f;
pointDistMax = 0.09f;
}
}
else if(backend == Optimizer::kTypeGTSAM)
{
// GTSAM soft-fixes the root via a tight PriorFactor (no native
// "setFixed" like g2o), so the gauge is slightly looser; the LM
// solver also stops at a relativeErrorTol that leaves a tiny bit of
// residual on the longest-range points. Both effects are sub-cm on
// the pose, ~few-cm on point cloud points that are 7-8 m from the
// root. Stereo variants converge to the same bounds as g2o because
// the depth constraint pins the gauge.
if(variant == BaVariant::kWithDepthNoLinksTuned)
{
poseDistMax = 0.015f;
pointDistMax = 0.015f;
}
else if(variant == BaVariant::kWithLidarDepthNoLinksTuned)
{
poseDistMax = 0.005f;
pointDistMax = 0.005f;
}
else if(variant == BaVariant::kWithDepth || variant == BaVariant::kWithDepthNoLinks)
{
// These two variants keep the DEFAULT noise weighting
// (PixelVariance=1, DisparityVariance=1), which the Tuned variants
// above exist to fix: weighting u/v and disparity equally makes the
// optimizer over-trust noisy disparity and push points along the
// depth axis, the "info-matrix asymmetry trap" described at the
// kWithDepthNoLinksTuned params. That is why these are the variants
// with 12.8 cm point error on g2o while the tuned ones reach 4 mm.
// The point bound is wide because of that depth-axis distortion,
// which is the behaviour these variants exist to pin down.
//
// These bounds now measure SHAPE only -- the global scale is solved
// and divided out below, and checked separately there. That is what
// makes them portable. Both GTSAM and Ceres used to fail these on
// macOS with a uniform 0.3-0.5% scale shrink and an otherwise
// near-perfect shape, which an absolute position bound turns into
// 5 cm of "error" on the poses 10 m from the root and nothing on the
// near ones. Once scale is factored out, the linux pose residual on
// this variant drops from 0.0058 to 0.0030 and the macOS ratios were
// uniform to 4 decimals, so both platforms land in the same place.
//
// The scale error itself was not a convergence failure: on linux the
// result is bit-identical across Optimizer/Epsilon from 1e-5 down to
// 0, Optimizer/Iterations from 200 to 2000, and PCG vs direct
// multifrontal Cholesky. It is the converged optimum of an
// ill-conditioned problem, and two different toolchains land on
// slightly different optima. Nothing to tighten, which is why it is
// measured rather than chased.
poseDistMax = 0.04f;
pointDistMax = roundPixels ? 0.04f : 0.15f;
}
else if(variant == BaVariant::kNoLinks)
{
// Pure mono reprojection. Mono BA has a 7-DOF gauge -- the
// recovered geometry is only correct up to scale -- but we
// solve for that scale against truth below before checking
// bounds, so the bounds match the clean-converged case.
poseDistMax = 0.015f;
pointDistMax = 0.02f;
}
else
{
// kDefault: mono BA + noisy chain. Scale handled below; the
// chain noise still pulls poses ~few cm.
poseDistMax = 0.04f;
pointDistMax = 0.04f;
}
}
else if(backend == Optimizer::kTypeCeres)
{
if(roundPixels &&
(variant == BaVariant::kWithDepth || variant == BaVariant::kWithDepthNoLinks))
{
// Ceres uses the generic 0.025 default everywhere else; these two
// variants get a little more room for the +-0.5 px u/v quantization
// stacked on the 1 px disparity noise. Both variants share the noise
// model, so they share the bound.
//
// This bound used to be doing double duty, absorbing a macOS scale
// bias as well: the recovered points came out a uniform 0.29% short
// (ratio 0.9971 on every failing point -- one global factor, not
// per-point noise), which is 2.6 cm at 9 m from the root against
// 0.0018 on linux. That is now handled where it belongs, by the
// scale solve below, so this only has to cover quantization again.
pointDistMax = 0.04f;
}
}
else if(backend == Optimizer::kTypeCVSBA)
{
poseDistMax = 0.06f;
pointDistMax = 0.04f;
}
if(roundPixels)
{
// +-0.5 px quantization noise hits the point residuals harder than
// the pose residuals (each point depends on its few observations,
// each pose averages over ~200). Loosen the point bound a bit.
// Note this only lifts bounds that are still below the floor: the
// branches above already set rounded-specific bounds where needed.
pointDistMax = std::max(pointDistMax, 0.02f);
}
// Gauge handling. Every variant is compared to truth after dividing out one
// global scale factor, solved in closed form from the camera positions in
// the root-relative frame:
// s = Σ(d_out · d_truth) / Σ(d_out · d_out)
// then applied to BOTH poses and points. This is the test-side counterpart
// to the brute-force scale scan in tools/Report/main.cpp; here we know the
// relationship is quadratic so the closed form is exact.
//
// For mono BA this is the only way to compare at all: fixing one pose still
// leaves a 1-DOF scale ambiguity the optimizer is free to land anywhere on.
// That covers the pure-mono variants, and also every variant on CVSBA --
// rtabmap's CVSBA path uses cvsba's Sba::run(), which takes 2D image points
// only, so it runs mono BA even when handed stereo-style observations.
//
// The depth variants are a different case: g2o, GTSAM and Ceres switch to a
// stereo cost function when depth+baseline are available, so scale IS
// observable there and a scale error is a real error rather than a gauge
// choice. They get the same treatment anyway, because it separates two
// failure modes that a raw position comparison conflates. A uniform scale
// bias and a distorted shape are very different defects, but an absolute
// position bound charges for both at once -- and since the error from a scale
// bias grows with distance from the root, it charges the far poses several
// times more than the near ones for the identical relative mistake. Dividing
// scale out means the position bounds below measure shape only, and scale
// gets its own flat check where a 0.5% bias reads as 0.5% wherever the pose
// sits.
//
// Splitting them is what makes these bounds portable. On macOS both GTSAM and
// Ceres recover this graph with a 0.3-0.5% scale shrink and an otherwise
// near-perfect shape -- the per-pose and per-point ratios agree to 4-5
// decimals -- which used to trip the absolute bounds only on the poses and
// points 8-10 m out, and only on the ill-conditioned default-weighting
// variants. Nothing about the shape was wrong.
const bool monoVariant = (variant == BaVariant::kDefault || variant == BaVariant::kNoLinks);
const bool monoOnlyBackend = (backend == Optimizer::kTypeCVSBA);
const bool monoBA = monoVariant || monoOnlyBackend;
float scale = 1.0f;
{
double num = 0.0;
double den = 0.0;
for(const auto & kv : g.truePoses)
{
if(kv.first == 1) continue;
if(!outPoses.count(kv.first)) continue;
const Transform truthRel = truthRootInv * kv.second;
const Transform outRel = outRootInv * outPoses.at(kv.first);
num += outRel.x()*truthRel.x() + outRel.y()*truthRel.y() + outRel.z()*truthRel.z();
den += outRel.x()*outRel.x() + outRel.y()*outRel.y() + outRel.z()*outRel.z();
}
if(den > 1e-12)
{
scale = static_cast<float>(num / den);
}
}
// Now that scale is factored out of the position checks, assert on it
// directly wherever it is observable -- otherwise dividing it out would
// silently discard a real failure mode (a broken baseline, a wrong Tx sign,
// or a bad disparity-to-depth conversion all show up as a scale error and
// nothing else). Mono BA has no scale to be wrong about, so it is exempt.
//
// 2% is loose next to the 0.5% worst case observed across platforms, but it
// is a flat bound on a quantity that should be exactly 1, and the failures
// it guards against are order-of-percent-to-2x, not fractions of a percent.
if(!monoBA)
{
EXPECT_NEAR(scale, 1.0f, 0.02f)
<< optimizerTypeName(backend) << " variant=" << (int)variant
<< " rounded=" << roundPixels
<< ": stereo/depth BA should pin scale, but the recovered geometry"
" is off by a global factor of " << scale;
}
// Recovered poses (relative to root). maxPoseDist and the radius of the
// pose that produced it are reported together below: these bounds are
// absolute metres applied to poses up to 10 m from the root, so the same
// bound is a much tighter relative constraint on the far poses than on the
// near ones, and the ratio is what tells a scale bias apart from noise.
float maxPoseDist = 0.0f;
float maxPoseDistRadius = 0.0f;
for(const auto & kv : g.truePoses)
{
const int id = kv.first;
if(id == 1) continue; // root is its own reference (delta = identity in both frames)
ASSERT_TRUE(outPoses.count(id));
const Transform truthRel = truthRootInv * kv.second;
Transform outRel = outRootInv * outPoses.at(id);
outRel.x() *= scale;
outRel.y() *= scale;
outRel.z() *= scale;
if(outRel.getDistance(truthRel) > maxPoseDist)
{
maxPoseDist = outRel.getDistance(truthRel);
maxPoseDistRadius = truthRel.getNorm();
}
EXPECT_LT(outRel.getDistance(truthRel), poseDistMax)
<< optimizerTypeName(backend) << " pose " << id
<< " got(rel)=" << outRel.prettyPrint()
<< " truth(rel)=" << truthRel.prettyPrint();
const float angDeg = outRel.getAngle(truthRel) * 180.0f / static_cast<float>(CV_PI);
EXPECT_LT(angDeg, poseAngMaxDeg)
<< optimizerTypeName(backend) << " pose " << id << " angle=" << angDeg << " deg";
}
// Recovered points (relative to root).
float maxPointDist = 0.0f;
for(const auto & kv : g.truePoints3D)
{
const int id = kv.first;
ASSERT_TRUE(outPoints.count(id));
const cv::Point3f truthRel = util3d::transformPoint(kv.second, truthRootInv);
cv::Point3f outRel = util3d::transformPoint(outPoints.at(id), outRootInv);
outRel.x *= scale;
outRel.y *= scale;
outRel.z *= scale;
const cv::Point3f diff = outRel - truthRel;
const float d = std::sqrt(diff.x*diff.x + diff.y*diff.y + diff.z*diff.z);
maxPointDist = std::max(maxPointDist, d);
EXPECT_LT(d, pointDistMax)
<< optimizerTypeName(backend) << " point " << id
<< " got(rel)=(" << outRel.x << "," << outRel.y << "," << outRel.z << ")"
<< " truth(rel)=(" << truthRel.x << "," << truthRel.y << "," << truthRel.z << ")";
}
std::cerr << "[bound] " << optimizerTypeName(backend) << " variant=" << (int)variant
<< " rounded=" << roundPixels << " maxPointDist=" << maxPointDist
<< " bound=" << pointDistMax
<< " | maxPoseDist=" << maxPoseDist << " bound=" << poseDistMax
<< " (worst pose is " << maxPoseDistRadius << " m from root, so "
<< (maxPoseDistRadius > 0.0f ? 100.0f*maxPoseDist/maxPoseDistRadius : 0.0f)
<< "% of its range)"
<< " | scale=" << scale << (monoBA ? " (gauge, unchecked)" : "")
<< "\n";
}
INSTANTIATE_TEST_SUITE_P(
BABackends,
BundleAdjustmentTest,
::testing::Values(
// Continuous-pixel observations (the test's original setup).
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kDefault, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kNoLinks, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepth, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepthNoLinks, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithLidarDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kDefault, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kNoLinks, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepth, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepthNoLinks, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithLidarDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kDefault, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kNoLinks, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepth, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepthNoLinks, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithLidarDepthNoLinksTuned, false),
std::make_tuple(Optimizer::kTypeCVSBA, BaVariant::kDefault, false),
// Same setups but with keypoints rounded to integer pixel
// coordinates (simulates a real detector).
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kDefault, true),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kNoLinks, true),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepth, true),
std::make_tuple(Optimizer::kTypeG2O, BaVariant::kWithDepthNoLinks, true),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kDefault, true),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kNoLinks, true),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepth, true),
std::make_tuple(Optimizer::kTypeGTSAM, BaVariant::kWithDepthNoLinks, true),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kDefault, true),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kNoLinks, true),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepth, true),
std::make_tuple(Optimizer::kTypeCeres, BaVariant::kWithDepthNoLinks, true),
std::make_tuple(Optimizer::kTypeCVSBA, BaVariant::kDefault, true)),
[](const ::testing::TestParamInfo<BaParam> & info)
{
std::string name = optimizerTypeName(std::get<0>(info.param));
name += "_";
name += baVariantName(std::get<1>(info.param));
if(std::get<2>(info.param)) name += "_Rounded";
return name;
});
// ---------------------------------------------------------------------------
// Regression test: wordReferences with negative ids. Memory.cpp assigns
// sequential negative ids (-1, -2, -3, ...) to features that aren't
// quantized into the visual vocabulary, and OdometryF2M forwards them
// straight into optimizeBA(). gtsam::Symbol packs the index into 56
// unsigned bits and used to overflow on these ("Symbol index is too
// large"); g2o remaps them via negVertexOffset; Ceres handles them too.
// Build the standard BA scenario, then negate every point id so the
// optimizer sees what F2M's wordReferences actually looks like in
// production.
// ---------------------------------------------------------------------------
class NegativeWordIdBaTest : public ::testing::TestWithParam<Optimizer::Type>
{
protected:
void SetUp() override
{
const Optimizer::Type t = GetParam();
if(!Optimizer::isAvailable(t))
{
GTEST_SKIP() << optimizerTypeName(t) << " not built in";
}
}
};
TEST_P(NegativeWordIdBaTest, OptimizeBaHandlesNegativeIds)
{
const Optimizer::Type backend = GetParam();
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kOptimizerIterations()] = "200";
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
BundleGraph g = buildBundleGraph(/*noisy=*/true, /*roundPixels=*/false);
// Remap every point id p -> -p in truePoints3D, initialPoints3D, and
// wordReferences. Pose ids stay positive — the bug only triggers on
// the landmark/feature side of the BA graph.
std::map<int, cv::Point3f> truePoints3D;
std::map<int, cv::Point3f> initialPoints3D;
std::map<int, std::map<int, FeatureBA>> wordReferences;
for(const auto & kv : g.truePoints3D) truePoints3D[-kv.first] = kv.second;
for(const auto & kv : g.initialPoints3D) initialPoints3D[-kv.first] = kv.second;
for(const auto & kv : g.wordReferences) wordReferences[-kv.first] = kv.second;
std::map<int, cv::Point3f> outPoints = initialPoints3D;
std::map<int, Transform> outPoses = opt->optimizeBA(
/*rootId=*/1, g.initialPoses, g.links, g.models, outPoints,
wordReferences);
ASSERT_FALSE(outPoses.empty())
<< optimizerTypeName(backend) << " optimizeBA returned no poses "
<< "with negative word ids";
ASSERT_EQ(outPoses.size(), g.truePoses.size());
// Sanity: optimized points should still match the (negated) ids in
// outPoints — i.e. the optimizer round-tripped the ids without
// dropping or aliasing them.
for(const auto & kv : truePoints3D)
{
EXPECT_TRUE(outPoints.count(kv.first))
<< optimizerTypeName(backend)
<< " dropped point id " << kv.first;
}
}
INSTANTIATE_TEST_SUITE_P(
BABackends,
NegativeWordIdBaTest,
::testing::Values(
Optimizer::kTypeG2O,
Optimizer::kTypeGTSAM,
Optimizer::kTypeCeres),
[](const ::testing::TestParamInfo<Optimizer::Type> & info)
{
return optimizerTypeName(info.param);
});
// ---------------------------------------------------------------------------
// PlanarBundleAdjustmentTest -- verifies that isSlam2d() in BA locks the
// recovered trajectory to its initial Z plane. g2o has supported this since
// forever via EdgeSBACamPrior; GTSAM and Ceres got matching planar
// constraints in this rev.
// ---------------------------------------------------------------------------
class PlanarBundleAdjustmentTest : public ::testing::TestWithParam<Optimizer::Type>
{
protected:
void SetUp() override
{
if(!Optimizer::isAvailable(GetParam()))
{
GTEST_SKIP() << optimizerTypeName(GetParam()) << " not built in";
}
}
};
TEST_P(PlanarBundleAdjustmentTest, RecoveredTrajectoryStaysOnInitialPlane)
{
const Optimizer::Type backend = GetParam();
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kOptimizerIterations()] = "200";
params[Parameters::kRegForce3DoF()] = "true"; // → isSlam2d() == true
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_TRUE(opt->isSlam2d());
BundleGraph g = buildBundleGraph(/*noisy=*/true, /*roundPixels=*/false);
// Planar test: initial poses reflect a real 2D-SLAM gauge -- the robot
// lives on a single horizontal plane at some non-zero height (z=0.5 m
// here, e.g. a camera mounted on a robot half a meter off the floor)
// with no roll/pitch. Strip the z/roll/pitch noise from the noisy
// initial poses and keep only the x/y/yaw noise. Using a non-zero
// reference z verifies the constraint locks to the INITIAL plane,
// not to z=0 by accident.
const float planeZ = 0.5f;
for(auto & kv : g.initialPoses)
{
float x, y, z, roll, pitch, yaw;
kv.second.getTranslationAndEulerAngles(x, y, z, roll, pitch, yaw);
kv.second = Transform(x, y, planeZ, 0.0f, 0.0f, yaw);
}
std::map<int, cv::Point3f> outPoints = g.initialPoints3D;
std::map<int, Transform> outPoses = opt->optimizeBA(
/*rootId=*/1, g.initialPoses, g.links, g.models, outPoints, g.wordReferences);
ASSERT_FALSE(outPoses.empty()) << "optimizeBA returned no poses";
ASSERT_EQ(outPoses.size(), g.truePoses.size());
// With the planar constraint locking every camera to its initial
// z = planeZ (weight 1e9 = sub-mm tolerance), no recovered pose
// should drift off the plane. Without the constraint, a 3D BA on
// this dataset can drift up to a few mm in z due to reprojection /
// chain residuals.
for(const auto & kv : outPoses)
{
EXPECT_NEAR(kv.second.z(), planeZ, 0.0005f) // 0.5 mm
<< optimizerTypeName(backend) << " pose " << kv.first
<< " drifted off the z=" << planeZ << " plane: z=" << kv.second.z();
}
}
INSTANTIATE_TEST_SUITE_P(
Backends,
PlanarBundleAdjustmentTest,
::testing::Values(Optimizer::kTypeG2O, Optimizer::kTypeGTSAM, Optimizer::kTypeCeres),
[](const ::testing::TestParamInfo<Optimizer::Type> & info)
{
return optimizerTypeName(info.param);
});
// ---------------------------------------------------------------------------
// MultiCamBundleAdjustmentTest -- 4 cameras at body origin pointing
// forward/left/backward/right, 30 poses on the same hilly 10 m circle as
// CircleGraphTest, random 3D points in a 20 m x 20 m x 1.5 m box. Each
// camera only "sees" points within 10 m of the body (avoids projecting
// across-trajectory points). Verifies that g2o and GTSAM multi-camera BA
// recovers the chain + point cloud. Ceres / CVSBA explicitly reject
// multi-cam (`models.size() > 1` in their optimizeBA) so they're not
// instantiated here.
// ---------------------------------------------------------------------------
namespace {
constexpr int kMultiCamNumPoses = 30;
constexpr float kMultiCamCircleRadius = 10.0f;
constexpr float kMultiCamHillAmp = 0.5f;
constexpr float kMultiCamHillFreq = 3.0f;
constexpr int kMultiCamNumPoints = 750;
constexpr float kMultiCamPointBoxXY = 20.0f;
constexpr float kMultiCamPointBoxZ = 1.5f;
constexpr float kMultiCamVisibilityRange = 10.0f;
constexpr int kMultiCamNumCameras = 4;
// Realistic robot body: vacuum-sized round chassis. Each side-facing
// camera sits on the rim at ±17.5 cm from the body center, on a small
// stalk 5 cm above the chassis plane. Opposite cameras are 35 cm apart
// (a real translational baseline that adds triangulation strength
// independent of the body's motion along the chain).
constexpr float kMultiCamRobotRadius = 0.175f;
constexpr float kMultiCamCameraZ = 0.05f;
// Narrow-FOV intrinsics: fx = 376 / tan(30°) ≈ 651 gives ~60° HFOV. With
// 4 cameras 90° apart this leaves ~15° gaps each side between neighbours
// -- no FOV overlap, so points are only ever observed by one camera per
// frame. Scale ambiguity returns (mono BA behavior) and the test has to
// fall back on scale-recovery + looser bounds. Used by the kNarrowFov
// variant below.
constexpr double kMultiCamNarrowFx = 651.0;
constexpr double kMultiCamNarrowFy = 651.0;
enum class MultiCamFov { kWide, kNarrow };
enum class MultiCamMode { kMono, kStereo };
enum class MultiCamLinks { kWithLinks, kNoLinks };
// Stereo baseline used when MultiCamMode::kStereo is set: 15 cm (e.g.
// ZED Mini / RealSense D435i tier). Each of the 4 cameras becomes its
// own stereo pair, so every observation carries a noisy disparity-derived
// depth in its FeatureBA (1 px sigma on the disparity channel).
constexpr double kMultiCamStereoBaseline = 0.15;
const char * multiCamFovName(MultiCamFov f)
{
return f == MultiCamFov::kWide ? "WideFov" : "NarrowFov";
}
const char * multiCamModeName(MultiCamMode m)
{
return m == MultiCamMode::kMono ? "Mono" : "Stereo";
}
const char * multiCamLinksName(MultiCamLinks l)
{
return l == MultiCamLinks::kWithLinks ? "WithLinks" : "NoLinks";
}
struct MultiCamBundleGraph
{
std::map<int, Transform> truePoses;
std::map<int, Transform> initialPoses;
std::map<int, cv::Point3f> truePoints3D;
std::map<int, cv::Point3f> initialPoints3D;
std::map<int, std::vector<CameraModel>> models;
std::multimap<int, Link> links;
std::map<int, std::map<int, FeatureBA>> wordReferences;
};
MultiCamBundleGraph buildMultiCamBundleGraph(MultiCamFov fov = MultiCamFov::kWide,
MultiCamMode mode = MultiCamMode::kMono)
{
MultiCamBundleGraph g;
const double fx = fov == MultiCamFov::kNarrow ? kMultiCamNarrowFx : kBaFx;
const double fy = fov == MultiCamFov::kNarrow ? kMultiCamNarrowFy : kBaFy;
// Negative Tx triggers the stereo-edge path in g2o / GTSAM BA; mono
// keeps it at 0 so depth observations are ignored.
const double Tx = mode == MultiCamMode::kStereo ? -kMultiCamStereoBaseline * fx : 0.0;
// 1) Hilly circular trajectory (matches buildCircleGraph's hill profile).
for(int i = 1; i <= kMultiCamNumPoses; ++i)
{
const float theta = 2.0f * static_cast<float>(CV_PI) * static_cast<float>(i - 1) / static_cast<float>(kMultiCamNumPoses);
const float x = kMultiCamCircleRadius * std::cos(theta);
const float y = kMultiCamCircleRadius * std::sin(theta);
const float z = kMultiCamHillAmp * std::sin(kMultiCamHillFreq * theta);
const float yaw = theta + static_cast<float>(CV_PI) / 2.0f; // tangent CCW
const float pitch = -kMultiCamHillAmp * kMultiCamHillFreq / kMultiCamCircleRadius
* std::cos(kMultiCamHillFreq * theta);
g.truePoses[i] = Transform(x, y, z, 0.0f, pitch, yaw);
}
// 2) Four cameras mounted on the round-robot rim pointing forward /
// left / backward / right. localTransform per camera =
// (rim offset in body frame) * (body-frame yaw) * opticalRotation
// so each camera's optical +Z aligns with the indicated body
// direction while sitting ~17.5 cm out from the body center.
const float R = kMultiCamRobotRadius;
const float Z = kMultiCamCameraZ;
const float multicamYaws[kMultiCamNumCameras] = {
0.0f, // 0: forward (body +X)
static_cast<float>(CV_PI) / 2.0f, // 1: left (body +Y)
static_cast<float>(CV_PI), // 2: back (body -X)
-static_cast<float>(CV_PI) / 2.0f // 3: right (body -Y)
};
const float multicamOffsetX[kMultiCamNumCameras] = { R, 0, -R, 0 };
const float multicamOffsetY[kMultiCamNumCameras] = { 0, R, 0, -R };
for(auto & kv : g.truePoses)
{
std::vector<CameraModel> rig;
for(int c = 0; c < kMultiCamNumCameras; ++c)
{
const Transform localTransform =
Transform(multicamOffsetX[c], multicamOffsetY[c], Z,
0.0f, 0.0f, multicamYaws[c])
* CameraModel::opticalRotation();
rig.emplace_back(fx, fy, kBaCx, kBaCy,
localTransform, Tx,
cv::Size(kBaImageWidth, kBaImageHeight));
}
g.models[kv.first] = std::move(rig);
}
// 3) Random points uniformly in a 20 m x 20 m x 1.5 m box. The box is
// wider than the 10 m trajectory radius, so points sit both inside
// the circle (mid-frame visibility) and in the outer annulus
// (back/side visibility from the nearer poses).
std::mt19937 rng(7);
std::uniform_real_distribution<float> distXY(-kMultiCamPointBoxXY, kMultiCamPointBoxXY);
std::uniform_real_distribution<float> distZ (-kMultiCamPointBoxZ, kMultiCamPointBoxZ);
for(int p = 1; p <= kMultiCamNumPoints; ++p)
{
const float px = distXY(rng);
const float py = distXY(rng);
const float pz = distZ(rng);
g.truePoints3D[p] = cv::Point3f(px, py, pz);
}
// 4) Project. Visibility filter: body-to-point distance < 10 m
// (suppresses projecting points from the far side of the circle
// through the robot), point in front of camera (z > 0.1 m), pixel
// in image bounds. In stereo mode each observation also gets a
// depth derived from a noisy disparity (1 px sigma) -- so depth
// error grows quadratically with range, matching a real stereo
// block matcher / SGM at long range.
std::mt19937 dispRng(13);
std::normal_distribution<double> dispNoise(0.0, 1.0);
for(const auto & ptkv : g.truePoints3D)
{
const int pointId = ptkv.first;
for(const auto & posekv : g.truePoses)
{
const int frameId = posekv.first;
const float dx = ptkv.second.x - posekv.second.x();
const float dy = ptkv.second.y - posekv.second.y();
const float dz = ptkv.second.z - posekv.second.z();
if(std::sqrt(dx*dx + dy*dy + dz*dz) > kMultiCamVisibilityRange)
{
continue;
}
for(int c = 0; c < kMultiCamNumCameras; ++c)
{
const CameraModel & m = g.models.at(frameId)[c];
const Transform world_to_optical = (posekv.second * m.localTransform()).inverse();
const cv::Point3f pc = util3d::transformPoint(ptkv.second, world_to_optical);
if(pc.z <= 0.1f)
{
continue;
}
float u, v;
m.reproject(pc.x, pc.y, pc.z, u, v);
if(u < 0.0f || u >= kBaImageWidth || v < 0.0f || v >= kBaImageHeight)
{
continue;
}
float depthForObs = 0.0f;
if(mode == MultiCamMode::kStereo)
{
// Realistic stereo depth: noisy disparity → depth via
// d = b*fx / disparity. 1 px sigma on disparity gives
// σ_depth ≈ z² / (b*fx) * σ_disp, so depth error
// grows quadratically with range.
const double dispTrue = kMultiCamStereoBaseline * fx / pc.z;
const double dispNoisy = dispTrue + dispNoise(dispRng);
depthForObs = static_cast<float>(
kMultiCamStereoBaseline * fx / std::max(dispNoisy, 0.1));
}
g.wordReferences[pointId].insert(
{frameId, FeatureBA(cv::KeyPoint(u, v, 1.0f), depthForObs, cv::Mat(), c)});
}
}
}
// 5) Neighbor links 1->2->...->N (no loop closure; the existing BA
// test scenes are open chains).
const cv::Mat info = cv::Mat::eye(6, 6, CV_64FC1);
for(int i = 1; i < kMultiCamNumPoses; ++i)
{
const Transform t = g.truePoses[i].inverse() * g.truePoses[i + 1];
g.links.insert({i, Link(i, i + 1, Link::kNeighbor, t, info)});
}
// 6) Noisy initial guess: 5 cm linear / 1 deg angular per pose (root
// held at truth), 5 cm linear per point.
std::mt19937 nRng(42);
std::normal_distribution<float> linNoise(0.0f, 0.05f);
std::normal_distribution<float> angNoise(0.0f, 1.0f * static_cast<float>(CV_PI) / 180.0f);
for(const auto & kv : g.truePoses)
{
if(kv.first == 1)
{
g.initialPoses[kv.first] = kv.second;
continue;
}
float x, y, z, roll, pitch, yaw;
kv.second.getTranslationAndEulerAngles(x, y, z, roll, pitch, yaw);
const float nx = x + linNoise(nRng);
const float ny = y + linNoise(nRng);
const float nz = z + linNoise(nRng);
const float nroll = roll + angNoise(nRng);
const float npitch = pitch + angNoise(nRng);
const float nyaw = yaw + angNoise(nRng);
g.initialPoses[kv.first] = Transform(nx, ny, nz, nroll, npitch, nyaw);
}
for(const auto & kv : g.truePoints3D)
{
const float ptx = kv.second.x + linNoise(nRng);
const float pty = kv.second.y + linNoise(nRng);
const float ptz = kv.second.z + linNoise(nRng);
g.initialPoints3D[kv.first] = cv::Point3f(ptx, pty, ptz);
}
return g;
}
} // namespace
using MultiCamParam = std::tuple<Optimizer::Type, MultiCamFov, MultiCamMode, MultiCamLinks>;
class MultiCamBundleAdjustmentTest : public ::testing::TestWithParam<MultiCamParam>
{
protected:
void SetUp() override
{
if(!Optimizer::isAvailable(std::get<0>(GetParam())))
{
GTEST_SKIP() << optimizerTypeName(std::get<0>(GetParam())) << " not built in";
}
}
};
TEST_P(MultiCamBundleAdjustmentTest, FourCameraRigRecoversPosesAndPoints)
{
const Optimizer::Type backend = std::get<0>(GetParam());
const MultiCamFov fov = std::get<1>(GetParam());
const MultiCamMode mode = std::get<2>(GetParam());
const MultiCamLinks links = std::get<3>(GetParam());
ParametersMap params;
params[Parameters::kOptimizerStrategy()] = uNumber2Str(static_cast<int>(backend));
params[Parameters::kOptimizerIterations()] = "200";
if(mode == MultiCamMode::kStereo)
{
// Sub-pixel u/v + ~1 px disparity tuning, same recipe as the
// kWithDepthNoLinksTuned single-cam BA variant. Default
// pv = dispVar = 1 falls into the info-matrix asymmetry trap at
// long range (10 m): observed u/v residuals are ~0 px but
// disparity has 1 px sigma; the optimizer over-trusts the
// noisier disparity channel and pushes points along the depth
// axis (multi-meter residuals). pv = 0.1 calibrates u/v as
// sub-pixel so both channels carry similar relative weight.
params[Parameters::kOptimizerPixelVariance()] = "0.1";
params[Parameters::kOptimizerDisparityVariance()] = "1.0";
}
std::unique_ptr<Optimizer> opt(Optimizer::create(params));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), backend);
MultiCamBundleGraph g = buildMultiCamBundleGraph(fov, mode);
if(links == MultiCamLinks::kNoLinks)
{
// Drop the pose-graph chain so BA has to recover the trajectory
// purely from reprojection (mono) or reprojection+disparity
// (stereo). For multi-cam mono this still converges to truth
// because the cross-camera, cross-frame triangulation pins
// scale; for stereo it's the disparity that anchors the gauge.
g.links.clear();
}
// Sanity: with 360° coverage + 10 m visibility, every point should be
// observed by many camera-frame pairs across the chain.
int totalObs = 0;
for(const auto & wkv : g.wordReferences) totalObs += static_cast<int>(wkv.second.size());
// Narrow FOV sees ~half the area of wide FOV per camera, so we relax
// the sanity floor accordingly.
const int minObs = fov == MultiCamFov::kWide ? 3000 : 1500;
ASSERT_GT(totalObs, minObs) << "Too few observations: " << totalObs;
std::map<int, cv::Point3f> outPoints = g.initialPoints3D;
std::map<int, Transform> outPoses = opt->optimizeBA(
/*rootId=*/1, g.initialPoses, g.links, g.models, outPoints, g.wordReferences);
ASSERT_FALSE(outPoses.empty()) << "optimizeBA returned no poses";
ASSERT_EQ(outPoses.size(), g.truePoses.size());
// Multi-camera BA is fully observable in BOTH FOV regimes -- even when
// adjacent cameras' fields of view don't overlap (narrow mode), the
// chain of 30 poses with rigidly-attached 4-camera rigs is enough to
// pin scale: a point first observed by the forward camera at pose K
// is observed by the left camera at some later pose K+n once the body
// has yawed, providing cross-camera triangulation across frames. No
// scale recovery needed -- both backends converge to µm-scale pose
// residuals against truth in MONO mode. STEREO loosens to ~2 cm
// because 1-px disparity noise on the 15 cm baseline propagates
// quadratically with range (σ_depth ≈ z²/(b·fx)·σ_disp = ~2 m at
// 10 m range per observation) and biases the pose estimates.
const Transform truthRootInv = g.truePoses.at(1).inverse();
const Transform outRootInv = outPoses.at(1).inverse();
// Stereo loosens to ~2 cm regardless of chain. Mono converges to µm
// with the chain anchor; without links the BA has to lock the gauge
// purely from reprojection, which costs a few mm of pose precision.
float poseBound;
if(mode == MultiCamMode::kStereo) poseBound = 0.03f;
else if(links == MultiCamLinks::kNoLinks) poseBound = 0.005f;
else poseBound = 0.001f;
const float angBound = mode == MultiCamMode::kStereo ? 0.05f : 0.01f;
for(const auto & kv : g.truePoses)
{
if(kv.first == 1) continue;
ASSERT_TRUE(outPoses.count(kv.first));
const Transform truthRel = truthRootInv * kv.second;
const Transform outRel = outRootInv * outPoses.at(kv.first);
EXPECT_LT(outRel.getDistance(truthRel), poseBound)
<< optimizerTypeName(backend) << " pose " << kv.first
<< " got(rel)=" << outRel.prettyPrint()
<< " truth(rel)=" << truthRel.prettyPrint();
const float angDeg = outRel.getAngle(truthRel) * 180.0f / static_cast<float>(CV_PI);
EXPECT_LT(angDeg, angBound)
<< optimizerTypeName(backend) << " pose " << kv.first << " angle=" << angDeg << " deg";
}
int verifiedPoints = 0;
for(const auto & kv : g.truePoints3D)
{
// Only check points that are well-triangulated -- the random
// scatter includes far-corner points that no camera sees within
// the 10 m visibility cap (NaN-filled by g2o / left at noisy
// initial by GTSAM), plus single-observation points that BA
// can't constrain in any meaningful way (mono: stays at noisy
// initial; stereo: ~σ_depth ≈ z²/(b·fx)·σ_disp ≈ 8 m at 20 m
// range from one disparity sample). Require ≥3 obs.
const auto wkv = g.wordReferences.find(kv.first);
if(wkv == g.wordReferences.end() || wkv->second.size() < 3) continue;
if(!outPoints.count(kv.first)) continue;
const cv::Point3f outRaw = outPoints.at(kv.first);
if(!std::isfinite(outRaw.x) || !std::isfinite(outRaw.y) || !std::isfinite(outRaw.z)) continue;
const cv::Point3f truthRel = util3d::transformPoint(kv.second, truthRootInv);
const cv::Point3f outRel = util3d::transformPoint(outRaw, outRootInv);
const cv::Point3f diff = outRel - truthRel;
const float d = std::sqrt(diff.x*diff.x + diff.y*diff.y + diff.z*diff.z);
// Point residuals are dominated by triangulation noise at the
// edge of the visibility cone (~10 m from the observing pose).
// Mono Wide: ~9 cm worst; Mono Narrow: ~11 cm worst (slightly
// weaker triangulation -- no simultaneous overlap means each
// point is effectively reconstructed from inter-frame baselines
// only). Stereo direct depth tightens both.
float pointBound;
if(mode == MultiCamMode::kStereo)
{
pointBound = fov == MultiCamFov::kWide ? 0.08f : 0.10f;
}
else
{
pointBound = fov == MultiCamFov::kWide ? 0.10f : 0.12f;
}
EXPECT_LT(d, pointBound)
<< optimizerTypeName(backend) << " point " << kv.first
<< " got(rel)=(" << outRel.x << "," << outRel.y << "," << outRel.z << ")"
<< " truth(rel)=(" << truthRel.x << "," << truthRel.y << "," << truthRel.z << ")";
++verifiedPoints;
}
// Sanity that we're actually verifying something -- if the visibility
// filter is too aggressive, we'd silently pass with no point checks.
EXPECT_GT(verifiedPoints, 200);
}
INSTANTIATE_TEST_SUITE_P(
Backends,
MultiCamBundleAdjustmentTest,
// g2o, GTSAM, and Ceres all support multi-camera BA. CVSBA
// doesn't (its underlying Sba::run() API is single-camera).
// Cross every FOV regime (wide 100° / narrow 60°) with mono /
// stereo (15 cm baseline + noisy disparity-derived depth) with
// or without the pose-graph chain. 24 variants total.
::testing::Combine(
::testing::Values(Optimizer::kTypeG2O, Optimizer::kTypeGTSAM, Optimizer::kTypeCeres),
::testing::Values(MultiCamFov::kWide, MultiCamFov::kNarrow),
::testing::Values(MultiCamMode::kMono, MultiCamMode::kStereo),
::testing::Values(MultiCamLinks::kWithLinks, MultiCamLinks::kNoLinks)),
[](const ::testing::TestParamInfo<MultiCamParam> & info)
{
return std::string(optimizerTypeName(std::get<0>(info.param))) + "_"
+ multiCamFovName(std::get<1>(info.param)) + "_"
+ multiCamModeName(std::get<2>(info.param)) + "_"
+ multiCamLinksName(std::get<3>(info.param));
});
TEST(OptimizerTest, CvsbaPoseGraphOptimizeReturnsEmpty)
{
// CVSBA only overrides optimizeBA() -- it doesn't implement pose-graph
// optimize(). Calling optimize() on a CVSBA instance hits the base
// Optimizer::optimize() which logs an error and returns an empty map.
// Pin that contract so callers can safely fall back.
if(!Optimizer::isAvailable(Optimizer::kTypeCVSBA))
{
GTEST_SKIP() << "CVSBA not built in";
}
std::unique_ptr<Optimizer> opt(Optimizer::create(Optimizer::kTypeCVSBA));
ASSERT_NE(opt.get(), nullptr);
ASSERT_EQ(opt->type(), Optimizer::kTypeCVSBA);
std::map<int, Transform> in;
std::multimap<int, Link> inLinks;
makeChain3(in, inLinks);
std::map<int, Transform> out = opt->optimize(1, in, inLinks);
EXPECT_TRUE(out.empty()) << "CVSBA should not handle pose-graph optimize()";
}