Adding doc and tests (#1492)

* added doc and tests for util2d.h

* updated cmake-ros ci

* Added util3d.h doc and tests

* util3d_transforms.h: Added doc and tests

* util3d_filtering.h: started doc and test

* util3d_filtering.h: more tests and doc

* Added more doc/tests

* finished util3d_filtering doc and tests

* added test for util2d::depthBleedingFiltering

* Added util3d_registration tests

* Added util3d_features.h doc/tests

* added doc/tests for util3d_correspondences.h

* added doc/gtest for util3d_mapping.h (missing hpp functions)

* finished testing util3d_mapping.hpp

* Added util3d_motion_estimation.h tests (2D->3D done)

* finished util3d_motion_estimation.h tests

* minimal util3d_surface.h

* Added Transform and VisualWord tests

* Added doc for CameraModel and StereoCameraModel

* Added more logs in ros ci

* Passing tests on fical

* improved all devcontainer

* added devcontainer kilted, fixed source setup.bash, removed ldconfig in ros-cmake workflow

* cleanup

* source ros

* Added utilite tests

* Added testing to appveyor, github actions cancellable on re-commit on same branch

* appveyor testing without all targets

* appveyor: specifying ALL_BUILD target

* Fixed Util2dTest.NMSImageBoundsRespected test

* Fixing PCL Indices error on old pcl

* Added VWDictionary tests and doc. Fixed LSH not working (fix from https://github.com/flann-lib/flann/pull/472

* fixing some appveyor CI errors, added test to check dictionary serialization against all type

* Added StereoDense, StereoBM and StereoSGBM doc and tests

* Added Stereo tests

* Added CameraModel and StereoCameraModel tests

* Added doc and test for Statistics

* Added doc/tests for Signature

* Added doc/test for SensorEvent, added doc for SensorCaptureInfo

* Added doc to SensorData

* Added SensorData tests

* Added SensorCapture and SensorCaptureThread doc and tests

* fixed sensordata test

* updated SSC test and doc

* Added doc and tests for BayesFilter class

* Enabled testing on mac, updated windows testing like on linux

* added test_link

* fixed unresolved on windows

* fixed ThreadHandle error on macos ci

* Added GPS and GeodeticCoords tests

* Added tests for compression

* Added Odometry tests (base class only)

* Added DBDriver tests

* Added coverage report

* uniformized test names

* fixing concurancy and coverage ci

* dont built tools, examples and app for coverage build

* fixed report tool rebuilt without qt compilation error

* updated coverage option

* updated coverage config

* added doc CI job

* fixing windows and mac ci errors

* Added DBDriverSqlite3 tests

* Added IMU tests

* Added Graph tests

* fixing flaky macos test

* Added IMUThread and IMUFilter tests

* Added Landmarks tests

* Added LASWriter tests

* fixing seed flaky test

* fixing flaky macos timing tests

* Added LocalGrid tests

* Added LocalGridMaker tests

* fixing ci errors

* Added GlobalMap tests

* Added doc for EnvSensor

* Added Features2D tests

* Added Registration tests

* Added RegistrationVis tests

* Added doc for Rtabmap and Memory classes

* Added Memory and Rtabmap tests

* making some tests less flaky

* lcov 1.14 support

* updated compatible tool arguments

* Added integration tests (RGB-D, Stereo, Lidar2d, Lidar3d)

* More octomap checks

* Refactored how/when python interpretor is created to simplify library usage

* Added python tests

* fixed some flaky tests

* suppressed some third party related warnings

* fixed ceres tests

* more flaky fixes

* Fixing tests without libpointmatcher

* Added RANSAC rejection filter to PCL ICP

* fixing multi platform flakiness

* Added test to detect regression

* Fixing windows pcl link error

* fixed some macos flakiness

* bigger 2D2D registration error on opencv 4.6.0

* flakiness

* fixing flaky tests on windows and mac

* flaky thread test on slow mac VM

* windows slow test

* fixing more ci erros

* fxing temp dir on windows

* Added Optimizer tests and discovered some bugs (fixed)

* fixing flaky tests in mac and windows

* Added Optimizer doc

* Added GTSAM BA, updated Ceres to use g2o ba parameters. Renamed g2o's ba related parameters to Optimizer group and used by both gtsam and ceres.

* fixing build without gtsam

* fixing home dir

* fixing python ci isssues

* Added multicam ba tests

* Added Ceres multicam BA support

* Aligned BundleAdjustment parameters with Optimizer/Strategy to avoid confusion in the code

* Added BA integration test

* Added robust graph optimization integration test

* Added loop3it test

* Added stereo20Hz test

* Added smartfactor gtsam

* Fixed bugged check and warn if python didn't return any descriptors

* Fixing gtsam version build issues

* fixing tilt on windows ci

* loosing ceres integration test for ci

* mac ci flakiness

* updating missing param in gui

* updating test bound for mac

* added appearance-based tests, set min gftt quality to quality level

* testing more stuff

* improving features2d tests

* ci flakiness

* fixing flaky ci

* ci fixes

* flaky fixes

* Added RegistrationIcp tests

* Added icp integration test with real-worl corridor like env

* intermediate nodes

* fixing enum

* Updated test to catch #1714

* Fixed 2d corridor failing on pcl

* flaky pnp test

* flaky brisk test

* Set rtabmap_integration test as long

* updating loop closure test

* flaky ci tests

* TEsting roundtrip g2o/toro save/load

* loosing test bound

* fixed cuda capable checks

* flaky tests

* Debugging test hanging

* more debugging stuff

* updating limit

* windows: disabled cuda on ci to avoid incompatible driver issue. Fixing a bad test mem allocation

* trying fixing cuda hanging issue

* fixing ci flakyness

* flaky tests

* Updated BOW flaky tests by checking min precision/recall instead of recall@100precision. Fixed signature test

* CameraModel::load() test initRectificationMap param

* test dbdriver load dictionary idsOnly

* Memory: test keepLinkedInDb param

* added dummyDictionary tests

* test intermediate nodes count

* Added MarkerDetector tests

* reverted breaking change of UMutex and USemaphore

* Features2d: fixed compiltion warnings with clang about override

* clang warnings

* fixing test build with pcl 1.8

* g2o and gtsam build errors on android

* opencv5 test fixes

* disabled testing for ios and android builds

* normalized endline characters for easier diff

* added LF CRLF rule

* bump 0.23.10. fixing doc version

* Publish rtabmap website doc from ci

* fixing MSCVC build error

* macos icp flaky test

* fixing ceres macos test bound

* ficing more flaky tests

* fixing opencv5 related test errors. Also fixed an actual bug in ENU_WGS84ToGeocentric_WGS84()

* added comment about mrpt change

* removed rosdoc2 (will add it for rtabmap_ros later)

* fixing website style

* updated download links

* locally deployable website with api

* sweep doxygen issues

* improved/revised doxygen main pages

* removed examples empty page

* Updated doxygen style

* more concise doxygen groups

* added api link on main readme

* fixing utilite test error

* fixing CommonFilteringGroundNormalsUp test

* updated precisionRecall test bounds for Freak and brief descriptors

* fixing scale check in ba tests

* disabled tests on windows cuda build (missing dlls amd runner cannot test cuda anyway)

* ceres: missing suitesparse dep in windows ci

* adjusting recall thr for fast/freak

* ficing more flaky tests

* fixing flaky tests

* disabled coverage in ros ci

* Enable integration tests for ros ci jobs

* loosing up some threshold for failing tests

* trigger cache

* fixing test data in ros ci. Updated flaky test for mac

* slaking some test limit

* Fixed rtabmap-detectMoreLoopClosures inverted output value

* loosing up sift recall on mac

* optimizer re-ordered distribution for reproducible results (mac g2o)

* macos dump test crash log

* combining all tests to save time on shared library reload. Also fixed Logs with missing arguments.

* Added ENABLE_FORMAT_ERRORS cmake option

* do test only one time

* fixed all format warnings

* format security android build errors

* less verbose tests

* updated ImuUThread test

* fixed a log

* Fixed libpointmatcher 2d normals eigen issue

* Fixing libpointmatcher conversion issues

* fixing libpointmatcher test on windows ci

* cleanup comments, relax some test thr

* disabled sequoia-intel ci build (too flaky, would need extensive testing directly on that machine)

corelib/src/VWDictionary.cpp

@@ -138,7 +138,7 @@ void VWDictionary::setIncrementalDictionary()
 		_incrementalDictionary = true;
 		if(_visualWords.size())
 		{
-			UWARN("Incremental dictionary set: already loaded visual words (%d) from the fixed dictionary will be included in the incremental one.", _visualWords.size());
+			UWARN("Incremental dictionary set: already loaded visual words (%d) from the fixed dictionary will be included in the incremental one.", (int)_visualWords.size());
 		}
 	}
 	_dictionaryPath = "";
@@ -259,7 +259,7 @@ void VWDictionary::setFixedDictionary(const std::string & dictionaryPath)
 			if(_visualWords.size() == 0)
 			{
 				_incrementalDictionary = _visualWords.size()==0;
-				UWARN("No words loaded, cannot set a fixed dictionary.", (int)_visualWords.size());
+				UWARN("No words loaded, cannot set a fixed dictionary.");
 			}
 			else
 			{
@@ -276,7 +276,7 @@ void VWDictionary::setFixedDictionary(const std::string & dictionaryPath)
 		}
 		else
 		{
-			UERROR("Cannot change to a fixed dictionary if there are already words (%d) in the incremental one.", _visualWords.size());
+			UERROR("Cannot change to a fixed dictionary if there are already words (%d) in the incremental one.", (int)_visualWords.size());
 		}
 	}
 	else if(_incrementalDictionary && _visualWords.size())
@@ -300,7 +300,7 @@ bool VWDictionary::setNNStrategy(NNStrategy strategy)
 {
 #if CV_MAJOR_VERSION < 3
 #ifdef HAVE_OPENCV_GPU
-	if(strategy == kNNBruteForceGPU && !cv::gpu::getCudaEnabledDeviceCount())
+	if(strategy == kNNBruteForceGPU && cv::gpu::getCudaEnabledDeviceCount() <= 0)
 	{
 		UERROR("Nearest neighobr strategy \"kNNBruteForceGPU\" chosen but no CUDA devices found! Doing \"kNNBruteForce\" instead.");
 		strategy = kNNBruteForce;
@@ -314,7 +314,7 @@ bool VWDictionary::setNNStrategy(NNStrategy strategy)
 #endif
 #else
 #ifdef HAVE_OPENCV_CUDAFEATURES2D
-	if(strategy == kNNBruteForceGPU && !cv::cuda::getCudaEnabledDeviceCount())
+	if(strategy == kNNBruteForceGPU && cv::cuda::getCudaEnabledDeviceCount() <= 0)
 	{
 		UERROR("Nearest neighobr strategy \"kNNBruteForceGPU\" chosen but no CUDA devices found! Doing \"kNNBruteForce\" instead.");
 		strategy = kNNBruteForce;
@@ -330,7 +330,7 @@ bool VWDictionary::setNNStrategy(NNStrategy strategy)
 
 	if(strategy>=kNNUndef)
 	{
-		UERROR("Nearest neighobr strategy \"%d\" chosen but this strategy cannot be used with a dictionary! Doing \"kNNBruteForce\" instead.");
+		UERROR("Nearest neighbor strategy \"%d\" chosen but this strategy cannot be used with a dictionary! Doing \"kNNBruteForce\" instead.", (int)strategy);
 		strategy = kNNBruteForce;
 	}
 
@@ -377,7 +377,10 @@ unsigned long VWDictionary::getMemoryUsed() const
 {
 	long memoryUsage = sizeof(VWDictionary);
 	memoryUsage += getIndexMemoryUsed();
-	memoryUsage += _dataTree.total()*_dataTree.elemSize();
+	if(!_dataTree.empty())
+	{
+		memoryUsage += _dataTree.total()*_dataTree.elemSize();
+	}
 	if(!_visualWords.empty())
 	{
 		memoryUsage += _visualWords.size()*(sizeof(int) + _visualWords.rbegin()->second->getMemoryUsed() + sizeof(std::map<int, VisualWord *>::iterator)) + sizeof(std::map<int, VisualWord *>);
@@ -516,7 +519,7 @@ void VWDictionary::update()
 			{
 				UTimer timer;
 				timer.start();
-				ULOGGER_DEBUG("Incremental FLANN: Inserting %d words...", (int)_notIndexedWords.size(), _byteToFloat?"true":"false");
+				ULOGGER_DEBUG("Incremental FLANN: Inserting %d words (byteToFloat=%s)...", (int)_notIndexedWords.size(), _byteToFloat?"true":"false");
 				for(std::set<int>::iterator iter=_notIndexedWords.begin(); iter!=_notIndexedWords.end(); ++iter)
 				{
 					VisualWord* w = uValue(_visualWords, *iter, (VisualWord*)0);
@@ -675,21 +678,24 @@ void VWDictionary::update()
 					_mapIdIndex.insert(_mapIdIndex.end(), std::pair<int, int>(iter->second->id(), i));
 				}
 
-				ULOGGER_DEBUG("_mapIndexId.size() = %d, words.size()=%d, _dim=%d",_mapIndexId.size(), _visualWords.size(), dim);
+				ULOGGER_DEBUG("_mapIndexId.size() = %d, words.size()=%d, _dim=%d",(int)_mapIndexId.size(), (int)_visualWords.size(), dim);
 				ULOGGER_DEBUG("copying data = %f s", timer.ticks());
 
-				_flannIndex->buildIndex(
-					_strategy == kNNFlannNaive ? FlannIndex::FLANN_INDEX_LINEAR:
-						_strategy == kNNFlannLSH ? FlannIndex::FLANN_INDEX_LSH:
-							FlannIndex::FLANN_INDEX_KDTREE, // kNNFlannKdTree
-					_dataTree,
-					useDistanceL1_,
-					_incrementalDictionary&&_incrementalFlann?_rebalancingFactor:1);
-				ULOGGER_DEBUG("Time to create kd tree = %f s", timer.ticks());
+				if(_strategy < kNNBruteForce)
+				{
+					_flannIndex->buildIndex(
+						_strategy == kNNFlannNaive ? FlannIndex::FLANN_INDEX_LINEAR:
+							_strategy == kNNFlannLSH ? FlannIndex::FLANN_INDEX_LSH:
+								FlannIndex::FLANN_INDEX_KDTREE, // kNNFlannKdTree
+						_dataTree,
+						useDistanceL1_,
+						_incrementalDictionary&&_incrementalFlann?_rebalancingFactor:1);
+					ULOGGER_DEBUG("Time to create kd tree = %f s", timer.ticks());
+				}
 			}
 		}
 		UDEBUG("Dictionary updated! (size=%d added=%d removed=%d)",
-				_dataTree.rows, _notIndexedWords.size(), _removedIndexedWords.size());
+				_dataTree.rows, (int)_notIndexedWords.size(), (int)_removedIndexedWords.size());
 	}
 	else
 	{
@@ -714,38 +720,38 @@ std::vector<unsigned char> VWDictionary::serializeIndex() const
 	return _flannIndex->serializeIndex(_serializeWithChecksum);
 }
 
-void VWDictionary::deserializeIndex(const std::vector<unsigned char> & data)
+bool VWDictionary::deserializeIndex(const std::vector<unsigned char> & data)
 {
-	deserializeIndex(data.data(), data.size());
+	return deserializeIndex(data.data(), data.size());
 }
 
-void VWDictionary::deserializeIndex(const unsigned char * data, size_t size)
+bool VWDictionary::deserializeIndex(const unsigned char * data, size_t size)
 {
 	if(data== NULL || size == 0)
 	{
 		UWARN("Trying to deserialize empty data, aborting.");
-		return;
+		return false;
 	}
 	UDEBUG("Loading flann index... (data size=%ld bytes)", size);
 	if(_strategy >= kNNBruteForce) {
 		//ignore
-		return;
+		return false;
 	}
 
 	if(_flannIndex->isBuilt()) {
 		UERROR("Flann index is already built, cannot deserialize data!");
-		return;
+		return false;
 	}
 
 	if(_visualWords.empty()) {
 		UERROR("Descriptors should be added before deserializing flann index! See VWDictionary::addWord()");
-		return;
+		return false;
 	}
 
 	if(!(_removedIndexedWords.empty() && _visualWords.size() == _notIndexedWords.size())) {
 		UERROR("State of dictionary not as expected before deserializing. (removed words=%ld, words=%ld, not indexed=%ld)", 
 			_removedIndexedWords.size(), _visualWords.size(), _notIndexedWords.size());
-		return;
+		return false;
 	}
 		
 	std::map<int, int> mapIndexId;
@@ -811,7 +817,7 @@ void VWDictionary::deserializeIndex(const unsigned char * data, size_t size)
 		mapIdIndex.insert(mapIdIndex.end(), std::pair<int, int>(iter->second->id(), i));
 	}
 
-	ULOGGER_DEBUG("mapIndexId.size() = %d, words.size()=%d, dim=%d", mapIndexId.size(), _visualWords.size(), dim);
+	ULOGGER_DEBUG("mapIndexId.size() = %d, words.size()=%d, dim=%d", (int)mapIndexId.size(), (int)_visualWords.size(), dim);
 	ULOGGER_DEBUG("copying data = %f s", timer.ticks());
 
 	std::string errorMsg;
@@ -835,9 +841,11 @@ void VWDictionary::deserializeIndex(const unsigned char * data, size_t size)
 	else {
 		UWARN("Failed deserializing flann index data (error: %s), the index will be rebuilt on next update.", errorMsg.c_str());
 		_flannIndex->release(); // reset to initial state
+		return false;
 	}
 
 	ULOGGER_DEBUG("Time to load flann index = %f s", timer.ticks());
+	return true;
 }
 
 void VWDictionary::clear(bool printWarningsIfNotEmpty)
@@ -1094,14 +1102,9 @@ std::list<int> VWDictionary::addNewWords(
 			for(int j=0; j<dists.cols; ++j)
 			{
 				float d = dists.at<float>(i,j);
-				int index;
-				if (sizeof(size_t) == 8)
-				{
-					index = *((size_t*)&results.at<double>(i, j));
-				}
-				else
-				{
-					index = *((size_t*)&results.at<int>(i, j));
+				int index = results.at<int>(i, j);
+				if(index<0) {
+					continue;
 				}
 				int id = uValue(_mapIndexId, index);
 				if(d >= 0.0f && id != 0)
@@ -1219,7 +1222,7 @@ std::list<int> VWDictionary::addNewWords(
 	}
 	ULOGGER_DEBUG("naive search and add ref/words time = %f s", timerLocal.ticks());
 
-	ULOGGER_DEBUG("%d new words added...", _notIndexedWords.size());
+	ULOGGER_DEBUG("%d new words added...", (int)_notIndexedWords.size());
 	ULOGGER_DEBUG("%d duplicated words added (from current image = %d)...",
 			dupWordsCountFromDict+dupWordsCountFromLast, dupWordsCountFromLast);
 	UDEBUG("total time %fs", timer.ticks());
@@ -1459,15 +1462,9 @@ std::vector<int> VWDictionary::findNN(const cv::Mat & queryIn) const
 				for(int j=0; j<dists.cols; ++j)
 				{
 					float d = dists.at<float>(i,j);
-					int index;
-
-					if (sizeof(size_t) == 8)
-					{
-						index = *((size_t*)&results.at<double>(i, j));
-					}
-					else
-					{
-						index = *((size_t*)&results.at<int>(i, j));
+					int index = results.at<int>(i, j);
+					if(index < 0) {
+						continue;
 					}
 					int id = uValue(_mapIndexId, index);
 					if(d >= 0.0f && id != 0)
@@ -1656,7 +1653,7 @@ void VWDictionary::exportDictionary(const char * fileNameReferences, const char
 		}
 	}
 
-	UDEBUG("Export %d words...", _visualWords.size());
+	UDEBUG("Export %d words...", (int)_visualWords.size());
     for(std::map<int, VisualWord *>::const_iterator iter=_visualWords.begin(); iter!=_visualWords.end(); ++iter)
     {
     	// References