Rebuild my 3D world [open source] [serialization-3] [comparison between colmap and openmvg]

stay Rebuild my 3D The world Series of Second articles , I mentioned about SFM Part is choosing Colmap still OpenMVG The problem of , And decided to practice the truth .Colmap The official website provides a test program with an interface , It's very convenient to use , I downloaded. 3.6 edition , The address is colmap 3.6.
and OpenMVG It can be compiled easily , It does not provide programs with interfaces , There are console programs that can be tested , The results can be imported Colmap Visualization in the interface program .

0. Preface

This test is generally a shallow test , I simply reconstruct the visual effect from the scene 、 Reconstruction efficiency 、 Reprojection error 、 Image pose restoration integrity, etc . In fact, some more professional evaluation indicators , Such as the difference between the point cloud or control point of the scene and the true value 、 The difference between pose and true value , Not evaluated in this experiment , So I'm sorry to remind you that this is not a professional test , The final result may not be so professional or accurate , Please forgive me. .

1. Data selection

After the procedures are deployed , The first step is to select test data , Here I choose according to the type of data 6 Group data , Not a lot of data （ Not more than 1000 Zhang , The amount of data is too large, and the test is too time-consuming ）, The scenes are different , There are farmland and buildings ; There are outdoor and indoor ; There are drones and digital cameras .

What needs to be added is , In order to fit our theme , I just hope to use some simple equipment to rebuild the surrounding 3D The world , So I also took two sets of mobile phone photo collections , To test the SFM The effect of , One is a running shoe , The second is a dinosaur model .

Below is an overview of all the data ：

Data name	Number of images	Size	Camera type	Scene description	Yes no GPS
Small Buildings	36	4592x3056	Unmanned aerial vehicle (uav) （AscTecFalcon 8）	Architecture	Yes
Urban	113	5472x3648	Unmanned aerial vehicle (uav) （senseFly）	Farmland , Architecture , Naked land	Yes
Person-hall	330	5616x3744	Digital camera （Canon）	Architecture 、 tree 、 Close up	nothing
Graham-hall	711	5316x3744	Digital camera （Canon）	indoor , Close up	nothing
Shoe	42	4032x3024	mobile phone （ samsung S20+）	Articles for daily use	nothing
Dinosaur	55	4032x3024	mobile phone （ samsung S20+）	Model	nothing

Small Buildings	Urban
Person-hall	Graham-hall
Shoe	Dinosaur

2. Test environment

All tests are conducted on my desktop , The configuration is as follows

platform	model
CPU	Intel I7-8700K 3.7GHz
GPU	NVIDIA GTX 1080 8G

3. Parameter setting

Colmap All use default parameters , Small vocabulary trees are used in feature matching , Applicable to less than 1000 Image set of . Download address ：vocab_tree_flickr100K_words32K

OpenMVG All use default parameters , When the sensor model cannot be recognized , Provide initial focal length $\max (w,h)\ast 1.2$. Please refer to ：openMVG softwares & tools. And use the generated results OpenMVG Turn your own tools into Colmap Of SFM Format in Colmap The program shows .

Both use Incremental SFM , And they are not used GPS auxiliary SFM, Although some data have bands GPS And OpenMVG Provides assistance GPS Of SFM, But because of Colmap There is no module , So to be fair , They are not used in the test .

In order to verify , Some test data will be used in the middle Agisoft Metashape Run again ,Agisoft Metashape The predecessor was Agisoft Photoscan, With SFM Grow perceptibly .

4. test result

1 Small Buildings 36p 4592x3056

Colmap

step	Time ( branch )
feature extraction Feature Extraction	0.094
Feature matching Feature Matching	1.723
Beam adjustment Bundle adjustment	0.996
SFM Total time	2.813
Re projection error average	0.89
Number of reconstructed images / Sparse points	36 / 20972

OpenMVG

step	Time ( branch )
feature extraction Feature Extraction	1.867
Feature matching Feature Matching	0.583
Beam adjustment Bundle adjustment	1.067
SFM Total time	3.517
Re projection error average	0.28
Number of reconstructed images / Sparse points	36 / 69791

There is not much difference between the two figures ,OpenMVG Reconstruction efficiency is slightly lower , More sparse points are generated , The average re projection error is also smaller .

2 Urban 113p 5472x3648

Colmap

step	Time ( branch )
feature extraction 【Feature Extraction】	1.195
Feature matching 【Feature Matching】	7.414
Beam adjustment Bundle adjustment	2.898
SFM Total time	11.507
Re projection error average	1.07
Number of reconstructed images / Sparse points	113 / 73691

The scene looks a little crooked , In order to verify , I use commercial software Agisoft Metashape Also ran once , Time consuming 102.344 second （1.706 branch ）, Very efficient , Reconstruction image 113 Zhang , common 173095 Sparse , give the result as follows ：

The scene is flat , It can be judged Colmap The reconstructed scene is a little drifting （Drift） Of . But it should be noted that ,Metashape There is assistance GPS, and Colmap not used GPS auxiliary .

OpenMVG

step	Time ( branch )
feature extraction Feature Extraction	8.200
Feature matching Feature Matching	23.13
Beam adjustment Bundle adjustment	12.13
SFM Total time	43.46
Re projection error average	0.37
Number of reconstructed images / Sparse points	113/ 408068

OpenMVG The reconstructed scene also has a bending trend , The degree of curvature looks better than Colmap It will be better , But it didn't reach Metashape The level of （ Whereas Metashape It was used GPS Auxiliary and OpenMVG No, , This result is acceptable ）.

OpenMVG The reconstruction time is significantly longer than Colmap Of （ near 4 times ）, Even though the algorithm has used multithreading ,CPU Also basically in a busy state .

3 Person-hall 330p 5616x3744

Colmap

step	Time ( branch )
feature extraction 【Feature Extraction】	1.169
Feature matching 【Feature Matching】	36.197
Beam adjustment Bundle adjustment	22.108
SFM Total time	59.474
Re projection error average	0.75
Number of reconstructed images / Sparse points	330 / 179134

The effect looks good , The structure recovery is relatively regular . We also use Agisoft Metashape Run it over , Because the photo itself does not GPS, So let's see if GPS when Metashape How are they performing? ：

Equally robust reconstruction results , Time consuming 509.577 second （8.493 branch ）, Very efficient , Reconstruction image 330 Zhang , Generate sparse points 226761.

OpenMVG

step	Time ( branch )
feature extraction Feature Extraction	29.91
Feature matching Feature Matching	247.78
Beam adjustment Bundle adjustment	130.38
SFM Total time	408.07
Reprojection error	0.26
Number of reconstructed images / Sparse points	330/ 900759

OpenMVG The reconstruction of the scene is intuitive and Colmap And Metashape Are relatively similar , It's just near 7 Hours The time is really too “ Very long ” 了 .

4 Graham-hall 711p 5616x3744

Colmap

step	Time ( branch )
feature extraction 【Feature Extraction】	2.317
Feature matching 【Feature Matching】	25.141
Beam adjustment Bundle adjustment	48.798
SFM Total time	76.256
Re projection error average	0.97
Number of reconstructed images / Sparse points	691/ 171956

The effect is good ,

OpenMVG

In view of the previous group 330 Zhang took nearly 7 Hours , This group of experiments will not be done .

PS. The following two groups are for more intuitive observation of the reconstruction effect , about Shoe Data and Dinosaur data , I use Colmap Dense point clouds are reconstructed .

5 Shoe 42p 4032x3024

Colmap

step	Time ( branch )
feature extraction 【Feature Extraction】	0.085
Feature matching 【Feature Matching】	1.112
Beam adjustment Bundle adjustment	0.259
SFM Total time	1.456
Re projection error average	0.68
Number of reconstructed images / Sparse points	38 / 5059

OpenMVG

step	Time ( branch )
feature extraction Feature Extraction	1.63
Feature matching Feature Matching	0.37
Beam adjustment Bundle adjustment	0.32
SFM Total time	2.32
Re projection error average	0.24
Number of reconstructed images / Sparse points	39/ 19794

This data does not show much difference between the two ,OpenMVF The re projection error will be smaller .

Colmap Rebuild the dense point cloud as follows ：

step	Time ( branch )
Stereo matching 【Stereo】	33.727
Point cloud fusion 【Fusion】	1.494

6 Dinosaur 55p 4032x3024

step	Time ( branch )
feature extraction 【Feature Extraction】	0.096
Feature matching 【Feature Matching】	1.098
Beam adjustment Bundle adjustment	0.488
SFM Total time	1.682
Re projection error average	0.91
Number of reconstructed images / Sparse points	55 / 10383

OpenMVG

step	Time ( branch )
feature extraction Feature Extraction	2.00
Feature matching Feature Matching	0.05
Beam adjustment Bundle adjustment	0.40
SFM Total time	2.45
Re projection error average	0.34
Number of reconstructed images / Sparse points	55/ 15796

This data does not show much difference between the two ,OpenMVF The re projection error will still be smaller .

Colmap Rebuild the dense point cloud as follows ：

step	Time ( branch )
Stereo matching 【Stereo】	48.093
Point cloud fusion 【Fusion】	1.945

5 summary

Colmap

In the absence of GPS In the case of assistance ,Colmap For small scenes （<1000 photos） The reconstruction quality of is still good , Basically, the scene structure can be restored completely and correctly , In addition to a set of ground data, some scenes drift , But I think if supplemented by GPS Information , This problem will be well solved .

The efficiency under the default parameters is low , And commercial software Metashape There is a big gap ,Colmap The default parameters described in the document aim at medium to high quality reconstruction , So the efficiency may be improved by setting some parameters , But it may also sacrifice accuracy to some extent .

except SFM,Colmap Dense reconstruction of is also very slow , It mainly consumes a lot of time on stereo matching , The effect is OK , The point cloud is thin , But support for weak textures is generally , Of course, this is also a general reconstruction problem . Because there is no such thing as OpenMVS Optimize the grid , So the final reconstructed mesh is not fine and smooth .

OpenMVG

Again , In the absence of GPS In the case of assistance , The reconstruction quality of small scenes is good . after “ Very long ” Reconstruction process of , Re projection error ratio Colmap smaller , And the drift phenomenon should be slightly better .

Whole SFM Pipeline Efficiency is really very low , So that bloggers gave up 700 Multiple data tests , near 7 Hour reconstruction 330 Zhang is unacceptable .

OpenMVG One of the reasons for low efficiency is that feature extraction adopts CPU Version of SIFT, Even after multithreading acceleration , Efficiency cannot match Colmap Of GPU edition SIFT comparison . Another reason is that enumeration method is used by default to establish feature matching , The number of matching pairs is very large and redundant , and Colmap The word tree is selected , A method that can quickly establish an appropriate number of matching pairs , There will be fewer matches .

choice

Based on the existing test results , I tend to use Colmap Instead of OpenMVG To complete SFM, The efficiency advantage is very obvious , And there is no obvious disadvantage in the integrity and accuracy of structural recovery .

Tips

1. stay SFM in ,GPS Is a very important auxiliary information , On the one hand, it can be used to restore scale information ; On the other hand, it can effectively prevent scene drift ; In addition, it can also be used as auxiliary information when screening matching image pairs , Indirectly improve the matching efficiency .
2. Improving the efficiency of feature extraction algorithm can effectively improve SFM The efficiency of , Generally speaking, based on GPU Our algorithm will have more advantages .
3. Effectively reducing image matching image logarithm can also effectively improve SFM The efficiency of , The image retrieval method based on vocabulary tree can quickly screen candidate matching image pairs through image similarity , Effectively reduce matching , Improve matching efficiency . Another effective strategy is to reduce the sampling resolution when using the vocabulary tree strategy .

If you have any suggestions or opinions, please leave a message in the comment area ！ Welcome to like collection ！

Project source code address ：TellusBuilder

Interested students , It is suggested to click star and watch , Of course, based on my work, what I want to do in the future is also welcome Fork ah .**

Follow bloggers ！

About bloggers ：
Li Yingsong （ You know ： Li Yingsong ）
Wuhan University Doctor of photogrammetry and remote sensing
Main direction Stereo matching 、 Three dimensional reconstruction
2019 Won the first prize of scientific and technological progress in surveying and mapping in （ Provincial and ministerial level ）

3D sharing open source *
GitHub： https://github.com/ethan-li-coding

Personal wechat ：（ Add me to the Group ）