Pangolin Library: subgraph

Subgraphs

Actual use SLAM When visualizing point cloud information , It is hoped that the current image information of the camera and feature point tracking can be displayed at the same time . At this point, you need to build the subgraph object , Display the point cloud information while displaying the rest of the information .

Build subgraph objects

The sub graph objects are constructed as follows ：

pangolin::View& sub_viewer = pangolin::Display(const std::string& name)
            .SetBounds(Attach bottom, Attach top, Attach left, Attach right, double aspect)
            .SetLock(Lock horizontal, Lock vertical );

• name： Name of the view or subgraph
• SetBounds： Used to set the position of the subgraph , Same as interactive view
• SetLock： Set the lock position of the subgraph , When zooming the window , Automatically lock the subgraph to the specified position
  • horizontal： Horizontal locking position , Fill in the following contents ：
    • LockLeft： left
    • LockCenter： The central
    • LockRight： On the right side
  • vertical： Vertical lock position , Fill in the following contents ：
    • LockBottom： Bottom
    • LockCenter： The central
    • LockTop： Top

Image container

For the image content to be displayed in the subgraph , You should first create a container for loading ：

pangolin::GlTexture imgTexture(GLint width, GLint height, GLint internal_format, bool sampling_linear, int border, GLenum glformat, GLenum gltype, GLvoid* data )

• width、height： Loaded image width 、 Height , Same as the width of the loaded image 、 Highly consistent
• internal_format：Pangolin Internal image storage format , Generally, it is filled in by default GL_RGB
• sampling_linear： Whether to start the current sampling , Default filling false
• border： Boundary size （ Pixels ）, Default filling 0
• glformat：GL Image storage format , It is generally used OpenCV Read the image , Therefore, fill in GL_BGR
• gltype：GL Image data format ,OpenCV The format is UInt type , Therefore, fill in GL_UNSIGNED_BYTE
• data：GL Image data

Loading images

Load image information into the container

imgTexture.Upload(const void* data, GLenum data_format, GLenum data_type)

• data： Image data
• data_format： Image storage format
• data_type： Image data format

Show subgraphs

Last , Display subgraph information

//  Activate subgraph 
sub_viewer.Activate();
//  Background color setting of subgraph 
glColor3f(1.0f, 1.0f, 1.0f);
//  reverse Y Axis , Otherwise, the output is inverted 
imgTexture.RenderToViewportFlipY();

here , Because the view origin position is in the lower left corner , and OpenCV The origin is in the upper left corner , Therefore, we need to reverse Y Axis .

comprehensive ： Subgraph display

in summary , The whole process of building the subgraph is as follows ：

CMake

Due to the use of OpenCV library , Need to introduce CV library ：

The root directory CMakeLists.txt The documents are as follows ：

# cmake version
cmake_minimum_required(VERSION 3.21)
# project name
project(Study)
# cpp version
set(CMAKE_CXX_STANDARD 14)
# eigen
include_directories("/usr/include/eigen3")
# Sophus
find_package(Sophus REQUIRED)
include_directories(${
    Sophus_INCLUDE_DIRS})
# pangolin
find_package(Pangolin REQUIRED)
include_directories(${
    Pangolin_INCLUDE_DIRS})
# opencv
find_package(OpenCV REQUIRED)
include_directories(${
    OpenCV_INCLUDE_DIRS})
# incldue
include_directories(include)
# src
add_subdirectory(src)

src Next CMakeLists.txt The documents are as follows ：

# exec
add_executable(Study main.cpp)
# link pangolin
target_link_libraries(Study ${
    Pangolin_LIBRARIES})
# link opencv
target_link_libraries(Study ${
    OpenCV_LIBRARIES})

The header file

Include Next file main.h as follows ：

#ifndef STUDY_MAIN_H
#define STUDY_MAIN_H

#include <iostream>
#include <cmath>
#include <unistd.h>
// Eigen
#include <Eigen/Core>
#include <Eigen/Dense>
#include <Eigen/Geometry>
// Sophus
#include <sophus/so3.hpp>
#include <sophus/se3.hpp>
// Pangolin
#include <pangolin/pangolin.h>
// OpenCV
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>

// namespace
using namespace std;
using namespace Eigen;


#endif //STUDY_MAIN_H

Source code

src Next source file main.cpp as follows ：

#include <string>
#include <iostream>
#include "main.h"

int main()
{
    
    /*-------- Pangolin The camera  --------*/
    //  window 
    pangolin::CreateWindowAndBind("MultiImage", 752, 480);
    //  Start the depth test 
    glEnable(GL_DEPTH_TEST);
    //  The camera 
    pangolin::OpenGlRenderState s_cam(
            pangolin::ProjectionMatrix(752, 480, 420, 420, 320, 320, 0.1, 1000),
            pangolin::ModelViewLookAt(-2, 0, -2, 0, 0, 0, pangolin::AxisY)
    );
    /*--------  View 、 Subgraphs  --------*/
    //  Main view cam, The tensile 
    pangolin::View& d_cam = pangolin::Display("cam")
            .SetBounds(0., 1., 0., 1., -752/480.)
            .SetHandler(new pangolin::Handler3D(s_cam));
    //  Subgraphs 1 image_1, tailoring , Lock the upper left corner 
    pangolin::View& cv_img_1 = pangolin::Display("image_1")
            .SetBounds(2/3.0f, 1.0f, 0., 1/3.0f, 752/480.)
            .SetLock(pangolin::LockLeft, pangolin::LockTop);
    //  Subgraphs 2 image_2, tailoring , Lock the lower right corner 
    pangolin::View& cv_img_2 = pangolin::Display("image_2")
            .SetBounds(0., 1/3.0f, 2/3.0f, 1.0, 752/480.)
            .SetLock(pangolin::LockRight, pangolin::LockBottom);

    // glTexture Containers , For reading images 
    pangolin::GlTexture imgTexture1(752, 480, GL_RGB, false, 0, GL_BGR, GL_UNSIGNED_BYTE);
    pangolin::GlTexture imgTexture2(752, 480, GL_RGB, false, 0, GL_BGR, GL_UNSIGNED_BYTE);

    while(!pangolin::ShouldQuit()){
    
        //  Clear color 、 Deep cache 
        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
        //  Background color 
        glClearColor(0.2, 0.2, 0.2, 0.3);
        //  Activate the camera 
        d_cam.Activate(s_cam);

        /*--------  Draw the contents of the main view  --------*/
        //  Origin cube 
        glColor3f(1.0f, 1.0f, 1.0f);
        pangolin::glDrawColouredCube();

        /*--------  Subgraph information  --------*/
        //  Read images 
        cv::Mat img1 = cv::imread("/home/jasonli/workspace/Study/pic/01.png");
        cv::Mat img2 = cv::imread("/home/jasonli/workspace/Study/pic/02.png");
        //  Loading images 
        imgTexture1.Upload(img1.data, GL_BGR, GL_UNSIGNED_BYTE);
        imgTexture2.Upload(img2.data, GL_BGR, GL_UNSIGNED_BYTE);

        /*--------  Subgraphs 1 To configure  --------*/
        //  Activate subgraph 1
        cv_img_1.Activate();
        //  Subgraphs 1 Background color settings 
        glColor3f(1.0f, 1.0f, 1.0f);
        //  reverse Y Axis , Otherwise, the output is inverted 
        imgTexture1.RenderToViewportFlipY();

        /*--------  Subgraphs 2 To configure  --------*/
        //  Activate subgraph 2
        cv_img_2.Activate();
        //  Subgraphs 1 Background color settings 
        glColor3f(1.0f, 1.0f, 1.0f);
        //  reverse Y Axis , Otherwise, the output is inverted 
        imgTexture2.RenderToViewportFlipY();

        //  Frame cycle 
        pangolin::FinishFrame();
    }

    return 0;
}

The effect is as follows ：