当前位置:网站首页>R3live notes: image processing section
R3live notes: image processing section
2022-07-06 21:17:00 【How much is the code? Oneortwo】
1. IMAGE_topic
subscribe :
sub_img = m_ros_node_handle.subscribe(IMAGE_topic.c_str(), 1000000, &R3LIVE::image_callback, this, ros::TransportHints().tcpNoDelay());
Images topic When data is transferred in , Callback function image_callback:
void R3LIVE::image_callback( const sensor_msgs::ImageConstPtr &msg )
{
std::unique_lock< std::mutex > lock( mutex_image_callback );
if ( sub_image_typed == 2 )
{
return; // Avoid subscribe the same image twice.
}
sub_image_typed = 1;
// Determine whether it is the first frame
if ( g_flag_if_first_rec_img )
{
g_flag_if_first_rec_img = 0;
m_thread_pool_ptr->commit_task( &R3LIVE::service_process_img_buffer, this );
}
// Convert the image encoding to BGR8
cv::Mat temp_img = cv_bridge::toCvCopy( msg, sensor_msgs::image_encodings::BGR8 )->image.clone();
// Image data processing
process_image( temp_img, msg->header.stamp.toSec() );
}
Image data processing process_image:
void R3LIVE::process_image( cv::Mat &temp_img, double msg_time )
{
cv::Mat img_get;
if ( temp_img.rows == 0 )
{
cout << "Process image error, image rows =0 " << endl;
return;
}
// The timestamp of this frame is smaller than that of the previous frame error
if ( msg_time < last_accept_time )
{
cout << "Error, image time revert!!" << endl;
return;
}
// The interval between the timestamp of this frame and the timestamp of the previous frame is less than the release frequency error
if ( ( msg_time - last_accept_time ) < ( 1.0 / m_control_image_freq ) * 0.9 )
{
cout << "Error, image time interval wrong!!" << endl;
return;
}
last_accept_time = msg_time;
// In fact, it is the initialization operation when the first image comes in
if ( m_camera_start_ros_tim < 0 )
{
m_camera_start_ros_tim = msg_time;
m_vio_scale_factor = 1280 * m_image_downsample_ratio / temp_img.cols; // 320 * 24
// load_vio_parameters();
// Camera parameter initialization , Including internal reference 、 External reference 、 Distortion parameters, etc
set_initial_camera_parameter( g_lio_state, m_camera_intrinsic.data(), m_camera_dist_coeffs.data(), m_camera_ext_R.data(),
m_camera_ext_t.data(), m_vio_scale_factor );
cv::eigen2cv( g_cam_K, intrinsic );
cv::eigen2cv( g_cam_dist, dist_coeffs );
// TODO QJF: Distortion correction ,opencv Provided correction algorithm that can be used directly , and remap() Combine to handle
// This function is used to calculate the distortion free and corrected conversion relationship , To remap , Express the result in the form of mapping . The undistorted image looks like the original image , It's like this image uses internal parameters as newCameraMatrix And distortion free camera acquisition .
initUndistortRectifyMap( intrinsic, dist_coeffs, cv::Mat(), intrinsic, cv::Size( 1280 / m_vio_scale_factor, 1024 / m_vio_scale_factor ),
CV_16SC2, m_ud_map1, m_ud_map2 );
m_thread_pool_ptr->commit_task( &R3LIVE::service_pub_rgb_maps, this);
m_thread_pool_ptr->commit_task( &R3LIVE::service_VIO_update, this);
m_mvs_recorder.init( g_cam_K, 1280 / m_vio_scale_factor, &m_map_rgb_pts );
m_mvs_recorder.set_working_dir( m_map_output_dir );
}
// Downsampling => Image size again Set up
if ( m_image_downsample_ratio != 1.0 )
{
// Image size again Set up
cv::resize( temp_img, img_get, cv::Size( 1280 / m_vio_scale_factor, 1024 / m_vio_scale_factor ) );
}
else
{
img_get = temp_img; // clone ?
}
std::shared_ptr< Image_frame > img_pose = std::make_shared< Image_frame >( g_cam_K );
// Copy the original image to img_pose
if ( m_if_pub_raw_img )
{
img_pose->m_raw_img = img_get;
}
// Remap , It is the process of placing a pixel at a certain position in an image to a specified position in another image .
// TODO What's the role ?
// INTER_LINEAR – Bilinear interpolation
cv::remap( img_get, img_pose->m_img, m_ud_map1, m_ud_map2, cv::INTER_LINEAR );
cv::imshow("sub Img", img_pose->m_img);
img_pose->m_timestamp = msg_time;
img_pose->init_cubic_interpolation();// Cubic interpolation Cubic interpolation initialization
img_pose->image_equalize();// Histogram equalization processing
m_camera_data_mutex.lock();
m_queue_image_with_pose.push_back( img_pose );
m_camera_data_mutex.unlock();
total_frame_count++;
if ( m_queue_image_with_pose.size() > buffer_max_frame )
{
buffer_max_frame = m_queue_image_with_pose.size();
}
// cout << "Image queue size = " << m_queue_image_with_pose.size() << endl;
}
2. IMAGE_topic_compressed
decompression compressed Format image
subscribe :
sub_img_comp = m_ros_node_handle.subscribe(IMAGE_topic_compressed.c_str(), 1000000, &R3LIVE::image_comp_callback, this, ros::TransportHints().tcpNoDelay());
Callback :
void R3LIVE::image_comp_callback( const sensor_msgs::CompressedImageConstPtr &msg )
{
// The mutex
std::unique_lock< std::mutex > lock2( mutex_image_callback );
if ( sub_image_typed == 1 ) // 0: TBD 1: sub_raw, 2: sub_comp
{
return; // Avoid subscribe the same image twice.
}
sub_image_typed = 2;
g_received_compressed_img_msg.push_back( msg );
if ( g_flag_if_first_rec_img )
{
g_flag_if_first_rec_img = 0;
m_thread_pool_ptr->commit_task( &R3LIVE::service_process_img_buffer, this );
}
return;
}
边栏推荐
- WEB功能测试说明
- for循环中break与continue的区别——break-完全结束循环 & continue-终止本次循环
- 首批入选!腾讯安全天御风控获信通院业务安全能力认证
- Acdreamoj1110 (multiple backpacks)
- Pat 1078 hashing (25 points) ⼆ times ⽅ exploration method
- No Yum source to install SPuG monitoring
- Reference frame generation based on deep learning
- @Detailed differences among getmapping, @postmapping and @requestmapping, with actual combat code (all)
- R language for text mining Part4 text classification
- Tips for web development: skillfully use ThreadLocal to avoid layer by layer value transmission
猜你喜欢
After working for 5 years, this experience is left when you reach P7. You have helped your friends get 10 offers
HMS core machine learning service creates a new "sound" state of simultaneous interpreting translation, and AI makes international exchanges smoother
2022菲尔兹奖揭晓!首位韩裔许埈珥上榜,四位80后得奖,乌克兰女数学家成史上唯二获奖女性
Deployment of external server area and dual machine hot standby of firewall Foundation
1500萬員工輕松管理,雲原生數據庫GaussDB讓HR辦公更高效
【论文解读】用于白内障分级/分类的机器学习技术
Swagger UI tutorial API document artifact
Is this the feeling of being spoiled by bytes?
[interpretation of the paper] machine learning technology for Cataract Classification / classification
Interviewer: what is the internal implementation of ordered collection in redis?
随机推荐
[MySQL] basic use of cursor
【论文解读】用于白内障分级/分类的机器学习技术
The use method of string is startwith () - start with XX, endswith () - end with XX, trim () - delete spaces at both ends
代理和反向代理
b站视频链接快速获取
Acdreamoj1110 (multiple backpacks)
Three schemes of SVM to realize multi classification
Pat 1078 hashing (25 points) ⼆ times ⽅ exploration method
Set up a time server
el-table表格——获取单击的是第几行和第几列 & 表格排序之el-table与sort-change、el-table-column与sort-method & 清除排序-clearSort
Introduction to the use of SAP Fiori application index tool and SAP Fiori tools
3D人脸重建:从基础知识到识别/重建方法!
Performance test process and plan
1500萬員工輕松管理,雲原生數據庫GaussDB讓HR辦公更高效
HMS core machine learning service creates a new "sound" state of simultaneous interpreting translation, and AI makes international exchanges smoother
Reflection operation exercise
Reference frame generation based on deep learning
039. (2.8) thoughts in the ward
分糖果
[MySQL] trigger