Any Alternative to .bmp regarding speed of cv::imwrite

534 Views Asked by At

I am trying to subscribe topics coming from kinect2 camera and save these images. I am using iai_kinect2 package with libfreenect2. My problem is except from .bmp format, all other formats give some jumps between timestamps. I do not want to use .bmp format because apparently it does not give correct depth image. I have just zeros and 255 in depth image when i try to open it in matlab. Here is a timestamp example for .png:

1533061840.176977104
1533061840.210227483
1533061840.243339200
1533061840.376478700 //see the jump
1533061840.476526118
1533061840.510053089
1533061840.576983109

All file extensions have the same kind of jumps except .bmp. I can not use .bmp because it only gives 0 or 255 for depth image here is an example: 000001.bmp gives incorrect depth image just zeros and 255 On the other hand .png for example gives correct depth image like this: 000005.png gives correct depth image

My question is how to fix this timestamp issue with .png or any other file extension? OR How to use .bmp image to produce correct depth image? When i add this .bmp image to matlab it says it has both image and color map.

I appreciate for your help.

You can find my source code at the below.

#include "kinect2_recorder/kinect2_recorder.h"

namespace kinect2_recorder {
// Constructor definition
RecordData::RecordData(const ros::NodeHandle & nh1, const std::string &save_dir_, const std::string &object_name_) : global_image_counter_(1), kinect_node_(nh1), it_(kinect_node_), sub_color_(it_, "color", 5), sub_depth_(it_,"depth", 5) , sync( MySyncPolicy_( 10 ), sub_depth_, sub_color_ ) , local_image_index_(1), padding_digit_number_(6)  
{
    /* Some code*/
    // Exact time policy
  sync.registerCallback(boost::bind(&RecordData::masterkinectCallback, this, _1, _2 ) );              

void RecordData::masterkinectCallback(const sensor_msgs::ImageConstPtr& depth_msg, const sensor_msgs::ImageConstPtr& color_msg) {
    ros::Time last_time1 = ros::Time::now();  
    cv_bridge::CvImageConstPtr cv_depthptr; 
    try{
    cv_depthptr = cv_bridge::toCvShare(depth_msg, sensor_msgs::image_encodings::TYPE_16UC1);
        //cv_depthptr->image.convertTo(cv_depthptr->image, CV_16U);
        //cv::normalize(cv_depthptr->image, cv_depthptr->image, 1, 0, cv::NORM_MINMAX);
    }
    catch (cv_bridge::Exception& e2){
        ROS_ERROR("Could not convert from '%s' to '16UC1'.", e2.what());  
    return;         
    cv_bridge::CvImageConstPtr cv_colorptr;     
    try{
        cv_colorptr = cv_bridge::toCvShare(color_msg, sensor_msgs::image_encodings::BGR8);
    }
    catch (cv_bridge::Exception& e1){
        ROS_ERROR("Could not convert from '%s' to 'BGR8'.", e1.what());
    return;
    } 
    ros::Time last_time2 = ros::Time::now();
    // Save images with increasing index names
    std::ostringstream str_localindex;
    // Ignore First 10 Frames
    if (global_image_counter_ <= 10) {} //do nothing
    else {
    // Save depth images
    std::cout << " Before depth: " << ros::Time::now();
    str_localindex << std::setw(padding_digit_number_) << std::setfill('0') << local_image_index_; 
    cv::imwrite(depth_dir_ + str_localindex.str() + ".bmp",  cv_depthptr->image);
    std::cout << " After depth: " << ros::Time::now();
    std::cout << "saved_depth_index: " << local_image_index_ << std::endl;
    // Save color images    
    cv::imwrite(color_dir_ + str_localindex.str() + ".bmp",  cv_colorptr->image); 
    std::cout << " After color: " << ros::Time::now();
    // Save timestamps
    textfile_ << cv_depthptr->header.stamp << std::endl;
    local_image_index_++;
    }
    global_image_counter_++;

ros::Time last_time3 = ros::Time::now();
    std::cout << "First time difference: " << last_time2-last_time1<< std::endl;
    std::cout << "Second time difference: " << last_time3-last_time2<< std::endl;
    std::cout << "Third time difference: " << last_time3-last_time1<< std::endl;
}    
} //end of namespace 

EDIT: I understand the problem. The saving procedure takes longer time than the time that i am receiving information from kinect2. What can i do to fix this? Here is the last output:

First time difference: 0.000049324
Second time difference: 0.000002948
Third time difference: 0.000052272
saved_depth_index: 1
First time difference: 0.000040741
Second time difference: 0.051224336
Third time difference: 0.051265077
saved_depth_index: 2
First time difference: 0.000033076
Second time difference: 0.042704145
Third time difference: 0.042737221
saved_depth_index: 3
First time difference: 0.000033785
Second time difference: 0.042851813
Third time difference: 0.042885598

Here is the output to print timing for .png:

      Before depth: 1533067649.437008647 After depth: 1533067649.449930403saved_depth_index: 1
 After color: 1533067649.479346684 Before depth: 1533067649.479786642 After depth: 1533067649.490577579saved_depth_index: 2
 After color: 1533067649.520330356 Before depth: 1533067649.520770286 After depth: 1533067649.531645103saved_depth_index: 3
 After color: 1533067649.561450753 Before depth: 1533067649.570491592 After depth: 1533067649.583154752saved_depth_index: 4
 After color: 1533067649.612989915 Before depth: 1533067649.614441724 After depth: 1533067649.625398696saved_depth_index: 5
 After color: 1533067649.654874365 Before depth: 1533067649.655352960 After depth: 1533067649.666162400saved_depth_index: 6
 After color: 1533067649.695561608 Before depth: 1533067649.696115115 After depth: 1533067649.708304851saved_depth_index: 7
 After color: 1533067649.736989649 Before depth: 1533067649.738013842 After depth: 1533067649.751335296saved_depth_index: 8
 After color: 1533067649.780984546 Before depth: 1533067649.781488371 After depth: 1533067649.792345625saved_depth_index: 9
 After color: 1533067649.822243383 Before depth: 1533067649.822725751 After depth: 1533067649.833530992saved_depth_index: 10
 After color: 1533067649.862818010 Before depth: 1533067649.863264236 After depth: 1533067649.875531476saved_depth_index: 11
 After color: 1533067649.904058584 Before depth: 1533067649.904558682 After depth: 1533067649.917082444saved_depth_index: 12
 After color: 1533067649.946167277 Before depth: 1533067649.946650618 After depth: 1533067649.957500888saved_depth_index: 13
 After color: 1533067649.987059465 Before depth: 1533067649.989056085 After depth: 1533067649.999987811saved_depth_index: 14
 After color: 1533067650.029481919 Before depth: 1533067650.070665790 After depth: 1533067650.083509494saved_depth_index: 15
 After color: 1533067650.113966288 Before depth: 1533067650.115590415 After depth: 1533067650.126528055saved_depth_index: 16
 After color: 1533067650.156366989 Before depth: 1533067650.237325222 After depth: 1533067650.250013465saved_depth_index: 17
 After color: 1533067650.280332343 Before depth: 1533067650.404256322 After depth: 1533067650.416994490saved_depth_index: 18

Here is the corresponding timestamp.txt lines for .png:

    1533067649.405156137
    1533067649.438584768
    1533067649.471184334
    1533067649.538291826
    1533067649.571496466
    1533067649.604620511
    1533067649.637874297
    1533067649.671323063
    1533067649.704726902
    1533067649.738140404
    1533067649.771370780
    1533067649.804495997
    1533067649.837931434
    1533067649.938107491
    1533067650.037869503
    1533067650.071517274
    1533067650.204548131

Here is the output for .bmp:

Before depth: 1533068550.007569541 After depth: 1533068550.039293417saved_depth_index: 1
After color: 1533068550.040323334 Before depth: 1533068550.041525631 After depth: 1533068550.041995796saved_depth_index: 2
 After color: 1533068550.043473161 Before depth: 1533068550.074098761 After depth: 1533068550.074672501saved_depth_index: 3
 After color: 1533068550.075766884 Before depth: 1533068550.106968347 After depth: 1533068550.107476638saved_depth_index: 4
 After color: 1533068550.108505096 Before depth: 1533068550.141064749 After depth: 1533068550.141878365saved_depth_index: 5
 After color: 1533068550.144248827 Before depth: 1533068550.174576245 After depth: 1533068550.175276900saved_depth_index: 6
 After color: 1533068550.176800271 Before depth: 1533068550.207376338 After depth: 1533068550.207835761saved_depth_index: 7
 After color: 1533068550.209161683 Before depth: 1533068550.242071242 After depth: 1533068550.242618620saved_depth_index: 8
 After color: 1533068550.243642589 Before depth: 1533068550.275326665 After depth: 1533068550.275932215saved_depth_index: 9
 After color: 1533068550.278269832 Before depth: 1533068550.307122129 After depth: 1533068550.307663711saved_depth_index: 10
 After color: 1533068550.309189049 Before depth: 1533068550.341019501 After depth: 1533068550.341644754saved_depth_index: 11
 After color: 1533068550.343060063 Before depth: 1533068550.374125217 After depth: 1533068550.374740065saved_depth_index: 12
 After color: 1533068550.376692289 Before depth: 1533068550.407086067 After depth: 1533068550.407683886saved_depth_index: 13
 After color: 1533068550.409053003 Before depth: 1533068550.441498083 After depth: 1533068550.442042331saved_depth_index: 14
 After color: 1533068550.444182590 Before depth: 1533068550.507368335 After depth: 1533068550.507842169saved_depth_index: 15
 After color: 1533068550.509398491 Before depth: 1533068550.541008375 After depth: 1533068550.541669296saved_depth_index: 16
 After color: 1533068550.543557788 Before depth: 1533068550.575678313 After depth: 1533068550.576168080saved_depth_index: 17
 After color: 1533068550.577603736 Before depth: 1533068550.608481247 After depth: 1533068550.608980560saved_depth_index: 18
 After color: 1533068550.610480415 Before depth: 1533068550.674148550 After depth: 1533068550.674846522saved_depth_index: 19
 After color: 1533068550.676181557 Before depth: 1533068550.707998985 After depth: 1533068550.708643165saved_depth_index: 20
0

There are 0 best solutions below