将数据从glReadPixels（）转换为OpenCV :: Mat

I want to get every OpenGL frame from an animation with glReadPixels() and convert the data to OpenCV::Mat. I know that glReadPixels() gets the data by rows from the lower one to upper one, from left to right. On the other hand, OpenCV stores the data differently.

我想通过glReadPixels()从动画中获取每个OpenGL帧,并将数据转换为OpenCV :: Mat。我知道glReadPixels()从左到右按行从下一行到上一行获取数据。另一方面,OpenCV以不同方式存储数据。

Does anybody know any library or any tutorial/example that helps me to convert data from glReadPixels to a OpenCV:Mat in C++?

有没有人知道任何库或任何帮助我将数据从glReadPixels转换为OpenCV的教程/示例:C ++中的Mat?

SUMMARY

 OpenGL frame      ----------------------->        CV::Mat

Data from left to right,                    Data from left to right,
bottom to top.                              top to bottom.

2 个解决方案

#1

First we create an empty (or unititialized) cv::Mat for our data to be read into directly. This can be done once at startup, but on the other hand cv::Mat::create doesn't really cost much when the image already has matching size and type. The type depends on your needs, usually it's something like CV_8UC3 for a 24-bit color image.

首先,我们创建一个空的(或unititialized)cv :: Mat,以便我们直接读取数据。这可以在启动时完成一次,但另一方面,当图像已经具有匹配的大小和类型时,cv :: Mat :: create并不会花费太多。类型取决于您的需求,通常它类似于CV_8UC3,用于24位彩色图像。

cv::Mat img(height, width, CV_8UC3);

img.create(height, width, CV_8UC3);

Then you have to account for cv::Mat not neccessarily storing image rows contiguously. There might be a small padding value at the end of each row to make rows 4-byte aligned (or 8?). So you need to mess with the pixel storage modes:

然后你必须考虑cv :: Mat不必连续存储图像行。每行末尾可能有一个小的填充值,使行4字节对齐(或8?)。所以你需要搞乱像素存储模式:

//use fast 4-byte alignment (default anyway) if possible
glPixelStorei(GL_PACK_ALIGNMENT, (img.step & 3) ? 1 : 4);

//set length of one complete row in destination data (doesn't need to equal img.cols)
glPixelStorei(GL_PACK_ROW_LENGTH, img.step/img.elemSize());

Next, the type of the matrix influences the format and type parameters of glReadPixels. If you want color images you have to keep in mind that OpenCV usually stores color values in BGR order, so you need to use GL_BGR(A) (which were added with OpenGL 1.2) instead of GL_RGB(A). For one component images use either GL_LUMINANCE (which sums the individual color components) or GL_RED, GL_GREEN, ... (to get an individual component). So for our CV_8UC3 image the final call to read it directly into the cv::Mat would be:

接下来,矩阵的类型会影响glReadPixels的格式和类型参数。如果你想要彩色图像,你必须记住,OpenCV通常以BGR顺序存储颜色值,因此你需要使用GL_BGR(A)(用OpenGL 1.2添加)而不是GL_RGB(A)。对于一个组件图像,使用GL_LUMINANCE(对各个颜色组件求和)或GL_RED,GL_GREEN,...(以获得单个组件)。因此,对于我们的CV_8UC3图像,最终调用将其直接读入cv :: Mat将是:

glReadPixels(0, 0, img.cols, img.rows, GL_BGR, GL_UNSIGNED_BYTE, img.data);

Finally, OpenCV stores images from top to bottom. So you may need to either flip them after getting them or render them flipped in OpenGL in the first place (this can be done by adjusting the projection matrix, but keep an eye on triangle orientation in this case). To flip a cv::Mat vertically, you can use cv::flip:

最后,OpenCV从上到下存储图像。所以你可能需要在获得它们之后翻转它们或者首先在OpenGL中渲染它们(这可以通过调整投影矩阵来完成,但在这种情况下要注意三角形方向)。要垂直翻转cv :: Mat,可以使用cv :: flip:

cv::flip(img, flipped, 0);

So to keep in mind OpenCV:

所以要记住OpenCV:

stores images from top to bottom, left to right

存储图像从上到下,从左到右

stores color images in BGR order

以BGR顺序存储彩色图像

might not store image rows tightly packed

可能无法存储紧密打包的图像行

#2

unsigned char* getPixelData( int x1, int y1, int x2, int y2 )
{
    int y_low, y_hi;
    int x_low, x_hi;

    if ( y1 < y2 )
    {
        y_low = y1;
        y_hi  = y2;
    }
    else
    {
        y_low = y2;
        y_hi  = y1;
    }

    if ( x1 < x2 )
    {
        x_low = x1;
        x_hi  = x2;
    }
    else
    {
        x_low = x2;
        x_hi  = x1;
    }

    while ( glGetError() != GL_NO_ERROR )
    {
        ;
    }

    glReadBuffer( GL_BACK_LEFT );

    glDisable( GL_TEXTURE_2D );

    glPixelStorei( GL_PACK_ALIGNMENT, 1 );

    unsigned char *data = new unsigned char[ ( x_hi - x_low + 1 ) * ( y_hi - y_low + 1 ) * 3 ];

    glReadPixels( x_low, y_low, x_hi-x_low+1, y_hi-y_low+1, GL_RGB, GL_UNSIGNED_BYTE, data );

    if ( glGetError() != GL_NO_ERROR )
    {
        delete[] data;
        return 0;
    }
    else
    {
        return data;
    }
}

use:

CvSize size = cvSize( 320, 240 );

unsigned char *pixel_buf = getPixelData( 0, 0, size.width - 1, size.height - 1 );

if ( pixel_buf == 0 )
    return 0;

IplImage *result = cvCreateImage( size, IPL_DEPTH_8U, 3 );
memcpy( result->imageData, pixel_buf, size.width * size.height * 3 );
delete[] pixel_buf;

#1

cv::Mat img(height, width, CV_8UC3);

img.create(height, width, CV_8UC3);