Building Computer Vision Projects with OpenCV 4 and C++

上QQ阅读APP看书，第一时间看更新

OpenGL support

OpenCV includes OpenGL support. OpenGL is a graphical library integrated in almost all graphical cards as a standard. OpenGL allows us to draw 2D up to complex 3D scenes. OpenCV includes OpenGL support due to the importance of representing 3D spaces in a number of tasks. To allow window support in OpenGL, we have to set up the WINDOW_OPENGL flag when we create the window using the namedWindow call.

The following code creates a window with OpenGL support and draws a rotate plane where we are going to show the web camera frames:

Mat frame; 
GLfloat angle= 0.0; 
GLuint texture;  
VideoCapture camera; 
 
int loadTexture() { 
 
    if (frame.data==NULL) return -1; 

   glBindTexture(GL_TEXTURE_2D, texture);  
   glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR); 
   glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR); 
   glPixelStorei(GL_UNPACK_ALIGNMENT, 1); 
 
   glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, frame.cols, frame.rows,0, GL_BGR, GL_UNSIGNED_BYTE, frame.data); 
   return 0; 
 
} 
 
void on_opengl(void* param) 
{ 
    glLoadIdentity();   
    // Load frame Texture 
    glBindTexture(GL_TEXTURE_2D, texture);  
    // Rotate plane before draw 
    glRotatef(angle, 1.0f, 1.0f, 1.0f); 
    // Create the plane and set the texture coordinates 
    glBegin (GL_QUADS); 
        // first point and coordinate texture 
     glTexCoord2d(0.0,0.0);  
     glVertex2d(-1.0,-1.0);  
        // second point and coordinate texture 
     glTexCoord2d(1.0,0.0);  
     glVertex2d(+1.0,-1.0);  
        // third point and coordinate texture 
     glTexCoord2d(1.0,1.0);  
     glVertex2d(+1.0,+1.0); 
        // last point and coordinate texture 
     glTexCoord2d(0.0,1.0);  
     glVertex2d(-1.0,+1.0); 
    glEnd(); 
 
} 
 
int main(int argc, const char** argv) 
{ 
    // Open WebCam 
    camera.open(0); 
    if(!camera.isOpened()) 
        return -1; 
 
    // Create new windows 
    namedWindow("OpenGL Camera", WINDOW_OPENGL); 
     
    // Enable texture 
    glEnable( GL_TEXTURE_2D );
    glGenTextures(1, &texture); 
    setOpenGlDrawCallback("OpenGL Camera", on_opengl); 
    while(waitKey(30)!='q'){ 
        camera >> frame; 
        // Create first texture 
        loadTexture();     
        updateWindow("OpenGL Camera"); 
        angle =angle+4; 
    } 
    // Destroy the windows 
    destroyWindow("OpenGL Camera"); 
    return 0; 
}

Let's understand the code!

The first task is to create the required global variables, where we store the video capture, save the frames, and control the animation angle plane and the OpenGL texture:

Mat frame; 
GLfloat angle= 0.0; 
GLuint texture;  
VideoCapture camera;

In our main function, we have to create the video camera capture to retrieve the camera frames:

camera.open(0); 
    if(!camera.isOpened()) 
        return -1;

If the camera is opened correctly, we can create our window with OpenGL support using the WINDOW_OPENGL flag:

// Create new windows 
namedWindow("OpenGL Camera", WINDOW_OPENGL);

In our example, we want to draw the images that come from the web camera in a plane; then, we need to enable the OpenGL textures:

// Enable texture 
glEnable(GL_TEXTURE_2D);

Now we are ready to draw with OpenGL in our window, but we need to set up a draw OpenGL callback like a typical OpenGL application. OpenCV gives us the setOpenGLDrawCallback function which has two parameters – the window name and the callback function:

setOpenGlDrawCallback("OpenGL Camera", on_opengl);

With the OpenCV window and callback function defined, we need to create a loop to load the texture, update the window content calling the OpenGL draw callback, and finally update the angle position. To update the window content, we use the OpenCV function update window with the window name as a parameter:

while(waitKey(30)!='q'){ 
        camera >> frame; 
        // Create first texture 
        loadTexture(); 
        updateWindow("OpenGL Camera"); 
        angle =angle+4; 
    }

We are in the loop when the user presses the Q key. Before compiling our application sample, we need to define the loadTexture function and our on_opengl callback draw function. The loadTexture function converts our Mat frame to an OpenGL texture image ready to load and use in each callback drawing. Before loading the image as a texture, we have to ensure that we have data in our frame matrix, checking that the data variable object is not empty:

if (frame.data==NULL) return -1;

If we have data in our matrix frame, then we can create the OpenGL texture binding and set the OpenGL texture parameter as a linear interpolation:

glGenTextures(1, &texture); 
 
glBindTexture(GL_TEXTURE_2D, texture); 
    glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR); 
    glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);

Now, we have to define how the pixels are stored in our matrix and generate the pixels with the OpenGL glTexImage2D function. It's very important to note that OpenGL uses the RGB format, and OpenCV the BGR format, by default, and we have to set up the correct format in this function:

glPixelStorei(GL_UNPACK_ALIGNMENT, 1); 
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, frame.cols, frame.rows,0, GL_BGR, GL_UNSIGNED_BYTE, frame.data); 
    return 0;

Now, we only need to finish drawing our plane on every callback when we call updateWindow in the main loop. We use the common OpenGL functions, and then we load the identity OpenGL matrix to reset all our previous changes:

glLoadIdentity();

We also have to bring the frame texture to memory:

    // Load Texture 
    glBindTexture(GL_TEXTURE_2D, texture);

Before drawing our plane, we apply all transformations to our scene. In our case, we are going to rotate our plane in the 1,1,1 axis:

    // Rotate plane 
    glRotatef(angle, 1.0f, 1.0f, 1.0f);

Now that we have the scene correctly set to draw our plane, we are going to draw quads faces (faces with four vertices) and use glBegin (GL_QUADS) for this purpose:

// Create the plane and set the texture coordinates 
    glBegin (GL_QUADS);

Next, we will draw a plane centered in the 0,0 position, which is 2 units in size. Then, we have to define the texture coordinate to use and the vertex position using the glTextCoord2D and glVertex2D functions:

    // first point and coordinate texture 
 glTexCoord2d(0.0,0.0);  
 glVertex2d(-1.0,-1.0);  
    // seccond point and coordinate texture 
 glTexCoord2d(1.0,0.0);  
 glVertex2d(+1.0,-1.0);  
    // third point and coordinate texture 
 glTexCoord2d(1.0,1.0);  
 glVertex2d(+1.0,+1.0); 
    // last point and coordinate texture 
 glTexCoord2d(0.0,1.0);  
 glVertex2d(-1.0,+1.0); 
    glEnd();

This OpenGL code becomes obsolete, but it is appropriated to understand better the OpenCV and OpenGL integration without complex OpenGL code. By way of an introduction to modern OpenGL, read Introduction to Modern OpenGL, from Packt Publishing.

We can see the result in the following image: