Real-time webcam detection

The previous webcam_pipeline is not a real-time detection system because it just takes snapshots and applies detection to the single taken image. This is a necessary limitation because dealing with webcam streaming requires intensive I/O data exchange. In particular, the problem is the queue of images arriving from the webcam to the Python interpreter that locks down Python until the transfer is completed. Adrian Rosebrock on his website pyimagesearch proposes a simple solution based on threads that you can read about at this Web address: http://www.pyimagesearch.com/2015/12/21/increasing-webcam-fps-with-python-and-opencv/.

The idea is very simple. In Python,  because of the global interpreter lock (GIL), only one thread can execute at a time. If there is some I/O operation that blocks the thread (such as downloading a file or getting an image from the webcam), all the remaining commands are just delayed for it to complete causing a very slow execution of the program itself. It is then a good solution to move the blocking I/O operation to another thread. Since threads share the same memory, the program thread can proceed with its instructions and inquiry from time to time the I/O thread in order to check if it has completed its operations.  Therefore, if moving images from the webcam to the memory of the program is a blocking operation, letting another thread dealing with I/O could be the solution. The main program will just inquiry the I/O thread, pick the image from a buffer containing only the latest received image and plot it on the screen.  

from tensorflow_detection import DetectionObj
from threading import Thread
import cv2

def resize(image, new_width=None, new_height=None):
    """
    Resize an image based on a new width or new height
    keeping the original ratio
    """
    height, width, depth = image.shape
    if new_width:
        new_height = int((new_width / float(width)) * height)
    elif new_height:
        new_width = int((new_height / float(height)) * width)
    else:
        return image
    return cv2.resize(image, (new_width, new_height), \
                      interpolation=cv2.INTER_AREA)

class webcamStream:
    def __init__(self):
        # Initialize webcam
        self.stream = cv2.VideoCapture(0)
        # Starting TensorFlow API with SSD Mobilenet
        self.detection = DetectionObj(model=\
                        'ssd_mobilenet_v1_coco_11_06_2017')
        # Start capturing video so the Webca, will tune itself
        _, self.frame = self.stream.read()
        # Set the stop flag to False
        self.stop = False
        #
        Thread(target=self.refresh, args=()).start()

    def refresh(self):
        # Looping until an explicit stop is sent 
        # from outside the function
        while True:
            if self.stop:
                return
            _, self.frame = self.stream.read()

    def get(self):
        # returning the annotated image
        return self.detection.annotate_photogram(self.frame)

    def halt(self):
        # setting the halt flag
        self.stop = True

if __name__ == "__main__":
    stream = webcamStream()
    while True:
        # Grabbing the frame from the threaded video stream 
        # and resize it to have a maximum width of 400 pixels
        frame = resize(stream.get(), new_width=400)
        cv2.imshow("webcam", frame)
        # If the space bar is hit, the program will stop
        if cv2.waitKey(1) & 0xFF == ord(" "):
            # First stopping the streaming thread
            stream.halt()
            # Then halting the while loop
            break

The above code implements this solution using a webcamStream class that instantiates a thread for the webcam I/O, allowing the main Python program to always have at hand the latest received image, processed by the TensorFlow API (using ssd_mobilenet_v1_coco_11_06_2017). The processed image is fluidly plotted on the screen using an OpenCV function, listening to the space bar keystroke in order to terminate the program.