The Real-time Automated Target Tracking System system presents a tracking system that performs several algorithms to complete one final task: Shoot an incoming helicopter. The system is an integration of several areas within the Electrical Engineering field: Image Processing, Control and Communications, because it requires several mini-tasks to complete the final objective.
To be able to perform the shooting reasonably the system have several hardware parts: The camera (captures a video stream and sends it to the computer), the computer (which performs the algorithms), The micro-controller (makes the communication between computer and other components), the rotating base (which have all the components together, and also is able to rotate in 360 degrees), the stepper motor (to make the rotation of the base possible), the gun (that performs the shooting) and the servo-motors (to slightly move the gun in the x and y axis).
Figure 1. Hardware Setup Overview
The detection of the target is made by using the camera, that sends the video stream to a computer that does Image Processing to find the target from the frames. It was used an open source library made especially for Image Processing called OpenCV, that provides several built-in functions to process images, and the programming used the C++ language. First, we perform the Feature Detection, the detection of features of the helicopter, and for that, we used the Haar cascade classifiers[1], a classifier that have the information of the features of the helicopter. To do that, it needs training, with several samples pictures of the helicopter for the training program (provided by OpenCV) find its features. To do that, I run a program that captured each frame from the camera, rather than taking photos separately, to save time. The output of the training program is a file with all the characteristics of the helicopter. This was made for 4 views of the helicopter: right and left side and front-right and front-left side.
After the Feature Detection we still have false alarms due to noise from the camera and the shape of the helicopter, that is very simple. To solve that we do the Image Segmentation, filter the frames by the color characteristics. Basically we set some range of values of the color and what is in that range receives '1', if not, '0'. This was made by changing the format of the frame from the RGB to HSV (Hue, Saturation and Value) color format, because it is more robust and it is easier to deal with. The main objective is to filter between all of the results of the Feature Detection, finding the “real” helicopter by its color (i.e. the real helicopter must be a shape detected with at least, a good amount of it inside the color threshold set).
After the detection is made, it automatically changes to a tracking algorithm, called Camshift[2] that performs tracking based on the color and
motion. It is notably more robust and faster for tracking than Feature Detection, but it needs to have an input of the object to be tracked, so we still need to do the detection of the helicopter.
Figure 2 – Steps of Image Processing. Top-left is Feature Detection, followed by Image Segmentation. Bottom row is the Camshift algorithm and its background segmentation for the helicopter.
After all the Image Processing steps, is time to work with the hardware parts of the system, and the first step for that is the communication between the computer to the micro-controller. A communication protocol was made to define what the micro-controller would do for each set of characters the computer sends to it.
With the communication done, the micro-controller sends the required commands to the motors (stepper motor to rotate the base when the helicopter moves horizontally and the servo motors to move the gun when there is motion in both axis). The goal is to put the image of the helicopter close to the center of the camera frame, because in this range, the variation of position made by the servomotors to hit the target with the gun is linear.
And finally the final step: the shoot. Our program have allows the shooting to be automatic or manual. The automatic version shoots when the helicopter is in the same position (with little variation) for some frames of the stream. The manual simply activates the gun while tracking the helicopter.
(Click poster to enlarge) 
Implementation of Software Defined Radio
Automated Sorting Machine using Video Pro...
