In the initial Windows-based software, a frame grabber was used to convert a video tape (from a VCR) into a directory of bitmap files. These bitmaps were matched to the timestamps and the targets marked up by modifying individual pixels. The directory was subsequently emitted to a VCR and also converted into a digital video for computer presentation. The hardware cost of systems to implement this approach in real time was prohibitive.
For real time, instead, a video mixing unit superimposes the computer's display onto the camera image using a `bluescreen' mode to ensure that most of the computer display was invisible. The mixer output is a video signal that has threat objects marked in, ready for display on a security monitor and/or recording by a VCR for subsequent replay or digitization. The image in figure 1 was grabbed from such a tape.
A small program in Microsoft's Visual Basic painted the entire screen blue, then accepted a sequence of target coordinates and moved a color-selectable blob to the desired location. In operation, we found that the blob could only be moved at a fraction of the video framerate (about 10 fps), so that the motion appeared jerky and was late. This jerkiness made it difficult to match the blob against a crowded room and thereby determine which person is carrying the gun.
The low frame rate was in part due to the use of a Rapid Application Development tool, compared to a C compiler. The scientists and other non-programmers on the team were comfortable with VB or command line GCC, but found the Visual C environment overwhelming and didn't use it.
Under Linux, we implemented a trivial equivalent using the ncurses library and its suite of graphics characters. Even an old Pentium 60 was able to achieve three times the video framerate for position updates, so we have not yet optimized from that library to using the underlying terminfo directly. Although the positioning accuracy is one character cell, careful choice of the VGA console mode achieves resolution comparable to the video camera alignment.
The text-based Linux version offers the rapid and smooth tracking of the target that is critical to reliable identification.