Science fiction always precedes reality


The computing power of computers has increased by orders of magnitude since it was invented, but the way we interact with these devices has changed little over this period. Image processing systems based on stereo cameras are beginning to change this. Futuristic interfaces are created that go far beyond what was science fiction yesterday.

If you ask any passer-by on the street how he interacts with his computer, he will most likely name three methods: keyboard, mouse, touch screen. The mouse and keyboard have been the de facto standard for input devices since the Xerox Alto was developed in 1972. And even the touchscreen, which is ubiquitous in smartphones and tablet PCs today, has been on the market since the early 1970s - Elo TouchSystems announced the first model in 1974.
These touch control systems are very popular in heavy industry and also in the field of building automation. Here, the mouse and keyboard prove to be superior in terms of the intuitive, fast or precise interaction between the user and the screen display.
A completely new type of communication with the digital world is now made possible by stereoscopic image processing systems. They are more intuitive to use and in some ways are reminiscent of science fiction films such as the Tom Cruise - Blockbuster Minority Report from 2002.

Beyond science fiction
Minority Report anticipated a computer interface that is able to use hand and finger movements. In the film, gestures such as pointing, grasping, opening and swiping are interpreted in such a way that displays and data on the screen can be controlled and changed. The accuracy and increasing bandwidth of today's image processing cameras, however, allow more than just finger control.
An interface developed in Switzerland that was created through a collaboration between Atracsys and Sony Image Sensing Solutions is particularly interesting: the Interactive Communication Unit (ICU). The ICU is based on two highly specified IEEE1394.b cameras that can precisely track the three-dimensional movements of a user and even their emotions. The cameras are connected to a computer that immediately updates the image content on the monitor accordingly.
The ICU uses two Sony XCD-V60 cameras connected to a standard CPU. The system will soon be modernized with the XCD-SX90 cameras which contain 1/3 type PS IT sensors.
The ICU even recognizes subtle differences in facial expressions, making the image processing system ideal for use in marketing. The age and gender of a user can be determined, but also states of mind such as happiness, anger, sadness and surprise. This is accomplished by tracking down the corners of the mouth, for example, or opening the eyes wide, both of which are features associated with smiling. With the help of two cameras, this recognition can be carried out three-dimensionally in space, thus enabling a computer interface that brings a user even closer to an application, be it for industrial, marketing or gaming purposes.

Tracking options
The replacement of the mouse: With the help of a stereoscopic image processing system, it is possible to determine exactly where a person is pointing or even what this person is focusing on. A mouse pointer can be moved with your finger or even your gaze, and a second movement, e.g. B. a smile or nod, simulates the "mouse click".
When it is currently used for marketing purposes, the system takes advantage of B. to Internet content to call up additional product or service information. A person looks z. B. on a clock or other device in a shop window and thereby receives access to a website with further relevant information about the viewed object.
Similar to the multi-touch capability of touch screens, image processing systems can also interpret various types of gestures, e.g. B. circular menus where the cursor is controlled by moving the head (left / right or up / down).
Correct positioning: Whether it is a shop window or an industrial application - the viewer is seldom in the optimal position to see the displayed content well or to interact with it. However, stereo vision enables the computer to precisely calculate the position of the user. With the help of this information, it is possible to adapt the information displayed using 3D rendering effects in order to achieve an optimal display for the user.
Mood control: By recognizing the smallest movements in the face of the observed person, the ICU can interpret the behavior of this person. Since, according to Atracsys, marketing applications for this technology are a very promising market, this sensitivity in motion detection is crucial for future success. Content can then not only be adapted to demographic aspects, but also depending on the mood of the viewer.
Games: By moving their entire body, users can interact with game content easily and directly. With the help of simple algorithms that run in the CPU and are tailored to each application, lifelike actions can be imitated with avatars. So z. B. Space Invader, a classic arcade game from the 1970s, can be controlled via the right and left movements of a player, with which the spaceship is maneuvered accordingly. In addition to games and marketing applications, this type of interaction also offers numerous advantages in the industrial sector. There z. B. Remotely control robotic arms using simple hand or arm movements.

Future applications
An improvement of the computer interfaces, more intuitive and more involving the user, is essential. In addition to commercial applications, interfaces based on image processing systems could also play an important role in medical technology. In an operating room, the surgeon could use it to call up data without having to touch a computer mouse or keyboard - and thus maintain sterile conditions.
Stereoscopic image processing systems bring the greatest change in the field of computer interaction since the invention of the mouse and keyboard in the early 1970s. The technology behind the ICU can be used for almost all purposes and adapted to any application requirements. Limitations arise only from the abundance of conceivable content and from space restrictions that could complicate multi-user use.
The film "Minority Report" introduced the concept to the world. It took less than seven years to develop the technology from there and turn science fiction into reality.