For a humanoid robot, the perception of the users, their locations, identities, gestures, and other communicative cues is an essential necessity for efficient and safe interaction. It allows the robot to understand what users want, and to generate an appropriate response. As a member of the SFB588 “Humanoid Robots” we are working on the visual perception capabilities of the humanoid robot Armar-III with respect to the user and his activities.

As smart environments one can consider sensorized environments that can perceive the users and their activities in it, and which aim at supporting these users during their activities in these rooms. Examples are smart houses, smart meeting rooms or smart lecture rooms.

We have extensively worked on building perceptual technologies for smart environments, including smart offices, meeting and lectures rooms. Recently this has mainly be done in the framework of the European Integrated Project CHIL - Computers in the Human Interaction Loop - in which we were responsible for the overall scientific project coordination, and in which we contributed critically important perception modules, including person tracking, person identification, estimation of head pose and focus of attention as well as recognition of activities.

With the increasing amount of available multimedia data, it has become more and more important to be able to automatically analyze multimedia content and retrieve it according to the user's wishes. There are many technologies involved in such a task. First of all, the content of the multimedia data should be represented in a suitable way to facilitate successful retrieval. This requires the use of audio-visual and text analysis tools to derive descriptions of the multimedia data. Afterwards, the search can be conducted either in a fully automatic way or in an interactive way by adding the user to the decision loop.

Person tracking is one of the main topics of research of the CVHCI group. Many of the other topics investigated, such as face identification, body pose analysis, or focus of attention analysis often depend on the detection and localization of persons in the first place. The algorithms developed are designed to function in a fully automatic way, initializing tracks for one or several persons, adapting person models, handling inter-person dependences such as occlusions, etc.

Our aim is to build face recognition systems for real-life scenarios that:

• do not require elaborate face alignment,
• are less sensitive to illumination and background variations,
• can operate on both still images and video
• can detect & recognize unknown people
• are robust against face detection failures - partially detected faces, false detections.

[under construction]

Among the set of gestures intuitively performed by humans when communicating with each other, pointing gestures are especially interesting for communication with robots. They open up the possibility of intuitively indicating objects and locations, e.g., to make a robot change direction of its movement or to simply mark some object. This is particularly useful in combination with speech recognition as pointing gestures can be used to specify parameters of location in verbal statements (“put the cup there!”).