Human-Aware Intelligent Systems, that interact naturally with a human-inhabited environment, represent the next frontier of information technology. They extract information related to humans and their activities from sensor data, perform decision making, and act to make human living more safe, comfortable and entertaining.

People

Open Positions

Lectures

Sensor-based pedestrian protection
2000-now, Daimler R&D, Germany.

Our long standing effort is directed towards a system for real-time detection and tracking of pedestrians from a moving vehicle. The current pedestrian system combines pedestrian detection, trajectory estimation, risk assessment and driver warning or vehicle braking

Ph.D. Students: Christoph Keller, Nicolas Schneider

EU FP7: ADABTS project (“Automatic Detection of Abnormal Behaviour and Threats in Crowded Spaces”), 2009-2013, Univ. of Amsterdam, Netherlands

Whereas CASSANDRA (see past projects) considered only aggressive behaviour, the scope is enlarged in ADABTS to detect all kinds of abnormal behaviour, as resulting from threats of terrorism, crime, and riots. Consequently, both supervised and unsupervised learning frameworks are considered. A further extension is that ADABTS looks at larger groups of people.

Ph.D. Students: Martijn Liem and Julian Kooij

3D Human Shape Model Adaptation and Pose Estimation, 2009-2010, Univ. of Amsterdam, Netherlands

Automatic 3D human pose and shape estimation by a three-step procedure: first, recover initial poses over a sequence using an initial (generic) body model. Both model and poses then serve as input to an adaptation process based on frame selection and shape-texture optimization. Finally, a more accurate pose recovery is obtained by means of the adapted model.

See CVIU’11 article and videoclip (11 MB)

Multi-view 3D Human Pose Estimation in Complex Environment, 2005-2008, Univ. of Amsterdam, Netherlands

We introduced a framework for unconstrained 3D human upper body pose estimation from multiple camera views in complex environment. Its main novelty involved the integration of three components: single-frame pose recovery, temporal integration and model texture adaptation.

See IJCV’11 article and videoclip (27 MB)

CASSANDRA: Aggression Detection by Fusion Video and Audio, 2005-2009, Univ. of Amsterdam, Netherlands

This project pursues human activity recognition in dynamic environments, in particular, automatic aggression detection. Because events associated with the buildup or enactment of aggression are difficult to detect by a single sensor modality (e.g. shouting versus hitting-someone), CASSANDRA combines audio- and video-sensing.

See AVSS article.

The Chamfer System
1997-2007, Daimler R&D, Germany.

I worked a number of years on generic shape-based object detection based on hierarchical matching with distance transforms. The method was successfully applied in a variety of application domains ranging from intelligent vehicles to industrial inspection.

Multi-cue 3D Pedestrian Tracking
2002-2006, DaimlerChrysler R&D, Germany.

This work involves a spatio-temporal object representation termed Dynamic Point Distribution Models (DPDMs) which can deal with both continuous and discontinuous appearance changes and is learned automatically from training data. State propagation is achieved using a particle filter which  integrates shape, texture and depth information.

See Trans. on ITS’08 article and videoclip.

Real-time Dense Stereo for Intelligent Vehicles
2003-2005, University of Amsterdam, The Netherlands.

With recent hardware advances, dense stereo algorithms have become feasible for real-time implementation on general-purpose processors. We developed a framework of such algorithms based on a SIMD architecture and examined their performance-speed trade-offs.

See Trans. on ITS’06 article.

3D Human Body Tracking with Multiple Cameras
1993-1996, University of Maryland, USA.

First system for the vision-based 3D tracking of unconstrained whole-body movement, of that time. Using four cameras, the system recovered 3D body pose without requiring the human to wear special markers, as was (and still is) the norm in motion capture.
 

3D Head Model Acquisition
1996, MIT Media Lab, USA.

During my semester-long visit at the MIT Media Lab I worked on a “poor man’s Cyberware scanner”: a system that uses a single video-camera to create from a sequence of a user turning his head a realistic textured 3D head model.
 

Hermite Deformable Models
1995-1996, University of Maryland, USA.

This research introduces the Hermite contour representation for deformable shape tracking.  It combines a maximum-a-posteriori criterion for the energy function with a dynamical programming technique to find optimal solution of the resulting minimization problem.