Jump to: Projects | Personnel | Events


The People Image Analysis (PIA) Consortium develops and distributes technologies that process images and videos to detect, track, and understand people's face, body, and activities. The areas of technology that the PIA Consortium focuses on include detection and tracking of humans, face recognition, facial expression analysis, gait analysis, and activity recognition. The goal of the Consortium is to develop a comprehensive set of imaging and processing tools, systems, or subsystems that work in the real-world environment.

The Consortium's full and associate members contribute funds to support researchers at the Robotics Institute of Carnegie Mellon University. In return, members receive the advanced research and development for PIA technologies, as well as the results of related publicly-funded research. Members also receive the benefit of participating in workshops, sending visiting scientists to conduct research at the Robotics Institute, recruiting graduates, and networking. The Member Agreement Form can be downloaded at : PIA Consortium Agreement (PDF).

Directors: Takeo Kanade and Martial Hebert
Coordinator: Kris Kitani  ( kkitani at cs.cmu.edu )



  • PIA Workshop 2015 was held on December 21, 2015.
  • Visual odometry code released on June 18, 2015.
  • ZFace Tracker code released on March 30, 2015.
  • PIA Workshop 2014 was held on November 26, 2014.
  • CMU_Surveillance Research Dataset was released. (11/1/2014)
  • Source Code for Principal Nested Spheres for Shape Alignment was released for PIA members. (09/04/2014)
  • Source code of "Experiments on Viewpoint Invariant Person Re-identification" was released for PIA members. (12/26/2013)
  • CMU_Surveillance Research Dataset page was opened. (12/16/2013)
  • PIA (People Image Analysis) Workshop 2013 will be held on Dec. 17, 2013. (11/21/2013)
  • Cohn-Kanade (CK and CK+) database Download Site was opened. (08/10/2013)
  • PAINFUL DATA Download Site was released. (08/10/2013)
  • Hyperspectral Face Database was released. (06/13/2013)
  • Correlation Filters for Object Alignment was uploaded. (06/13/2013)
  • PIA Technical Report (Experiments on Viewpoint Invariant Person Re-identification) was uploaded to release page only for PIA members. (02/08/2013)
  • Advanced CMU's Face Tracking Library was uploaded to release page only for PIA members. (11/30/2012)
  • CMU's Face Tracking Library was uploaded to release page only for PIA members. (11/29/2012)

PIA and Affiliated Projects

Video Processing Techniques

Camera Calibration and Undistortion Using LCD
A system to calibrate camera and correct the non-linear distortion of camera lenses
( Source Code Available )
Local Feature Framework
Our local feature framework prototype system contains three major modules: feature extraction, matching, and localization
( Source Code Available )
Video Stabilization
A prototype system to stablize the video if the camera is shaking or moving. The output video will be stable as if the camera is virtually not moving at all.
( Source Code Available )
Multiple View Geometry Toolbox
A system to provides a group of functions to compute fundamental matrix, projective transformation, affine transformation between two images by robust estimation.
( Source Code Available )
Virtual Background Motion Detection
A Virtual Background Model to detect motion with a moving camera.
( Source Code Available )
Image Enhancement: Denight and Defog
A system to enhance image quality under night time or bad whether such as fog, rain, and snow.
( Source Code Available )
Flow-Based Background Estimation
A Bayesian framework for estimating dense optical flow over time that explicitly estimates a persistent model of background appearance.
Layered Detection
A system for detecting multiple overlapping objects from a real-time video stream.
Master-Slave Distant Human Identification (DHID)
A master-slave real-time surveillance system designed to detect, track, and acquire biometric imagery of s at distance.
( Source Code Available )
Video Surveillance and Monitoring (VSAM)
An end-to-end prototype system demonstrating a wide range of advanced automated video surveillance and monitoring techniques.
( Source Code Available )
Camera Calibration Algorithm Using Planar Calibration Patterns
A novel camera calibration method can increases not only an accuracy of intrinsic camera parameters but also an accuracy of stereo camera calibration by utilizing a single framework for square, circle, and ring planar calibration patterns.
( Source Code and Dataset Available for Everyone )
Image-Consistent Surface Triangulation
Given a set of 3D points that we know lie on the surface of an object, we can define many possible surfaces that pass through all of these points. Our goal is to find the particular surface that is closest to the true object surface.
( Source Code Available )

Detecting,Tracking & Recognizing

Articulated Motion Analysis
Human articulated poses are estimated from a monocular video by combining appearance-based and model-based approaches.
Omega-Shape Tracker
Techniques to track Head and Shoulder Line with a Omega-Shape model.
( Source Code Available )
Tracking Evaluation Testbed
A testbed system to evaluate and compare various tracking algorithms.
( Source Code Available )
Lucas-Kanade 20 Years On
An overview and extension to Lucas-Kanade image alignment algorithm, one of the most widely used techniques in computer vision.
( Source Code Available )
Human Detection and Tracking
Techniques to detect and tracking human Head/Torso/Arms/Legs by probabilistic human body representation.
( Source Code Available )
Camera Assisted Meeting Event Observer (CAMEO)
A sensory system designed to provide an electronic agent with physical awareness of the real world.
Event Detection in Videos
Our event detection method can detect a wide range of actions in video by correlating spatio-temporal shapes to over-segmented videos without background subtraction.
( Source Code Available )
Multi-People Tracking
Our multi-people tracking method can automatically initialize and terminate paths of people and follow multiple and changeable number of people on cluttered scenes over long time intervals.
Multi-view Car Detection and Registration
This method can detect cars with occlusions and varying viewpoints from a single still images by using multi-class boosting algorithm.
( Source Code Available )
Hand Tracking and 3-D Pose Estimation
A 2-D and 3-D model-based tracking method can track a human hand rapidly moving and deformed on complicated backgrounds and recover its 3-D pose parameters.
The CMU Motion of Body (MoBo) Database
The database contains 25 individuals walking on a treadmill in the CMU 3D room. The subjects perform four different walk patterns: slow walk, fast walk, incline walk and walking with a ball.
(Database Available )
Correlation Filters for Object Alignment
This method can align cars with occlusions and varying viewpoints from a single still images by using correlation filters for landmark detection.
( Source Code and Database Available )
CMU_Surveillance Research Dataset  New
This dataset is for research for researches related with surveillance cameras. We collected data with 8 synchronized camera on the path.
( Dataset Available for Everyone )

Face and Facial Expression

Principal Nested Spheres for Shape Alignment
Modeling face shapes using PNS.
( Source Code Available )
2D and 3D Face Alignment
Automatically identify key facial feature points on frontal or side view faces.
Face Tracking using AAM and Cylinder Head Tracker
A method to combine the AAM and the Cylinder Head Tracker so that we can robustly tracking the face on all view angles.
( Source Code Available )
3D Head Motion Recovery
A method to recover the full-motion (3 rotations and 3 translations) of the head using a cylindrical model.
( Source Code Available )
Hallucinating Faces
A Super-resolution algorithm to yield 4-8 fold improvements in resolution (16-64 times as many pixels) using as few as 2-3 images
( Source Code Available )
Face Detection
A neural network-based face detection system to detect faces at any degree of rotation in the image plane.
( Source Code Available )
AAM Face Modeling and Fitting
Techniques for building and fitting 2D and 3D models of human faces and heads.
Facial Expression Analysis
An interdisciplinary research group that covers expertise in facial expression analysis.
Pose, Illumination, and Expression (PIE) Face Database
A face database collection of 41,368 images of 68 people. Each face is under 13 different poses, 43 different illumination conditions, and with 4 different expressions.
Feature-based 3D Head Tracking
A feature-based head tracking algorithm can handle occlusions and fast motion of face.
Real-time Face Detection
A face detection system has an accurate detection rate and real time performance by using an ensemble of weak classifiers.
Spatio-Temporal Facial Expression Segmentation
A two-step approach temporally segment facial gestures from video sequences. It can register the rigid and non-rigid motion of the face.
All Pose Face Alignment Robust to Occlusion
A robust facial feature detection algorithm on all pose images with partial occlusions.
( Online Demonstration is under construction. )
( Source Code Available )
Robust 3D Head Tracking by View-based Feature Point Registration
A feature-based tracking method to recover the full-motion of the head using a cylindrical model.
( Source Code Available )
Hyperspectral Face Database
We collected hyperspectral face images covering the spectral range from 450nm to 1100 nm. To date the database contains 54 diverse faces at multiple sessions over a period of about two months.
( Dataset Available for Everyone )
Cohn-Kanade AU-Coded Expression Database
This dataset is for research in automatic facial image analysis and synthesis and for perceptual studies.
( Dataset Available for Everyone )
The UNBC-McMaster Shoulder Pain Expression Archive Database
This dataset is images of participant's faces (who were suffering from shoulder pain) while they were performing a series of active and passive range-of-motion tests.
( Dataset Available for Everyone )


  PIA (People Image Analysis) Workshop 2015 (12/21/2015)

  PIA (People Image Analysis) Workshop 2014 (11/26/2014)

  PIA (People Image Analysis) Workshop 2013 (12/17/2013)

  PIA (People Image Analysis) Workshop 2012 (12/13/2012)

  PIA Consortium Advisory Board Workshop 2009 (12/10/2009)

  PIA Consortium Advisory Board Workshop 2008 (12/11/2008)

  PIA Consortium Advisory Board Workshop 2007 (11/16/2007)

  PIA Consortium Advisory Board Workshop 2006 (10/06/2006)

  PIA Consortium Advisory Board Workshop 2005

  PIA Group Meeting