FaceTracker is a C/C++ API for real time generic non-rigid face alignment and tracking.


Non-rigid face alignment and tracking is a common problem in computer vision. It is the front-end to many algorithms that require registration, for example face and expression recognition. However, those working on algorithms for these higher level tasks are often unfamiliar with the tools and peculiarities regarding non-rigid registration (i.e. pure machine learning scientists, psychologists, etc.). Even those directly working on face alignment and tracking often find implementing an algorithm from published work to be a daunting task, not least because baseline code against which performance claims can be assessed does not exist. As such, the goal of FaceTracker is to provide source code and pre-trained models that can be used out-of-the-box, for the dual purpose of:

  1. 1. Promoting the advancement of higher level inference algorithms that require registration.

  2. 2. Providing baseline code to promote quantitative improvements in face registration.


  1. Real time: ranging from 20-30 fps (depending on processor, compiler and use of OpenMP)

  2. Generic: designed to work for most people under most conditions

  3. No training required: a pre-trained model is provided

  4. Detection based initialisation: no user intervention required

  5. Automatic failure detection: requires no user re-initialisation

  6. Camera or video input


FaceTracker is available for download (for research purposes only). The library includes the C/C++ API, example code for interfacing with the API, a pre-trained model and documentation. To download it, please send an email to Jason Saragih (Jason.saragih@csiro.au).

The Tracker:

The code requires OpenCV 2.0 and the provided model was trained using the MultiPIE database. The tracker is based on a modified version of the constrained local model described in:

J. Saragih, S. Lucey and J. Cohn, „Deformable Model Fitting by Regularized Landmark Mean-Shifts“, International Journal of Computer Vision (IJCV)

People Using FaceTracker:

Some people are using FaceTracker to do some really cool stuff:

  1. FaceOSC

  2. Face Cloning

  3. A music Video!

  4. Face Projection

  5. Being John Malkovich

  6. pkmFace


Below are some examples of difficult YouTube videos that were processed using FaceTracker with the pre-trained model. There was absolutely no user intervention in any of the videos.