Face Alignment Using K-cluster Regression Forests

In this work we propose a novel face alignment pipeline consisting of three stages: Affine Pose Regression, 3D-Affine Pose Regression and face alignment itself. The first two stages of the pipeline aims to improve the face shape initialization in terms of translation, scale and in-plane rotation. The second step further refines the initialization in terms of out-of-plane rotation. The combined pipeline achieves state-of-the-art results on the entire 300-W face alignment dataset. You can read an article about our method, published in IEEE Signal Processing Letters here.

Markerless motion capture and face animation

Together with Jacek Naruniec and a US based startup, we have created an application that captures the head pose, facial expression and gaze direction of any person and animates a face model that mimics the behavior of that person. It works in real time and requires only a standard webcam. Below you can see a couple of images that demonstrate its behavior.

www.ire.pw.edu.pl_jnaruniec_wp-content_uploads_2015_10_animacja1.jpg

Live 3D reconstruction using multiple Kinect v2 sensors

In a project financed by our faculty's Dean we have designed and implemented a system that uses multiple Kinect v2 devices for real time 3D reconstruction of a scene from multiple viewpoints. Each sensor is connected to a separate computer that communicates over ethernet with a server to allow calibration, noise removal and synchronized frame capture.

The system, has been described in “LiveScan3D: A Fast and Inexpensive 3D Data Acquisition System for Multiple Kinect v2 Sensors” which was presented on the 3DV 2015 conference. You can download the poster that accompanied the article here.

LiveScan3D is available as source on its website on GitHub and as a set of Windows binaries on its GitHub release page.

Publications

  1. Kowalski M., Naruniec J., Trzcinski T.: Deep Alignment Network: A convolutional neural network for robust face alignment. 2017 IEEE International Conference on Computer Vision and Pattern Recognition Workshop, 2017 [ code ]
  2. Wrobel K., Doroz R., Porwik P., Naruniec J., Kowalski M.: Using a Probabilistic Neural Network for Lip-based Biometric Verification. Engineering Applications of Artificial Intelligence, 2017
  3. Kowalski M., Naruniec J.: Face Alignment Using K-cluster Regression Forests With Weighted Splitting. Signal Processing Letters, 2016
  4. Naruniec J., Wieczorek M., Szlufik S., Koziorowski D., Tomaszewski M., Kowalski M., Przybyszewski A.: Webcam based system for video occulography. IET Computer Vision, 2016
  5. Kowalski M., Naruniec J., Daniluk M.: LiveScan3D: A Fast and Inexpensive 3D Data Acquisition System for Multiple Kinect v2 Sensors. 3D Vision (3DV), International Conference on, pp.318-325, Lyon, France, 2015
  6. Strupczewski A., Czupryński B., Skarbek W., Kowalski M., Naruniec J.: Head pose tracking from RGBD sensor based on direct motion estimation. Conference proceedings: 6th International Conference on Pattern Recognition and Machine Intelligence, Warsaw, Poland, 2015
  7. Naruniec J., Kowalski M., Daniluk M.: 3D Face Data Acquisition and Modelling Based on an RGBD Camera Matrix. Conference proceedings: 8th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, Warsaw, Poland, 2015
  8. Rybus T., Nicolau-Kukliński J., Seweryn K., Barciński T., Ciesielska M., Grassmann K., Grygorczuk J., Karczewski M., Kowalski M., Krzewski M., Kuciński T., Lisowski J., Przybyła R., Skup K., Szewczyk T., Wawrzaszek R.: New Planar Air-bearing Microgravity Simulator for Verification of Space Robotics Numerical Simulations and Control Algorithms. Conference proceedings: 12th Symposium on Advanced Space Technologies in Robotics and Automation (ASTRA 2013). ESTEC, Noordwijk, Netherlands, 2013
  9. Kowalski M., Naruniec J.: Evaluation of active appearance models in varying background conditions. Conference proceedings: Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2013. Wilga, Poland, 2013

Master Thesis