eMotion-GAN

Overview

eMotion-GAN is a novel motion‑based Generative Adversarial Network for photorealistic frontal view synthesis that preserves facial expressions. Our approach disentangles facial motion from identity and appearance, enabling high‑quality frontalization while maintaining the original expression dynamics. The method demonstrates state‑of‑the‑art performance on frontal view synthesis and cross‑subject facial motion transfer.

Overview of the eMotion-GAN framework

Abstract

Facial expression recognition (FER) systems frequently suffer significant performance degradation when confronted with head pose variations, a pervasive challenge in real-world applications ranging from healthcare monitoring to human–computer interaction. While existing frontal view synthesis (FVS) methods attempt to address this issue, they predominantly operate in the appearance domain, often introducing artifacts that distort the subtle motion patterns crucial for accurate expression analysis. We present eMotion-GAN, a two-stage generative motion-domain framework that fundamentally rethinks frontalization by decomposing facial dynamics into two distinct components: (1) expression-related motion stemming from muscle activity, and (2) pose-related motion acting as noise. We conducted extensive evaluations using several widely recognized dynamic FER datasets, which encompass sequences exhibiting various degrees of head pose variations in both intensity and orientation. Our results demonstrate the effectiveness of our approach in significantly reducing the FER performance gap between frontal and non-frontal faces. Specifically, we achieved a FER improvement of up to +5% for small pose variations and up to +20% improvement for larger pose variations.

Explainer Video

Watch our explainer video for detailed explanations and more results:

Interactive Demo

Try our interactive visualization demo using Google Colab (no GPU needed):

Open Colab Demo

Frontal View Synthesis & Expression Preservation

eMotion-GAN generates photorealistic frontal views while preserving the original facial expressions, even under extreme poses and lighting conditions.

Comparison of frontalization results with state‑of‑the‑art methods

Motion Frontalization & Expression Embedding

Cross-subject Facial Motion Transfer

Our model can transfer facial motion from a source subject to a target subject while maintaining the target's identity and the source's expression dynamics.

Motion transfer examples

Citation

If you find this work useful, please cite our paper: