Synthetic to Authentic:

Transferring Realism to 3D Face Renderings for Boosting Face Recognition

Parsa Rahimi1,2, Behrooz Razeghi2, Sébastien Marcel2,3
1EPFL, 2Idiap, 2UNIL
Paper arXiv Code Dataset

Syn2Auth We observe with limited data we can make the face images in the 3D face rendering datasets look more realistic, using off-the-shelf image-to-image translation tasks.

Abstract

In this paper, we investigate the potential of image-to-image translation (I2I) techniques for transferring realism to 3D-rendered facial images in the context of Face Recognition (FR) systems. The primary motivation for using 3D-rendered facial images lies in their ability to circumvent the challenges associated with collecting large real face datasets for training FR systems. These images are generated entirely by 3D rendering engines, facilitating the generation of synthetic identities.

However, it has been observed that FR systems trained on such synthetic datasets underperform when compared to those trained on real datasets, on various FR benchmarks. In this work, we demonstrate that by transferring the realism to 3D-rendered images (i.e., making the 3D-rendered images look more real), we can boost the performance of FR systems trained on these more photorealistic images. This improvement is evident when these systems are evaluated against FR benchmarks like IJB-C, LFW which utilize real-world data by 2% to 5%, thereby paving new pathways for employing synthetic data in real-world applications.

In each row the left images are presenting the original image from DigiFace1M and the right one is after realism transfer.

ROC Curve Before and After realism transfer

Here, we are observing after applying the realism the ROC curve becomes closer to the other methods which are using much more real data. Also note that in comparison to the DigiFace1M (solid red) how much the the verification accuracy is improving.

High Quality Image Benchmarks

In the following benchmarks we also observe that the accuracy increased significantly over the accuracy of the original DigiFace1M.

Dataset

We release the Realism Transferred DigiFace1M ( RealDigiFace) using two methods which were reported in the paper, CodeFormer and VSAIT.

The dataset is released in three formats, MX `rec` files, image folder tarified (i.e., for usage with ImageTar dataloader) or uncompressed image folder hierarchy.

Paper

BibTeX

@article{rahimi2024synthetic,
      title={Synthetic to Authentic: Transferring Realism to 3D Face Renderings for Boosting Face Recognition},
      author={Rahimi, Parsa and Razeghi, Behrooz and Marcel, Sebastien},
      journal={arXiv preprint arXiv:2407.07627},
      year={2024}
    }