eduzhai > Applied Sciences > Engineering >

Generalized Zero and Few-Shot Transfer for Facial Forgery Detection

  • Save

... pages left unread,continue reading

Document pages: 17 pages

Abstract: We propose Deep Distribution Transfer(DDT), a new transfer learning approachto address the problem of zero and few-shot transfer in the context of facialforgery detection. We examine how well a model (pre-)trained with one forgerycreation method generalizes towards a previously unseen manipulation techniqueor different dataset. To facilitate this transfer, we introduce a new mixturemodel-based loss formulation that learns a multi-modal distribution, with modescorresponding to class categories of the underlying data of the source forgerymethod. Our core idea is to first pre-train an encoder neural network, whichmaps each mode of this distribution to the respective class labels, i.e., realor fake images in the source domain by minimizing wasserstein distance betweenthem. In order to transfer this model to a new domain, we associate a fewtarget samples with one of the previously trained modes. In addition, wepropose a spatial mixup augmentation strategy that further helps generalizationacross domains. We find this learning strategy to be surprisingly effective atdomain transfer compared to a traditional classification or evenstate-of-the-art domain adaptation few-shot learning methods. For instance,compared to the best baseline, our method improves the classification accuracyby 4.88 for zero-shot and by 8.38 for the few-shot case transferred from theFaceForensics++ to Dessa dataset.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...