Edited by William Press, The University of Texas at Austin, Austin, TX; received September 23, 2022; accepted October 3, 2022.
November 23, 2022
119 (48) e2216035119
Abstract
Since their emergence a few years ago, artificial intelligence (AI)-synthesized media—so-called deep fakes—have dramatically increased in quality, sophistication, and ease of generation. Deep fakes have been weaponized for use in nonconsensual pornography, large-scale fraud, and disinformation campaigns. Of particular concern is how deep fakes will be weaponized against world leaders during election cycles or times of armed conflict. We describe an identity-based approach for protecting world leaders from deep-fake imposters. Trained on several hours of authentic video, this approach captures distinct facial, gestural, and vocal mannerisms that we show can distinguish a world leader from an impersonator or deep-fake imposter.
Data, Materials, and Software Availability
Some study data available (The data associated with this manuscripts includes training videos which we will make available. We prefer not to make available the other data in the form of the trained behavioral models because we fear that this could be used by an adversary to evaluate the realism of fake videos. We will, however, upon request make our model available to researchers working in the general space of digital forensics) (18).
Acknowledgments
Author contributions
M.B. and H. F. designed research, performed research, contributed new reagents/analytic tools, analyzed data, and wrote the paper.
Competing interests
The authors declare no competing interest.
References
2
B. Chesney, D. Citron, Deep fakes: A looming challenge for privacy, democracy, and national security. Calif. Law Rev. 107, 1753 (2019).
4
M. Groh, Z. Epstein, C. Firestone, R. Picard, Deepfake detection by human crowds, machines, and machine-informed crowds. Proc. Natl. Acad. Sci. U.S.A. 119, e2110013119 (2022).
5
P. Zhou, X. Han, V. I. Morariu, L. S. Davis, Two-stream neural networks for tampered face detection., International Conference on Computer Vision and Pattern Recognition (2017).
7
Y. Li et al., In ictu oculi: Exposing AI created fake videos by detecting eye blinking., International Workshop on Information Forensics and Security pp. 1–7. (2018).
8
S. Agarwal, H. Farid, O. Fried, M. Agrawala, Detecting deep-fake videos from phoneme-viseme mismatches in CVPR Workshop on Media Forensics. pp. 660–661. (2020).
9
S. Agarwal et al., Protecting world leaders against deep fakes in CVPR Workshop on Media Forensics. 1. (2019).
10
D. Cozzolino, A. Rössler, J. Thies, M. Nießner, L. Verdoliva, ID-reveal: Identity-aware deepfake video detection in International Conference on Computer Vision. pp. 15108–15117. (2021).
11
A. Rössler et al., FaceForensics++: Learning to detect manipulated facial images in International Conference on Computer Vision and Pattern Recognition. (2019).
12
T. Baltrusaitis, A. Zadeh, Y. C. Lim, L. P. Morency, Openface 2.0: Facial behavior analysis toolkit in IEEE International Conference on Automatic Face& Gesture Recognition. pp. 59–66. (2018).
13
R. Gross, I. Matthews, J. Cohn, T. Kanade, S. Baker, Multi-pie in IEEE International Conference on Automatic Face& Gesture Recognition. pp. 1–8. (2008).
14
P. Ekman, W. V. Friesen, Measuring facial movement. Environ. Psychol. Nonverbal Behav. 1, 56–75 (1976).
17
M. Boháček, M. Hrúz, Sign pose-based transformer for word-level sign language recognition in IEEE/CVF Winter Conference on Applications of Computer Vision Workshops. pp. 182–191. (2022).
18
A. Bauer, The Use of Signing Space in a Shared Sign Language of Australia (De Gruyter, ed. 1, 2014).
19
S. Davis, P. Mermelstein, Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoust. Speech Signal Process. 28, 357–366 (1980).
20
F. Pedregosa et al., Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
Information & Authors
Information
Published in
Proceedings of the National Academy of Sciences
Vol. 119 | No. 48
November 29, 2022
Classifications
Copyright
Data, Materials, and Software Availability
Some study data available (The data associated with this manuscripts includes training videos which we will make available. We prefer not to make available the other data in the form of the trained behavioral models because we fear that this could be used by an adversary to evaluate the realism of fake videos. We will, however, upon request make our model available to researchers working in the general space of digital forensics) (18).
Submission history
Received: September 23, 2022
Accepted: October 3, 2022
Published online: November 23, 2022
Published in issue: November 29, 2022
Keywords
- synthetic media
- deep fakes
- disinformation
- digital forensics
Acknowledgments
Author Contributions
M.B. and H. F. designed research, performed research, contributed new reagents/analytic tools, analyzed data, and wrote the paper.
Competing Interests
The authors declare no competing interest.
Notes
‡Two aspects of the default OpenFace2 configuration are changed to ensure more consistent facial tracking: 1) the Multi-PIE (10) model is always used and 2) the temporal smoothing of action units is disabled.
Authors
Affiliations
Matyáš Boháček1
Gymnasium of Johannes Kepler 169 00 Prague, Czech Republic
Department of Electrical and Computer Sciences, School of Information, University of California, Berkeley, CA 94708
Notes
1M.B. and H.F. contributed equally to this work.
Metrics & Citations
Metrics
Citation statements
Altmetrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
View Options
View options
PDF format
Download this article as a PDF file
DOWNLOAD PDF
Media
Figures
Tables
Other
RSS - Posts
