Foundations and Trends® in Computer Graphics and Vision > Vol 12 > Issue 4

Deep Learning for Multimedia Forensics

By Irene Amerini, Sapienza University of Rome, Italy, amerini@diag.uniroma1.it | Aris Anagnostopoulos, Sapienza University of Rome, Italy, aris@diag.uniroma1.it | Luca Maiano, Sapienza University of Rome and ELIS Innovation Hub, Italy, maiano@diag.uniroma1.it | Lorenzo Ricciardi Celsi, ELIS Innovation Hub, Italy, l.ricciardicelsi@elis.org

 
Suggested Citation
Irene Amerini, Aris Anagnostopoulos, Luca Maiano and Lorenzo Ricciardi Celsi (2021), "Deep Learning for Multimedia Forensics", Foundations and TrendsĀ® in Computer Graphics and Vision: Vol. 12: No. 4, pp 309-457. http://dx.doi.org/10.1561/0600000096

Publication Date: 31 Aug 2021
© 2021 Irene Amerini, Aris Anagnostopoulos, Luca Maiano and Lorenzo Ricciardi Celsi
 
Subjects
Forensics,  E-Crime,  Signal processing for security and forensic analysis,  Image and video processing,  Speech/audio/image/video compression
 

Free Preview:

Download extract

Share

Download article
In this article:
1. Introduction
2. Generating Fake Image and Video Content
3. Forgery Detection on Images and Videos
4. Assessing the Origin of Multimedia Content
5. Deepfakes: Strategies to Detect Artificially Generated Content
6. Evaluation Metrics for Multimedia Forensics
7. Multimedia Forensic Datasets
8. Discussion and Conclusions
Appendices
References

Abstract

In the last two decades, we have witnessed an immense increase in the use of multimedia content on the internet, for multiple applications ranging from the most innocuous to very critical ones. Naturally, this emergence has given rise to many types of threats posed when this content can be manipulated/used for malicious purposes. For example, fake media can be used to drive personal opinions, ruining the image of a public figure, or for criminal activities such as terrorist propaganda and cyberbullying. The research community has of course moved to counter attack these threats by designing manipulation-detection systems based on a variety of techniques, such as signal processing, statistics, and machine learning. This research and practice activity has given rise to the field of multimedia forensics.

The success of deep learning in the last decade has led to its use in multimedia forensics as well. In this survey, we look at the latest trends and deep-learning-based techniques introduced to solve three main questions investigated in the field of multimedia forensics. We begin by examining the manipulations of images and videos produced with editing tools, reporting the deep-learning approaches adopted to counter these attacks. Next, we move on to the issue of the source camera model and device identification, as well as the more recent problem of monitoring image and video sharing on social media. Finally, we look at the most recent challenge that has emerged in recent years: recognizing deepfakes, which we use to describe any content generated using artificial-intelligence techniques; we present the methods that have been introduced to show the existence of traces left in deepfake content and to detect them. For each problem, we also report the most popular metrics and datasets used today.

DOI:10.1561/0600000096
ISBN: 978-1-68083-854-1
165 pp. $99.00
Buy book (pb)
 
ISBN: 978-1-68083-855-8
165 pp. $140.00
Buy E-book (.pdf)
Table of contents:
1. Introduction
2. Generating Fake Image and Video Content
3. Forgery Detection on Images and Videos
4. Assessing the Origin of Multimedia Content
5. Deepfakes: Strategies to Detect Artificially Generated Content
6. Evaluation Metrics for Multimedia Forensics
7. Multimedia Forensic Datasets
8. Discussion and Conclusions
Appendices
References

Deep Learning for Multimedia Forensics

Over the past few decades, we have witnessed a huge increase in the use of multimedia content on the internet, for multiple applications ranging from innocuous to critical ones. This development has led to threats posed when content can be manipulated/used for malicious purposes. For example, fake media can be used to drive personal opinions, or for criminal activities such as terrorist propaganda and cyberbullying. This research and practice activity gave rise to the creation of the multimedia forensics field.

In this survey, the latest trends and deep-learning-based techniques for multimedia forensics are introduced, in both architectural and data-processing. Firstly, different techniques used to manipulate content are presented, followed by image and video forgery techniques. Thereafter, deep learning methods for source identification and recent solutions for deepfake detection are covered. Datasets and evaluation metrics are included, and conclusions are presented.

The publication is intended for researchers, students and professionals active in the fields of Deep Learning and Multimedia Forensics.

 
CGV-096