Foundations and Trends® in Computer Graphics and Vision > Vol 14 > Issue 1-2

Semantic Image Segmentation: Two Decades of Research

By Gabriela Csurka, Naver Labs Europe, France, Gabriela.Csurka@naverlabs.com | Riccardo Volpi, Naver Labs Europe, France, Riccardo.Volpi@naverlabs.com | Boris Chidlovskii, Naver Labs Europe, France, Boris.Chidlovskii@naverlabs.com

 
Suggested Citation
Gabriela Csurka, Riccardo Volpi and Boris Chidlovskii (2022), "Semantic Image Segmentation: Two Decades of Research", Foundations and TrendsĀ® in Computer Graphics and Vision: Vol. 14: No. 1-2, pp 1-162. http://dx.doi.org/10.1561/0600000095

Publication Date: 19 Oct 2022
© 2022 G. Csurka et al.
 
Subjects
Deep learning,  Pattern recognition and learning,  Learning and statistical methods,  Segmentation and grouping
 

Free Preview:

Download extract

Share

Download article
In this article:
Preface
1. Semantic Image Segmentation (SiS)
2. Domain Adaptation for SiS (DASiS)
3. Datasets and Benchmarks
4. Related Segmentation Tasks
5. Summary and Perspectives
Abbreviations
References

Abstract

Semantic image segmentation (SiS) plays a fundamental role in a broad variety of computer vision applications, providing key information for the global understanding of an image. This survey is an effort to summarize two decades of research in the field of SiS, where we propose a literature review of solutions starting from early historical methods followed by an overview of more recent deep learning methods including the latest trend of using transformers. We complement the review by discussing particular cases of the weak supervision and side machine learning techniques that can be used to improve the semantic segmentation such as curriculum, incremental or self-supervised learning.

State-of-the-art SiS models rely on a large amount of annotated samples, which are more expensive to obtain than labels for tasks such as image classification. Since unlabeled data is instead significantly cheaper to obtain, it is not surprising that Unsupervised Domain Adaptation (UDA) reached a broad success within the semantic segmentation community. Therefore, a second core contribution of this monograph is to summarize five years of a rapidly growing field, Domain Adaptation for Semantic Image Segmentation (DASiS) which embraces the importance of semantic segmentation itself and a critical need of adapting segmentation models to new environments. In addition to providing a comprehensive survey on DASiS techniques, we unveil also newer trends such as multi-domain learning, domain generalization, domain incremental learning, test-time adaptation and source-free domain adaptation. Finally, we conclude this survey by describing datasets and benchmarks most widely used in SiS and DASiS and briefly discuss related tasks such as instance and panoptic image segmentation, as well as applications such as medical image segmentation.

We hope that this monograph will provide researchers across academia and industry with a comprehensive reference guide and will help them in fostering new research directions in the field.

DOI:10.1561/0600000095
ISBN: 978-1-63828-076-7
172 pp. $99.00
Buy book (pb)
 
ISBN: 978-1-63828-077-4
172 pp. $290.00
Buy E-book (.pdf)
Table of contents:
Preface
1. Semantic Image Segmentation (SiS)
2. Domain Adaptation for SiS (DASiS)
3. Datasets and Benchmarks
4. Related Segmentation Tasksn
5. Summary and Perspectives
Abbreviations
References

Semantic Image Segmentation: Two Decades of Research

Semantic image segmentation (SiS) plays a fundamental role towards a general understanding of the image content and context, in a broad variety of computer vision applications, thus providing key information for the global understanding of an image.

This monograph summarizes two decades of research in the field of SiS, where a literature review of solutions starting from early historical methods is proposed, followed by an overview of more recent deep learning methods, including the latest trend of using transformers.

The publication is complemented by presenting particular cases of the weak supervision and side machine learning techniques that can be used to improve the semantic segmentation, such as curriculum, incremental or self-supervised learning. State-of-the-art SiS models rely on a large amount of annotated samples, which are more expensive to obtain than labels for tasks such as image classification. Since unlabeled data is significantly cheaper to obtain, it is not surprising that Unsupervised Domain Adaptation (UDA) reached a broad success within the semantic segmentation community. Therefore, a second core contribution of this monograph is to summarize five years of a rapidly growing field, Domain Adaptation for Semantic Image Segmentation (DASiS), which embraces the importance of semantic segmentation itself and a critical need of adapting segmentation models to new environments. In addition to providing a comprehensive survey on DASiS techniques, newer trends such as multi-domain learning, domain generalization, domain incremental learning, test-time adaptation and source-free domain adaptation are also presented. The publication concludes by describing datasets and benchmarks most widely used in SiS and DASiS and briefly discusses related tasks such as instance and panoptic image segmentation, as well as applications such as medical image segmentation.

This monograph should provide researchers across academia and industry with a comprehensive reference guide, and will help them in fostering new research directions in the field.

 
CGV-095