Foundations and Trends® in Robotics > Vol 8 > Issue 1–2

Semantics for Robotic Mapping, Perception and Interaction: A Survey

By Sourav Garg, Queensland University of Technology, Australia | Niko Sünderhauf, Queensland University of Technology, Australia | Feras Dayoub, Queensland University of Technology, Australia | Douglas Morrison, Queensland University of Technology, Australia | Akansel Cosgun, Monash University, Australia | Gustavo Carneiro, University of Adelaide, Australia | Qi Wu, University of Adelaide, Australia | Tat-Jun Chin, University of Adelaide, Australia | Ian Reid, University of Adelaide, Australia | Stephen Gould, Australian National University, Australia | Peter Corke, Queensland University of Technology, Australia | Michael Milford, Queensland University of Technology, Australia, michael.milford@qut.edu.au

 
Suggested Citation
Sourav Garg, Niko Sünderhauf, Feras Dayoub, Douglas Morrison, Akansel Cosgun, Gustavo Carneiro, Qi Wu, Tat-Jun Chin, Ian Reid, Stephen Gould, Peter Corke and Michael Milford (2020), "Semantics for Robotic Mapping, Perception and Interaction: A Survey", Foundations and Trends® in Robotics: Vol. 8: No. 1–2, pp 1-224. http://dx.doi.org/10.1561/2300000059

Publication Date: 23 Dec 2020
© 2020 Sourav Garg et al.
 
Subjects
 

Free Preview:

Download extract

Share

Download article
In this article:
1. Introduction
2. Static and Un-Embodied Scene Understanding
3. Dynamic Environment Understanding and Mapping
4. Interacting with Humans and the World
5. Improving Task Capability
6. Practical Aspects: Applications and Enhancers
7. Discussion and Conclusion
Acknowledgments
References

Abstract

For robots to navigate and interact more richly with the world around them, they will likely require a deeper understanding of the world in which they operate. In robotics and related research fields, the study of understanding is often referred to as semantics, which dictateswhat does the world “mean” to a robot, and is strongly tied to the question ofhow to represent that meaning. With humans and robots increasingly operating in the same world, the prospects of human–robot interaction also bringsemantics and ontology of natural language into the picture. Driven by need, as well as by enablers like increasing availability of training data and computational resources, semantics is a rapidly growing research area in robotics. The field has received significant attention in the research literature to date, but most reviews and surveys have focused on particular aspects of the topic: the technical research issues regarding its use in specific robotic topics like mapping or segmentation, or its relevance to one particular application domain like autonomous driving. A new treatment is therefore required, and is also timely because so much relevant research has occurred since many of the key surveys were published. This survey therefore provides an overarching snapshot of where semantics in robotics stands today. We establish a taxonomy for semantics research in or relevant to robotics, split into four broad categories of activity, in which semantics are extracted, used, or both. Within these broad categories we survey dozens of major topics including fundamentals from the computer vision field and key robotics research areas utilizing semantics, including mapping, navigation and interaction with the world. The survey also covers key practical considerations, including enablers like increased data availability and improved computational hardware, and major application areas where semantics is or is likely to play a key role. In creating this survey, we hope to provide researchers across academia and industry with a comprehensive reference that helps facilitate future research in this exciting field.

DOI:10.1561/2300000059
ISBN: 978-1-68083-768-1
236 pp. $99.00
Buy book (pb)
 
ISBN: 978-1-68083-769-8
236 pp. $280.00
Buy E-book (.pdf)
Table of contents:
1. Introduction
2. Static and Un-Embodied Scene Understanding
3. Dynamic Environment Understanding and Mapping
4. Interacting with Humans and the World
5. Improving Task Capability
6. Practical Aspects: Applications and Enhancers
7. Discussion and Conclusion
Acknowledgments
References

Semantics for Robotic Mapping, Perception and Interaction: A Survey

For robots to navigate and interact more richly with the world around them, they will likely require a deeper understanding of the world in which they operate. In robotics and related research fields, the study of understanding is often referred to as semantics, which dictates what does the world ‘mean’ to a robot, and is strongly tied to the question of how to represent that meaning. With humans and robots increasingly operating in the same world, the prospects of human-robot interaction also bring semantics and ontology of natural language into the picture.

Driven by need, as well as by enablers like increasing availability of training data and computational resources, semantics is a rapidly growing research area in robotics. The field has received significant attention in the research literature to date, but most reviews and surveys have focused on particular aspects of the topic: the technical research issues regarding its use in specific robotic topics like mapping or segmentation, or its relevance to one particular application domain like autonomous driving. A new treatment is therefore required, and is also timely because so much relevant research has occurred since many of the key surveys were published.

This survey provides an overarching snapshot of where semantics in robotics stands today. We establish a taxonomy for semantics research in or relevant to robotics, split into four broad categories of activity in which semantics are extracted, used, or both. Within these broad categories, we survey dozens of major topics including fundamentals from the computer vision field and key robotics research areas utilizing semantics such as mapping, navigation and interaction with the world. The survey also covers key practical considerations, including enablers like increased data availability and improved computational hardware, and major application areas where semantics is or is likely to play a key role. In creating this survey, we hope to provide researchers across academia and industry with a comprehensive reference that helps facilitate future research in this exciting field.

 
ROB-059