APSIPA Transactions on Signal and Information Processing > Vol 13 > Issue 5

Estimating 3D Hand Poses and Shapes from Silhouettes

Li-Jen Chang, National Tsing Hua University, Taiwan, Yu-Cheng Liao, National Tsing Hua University, Taiwan, Chia-Hui Lin, National Tsing Hua University, Taiwan, Shys-Fang Yang-Mao, National Tsing Hua University, Taiwan, Hwann-Tzong Chen, National Tsing Hua University, Taiwan, htchen@cs.nthu.edu.tw
 
Suggested Citation
Li-Jen Chang, Yu-Cheng Liao, Chia-Hui Lin, Shys-Fang Yang-Mao and Hwann-Tzong Chen (2024), "Estimating 3D Hand Poses and Shapes from Silhouettes", APSIPA Transactions on Signal and Information Processing: Vol. 13: No. 5, e403. http://dx.doi.org/10.1561/116.20240013

Publication Date: 07 Oct 2024
© 2024 L.-J. Chang, Y.-C. Liao, C.-H. Lin, S.-F. Yang-Mao and H.-T. Chen
 
Subjects
3D reconstruction and image-based modeling,  Rendering,  Shape,  Shape representation,  Learning and statistical methods
 
Keywords
Hand pose estimationhand shape estimationdifferentiable rendering
 

Share

Open Access

This is published under the terms of CC BY-NC.

Downloaded: 212 times

In this article:
Introduction 
Related Work 
Our Approach 
Experiments 
Limitations 
Conclusion 
References 

Abstract

We present Mask2Hand, a self-trainable method for predicting 3D hand pose and shape from a single 2D binary silhouette. Without additional manual annotations, our method uses differentiable rendering to project 3D estimations onto the 2D silhouette. A tailored loss function, applied between the rendered and input silhouettes, provides a self-guidance mechanism during end-to-end optimization, which constrains global mesh registration and hand pose estimation. Our experiments show that Mask2Hand, using only a binary mask input, achieves accuracy comparable to state-ofthe- art methods requiring RGB or depth inputs on both unaligned and aligned datasets.

DOI:10.1561/116.20240013

Companion

APSIPA Transactions on Signal and Information Processing Special Issue - Invited Papers from APSIPA ASC 2023
See the other articles that are part of this special issue.