APSIPA Transactions on Signal and Information Processing > Vol 12 > Issue 1

Lightweight Quality Evaluation of Generated Samples and Generative Models

Ganning Zhao, University of Southern California, USA, ganningz@usc.edu , Vasileios Magoulianitis, University of Southern California, USA, Suya You, DEVCOM Army Research Laboratory, USA, C.-C. Jay Kuo, University of Southern California, USA
 
Suggested Citation
Ganning Zhao, Vasileios Magoulianitis, Suya You and C.-C. Jay Kuo (2023), "Lightweight Quality Evaluation of Generated Samples and Generative Models", APSIPA Transactions on Signal and Information Processing: Vol. 12: No. 1, e33. http://dx.doi.org/10.1561/116.00000076

Publication Date: 25 Jul 2023
© 2023 G. Zhao, V. Magoulianitis, S. You and C.-C. J. Kuo
 
Subjects
 
Keywords
Generative modelquality evaluationquality controlgreen learninggreen AI
 

Share

Open Access

This is published under the terms of CC BY-NC.

Downloaded: 556 times

In this article:
Introduction 
Related Work 
Proposed LGSQE Method 
Experiments 
Conclusion and Future Work 
References 

Abstract

Although there are metrics to evaluate the performance of generative models, little research is conducted on the quality evaluation of individual generated samples. A lightweight generated sample quality evaluation (LGSQE) method is proposed in this work. LGSQE trains a binary classifier to differentiate real and synthetic images from a generative model and, then, uses it to assign a soft label between zero and one to a generated sample as its quality index. LGSQE can reject poor generations and serve as a post-processing module for quality control. Furthermore, by aggregating quality indices of a large number of generated samples, LGSQE offers four metrics (i.e., classification accuracy (Acc), the area under the curve (AUC), precision, and recall) to evaluate the performance of a generative model as a byproduct. LGSQE demands a significantly smaller memory size and faster evaluation time while preserving the same rank order predicted by the Fr├ęchet Inception Distance (FID). Extensive experiments are conducted to demonstrate the effectiveness and efficiency of LGSQE.

DOI:10.1561/116.00000076