A Survey of Metrics Used for Image Colorization Assessment
Nicolas Maignan, Fabien Pierre, Frédéric Sur
⚠ This is a preprint. It may change before it is accepted for publication.

Abstract

The objective evaluation of image and video colorization remains challenging. Metrics such as PSNR and SSIM have limited ability to capture perceptual quality and typical colorization artifacts. Furthermore, they require a reference color image, which is generally unavailable for the colorization task. Alternatively, referenceless evaluation metrics have been proposed, aiming to better reflect human perception, yet few studies have analyzed their effectiveness. In this paper, we provide a comprehensive review of the evaluation metrics employed by recent colorization methods. We introduce a framework to assess these metrics based on their sensitivity to synthetic transformations that simulate common colorization defects, such as color bleeding or non-semantic colors. This evaluation is conducted both in the static case of images and in the temporal dimension for video colorization. Besides, we propose extending to color images two computationally lightweight perceptual metrics previously defined in the literature for grayscale images.

Download