ipol.im » pub » algo »  
IPOL : Retinex Poisson Equation : a Model for Color Perception
  1. Contacts
  2. References
  3. Overview
  4. Online Demo
  5. Software
  6. The PDE-Retinex Model
  7. The Algorithm
  8. Examples

Contacts

References

  1. Jean-Michel Morel, Ana Belén Petro and Catalina Sbert, "PDE Formalization of the Retinex Theory".
    Submitted to IEEE Trans. on Image Processing.
  2. Jean-Michel Morel, Ana Belén Petro and Catalina Sbert, "Fast Implementation of color constancy algorithms".
    Color Imaging XIV: Displaying, Processing, Hardcopy and Application.
    Proc. of Electronic Imaging SPIE, vol 7241. January 2009. (preprint)
  3. P. Pérez, M. Gangnet and A. Blake, "Poisson Image Editing". ACM Transactions on Image Processing.
    Proc. of ACM SIGGRAPH 2003, vol 22, Issue 3 (July 2003). Pages: 313 - 318

Overview

In 1964 Edwin H. Land formulated the Retinex theory, the first attempt to simulate and explain how the human visual system perceives color. His theory and an extension, the ``reset Retinex'' were further formalized by Land and McCann. Several Retinex algorithms have been developed ever since. These color constancy algorithms modify the RGB values at each pixel to give an estimate of the physical color independent of the shading.

Unfortunately, the Retinex original algorithm is both complex and not fully specified. Indeed, this algorithm computes at each pixel an average of a very large and unspecified set of paths on the image. For this reason, Retinex has received several interpretations and implementations which, among other aims, attempt to tune down its excessive complexity.

But, as shown in the references, the original Retinex algorithm can be formalized as a (discrete) partial differential equation. More precisely, it can be shown that if the Retinex paths are interpreted as symmetric random walks, then Retinex is equivalent to a Neumann problem for a linear screened Poisson equation. This result gives a fast algorithm involving just one parameter, also present in the original theory. The obtained screened Poisson equation turns out to be very similar to Horn’s and Blake’s equations, which were proposed as alternatives to Retinex. To the best of our knowledge, this striking similarity of the three models had not yet been pointed out. The obtained Retinex PDE also turns out to be one of the “Poisson editing” equations proposed in Perez et al. (see references). We can directly quote from these authors:

“First, it is well known to psychologists [Land and McCann 1971] that slow gradients of intensity, which are suppressed by the Laplacian operator, can be superimposed on an image with barely noticeable effect. Conversely, the second-order variations extracted by the Laplacian operator are the most significant perceptually. Secondly, a scalar function on a bounded domain is uniquely defined by its values on the boundary and its Laplacian in the interior. The Poisson equation therefore has a unique solution and this leads to a sound algorithm.”

This yields an exact and fast implementation of the Land-McCann theory using only two DFT's. Test the theory on line on your own color images!

Online Demo

An online demo allows you to try Retinex with your own images. The demo has only one parameter, the contrast threshold present in the original theory. The images can be slightly resized for an efficient Fourier transform.

As the the input image has to be normalized after the Retinex algorithm, the demo also provides the result of this normalization without Retinex. We use the "Simplest Color Balance" algorithm for this normalization.

Software

The Retinex software that allows you to try the algorithm on your machine is available.

It should compile on any system since it's only ANSI C. This implementation is used in the online demo.

This code requires libtiff for TIFF input/output and libfftw3 to process the Fourier transforms.

Compilation and usage instructions are provided in the README.txt file.

The PDE-Retinex Model

In the IEEE article it is proven that the output of the retinex algorithm proposed by Land and McCann is the solution of the discrete partial differential equation with Neumann boundary conditions

- \Delta_d u(i,j) + \frac{4}{\dim - 1} u(i,j) = F(i,j)

where

- \Delta_d u(i,j) = u(i+1,j) + u(i-1,j) + u(i,j+1) + u(i,j-1) - 4 u(i,j)

is the discrete Laplacian, dim = N M is the size of the image,

F(i,j) = f(I(i,j) - I(i+1,j)) + f(I(i,j) - I(i-1,j)) + f(I(i,j) - I(i,j+1)) + f(I(i,j) - I(i,j-1))

and f(x) is a threshold function, whose value is zero if |x| < t and the identity in other case and I is the image to process. This function f eliminates the small variations of the intensity image I.

The parameter t (the threshold) is by default t = 3 but you can choose the value depending of the variations you want to eliminate.

The Algorithm

The output of the algorithm are two images: the first one is the white balance of the original color image (on each channel the darkest pixels are put to zero and the brightest ones are put to 255); the second image is the result of the Retinex PDE applied to the white balanced image.

The discrete partial differential equation is easily solved by fast Fourier transform. Applying the Fourier transform to the discrete partial differential equation yields

\hat{u}(k,l) \cdot \left(4 + \frac{4}{\dim - 1} - 2 \cos\left(\frac{2k\pi}{N}\right) - 2 \cos\left(\frac{2l\pi}{M}\right)\right) = \hat{F}(k,l)

The algorithm is

  1. Compute F(i,j);
  2. Compute the Fourier transform of F by DFT;
  3. Deduce the Fourier transform of u using the formula above;
  4. Compute the final solution u using the inverse DFT.

Examples

Here there are some examples, note that Retinex is not a model for image enhancement or image improvement; it is only an algorithm to mimic our color perception. Thus, Retinex will enhance an effect that our perception does anyway.

The role of the t threshold is to eliminate the small intensity variations due to shading. Thus t cannot be too large (less than 10 typically) to avoid removing significant details. But is must be large enough to eliminate light shading effects. This is not always possible, since shadows can very contrasted.

Adelson's checker

A first classic example, shows Adelson's checker shadow illusion. In the left image a green cylinder standing on a black and white checker-board casts a diagonal shadow across the board.

The image has been so constructed that the white squares in the shadow, one of which is labeled "B," have actually the very same gray value as the black squares outside the shadow, one of which is labeled "A."

If Retinex is faithful to human perception, it should make B much brighter than A, and it does.

original image
(A and B have a gray value of 120) 		white balance image
(A and B have gray value of 88)
Retinex result with t=3
(A has a gray value of 75 and B of 100) 		Retinex result with t=8

Circles on a Gradient

Simultaneous contrast, namely the fact that the appearance of a color depends on the colors surrounding it. The original image (already white balanced) shows a background with a smooth variation and two circles with the same gray value (170). One of them is placed in the darker part of the image, and the other one in the brighter part.

The usual perception is that the circle in the darker part looks conspicuously brighter than the other. If we use a threshold t=3 larger than the background variation, the result is an image with constant background (130). The left circle gets a 105 gray value and the right circle a 235 gray value, which predicts well with our perception.

original image, white balanced Retinex result

Color Illusion

This example shows the successful simulation of a color illusion. On the left image, two X's with exactly the same color are put on different backgrounds, one yellow and the other purple.

The striking illusion is that the X on the left seems to have a color similar to the purple background color on the right, while the X on the right has an apparent color similar to the yellow background color on the left. Applying the Retinex algorithm the X's really get these illusory colors.

Noisy Image

To understand the effect of the threshold t the next image shows a noisy original and the result of Retinex with increasing threshold values t=0, t=10 and t=15.

The background clutter and the shades are progressively filtered out when t increases, but the main edges are kept. At t=15, however, edges start loosing contrast and low contrasted details could disappear.

original image White balance Retinex result with t=0 Retinex result with t=10 Retinex result with t=15

Dark Image

An example with a dark image. There is no significant difference between the white balanced image and the Retinex result. A careful examination shows nonetheless the color on homogeneous regions becoming more constant.

original Image white Balance image Retinex result

Shadows remove

These two images demonstrate how retinex can be used for removing shadows. It is not always effective, but here we can observe two good examples.

original image White balance Retinex result
original image White balance Retinex result

Lena

Application to Lena.

The smooth shading variations on the shoulder or the face and in the background fade out when the threshold t increases.

original Image white balance image Retinex result with t=8 Retinex result with t=12

credits

Edward H. Adelson
the authors CC-BY
standard test image

π