Cartoon+Texture Image Decomposition

• algorithm
• demonstration
• archive

• Antoni Buades toni.buades@uib.es
• Triet Le triet.le@yale.edu
• Jean-Michel Morel morel@cmla.ens-cachan.fr
• Luminita Vese lvese@math.ucla.edu

Overview

This algorithm is based on a theory proposed by Yves Meyer in Oscillating patterns in image processing and nonlinear evolution equations: the fifteenth Dean Jacqueline B. Lewis memorial lectures, 2001, American Mathematical Society.

The cartoon+texture algorithm decomposes any image into the sum of a cartoon part, where only the image contrasted shapes appear, and a textural part with the oscillating patterns. Such a decomposition is analogous to the classical signal processing low pass-high pass filter decomposition. However, the cartoon part of an image actually contains strong edges, and therefore all frequencies, up to the high ones, while a texture can also contain middle and high frequencies. Thus, a linear decomposition is never clear cut. It blurs out edges, and leaves behind some texture in the low pass filtered part.

To overcome this difficulty, the algorithm proposed by the above authors is a nonlinear low pass-high pass filter pair. At each point, a decision is made of whether it belongs to the cartoon part or to the textural part. This decision is made by computing a local total variation of the image around the point, and comparing it to the local total variation after a low pass filter has been applied.

Edge points in an image tend to have a slowly varying local total variation when the image is convolved by a low pass filter. Textural points instead show a strong decay of their local total variation by convolution with a low pass filter. The cartoon+texture algorithm is based on this simple observation.

The cartoon part keeps the original image values at points termed as non-textural points. At points identified as texture points, the cartoon part takes the filtered value. At points where the cartoon/texture decision is ambiguous, a weighted average of them is given. The texture part simply is the difference between the original image and its cartoon part.

The presented algorithm deals with color images. The cartoon-texture decomposition is made independently on all channels.

The Scale Parameter

There is no unique decomposition of an image into texture and cartoon. A texture seen at close range is just a set of well-distinguished objects, such as leaves, or stripes, or straws. Thus, it can be viewed in the cartoon part for low values of the scale parameter, and will pass over to the textural part for larger scales.

The scale parameter in the demo is therefore crucial, and must be chosen by the user. However, a default value is proposed in the first trial. The scale parameter is measured in pixel size. Thus σ = 2 means roughly the texture half-period is 2 pixels. With σ = 2, only the finest textures are distinguished.

In general, humans perceive image regions as textures for values ranging from σ = 3 to 6. Over this value, the textures are made of well distinguished and contrasted objects, and the decision to view them as a texture is definitely subjective.

References

1. Antoni Buades, Triet Le, Jean-Michel Morel, Luminita Vese, Fast cartoon + texture image filters, preprint, to be submitted (2009).

On Line Demo: Try It!

An on-line demo of this algorithm is available.

The demo permits to upload any color image and to see the decomposition, depending on the scale parameter.

Source Code

A C/C++ implementation is provided.

• source code : zip tar/gz
• source code documentation : zip tar/gz online

It should compile on any system since it's only strict ANSI C/C++ and is distributed under the GPL licence.

Basic compilation and usage instructions are included in the README.txt file. This code requires libtiff and is limited to 8bit RGB TIFF image files.

The same code is used for the online demo.

Algorithm

The main characteristics of a textured region is its high total variation. This total variation decreases very fast under low pass filtering. Formalizing these remarks leads to define the local total variation (LTV)

where G_σ is a Gaussian kernel with standard deviation sigma and L_σ is a sharper low pass filter defined in the Fourier domain by

The relative reduction rate of LTV is defined by

which gives us the local oscillatory behavior of the function. If λ is close to 0, we have which means that there is little relative reduction of the local total variation by the low pass filter.

If instead λ is close to 1 the reduction is strong, which means that the considered point belongs to a textured region. Thus, a fast nonlinear low pass and high pass filter pair can be computed by doing weighted averages of f and L_σ ∗ f depending on the relative reduction of LTV

where w(x) : [0, 1] → [0, 1] is a increasing function that is constant and equal to zero near zero and constant and equal to 1 near 1.

In all experiments the soft threshold parameters defining w (see the figure below) have been fixed to a₁ = 0.25 and a₂ = 0.5. If λ(x) is small the function f is non-oscillatory around x and therefore the function is BV around x. Thus u(x) = f(x) is the right choice. If instead λ is large, the function f is locally oscillatory and locally replaced by L_σ ∗ f.

The choice of λ = 1/2 as underlying hard threshold is conservative: it permits to keep all step edges on the cartoon side, but puts all fine structures on the texture side, as soon as they oscillate more than once. Since it is desirable to have a one-parameter method, λ is fixed once and for all in that way the method keeps the scale of the texture as the only method parameter.

the soft threshold function w(s)

Implementation

As indicated in the algorithm, the cartoon+texture decomposition only requires the application on the gradient image of two low-pass filters, which are performed directly by a discrete convolution. The gradient is computed by the simplest centered difference scheme. The main steps are:

1. Apply the low pass filter L_σ to the image f
2. Compute the modulus of the image gradients of f and L_σ ∗ f
3. Convolve these moduli with the Gaussian G_σ to get the local total variation of f and L_σ ∗ f
4. Deduce the value of λ(x) at each point in the image
5. Deduce the value of the cartoon image as a weighted average of f and L_σ ∗ f
6. Compute the texture as the difference u-f

Examples

Cactus

The examples below illustrate several aspects of the decomposition. On the cactus image, which is large, a scale 5 is just enough to remove some detail. The borders of the cactus leaves are in no way step edges. In fact for many of them the color oscillates strongly near the edges, and is considered as texture.

sample	cartoon	texture
cactus	cartoon, scale=5	texture

Noisy Square

The second image, a noisy square on noisy background, is correctly divided into a smooth cartoon with sharp boundary and an noise texture. Notice an undesirable adhesion effect: points near the edges are considered edge points, and therefore their texture is kept in the cartoon part. This adhesion effect is observable in all images and would only be disappear if the isotropic filters are replaced by anisotropic ones.

sample	cartoon	texture
noisy square	cartoon scale=3, adhesion problem near edge	texture

Dolphin

This photograph of a logo on a boat is an almost perfect cartoon! The algorithm does well in keeping it almost entirely in the cartoon part.

sample	cartoon	texture
Dolphin: almost a cartoon	cartoon scale=4, almost identical	texture

Fingerprint

In this fingerprint image, the cartoon part only contains, as expected, a step function indicating the locus of the fingerprint.

sample	cartoon	texture
fingerprint	cartoon scale=2	texture: almost everything

Textured Square

In this image, everything is texture, and the averages of the textures are equal. So the cartoon only contains a smoothed version of the original, and all details move on the textured part.

sample	cartoon	texture
texture square	cartoon scale =3, blurred	texture: almost everything

More Examples