EUROGRAPHICS 2011 / M. Chen and O. Deussen
(Guest Editors)
Volume 30 (2011), Number 2
Image and Video Abstraction by
Coherence-Enhancing Filtering*
Jan Eric Kyprianidis1 Henry Kang2
1 Hasso-Plattner-Institut, Germany
2 University of Missouri, St. Louis, USA
Abstract
In this work, we present a non-photorealistic rendering technique to create stylized abstractions from color images
and videos. Our approach is based on adaptive line integral convolution in combination with directional shock
filtering. The smoothing process regularizes directional image features while the shock filter provides a sharpening
effect. Both operations are guided by a flow field derived from the structure tensor. To obtain a high-quality flow field,
we present a novel smoothing scheme for the structure tensor based on Poisson’s equation. Our approach effectively
regularizes anisotropic image regions while preserving the overall image structure and achieving a consistent level
of abstraction. Moreover, it is suitable for per-frame filtering of video and can be efficiently implemented to process
content in real-time.
Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Picture/Image Generation—
Display algorithms
1. Introduction
Directional features and flow-like structures are considered
pleasant, harmonic, or at least interesting by most humans
[Wei99]. They are also a highly sought-after property in many
of the traditional art forms, such as paintings and illustrations.
Enhancing directional coherence in the image helps to clarify
region boundaries and features. As exemplified by Expres-
sionism, it also helps to evoke mood or ideas and even elicit
* The definitive version is available at diglib.eg.org and
www.blackwell-synergy.com.
emotional response from the viewer [Wik10]. Particular ex-
amples include van Gogh and Munch, who have emphasized
these features in their paintings. In this work, we present a
new image and video abstraction technique that places empha-
sis on enhancing the directional coherence of features. The
most notable related work in this category is image abstrac-
tion and stylization based on partial differential equations
(PDE), in particular, shape-simplifying image abstraction by
Kang and Lee [KL08] and Weickert’s coherence-enhancing
shock filter [Wei03]. However, such PDE-based techniques
may require a large number of iterations and tend to be unsta-
ble when used for video processing [Par08].
© 2011 The Author(s)
Journal compilation © 2011 The Eurographics Association and Blackwell Publishing Ltd.
Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and
350 Main Street, Malden, MA 02148, USA.

J. E. Kyprianidis & H. Kang / Image and Video Abstraction by Coherence-Enhancing Filtering
(a) Original image (b) Proposed method (N D 2) (c) Proposed method (N D 5) (d) Proposed method (N D 10/
(e) Bilateral filter [TM98] (f) Anisotropic Kuwahara Filter [KKD09] (g) Shape-simplifying image
abstraction [KL08]
(h) Coherence-enhancing shock filter [Wei03]
Figure 1: Comparison of our approach with other popular image abstraction techniques. Top row shows results of the algorithm
for different number of iterations (N D 2; 5; 10).
We build upon the idea of combining diffusion with shock
filtering for image abstraction, but our approach is, in a sense,
contrary to that of [KL08], which our technique outperforms
in terms of speed, temporal coherence and stability. Instead
of simplifying the shape of the image features, we aim to
preserve the shape by using a curvature preserving smooth-
ing method that enhances coherence. More specifically, our
approach performs smoothing, in the direction of the smallest
change, and sharpening, in the orthogonal direction. Instead
of modeling this process by a PDE and solving it, we use ap-
proximations that operate as local filters on a neighborhood
of a pixel. Therefore, good abstraction results are already
achieved in a few iterations. This makes it possible to process
images and video at real-time rates on a GPU. It also results
in a much more stable algorithm that enables temporally-
coherent video processing. Compared to the conventional
abstraction approaches [WOG06, OBBT07, KKD09], we pro-
vide a good balance between the enhancement of directional
features and the smoothing of isotropic regions. As shown in
Figure 1, our technique preserves and enhances directional
features better and creates stronger contrast, which helps to
clarify boundaries and features. Furthermore, our approach
facilitates easy control over the level of abstractions.
2. Related Work
A common approach to image abstraction is segmenta-
tion. A classical example is the work by DeCarlo and San-
tella [DS02], where eye-tracking data is used to guide image
abstraction based on mean shift segmentation at different
scales.
Another popular approach to image abstraction is the use
of edge-preserving smoothing and enhancement filters. Tech-
niques of this type commonly remove detail in low-contrast
regions without filtering across discontinuities and, thus,
leave the overall structure of the input image unaffected.
A well-known example is the bilateral filter [TM98]. Win-
nemöller et al. [WOG06] combine bilateral filtering with
color quantization and difference of Gaussians edges, to cre-
ate cartoon-style abstractions from images and videos. Kypri-
anidis and Döllner [KD08] extend this approach and present
separable implementations of the bilateral and difference of
Gaussians filters that are aligned to the local image structure.
Kang et al. [KLC09] present a similar system, where the filter
shapes of the bilateral and difference of Gaussians filters are
deformed to follow a vector field derived from the salient
image features. Since methods based on the bilateral filter
preserve high-contrast edges, they generally fail for high-
contrast images where either no abstraction is performed
or too much detail is removed. This typically results in an
inconsistent abstraction.
The Kuwahara filter [KHEK76] is another popular edge-
preserving filter. It produces clearly noticeable artifacts due to
the use of rectangular subregions. In addition, the subregion
selection process is unstable if noise is present or if the subre-
gions have the same variance. This results in randomly chosen
subregions and corresponding artifacts. Several attempts have
been made to address the limitations of the Kuwahara filter.
Papari et al. [PPC07] define a new criterion to overcome
the limitations of the unstable subregion selection process
and replace the rectangular subregions by smooth weight-
ing functions defined over sectors of a disc. The anisotropic
© 2011 The Author(s)
Journal compilation © 2011 The Eurographics Association and Blackwell Publishing Ltd.