Diffusion Processes and Light Installations: Mathematics, Visualisation, and Perception

Year: 2008 Authors: Mike Kostner; Arjan Kuijper; Franz Schubert

Core claim

Gaussian filtering and scale space explain how light-installation details merge into larger perceived structures at different viewing distances.

Topics

light installation perception, Gaussian scale space, visual hierarchy

Domains

diffusion processes, image processing, scale space theory, light art, visualisation, transmedial art

Methods

Gaussian blur, scale-space layering, hierarchical tree analysis

Media

neon lines, tarpaulin print, mirror cabinet

Paper text

The text below is the locally extracted OCR/Markdown version of the paper. Raw PDF files remain local and are not published here.

Mike Kostner University of Applied Arts Vienna, Austria

Arjan Kuijper RICAM Linz, Austria *

Franz Schubert University of Applied Arts Vienna, Austria

Abstract

This work discusses perceptual and mathematical aspects of light installations made by the Austrian artist Brigitte Kowanz. The visual realisation of such installations can be modelled by a diffusion process. One aspect of such a process is that details merge to form larger structures - related to the distance from which the object is observed. We give examples of this process from the artistic, perceptional, and mathematical viewpoints.

1 Light installations

The Austrian artist Brigitte Kowanz [5] is famous for her light installations that are installed and exhibited all over the world, see e.g. Figure 1. The light installation we will discuss in detail in this work is a multipiece artwork exhibited in and on the building of LGT bank in Liechtenstein. The final permanent light artwork consists of several short and long neon lines building a German sentence encoded in Morse: “…DIE EINMALIGE ERSCHEINUNG EINER FERNE SO NAH SIE SEIN MAG” (…the single appearance of a distance as close as it may be). A cabinet consisting of a mirror and a two-way-mirror on the front side multiplies the Morse code visually in the third dimension until fading out in the infinity, a visual dramaturgy, which refers directly to the written sentence (Figure 1, left).

Figure 1: Two typical light installations as made by Brigitte Kowanz.

The part of artwork that we discuss is mounted without mirrors on the wall of an inner courtyard. It substitutes an interior temporary visualisation of the artwork itself, which was printed on a tarpaulin: a digitally elaborated picture on the tarpaulin simulates the succeeding light installation (Figure 2).

Figure 2: Artwork observed at different distances

2 Perceptional and artistic analysis

In fact the graphic seems to simulate not only the artwork itself, the tarpaulin appears visually like a semitransparent curtain covering the artwork. Therefore the simulacrum oscillates in our perception and reveals the observation object as different appearances: a real situation, a real illustration, a real simulation, a simulated situation, a simulated simulation (simulation of an iconographic figure). To create this graphic with such visual phenomena the artist uses software tools, which normally are designed for retouching photos and handle an almost subtractive work flow. The discussed graphic was rendered with common Gaussian filter operations.

The difficulty of creating a graphic lies in picturing the perception of the fine-gradient light features. Earlier trials capturing such phenomena with photographic methods failed because of the missing dynamic range of the photosensoric devices, even with HDR-processing tools, which combine different exposed pictures to one “super picture” the results are inadequate because perceptional sub-phenomena do not stick out in an optical capturing system.

2.1 Creating a light visualisation using Gaussian filters

The initial step is to picture the Morse neon-lines as white lines on a black background. We name the resulting picture ‘layer one’ (Figure 3, top-left). The drawing is similar to a design scheme with a color depth of one bit. The grammar of Morse signals determines a sentence with well-proportioned spacings. On this stage the script semantics are already finished and don’t require other visual explanation, but the artistic issue isn’t performed concerning the game between simulacrum, light and textual content.

The next steps are developed from the first one-bit picture using Gaussian filters and contrast increasing. Layer two was created with a 4-pixel Gaussian blur step (Figure 3, bottom-left). The outline of the ‘neon’ fades out and simulates halo effects on the contours of the light source. In nature the eye with its components like the cornea, lens, tear fluid amplify the outshining effect around bright light areas. If layer one and layer two are combined a blooming effect is produced, our vision recognises bright lights. Visual phenomena were added to the picture by simulating perceptional deficiency, which guides us paradoxically to an easier object recognition, in this case the recognition of a bright neon line.

The halo of layer three includes the line structure of the Morse script. That is in conformity with the

Figure 3: Left: Layers one (original) and two (minimal blurring). Right: Layers three and four - increased amount of blurring. These layers are the original layers from the printed artwork.

dynamic and orientation of reading procedures (Figure 3, top-right). On layer four structural formation of the art installations hardware dissolves, the halos grow out from the Morse lettering and generate new clusters and forms (Figure 3, bottom-right). With layer four technical/script and atmospheric/emotional visualisation aspects interweave, an artistic emphasis which underlines the oscillating of art reception on formal and content layers.

Layer five displays strictly atmospheric effects, aerosols refracting and reflecting from the emitted source light. The step from layer 5 to 6 initiate the reverse drawing evolution by passing from the atmospheric immaterial halo phenomena back to a material referred visualisation: the perimeter of the installation cabinet coincidences with the image frame. This is visualised in the sixth layer. As shadow effects on border areas are usual visual phenomena the graphic gets after the installation into the courtyard a hyper realistic touch (Figure 4).

2.2 More scale space, more distances

The results are now six layers figuring different sizes of halos worked out from neon stripes. The size of the halo was determined by a Gaussian operation and fixed manually on a level, where the structure-complexity in the picture changed dramatically: as the initial picture seems to have ‘invisible’ structure elements appearing as different sized clusters the Gaussian operation limits the structure to a certain complexity related on a scale space effect. All six layers combined to one layer generate the final image (Figure 2). As the image now includes structures from details to larger structures the final picture gets a notable perceptional depth: the image visualises structures with different scales. The increasing of structural information evokes reminiscences of scale space aspects. In fact all six structure layers are based on a scale space processing, even the artistic intention of each layer operates on different perceptual effects. Mathematical operations like Gaussian filter allow the artist to use artistic techniques that not only simulate physical effects (blur, diffusion) but are also practicable for amplifying our imagination. In the next section we discuss these operations

Figure 4: Layers five and six - large amount of blurring.

in more detail.

3 Mathematical Analysis: Diffusion

Mathematical techniques are commonly used in imaging, a part of computer vision that deals with digital data that represent a recording of a part the real world. Traditionally, a distinction is made between image processing, in which one automatically aims to improve the quality of an image, e.g. enhancing, denoising, and / or deblurring, and image analysis, in which one aims to extract ‘relevant’ data from images, e.g. segmentation of structures. Many approaches rely on evolution processes that transform the image to a desired end result. These processes can be modelled by partial differential equations (PDEs) and are very successful in image processing tasks.

Scale space methods originate from the observation that the real world, and therefore most images, contain structures of various scales. Therefore, filter-based processes with fixed filter-scale may favour certain structures and may produce unwanted results. On the other hand, multi-scale filters avoid this and a scale space of filters yields an hierarchy of structures [6, 7].

Gaussian scale space was introduced by Koenderink [4] as a method to study the dependence of image structure on the level of resolution. Nowadays, scale space is a fairly well established domain in computer vision ${}^2$ and biannually scale space conferences are held ${}^3$ .

3.1 Gaussian scale space

Let $L:{\mathbb{R}}^n\to \mathbb{R}(\mathbf{x})$ be an image with $\mathbf{x}$ an $n$ -dimensional spatial variable (point) and $L(\mathbf{x})$ the intensity measured at a point $\mathbf{x}$ . The Gaussian scale space image $L(\mathbf{x};t)$ is defined as the convolution of $L$ with a

Figure 5: Applying a Gaussian filter with increasing width blurs the image. Taking all images at all widths (scales) yields a Gaussian scale space image. During the blurring process structures merge and disappear in a well-defined manner.

Gaussian:

There are several motivations for applying this convolution [9], for instance smoothing the - usually noisy - image. The Gaussian filter is the Greens’ function of the diffusion equation:

When an image is blurred with this Gaussian filter, the scale (i.e. the width or the variance of the filter) needs to be chosen. The Gaussian scale space paradigm [2,4] states in contrast that no scale should be chosen in advance. The $n-$ dimensional image is thus extended to an $(n+1)$-dimensional Gaussian scale space image (Figure 5). In the remainder we use $n=2$ for visualisation purposes.

3.1.1 Excursion: Image processing and non-linear diffusion equations

As Fig. 5 clearly shows, applying a Gaussian scale space is not satisfactory for processing purposes, e.g. if one wants to detect edges in images. For this reason, several non-linear scale space approaches were proposed. See e.g. [3,8,9] for more details. The simplest extension of Eq. (1) is the observation that it can be written as $L_t=\nabla \cdot \nabla L$ which extends to

The function $g(\cdot )$ in (3) is chosen such that it enhances the edges (where $|\nabla L|$ is large) and deblurs noisy (flat) regions (where $|\nabla L|$ is small). Note that by taking $g(\cdot )=1$, the Gaussian scale space is obtained. A typical example for $g$ is $g(|\nabla L{|}^2)=\frac{1}{1+|\nabla L{|}^2/{\lambda }^2}$.

Going back to the heat equation, Eq. (1), also another approach can be taken. It is known that this equation arises from minimising the integral

under suitable boundary condition. The minimum is given by setting the variation derivative to zero and a PDE is obtained by taking a steepest decent evolution towards the solution. One can design energy functionals that describe desired image properties and minimise them to obtain an enhanced solution, see e.g. [3, 9].

3.2 Image analysis: Deep structure in Gaussian scale space

Although the non-linear PDE approaches have achieved impressive results in image enhancing, the reason for introducing the linear diffusion equation has been mainly overlooked. In his seminal paper [4], Koenderink linked the use of Gaussian filters with increasing scale to the heat equation, but the focus of his paper was on the structural changes in the image as the scale increases.

269

Figure 6: Iso-intensity manifolds and critical curves in scale space, with scale on the vertical axis. From left to right the intensity increases. Several configurations occur: a) Two manifolds with each one top. Each one intersects an extremum branch. b) One manifold consisting of two parts connected at a scale space saddle. c) One iso-intensity manifold with two tops, at each extremum branch one. d) One manifold intersecting only the left extremum branch. e) The nesting of the manifolds for the four different intensities.

3.2.1 Deep Structure

With increasing scale $t$ , the image becomes more and more blurred, until one blob-like structure remains. During this evolution, structure disappears due to the pair-wise annihilation of pairs of critical points. A critical point at scale $t$ is defined as ${\nabla }_{\mathbf{x}}L(\mathbf{x};t)=0$ .

The type of the critical points is determined by the Hessian matrix, $H(\mathbf{x};\mathbf{t})={\nabla }_{\mathbf{x}}{\nabla }_{\mathbf{x}}^{T}L(\mathbf{x};\mathbf{t})$ . If all eigenvalues have the same sign, the point is an extremum, a minimum if all are positive and a maximum if all are negative. If the eigenvalues are both positive and negative, the point is a saddle point. If an eigenvalue is zero (and $\det H=0$ ), the point is called catastrophe point. For $n-D$ images such points are non-generic, as these points are defined by $n+1$ constraints. Since Gaussian scale space images have dimension $n+1$ , both annihilations and creations are generic events, i.e. stable under small perturbations [1].

Critical curves are one-dimensional manifolds in the scale space image that satisfy ${\nabla }_{x}L(\mathbf{x};.)=0$

Parts of critical curves containing extrema are called extremum branches, while ones containing saddle points are called saddle branches. The catastrophe points are the points where an extremum branch and a saddle branch are connected [1]. At these points generically exactly one extremum and one saddle point are annihilated or created at increasing scale. Consequently, critical curves do not intersect.

An extension of critical points at a specific scale $t$ is given by the critical points in the scale space image: At scale space critical points ${\nabla }_{\mathbf{x}}L(\mathbf{x};t)=0\wedge {\partial }_{t}L(\mathbf{x};t)=0$ . One can prove that scale space critical points are always scale space saddles.

Approaches based on these special points can be used for image indexing and retrieval, reconstruction, or editing medical data, to mention some. To use the full multi-scale structure a mechanism is needed that is based on Iso-intensity manifolds, defined as $L(\mathbf{x};t)=c$ for $c\in \mathbb{R}$ . In $(\mathbf{x};t)$ -space they form closed “realms” [4]. This is visualised in Figure 6, together with two critical curves. Realms intersect critical curves. The local tops of a realm are spatial extrema.

3.2.2 Scale space hierarchy

As Figure 6 indicates, the nesting of the iso-intensity manifolds imposes a hierarchical structure on the original image as described in works by Kuijper and Florack [6,7]. Unique segments in scale space, bounded by the critical and dual manifolds, can be assigned to each extremum in the initial image. The nesting of these segments results in the topological hierarchy tree. The tree is topological, since it relates to the topology of the initial image, and hierarchical due to its nested critical - dual labelling.

An example is given in Figure 7a, a close-up of Figure 6b. It shows two parts of a manifold joining at the scale space saddle. The critical curve through it also intersects the right part of the manifold. At the top

Figure 7: a) At a scale space saddle (SSS) a manifold can be separated into two distinct manifolds $(D$ and $C)$ . The $C$ -part (on the right) is intersected by the critical curve containing the SSS. b) The scale space saddle SSS forms a node with edges to two ordered children (the manifolds $C$ and $D$ ) and the parent, either an SSS determined by $D$ , or the root of the tree.

of the curve an annihilation takes place involving the saddle branch and the extremum branch, the former on the left, through the scale space saddle, and the latter on the right, intersecting the manifold at a local top. The right part of the manifold is called critical, the left part dual. The dual part intersects an other critical curve at its top.

Figure 7b shows the corresponding building block of the tree structure. The scale space saddle SSS is a node with edges to two ordered children and a parent. The children are either nodes (scale space saddles) or leaves (extrema in the initial image). The parent is either a node or the root (the remaining extremum). The edges represent the iso-intensity manifolds. At SSS two distinct ones merge. The right edge $C$ corresponds to the series of manifolds encapsulated by the critical manifold, the left edge $D$ to those encapsulated by the dual one. The tree that arises from this construction is rooted, binary and ordered.

3.2.3 Topological segmentation

An application of this description is visualised in Figure 8, taken from Kuijper et al. [6]. An MRI scan (a) is filtered with a Gaussian filter (b), revealing the large structures. Then Figure 8c shows the top of the hierarchy tree, taken for the first seven large objects, related to the seven extrema present in Figure 8b. The right image, Figure 8d, shows the segment of Figure 8a related to the left sub-tree of Figure 8c. This part is obtained by manually selecting a subtree and determining the related object.

Figure 8: a) MR image. b) MR Image at a certain scale. The dots denote critical points. c) Top part of the hierarchy tree. The labels relate to the extrema present in b). d) Selecting the sub-tree containing $e1-e4$ and marking the corresponding region in the initial image with white, equals to selecting an object - in this case the white matter of the brain. e) Same segment as d), but now on the scale of the scale space saddle generating this structure.

As one can see in Figure 8e, this object becomes smoothed when observed at the scale of the scale space saddle generating this structure. This relates the hierarchical tree to the approach described in Section 2.1: the merging of small structures is captured in sub-trees. The larger the scale is taken, the larger the objects (and thus the sub-trees) become.

4 Summary and discussion

Light installations are perceived differently when observed at different distances. Small objects - details - are smeared out and merged together, while at small scales the larger structures are visible as well - for humans. This appeals to the human visual system, which is designed such that it is able to observe small details and larger coherent structures simultaneously. This causes an effect which is related to diffusion based image processing. We showed the example of a Gaussian scale space, in which the smearing and merging is explicitly extracted from the process to form an hierarchical tree that can be used for image segmentation.

5 Acknowledgements

The work presented is supported by the 2006 grant “Mathematical Methods for Image Analysis and Processing in the Visual Arts” of the Vienna Science and Technology Fund (WWTF). We thank Prof. B. Kowanz, Transmedial Art, University of Applied Arts, Vienna and Prof. P.M. Markowich, DAMTP, University of Cambridge & Dept. of Mathematics, University of Vienna for providing us with the images.

References

• [1] J. Damon. Local Morse theory for solutions to the heat equation and Gaussian blurring. Journal of Differential Equations, 115(2):386–401, 1995.
• [2] L. M. J. Florack. Image Structure, volume 10 of Computational Imaging and Vision Series. Kluwer Academic Publishers, Dordrecht, The Netherlands, 1997.
• [3] B. M. ter Haar Romeny. Front-end vision and multi-scale image analysis, volume 27 of Computational Imaging and Vision Series. Kluwer Academic Publishers, Dordrecht, The Netherlands, 2003.
• [4] J. J. Koenderink. The structure of images. Biological Cybernetics, 50:363–370, 1984.
• [5] B. Kowanz. LDM, Light_Darkness_Movements. Jennoptik, 2004. see also http://www.kowanz.com/index.html
• [6] A. Kuijper, L. M. J. Florack, and M. A. Viergever. Scale space hierarchy. Journal of Mathematical Imaging and Vision, 18(2):169–189, April 2003.
• [7] A. Kuijper and L.M.J. Florack. The hierarchical structure of images. IEEE Transactions on Image Processing, 12(9):1067–1079, 2003.
• [8] P.M. Markowich. Applied Partial Differential Equations: A Visual Approach. Springer, 2007.
• [9] J. A. Weickert. Anisotropic Diffusion in Image Processing. Teubner, Stuttgart, 1998.