Seeing and Hearing the Eigenvectors of a Fluid

Year: 2017 Authors: Aaron Demby Jones; JoAnn Kuchera-Morin; Theodore Kim

Core claim

Empirical eigenvectors from turbulent fluid flow can produce compelling visual forms and a sonifiable audiovisual palette for artistic exploration.

Topics

eigenvectors and eigenvalues, computational fluid dynamics, sonification, audiovisual art

Domains

linear algebra, eigendecomposition, singular value decomposition, Navier-Stokes equations, sound art, generative visuals, multimodal composition, short film

Methods

empirical eigenvector computation, singular value decomposition, reduced-space simulation, subtractive synthesis

Media

CFD simulation, broadband noise, subspace trajectories, supplemental video

Paper text

The text below is the locally extracted OCR/Markdown version of the paper. Raw PDF files remain local and are not published here.

Bridges 2017 Conference Proceedings

Seeing and Hearing the Eigenvectors of a Fluid

Aaron Demby Jones, JoAnn Kuchera-Morin and Theodore Kim

Media Arts and Technology Program

University of California, Santa Barbara

aaron.demby.jones@mat.ucsb.edu, jkm@create.ucsb.edu, kim@mat.ucsb.edu

Abstract

The intricate shapes and sounds that arise from vibrating Chladni plates are a well-known phenomenon. They are also quantitatively well understood, as the spatial patterns correspond to the eigenvectors of the underlying plate, and the audio frequencies arise from the plate’s eigenvalues. We explore a generalization of the phenomenon by computing analogous quantities for a computational fluid dynamics simulation. Unlike the Chladni plate case, direct analytic expressions are not available, so we instead compute a set of “empirical” eigenvectors and eigenvalues. We find that these vectors form abstract, turbulent patterns in space. In another departure from the Chladni plate case, the eigenvalues no longer have a natural sonic mapping, so we construct a sonification that allows us to “listen” to the eigenvectors of the fluid. The united visual and sonic forms comprise a multimodal compositional palette that has great artistic potential.

Figure 1: Left: Chladni patterns realized through a physical experiment. Source: Wikimedia Commons. Right: Analytic 2D eigenvector of the Navier-Stokes equations using the method of de Witt et al. [1].

Introduction

Chladni plates reveal beautiful patterns when vibrated at specific frequencies (Figure 1). Both the spatial patterns and sonic frequencies that arise have long been known to have intimate connections to the eigenvectors and eigenvalues of an idealized rigid plate. For this idealized case, closed form expressions can be obtained, and the eigenvalues are known to correspond to the audio spectrum emitted by the plate. In this work, we seek to explore a generalization of this phenomenon by applying a similar procedure to a more complex scenario: a turbulent computational fluid dynamics (CFD) simulation.

Unlike the Chladni plate case, closed form expressions are not available for the eigenvectors of an arbitrary CFD simulation, so we instead discover a set of “empirical” eigenvectors. The natural connection between eigenvalues and audio frequencies in the Chladni plate case is also no longer present, so we instead construct a sonification to produce a mapping between fluid trajectories and sound. Using this approach, we obtain a variety of organic forms that have a unique visual character and generate associated sounds that unfold over rich spectral envelopes. Our ability to directly control the spatial and audio frequency

Jones, Kuchera-Morin and Kim

spectrum allows the potential for a mathematically principled artistic exploration of the audiovisual space through the medium of short film, which we demonstrate with several brief preliminary results.

Figure 2: Left: A 2D velocity field over a regular grid. Right: From top to bottom, the modes of vibration of a guitar string for $N=1,2,3$ .

Eigenvector Preliminaries

In order to better understand the role they play in this work, we start with a review of eigenvectors and values. The eigendecomposition of a square matrix $\mathbf{A}$ is usually written as $\mathbf{A}=\mathbf{Q}\Lambda {\mathbf{Q}}^T$ . The matrix $\Lambda$ is zero except along its diagonal, and each individual entry along the diagonal is an “eigenvalue,” where the German eigen roughly translates to “characteristic.” The matrix $\mathbf{Q}$ is generally not diagonal, and contains columns that are each considered an “eigenvector” of $\mathbf{A}$ . The essential character of the matrix is captured by these vectors and values because they satisfy a specific relationship. If we place the $i$ th column of $\mathbf{Q}$ into a vector $\mathbf{q}$ , and the $i$ th diagonal entry of $\Lambda$ into a scalar $\lambda$ , the relation $\mathbf{Aq}=\lambda \mathbf{q}$ will always hold. The vector $\mathbf{q}$ will be scaled by the value $\lambda$ , but will otherwise remain unchanged.

There are many different ways to understand this relationship, but the scenario of a vibrating string offers a clean physical interpretation. One way to describe the phenomenon of vibration is as one of a fixed shape that is repeatedly scaled. When a guitar string is plucked, it visibly forms the shape of a sine wave over $(0,\pi )$ . Over time, the amplitude of this shape scales up to some positive value, attenuates back to zero, and then scales down to some negative value that is symmetric with the positive one. This cyclical sequence of amplitudes in time encapsulates the visual phenomenon of vibration.

The $\mathbf{Aq}=\lambda \mathbf{q}$ relation can be understood to model exactly this behavior. If the entries of $\mathbf{q}$ are set to be a sine wave over $(0,\pi )$ , and the entries of the matrix $\mathbf{A}$ are set according to the correct physical principles ${}^1$ , then multiplying by $\mathbf{A}$ will produce a scaled version of $\mathbf{q}$ . Repeated multiplications will produce a sequence of scaled vectors, mimicking vibration. The eigenvectors thus describe a set of characteristic vibrational shapes that a string is capable of producing. The other columns of the matrix $\mathbf{Q}$ describe other shapes (“vibration modes”) that a string is capable of producing. For example, sine waves defined over $(0,2\pi )$ , $(0,3\pi )$ and up to $(0,N\pi )$ will all make an appearance (Figure 2). They are less dominant from a physical perspective, which is why the $(0,\pi )$ version is the one we most visually associate with a vibrating string. This dominance is reflected in the eigenvalue analysis—the $(0,\pi )$ sine wave appears as the first vector in $\mathbf{Q}$ , and each successive column yields progressively less visually and sonically prominent shapes.

This understanding can be extended to both 2D and 3D, but some effort is needed to rearrange these higher-dimensional phenomena so that they can be packed into a 1D vector $\mathbf{q}$ . For example, we can cut a

Seeing and Hearing the Eigenvectors of a Fluid

2D square plate into a regular grid, and rearrange the 2D values defined over this grid into a 1D vector $\mathbf{q}$ . A matrix $\mathbf{A}$ can again be assembled according to physical principles so that its eigenvectors correspond to the characteristic vibrations of a square plate. When these eigenvectors are rearranged into a 2D grid, their visual character is in close agreement with those found by laboratory experiments (Figure 1).

Figure 3: Left: Assembling the velocity fields column-wise into a matrix $\mathbf{X}$ . We use simulations of a plume moving toward each face of its bounding box. Right: Obtaining the empirical eigenvectors after the singular value decomposition is performed on $\mathbf{X}$ , yielding $\mathbf{U}$ .

Empirical Eigenvectors in Computational Fluid Dynamics

The eigenvector analysis that automatically discovers Chladni patterns does not extend directly to more complex phenomena. Nevertheless, we are interested in discovering analogous patterns that arise in turbulent fluid flows, as they may have artistic value. The equations for these flows are inherently non-linear, while the eigenvector approach is linear, so we will instead use the method of “empirical” eigenvectors. In order to motivate these techniques, we will introduce some concepts from computational fluid dynamics.

A fluid is usually defined using a velocity field, where a velocity vector is associated with each point in space. While there are many different ways to represent these fields, we take the perspective from the previous section where a bounded region of space has been diced into a set of regular squares (or, in 3D, cubes). Each cube is then assigned a corresponding velocity vector (Figure 2), and in order to generate an animation, the vectors must then be evolved over time. There are many different equations that can be used to specify this evolution in time, but we use the well-known incompressible Navier-Stokes equations for fluid flow. A detailed discussion of these equations is beyond the scope of the current work. We instead assume that we have divided time into a discrete number of steps, and that at each step, the vector inside each cube in our computational grid has been assigned an appropriate value. Further details on the numerical integration methods we employ can be found in the paper by Stam [10].

Unlike the classic Chladni pattern case, CFD simulations do not yield a single matrix $\mathbf{A}$ on which an eigenvalue analysis can be performed. The non-linear operations involved generate a new, time-dependent matrix $\mathbf{A}$ at every timestep. Instead of one canonical $\mathbf{A}$ that can be said to characterize the behavior of the entire system, there are instead infinitely many $\mathbf{A}$ s, none of which inherently take precedence over the

others. Fortunately, if some sort of precedence is imposed, a process akin to an eigenvalue analysis can still be applied. The method of “empirical” eigenvectors [7], also known as the Proper Orthogonal Decomposition, Karhunen-Love expansion, or Hotelling transform, establishes one such criterion. Informally, given an existing simulation, we can analyze the series of matrices that arose during that simulation and give those matrices precedence over all others. The space of infinite $\mathbf{A}$s is thus reduced to a tractable, finite size.

Returning to Figure 2, we first perform a simulation over a regular $N\times N$ grid for $T$ timesteps. While the simulations in our actual experiments were run in $3$D (Figure 4), we will limit our discussion to $2$D for simplicity. Each grid cell then contains a $2$D velocity vector which possesses an $x$ and $y$ component, so the entire grid contains $N\times N=N^2$ such vectors. We can rearrange these values so that they are packed into a $1$D vector $\mathbf{x}$, which then has $2N^2$ entries. We can perform this repacking for each of the $T$ velocity fields from the simulation and concatenate all of the vectors into a matrix $\mathbf{X}$ with $2N^2$ rows and $T$ columns (Figure 3, left).

Generally, $2N^2\ne T$, so the matrix $\mathbf{X}$ will be rectangular, and an eigenvalue analysis can only be performed on square matrices. However, the singular value decomposition (SVD) can always be performed regardless of dimension, and the results possess many eigenvalue-like qualities. Instead of the eigendecomposition $\mathbf{A}=\mathbf{Q}\Lambda {\mathbf{Q}}^T$, the SVD instead yields $\mathbf{X}=\mathbf{U}\Sigma {\mathbf{V}}^T$. Similar to the eigendecomposition, the middle matrix $\Sigma$ is diagonal, and its entries correspond to the “singular” values of the matrix $\mathbf{X}$. Again, similar to the eigenvalue case, the columns of the left matrix $\mathbf{U}$ form an ordered set of the most important shapes, or quasi-vibration modes, that appeared during the simulation (Figure 3, right). The matrix $\mathbf{V}$ is a $T$-dimensional rotation matrix that was applied to $\mathbf{X}$ in order to arrive at $\mathbf{U}$ and $\Sigma$; for our purposes, it can be discarded.

We are interested in the representation formed by $\mathbf{U}$ and $\Sigma$ for two reasons. First, these two quantities comprise a quasi-frequency spectrum. The shapes that are encoded in each column of $\mathbf{U}$ are roughly analogous to the sine waves from the string case. If we take a single step from the original fluid simulation, ${\mathbf{x}}_t$, and apply the matrix-vector multiply ${\mathbf{U}}^T{\mathbf{x}}_t={\mathbf{q}}_t$, then we have performed a quasi-Fourier transform that translates ${\mathbf{x}}_t$ into a quasi-frequency domain. It then becomes straightforward to start interpreting the entries of ${\mathbf{q}}_t$ as the amplitudes in some auditory representation. Second, an inverse-quasi-Fourier transform has also been defined. Given some arbitrary audio signal ${\mathbf{q}}_{\ast }$, we can convert back to a spatial shape by performing the operation $\mathbf{U}{\mathbf{q}}_{\ast }={\mathbf{x}}_{\ast }$. Given some sound unfolding over time, we can then generate a sequence of velocity fields to drive a fluid’s motion.

Finally, empirical eigenvectors are a topic of interest in engineering because running simulations in this quasi-frequency-domain can have certain computational advantages. These are often referred to as “subspace” simulations. In this work, we will use the simulator described in Kim and Delaney [3], and refer the reader to that reference for further details.

Visualization

One of the challenges when using empirical eigenvectors is the construction of an interesting set of eigenvectors. For example, if a set of smooth, featureless laminar flows are input into the SVD, there is no reason to believe that the shapes (a.k.a modes) corresponding to the resulting eigenvectors will be visually interesting. In order to ensure that a rich set of eigenvectors are produced, we ran six separate CFD simulations using a standard fluid simulator [10], where a turbulent plume of smoke was aimed at each face of a rectangular simulation domain. All of the simulation data was then concatenated into a matrix $\mathbf{X}$. To reveal the shape of the modes, we then ran a series of simulations with each of the modes sequentially isolated. A “delta” vector ${\mathbf{q}}_d$, similar to an impulse response, was generated at each timestep where the $d$th entry is set to 1 and the rest of the vector was set to zero. A selection of these results can be seen in Figure 4.

Other excitation strategies are also possible, including ones that are more closely guided by the physics

Seeing and Hearing the Eigenvectors of a Fluid

Figure 4: In reading order: An assortment of nine of the 150 empirical eigenvectors, from lowest singular value to highest, discovered by taking the combined SVD of six separate Navier-Stokes simulations.

Jones, Kuchera-Morin and Kim

(a) Singular values: $T=150$

(b) Remapped frequencies: $f=64\mathrm{Hz}$ , $s=1.75$ Figure 5: Singular values and their remapped frequencies. Note the logarithmic scale on both y-axes. The horizontal red lines on both plots indicate the bounds of the human audible frequency range.

that generated the original input data. We can gradually activate each eigenvector in sequence and then push the resulting vector $\mathbf{q}$ through a subspace simulation that approximates the Navier-Stokes equations [3]. Less physical approaches are also possible by treating $\mathbf{q}$ as a set of mixing weights. We can then construct arbitrary paths $\phi$ , sampled at $T$ time steps ${\phi }_1,\dots ,{\phi }_T$ , that determine the corresponding $T$ time steps of a full-coordinate velocity field. In this vein, we show the fluid simulation that results from a random walk over a higher-dimensional sphere of constant radius in the supplemental video [9].

Sonification

We will now devise a strategy for converting the motion of a fluid into a sound. As mentioned previously, one of the main advantages of the empirical eigenvectors approach is that it already yields a quasi-frequency spectrum. However, some additional choices need to be made before these vectors can be made audible. This sequence of subjective but judicious choices is known as sonification. More precisely, according to Hermann [2], sonification is the “data-dependent, systematic generation of sound.”

We begin by interpreting the $T$ singular values ${\sigma }_1,\dots ,{\sigma }_T$ from the singular value decomposition as characteristic frequencies. There are two immediate concerns about this raw data. First, we can see in Figure 5a that the values range over approximately 4 orders of magnitude, which is approximately 13 octaves, and decrease to numbers less than 1. However, the human audible frequency range begins at $20\mathrm{Hz}$ and spans approximately 10 octaves, up to $20000\mathrm{Hz}$ [6]. Thus, we must somehow normalize the data to this audible range. Secondly, the singular value spectrum begins with a maximum value and then decreases, whereas an audio spectrum starts at fundamental frequency and then increases. Hence, we invert the singular values. One way to construct a practical mapping is to specify a fundamental frequency $f$ and an octave scaling $s$ . Writing our audible frequencies as $f_1,\dots ,f_T$ , we can define our mapping from singular values to audible frequencies as follows:

$f_i$ $=f\cdot {\left(\frac{{\sigma }_i^{-1}}{{\sigma }_{\text{max}}^{-1}}\right)}^{\frac{1}{s}}$ (1) $=f\cdot {\left(\frac{{\sigma }_{\text{max}}}{{\sigma }_i}\right)}^{\frac{1}{s}},i=1,\dots ,T.$

The effect of this remapping can be seen in Figure 5(b), where we have used a fundamental frequency of $f=64$ Hz and an octave scaling of $s=1.75$. The spectrum now begins at the fundamental, $f=64$ Hz, and ranges up to a maximum of approximately $15000$ Hz, which is an acoustically acceptable spread.

With an audible spectrum established, we now choose amplitudes for each individual frequency. These can be mapped from a corresponding subspace vector $\mathbf{q}\in {\mathbb{R}}^T$, as each of its $T$ components, $q_1,\dots ,q_T$ can be thought of as an amplitude for the $T$ corresponding frequencies $f_1,\dots ,f_T$. Care must be taken here to ensure that sum of all the amplitudes does not exceed unity gain, as this would lead to clipping. More concretely, we constrain the $L_1$ norm of $\mathbf{q}$, $|\mathbf{q}{|}_1={\sum }_{i=1}^T|q_i|$, to be at most $1$. A typical subspace vector $\mathbf{q}$ may also contain negative components. However, these can simply be thought of as encoding a positive amplitude and a reversal of phase. The phase reversal can be discarded, as its perceivable effect is typically undesirable clicking artifacts.

With these considerations in hand, we design a mapping from a subspace vector $\mathbf{q}$ to an amplitude vector $\mathbf{a}$ of unit $L_1$ norm as follows:

$a_i=\frac{|q_i|}{|\mathbf{q}{|}_1},i=1,\dots ,T.$ (2)

Given a sequence $({\mathbf{q}}_t)$ of vectors that describe a sampled trajectory ${\mathbf{q}}_1,\dots ,{\mathbf{q}}_T$ through the subspace ${\mathbb{R}}^T$, we can normalize each ${\mathbf{a}}_t$ vector individually. Alternatively, we can determine which ${\mathbf{q}}_t$ has the maximum $L_1$ norm, which we denote ${\mathbf{q}}_{\text{max}}$, and normalize based on $|{\mathbf{q}}_{\text{max}}{|}_1$. The effect of the former is a relatively uniform volume level, while the latter yields a more variable volume envelope. Each approach has its own musical merit depending on the compositional situation.

Synthesis

With our mappings carried out, we can now produce an audible sound. To illustrate the basic mapping between the visual forms of the eigenvectors and their corresponding remapped frequencies, we move sequentially through the eigenvectors and their corresponding frequencies in the first supplemental video [8]. This mapping generates a musical scale, and compositional operations can be carried out at the note level. For example, by scrambling the order of the frequencies and adding some rhythmic variety, we can produce a short melody, as demonstrated in the second supplemental video [4].

The previous example essentially captures freeze frames of each individual mode. However, a smoother, more sophisticated auditory mapping is needed if we want to capture the flow of the smoke through mixtures of the modes. As a point of departure, we use subtractive synthesis, which passes a spectrally rich input signal through various filters in order to alter its timbre. In our case, we construct a filter bank that attenuates the input signal everywhere except at the resonant frequencies that correspond to the active modes. A particular amplitude vector $\mathbf{a}$ with component $a_i$ then tells us the strength with which each corresponding frequency $f_i$ will resonate. The nature of the input sound also strongly influences the resulting timbre: noise creates a more atmospheric effect, while impulses can create driving rhythmic textures. To match the fluid feel of the smoke propagation, we use broadband noise for the primary sound examples in this paper.

Time Evolution

Static sound, while interesting as a new timbre for a few seconds, eventually grows stale. Thus, we would like to capture the time evolution of the subspace trajectory as a dynamic sonic event. This can be achieved by cycling through the the sequence of amplitude vectors ${\mathbf{a}}_1,\dots ,{\mathbf{a}}_T$ corresponding to the subspace trajectory ${\mathbf{q}}_1,\dots ,{\mathbf{q}}_T$. Using subtractive synthesis, as described above, we can generate a corresponding sound signal that changes subtly at each time step $t$. As previously discussed, different trajectories through the subspace generate different sequences of amplitude vectors. Musically, the unfolding of these trajectories over time occurs on a micro-scale, but the overall effect is perceived smoothly, much as the individual frames of a video fuse together into a continuous motion.

Experimentally, we have explored several different categories of subspace trajectories. In the third supplemental video [5], we gradually activate each eigenvector in sequence and push the resulting subspace vector through the reduced-space Navier-Stokes equations. In the final supplemental video [9], we put aside the equations of fluid flow and walk the subspace vector $\mathbf{q}$ across the surface of a sphere in ${\mathbb{R}}^T$.

Conclusion

We have performed a preliminary aesthetic exploration of the visual and sonic patterns that arise from a generalized form of the Chladni plate experiment. Instead of studying the eigenvectors of a rigid vibration, we computed the empirical eigenvectors associated with fluid flow. The resulting forms are visually striking, and the corresponding spectrum can be sonified to produce a promising compositional palette. Suspending the underlying physics and performing an abstract subspace traversal produces intriguing results, which suggest many possible avenues for future artistic work. The interplay between the audio and the visual shapes seems to form a compositional palette that is worthy of further exploration.

References

• [1] Tyler de Witt, Christian Lessig, and Eugene Fiume. Fluid simulation using Laplacian eigenfunctions. ACM Trans. Graph., 31(1):10:1–10:11, 2012.
• [2] Thomas Hermann. Taxonomy and definitions for sonification and auditory display. In Proceedings of the 14th International Conference on Auditory Display (ICAD 2008), 2008.
• [3] Theodore Kim and John Delaney. Subspace fluid re-simulation. ACM Trans. Graph., 32(4):62:1–62:9, July 2013.
• [4] Melody. https://youtu.be/N6fzJXbn2ts.
• [5] Reduced. https://youtu.be/_DXRQZYLcdE.
• [6] Stuart Rosen and Peter Howell. Signals and systems for speech and hearing, volume 29. Brill, 2011.
• [7] D. Ryckelynck. A priori hyperreduction method: an adaptive approach. Journal of Computational Physics, 202(1):346–366, 2005.
• [8] Sequential. https://youtu.be/cB79S4NwCHc.
• [9] Sphere. https://youtu.be/y6zraBpH5PA.
• [10] Jos Stam. Stable fluids. In SIGGRAPH 1999, pages 121–128, 1999.