A Transformational Approach to Harmony Improvisation

Year: 2020 Authors: Meirav Amram; Etan Fisher; Shai Gul; Uzi Vishne

Core claim

A transformational, UTT-enhanced Markov decision model can improvise or recompose chord progressions while balancing average harmony and entropy.

Topics

harmony improvisation, Markov decision system, uniform triadic transformations, entropy optimization

Domains

group theory, Markov processes, semidirect product, entropy, music theory, algorithmic composition, computer music, robotic musicianship

Methods

interval-based transformations, first-order Markov model, decision parameter optimization, sequence matching

Media

triads, chord progressions, musical examples, audio-based guitar robot

Paper text

The text below is the locally extracted OCR/Markdown version of the paper. Raw PDF files remain local and are not published here.

A Transformational Approach to Harmony Improvisation

Meirav Amram^{1}, Etan Fisher ^{2}, Shai Gul ^{3} and Uzi Vishne ^{4}

^{1} Dept. of Mathematics, SCE College of Engineering, Israel; ^{2} Dept. of Mechanical Engineering, SCE College of Engineering, Israel; ^{3} Dept. of Applied Mathematics, Holon Institute of Technology, Israel; ^{4} Dept. of Mathematics, Bar Ilan University, Israel

Abstract

This paper introduces a transformational approach to harmony improvisation within the framework of a Markov decision system. Group theory provides the mathematical background of the transformational approach. While chord progressions are the acceptable basis for harmony composition using Markov models, the transformational approach is interval-based. The capabilities and limitations of the transformational approach are demonstrated and discussed, then enhanced using a UTT-based approach. A decision system optimizes the balance between compatibility, represented by average harmony, and variety, represented by entropy. Musical examples are presented, including sequence matching that demonstrates consistency and sensitivity to the decision parameters.

Introduction

The algorithm presented in this paper was developed for an ongoing research project involving audio-based synchronization of guitar playing robots. The interactive goal of the robotic system is to follow a given chord progression, played by a human musician, then modify it and develop it in an improvisational environment [8]. This demanded maximizing improvisation capabilities given a relatively short chord progression. A transformational approach was considered due to its firm mathematical basis and potential to expand improvisational capabilities.

Transformational music theory involves analysis, characterization and formulation of transformations in musical composition [10]. Hook [9] suggested a generalized formulation that includes transformations between major and minor triads. This formulation is termed the uniform triadic transformation (UTT), which addresses the transformation on major or minor triads separately, and will prove to be a key element of the transformational improvisation approach presented herewith. It is important to acknowledge that transformational music theory is atonal in nature. However, a certain degree of atonality would be acceptable or even necessary within any musical improvisation or composition tool. Chuan and Chew [5] presented a method of harmonization that uses neo-Riemannian transformations. Harmonization based on harmonic relations within tonal interval space appears in [4].

Markov processes have been applied to music for over almost a century, being versatile and easy to implement [1, 12, 7]. In [2], the factor oracle is the structure around which a Markov-based improvisation algorithm is based, and addresses harmony, rhythm, melody, and style. An interval-based approach is suggested for expanding the scope of improvisation. The harmony improvisation algorithm presented below is interval-based and combines compatibility and variety, through optimization of average source probability and a given factor of entropy, which are added to the system based on a first order Markov process.

Theoretical Background

In this section we describe triads and their operations in the language and notation provided by group theory, and as defined in [9].

The major triad with root $r$, which is the relative shift in terms of semi-tones, is composed of three tones, $(r,r+4,r+7)$. As $r$ represent the same tone as $r+12$, the respective intervals within the triad are $(4,3,5)$, with the third entry 5 being the distance from $r+7$ to $r+12$. Similarly, the minor triad with root $r$ is composed of the three tones $(r,r+3,r+7)$, inducing the intervals $(3,4,5)$.

Definition 1.

A triad is represented by the ordered pair $\Delta =(r,\sigma )$, where $r$ is the root of the triad, expressed as an integer (mod 12); and $\sigma$ is a sign representing its mode ($+$ for major, $-$ for minor). For example, $(0,+)$ represents C major, or C+, while $(8,-)$ represents G# minor, or G#-.

We denote the set of 24 major and minor triads by $\Gamma$. We may define a binary operation on $\Gamma$, by

$(r_1,{\sigma }_1)(r_2,{\sigma }_2)=((r_1+r_2)(\mathrm{mod}12),{\sigma }_1{\sigma }_2).$ (1)

Recall that a group is a mathematical structure consisting of a set of elements endowed with a binary operation $(x,y)\mapsto x\cdot y$, satisfying certain axioms. The group is abelian if $y\cdot x=x\cdot y$. For example, ${\mathbb{Z}}_{12}=\{0,1,\dots ,11\}$ with addition modulo 12 is an abelian group. The number of elements is the order of the group. The order of an element $x$ in the group is the minimal $m>0$ for which $x^m$ equals the identity element (where $x^m$ is repeated operation with $x$). The order of an element always divides the order of group.

Theorem 2.

The set $\Gamma$ forms an abelian group (isomorphic to ${\mathbb{Z}}_{12}\times {\mathbb{Z}}_2$).

Since the group is abelian, we refer to the operation as “addition”. The inverse in this group is given by $-(r,\sigma )=(12-r(\mathrm{mod}12),\sigma )$.

The transformational between triads, as defined in [9] is

Definition 3.

Given ${\delta }_1=(r_1,{\sigma }_1)$ and ${\delta }_2=(r_2,{\sigma }_2)$, the transposition level $t=r_2-r_1$ (mod 12) is the interval between the roots, and the sign factor $\sigma ={\sigma }_1{\sigma }_2$ is the change in sign, following usual multiplication of signs. The $\Gamma$-interval $\mathrm{int}({\delta }_1,{\delta }_2)$ is the ordered pair $(t,\sigma )$, where $t$ and $\sigma$ are the transposition level and sign factor, respectively. The $\Gamma$-interval is, therefore, the triadic transformation between ${\delta }_1$ and ${\delta }_2$ (namely the unique element of $\Gamma$ one would add to ${\delta }_1$ to obtain ${\delta }_2$).

A uniform triadic transformation (UTT) is an operation on the group $\Gamma$, shifting major triads in one way, and minor triads in another way. A UTT is denoted by $⟨\sigma ,t^{+},t^{-}⟩$, where $\sigma \in \{+,-\}$ is the sign factor, $t^{+}$ is the transposition level given a major triad, and $t^{-}$ is the transposition level given a minor triad. By definition, the UTT $(\sigma ,t^{+},t^{-})$ moves the triad $(r,o)$ to the triad $(r^{\prime },o^{\prime })$, where the new mode (i.e., major or minor) is $o^{\prime }=\sigma o$; and the new root is $r^{\prime }=r+t^o(\mathrm{mod}12)$ (namely $r^{\prime }=r+t^{+}$ if $o=+$ and $r^{\prime }=r+t^{-}$ if $o=-$). For more on this unified notation see Sections 1,2 in [9]. Although the details are not needed in the sequel, the UTTs form a group of order 288, isomorphic to a semidirect product ${\mathbb{Z}}_2⋉({\mathbb{Z}}_{12}\times {\mathbb{Z}}_{12})$ with respect to the switching action. The maximal order of elements in this group is 24.

3.4 Improvisation Method

The improvisation method is based on an extended Markov process, given a desired musical chord (triad) progression to be improvised upon. The method takes into account two competing factors:

• Compatibility: The outcome is musical-theoretically coherent with the initial progression. This property will be measured in terms of average probability.
• Variety: For the same initial progression on which the system is required to improvise, several possible outcomes are appropriate. This property will be measured in terms of entropy.

A Transformational Approach to Harmony Improvisation

Denote by $\vec{C}$ the initial progression or database. Let $h_{\vec{C}}$ be the distribution of triadic transformations between consecutive pairs of triads in $\vec{C}$.

A new distribution is defined that maximizes a linear combination of compatibility and variety. Let $\delta$ be a (discrete) distribution on $\Gamma$, that is a positive function $\delta :\Gamma \to [0,1]$ such that

Definition 4. The average probability of $\delta$ (with respect to $\vec{C}$) is

This is the expectancy of $\delta$ viewed as a random variable, with respect to the distribution $h_{\vec{C}}$.

Definition 5. The entropy of $\delta$ is

(Shannon’s entropy is a classical measure to the deviation of $\delta$ from a uniform distribution; indeed the entropy is maximized when $\delta$ is the uniform distribution).

The average favors repeating the most common triadic transformations of $\vec{C}$. On the other hand, the entropy favors a uniform distribution. To balance the two competing effects, what we propose is to let the distribution $\delta$ maximize the function

where the parameters ${\mu }_1,{\mu }_2$ are fixed (and satisfy ${\mu }_1+{\mu }_2=1$ and ${\mu }_1,{\mu }_2\ge 0$).

In the next subsection we give an explicit formula for the distribution that maximizes the target function. This formula will be applied for the proposed transformational system.

The optimal distribution

Let $\Delta$ be the space of distributions on $\Gamma$. We are interested in the values of $\delta$ for $\delta \in \Delta$, which maximizes the target function $T$ from (5). Write ${\delta }_i$ for the probability that ${\delta }_i$ assigns to $c_i\in \Gamma$ (keeping in mind the space of 24 triads).

Rewrite (5) as follows:

where $n$ denotes the number of possible states (triads of transformations). We seek to find a maximum for $T$ on the space $\Delta$, namely under the constraint $\sum {\delta }_i=1$.

Define the Lagrange function as follows:

Lagrange’s method for optimization under constraints shows that the vector that maximizes the target function is:

Amram et al.

Figure 1: Output probability vector, ${\delta }_j$ , given average transition probabilities, $h_j=\{.04,.09,.1,.22,.13,.17,.11,.03,.04,.07\}$ , for increasing values of ${\mu }_2$ .

Figure (1) shows two example rows of the average probability matrix and the effect of the optimal distribution on the respective rows of the output probability matrix. Notice that for ${\mu }_1=1$ , ${\mu }_2=0$ , the transition with highest probability will always be chosen. As ${\mu }_1$ decreases and ${\mu }_2$ increases, the transition probability flattens out into a uniform distribution.

Demonstration and Examples

Markov-based harmony composition analyzes chord progressions and creates a probability matrix accordingly. Extending this method to the transformations between chords allows for compositional diversity. The UTT-based method distinguishes between transformations after major chords and transformations after minor chords, and requires two probability matrices. A MATLAB library was written for creating new progressions given a single chord (triad) sequence. A short progression is used to explain the methods. The methods are then applied to a well-known chord progression. Readers are encouraged to use a musical instrument or musical software to listen to the examples. Consider the two bar progression in Figure (2).

Figure 2: Two bar progression

The triads on the stave represent the chords rather than actual voice-leading. Below the musical stave is the triadic notation of the progression, where $(9,-)$ is A- and so forth. Below the triads, slightly indented, are the triadic transformations. For example, the triadic transformation between $(9,-)$ and $(0,+)$ is $(3,-)$ . Repetition has been added to demonstrate transition after the progression has ended. The triadic transformation between the G+ triad at the end of the progression, $(7,+)$ , and the A- triad at the beginning of

A Transformational Approach to Harmony Improvisation

the progression, is $(2,-)$ . For demonstration purposes, and without loss of generality, the new progressions in this section were chosen to be eight bars long, with the first triad and transformation kept at their original values.

Transformational approach

The probability matrix for the transformations within the progression appears in Table (1).

Table 1: Average transition probability - transformations

The transformational approach can expand the possibilities for new progressions, given the same input progression. The transformation $(3,-)$ appears twice in the original progression, reducing the average probability matrix dimension. The first row of the average transition probability matrix indicates that after a $(3,-)$ transformation, there is an equal probability of a $(4,-)$ transformation or a $(2,-)$ transformation.

A possible output progression, given a high value of ${\mu }_1$ (and a low value of ${\mu }_2$ ), appears in Figure (3).

Figure 3: Output progression, high ${\mu }_1$ .

The first two bars are identical to the original progression. However, the choice of transformation between triads $(0,+)$ and $(4,-)$ , represented by the transformation $(4,-)$ , or transformation between triads $(0,+)$ and $(2,-)$ , represented by $(2,-)$ , adds the $(2,-)$ (or, D-) and $(5,+)$ (or, F+) triads to the list of possible triadic outputs. As the entropy parameter is increased, the possibility of the original transformations appearing in a different order gradually increases, adding more triads to the possible output. For example, choosing ${\mu }_1=0.8$ and ${\mu }_2=0.2$ yielded the progression in (Figure (4)).

Figure 4: Output example, ${\mu }_1=0.8$

After the first two bars, which happen to be identical to the original progression, the transformation $(4,-)$ leads to B- rather than to A-. B- leads to $\mathrm{D}+$ , following the original $(3,-)$ transformation; but then,

Amram et al.

a $(3,-)$ transformation is chosen by the decision system, which causes a possibly unwanted change. The UTT-based approach reduces or eliminates the probability of such changes.

UTT-based approach

Using this approach, during the analysis stage of the original input progression, two separate probability matrices are calculated. The first is for transformations given major triads and the second is for transformations given minor triads. For the above progression, the transformational probability matrices are:

Table 2: Average transition probability matrices given major and minor triads - UTT-based approach

Given a minor triad, which occurs in this example after either a $(4,-)$ or a $(2,-)$ transformation, the output transformation is always $(3,-)$ . Given a major triad, the inverse occurs. The major triads in the example follow a $(3,-)$ transformation, and the output transformation will be either $(4,-)$ or $(2,-)$ , with equal probability. Output results are, in this case, independent of the values of ${\mu }_1$ and ${\mu }_2$ . Possible output progressions include the original progression and the first output progression of the transformational method displayed above.

Figure 5: Output example using the UTT-based method.

In Figure (5) each minor triad is followed by a $(3,-)$ transformation, and each major triad is followed by $(4,-)$ transformation. This progression may be described by the UTT $⟨-,4,3⟩$ .

Hello, Goodbye (chorus) - The Beatles The chorus of the Beatles’ song ‘Hello, Goodbye’ (Figure (6)) is characterized by a harmonic progression with a descending bass line, and is simplified here to be represented by triads.

Given the transformational approach, the transformations $(5,+)$ and $(3,+)$ that have a very specific role in the original sequence, would begin to appear with increasing probability as ${\mu }_1$ decreases. Furthermore, the $(10,-)$ transformation occurring after the only minor triad in the sequence, would have a very different effect if placed after a major triad. This may result in a very atonal output chord progression. A more subtle example appears in Figure 7. An example output of the transformational method given ${\mu }_1=0.8$ appears in Figure 7. This example can be seen as the original sequence transposed two and a half tones upwards but beginning with the third bar of the sequence. However, the last chord is D- rather than D+. A similar result was produced using the UTT-based method, this time with the D+ in place.

In order to demonstrate the consistency of the methods and the relative advantage of the UTT-based method, sequence matching is used. Sequence matching is an essential tool in the fields of bio-informatics and genetics and is also used as a method for comparing musical compositions [11, 6]. The decision method

A Transformational Approach to Harmony Improvisation

Figure 6: Chorus of ‘Hello, Goodbye’ by The Beatles.

Figure 7: Transformational output example given ${\mu }_1=0.8$

was applied to the transformations of a given progression, for both the transformational approach and the UTT-based approach, and 0 < \mu_{1} < 1 ( 1 > \mu_{2} > 0 ). A thousand new progressions of the same length were created and compared to the original progressions of transformations and of the triads of the original progression.

Figure (8) shows the maximal matching of the transformations chosen (left graph) and the respective output triads (right graph). In this example, the UTT-based approach succeeds in recreating the original progression for all values of ${\mu }_1$ until just below 1. The transformational approach succeeds in recreating the original progression for values above approximately ${\mu }_1=0.7$ . However, when ${\mu }_1$ is very close to 1, the results deteriorate. This follows the fact that near ${\mu }_1=1$ , the (single) transformation with the highest probability is chosen, preventing the new progression from recreating the original progression (see Figure (1)).

Conclusion

An interval-based approach to chord-based harmony improvisation was suggested within the framework of a Markov-based decision system. The system optimizes the balance between compatibility, represented by average harmony, and variety, represented by entropy. Results show that the interval-based approach can be effectively used to improvise upon, or recompose, a given chord progression. The UTT-based approach was shown to improve the results. The decision method presented in this paper is not limited to chords or triads and may be applied to algorithmic composition of melody, rhythm, and possibly other parameters. Interval-based approaches may be extended to include more chord types as well as melody or rhythm.

Amram et al.

Figure 8: Maximal sequence matching of transformations (left) and triads (right).

References

[1] C. Ames. “The Markov Process as a Compositional Model: A survey and tutorial.” Leonardo, vol. 22, no. 2, 1989, pp. 175-187. [2] G. Assayag and S. Dubnov. “Using Factor Oracles for Machine Improvisation.” Soft Computing, vol. 8, no. 2, 2004, pp. 604-610. [3] I. Bica, Z. Zhai, R. Hu and M. H. Melnyk. “Sound Signature Detection by Probability Density Function of Normalized Amplitudes”, Bridges Conference Proceedings, Linz, Austria, Aug. 16-20, 2019, pp. 287-294. [4] G. Bernardes. “Conchord: An Application for Generating Musical Harmony by Navigating in a Perceptually Motivated Tonal Interval Space.” Proceedings of the 11th International Symposium on Computer Music Modeling and Retrieval (CMMR), Plymouth, UK, June 16-19, 2015, pp. 243-260. [5] C. H. Chuan and E. Chew. “Generating and Evaluating Musical Harmonizations that Emulate Style.” Computer Music Journal, vol. 35, no. 4, 2011, pp. 64-82. [6] S. Cunningham, V. Grout and H. Bergen. “Mozart to Metallica: A Comparison of Musical Sequences and Similarities.” Conference on Computer Applications in Industry and Engineering (CAINE), 2005. [7] D.J. Fernández and F. Vico. “AI Methods in Algorithmic Composition: A comprehensive survey.” Journal of Artificial Intelligence Research, 48, 2013, 513-582. [8] Y. Garcia, E. Fisher and J. Mendez. “Positioning for a Guitar Playing Robot Using a Single Image.” ICR2016, Herzliya, Israel, 2016. [9] J. Hook. “Uniform Triadic Transformations.” Journal of Music Theory, vol. 46, no. 1/2, 2002, pp. 57-126. [10] D. Lewin. Generalized Musical Intervals and Transformations. Yale University Press, 1987. [11] M. Mongeau and D. Sankoff. “Comparison of Musical Sequences.” Computers and the Humanities, vol. 24, no. 3, 1990, pp. 161–175. [12] G. Nierhaus. Algorithmic Composition: paradigms of automated music generation. Springer, 2009.