Use of Pseudo Functions in Digital Creation

Year: 2010 Authors: Philippe Bootz

Core claim

Parameterized Temporal Motifs can be expressed mathematically as equivalence classes of functions and used for synchronized digital creation in sound and visual media.

Topics

temporal semiotics, pseudo functions, digital poetry, sound-visual synchronization

Domains

equivalence classes, function modeling, mathematical formalization, digital poetry, experimental multimedia, visual music

Methods

mathematical modeling, computer algorithms, psychological experiment, media experimentation

Media

music, visual media, digital poem passage, sound file, generated visual

Paper text

The text below is the locally extracted OCR/Markdown version of the paper. Raw PDF files remain local and are not published here.

Bridges 2010: Mathematics, Music, Art, Architecture, Culture

Use of Pseudo Functions in Digital Creation

Philippe Bootz Laboratoire Musique et Informatique de Marseille (MIM) Cité de la Musique 4 rue Bernard du Bois BP 62042 13201 Marseille Cedex 1, France E-mail : univ_2009@bootz.fr

Abstract

Temporal semiotics is iconic of movements. It was found in music but seems to be applied in all kind of media. The paper briefly reminds its nature and displays a mathematical model of the signifier. This model uses classes of equivalence of functions. These classes are named “temporal functions”, they must be qualified of pseudo functions. The paper introduces some experiments made in a digital poem, which implements some of them.

Temporal Semiotics

The researches of the Laboratoire Musique et Informatique de Marseille, that began in 1984, have permitted to detect temporal semiotic units (UST) in music [1]. A UST is a fragment of sound that feels like a specific movement, independently of any other consideration. Examples of USTs can be heard on http://www.labo-mim.org/. Recently, during a pluridisciplinary project, I constructed with Xavier Hautbois of the University Paris 1, a mathematic model based on pseudo functions in order to describe the signifier of these USTs [2]. This model was called model of Parameterized Temporal Motifs (MTP).

The MTP model can easily be implemented in computer algorithms. Xavier Hautbois generated USTs and a psychological experiment validated this model: the audience made no difference between USTs that are excerpts of music and generated USTs [3].

The Model of Parameterized Temporal Motifs

A Parameterized Temporal Motif (MTP) is defined by a set of temporal functions acting on 2 abstract variables, called F and I. In the sound, F is relative to the main frequency perceived and I to the main sound volume. These functions have a fuzzy behavior and are defined by only some properties:

• Their general form, called profilem. The exact value of the function at a given time does not play any role in the signifier of the UST. Only the general form of this function is important. 13 profilems are enough to build all USTs.

• Their characteristic duration. The duration of a temporal function is not well defined. It is included inside a range of values, a class called characteristic duration. There are only 3 characteristic durations called T1 (300ms - 1s), T2 (1s - several seconds), and T3 (> 10 s).

501

Bootz

A MTP can be represented by a conventional graph or a formula similar to a chemistry formula. As it does not use specific musical vocabulary, it can be applied to all media.

The MIM created the USTs model to analyze music. Furthermore, by observing works of experimental cinema, video and multimedia, we also found USTs in these works. Therefore, several researchers analyzed visual and multimedia works using the MTP model [4].

The Use of MTP in Digital Creation

MTPs can also be used in creation. Their semi-quantitative nature is an asset because it is possible to add functions that create local variability around the main function of the MTP without loosing the temporal semiotics character. Furthermore, it makes possible to manage media and to create an adapted morphology out of the MTP generation process. So, creativity, variability and ordered construction can be reconciled.

Marcel Frémiot and I experimented temporal synonymy (same temporal behavior in music and in the visual) inside the digital poem passage [5]. These experimentations are documented in Les Cahiers du MIM [6]. In a section of the work, a UST begins in music and ends in the visual. In other parts, the USTs are used to ensure coherence between a sound file and a generated visual. It gives the impression that music always seems to be written for the visual, whatever solution is generated. We choose the exact same duration for the MTP in the sound and in the visual in order to enable synchronization between them.

Conclusion

USTs are true signs. Electrophysiological experiments have showed that a UST has a specific meaning and does not only constitute a formal structure [7]. The MTP model may open the door to use mathematics as a metalanguage to associate formal and natural languages.

References

[1] MIM. Les Unités Sémiotiques Temporelles, éléments nouveaux d’analyse musicale. Marseille: MIM, 1996. [2] Ph. Bootz & X. Hautbois, ‘Times Measures in Documents: The model of “Motifs Temporels Paramétrés”, in Roswitha Skare, Niels Windfeld Lund, Andreas Vårheim (éds.) A Document (Re)turn, Frankfurt am Main : Peter Lang, pp. 197 - 222, 2007 [3] A. Frey, A. Daquet, S. Poitrenaud, C. Tijus, M. Frémiot, M. Formosa, L. Prod’homme, J. Mandelbrojt, M. Timsit-Berthier, Ph. Bootz, X. Hautbois & M. Besson, « Pertinence cognitive des Unités Temporelles Sémiotiques », Musicae Scientiae vol. XIII n° 2, pp. 415-440, fall 2009. [4] X. Hautbois, musimediane n° 5, http://www.musimediane.com, mars 2010. Accessed on 06/03/2010 [5] Ph. Bootz & M. Frémiot, passage 2009, http://www.labo-mim.org/site/index.php?2009/09/04/151-passage-en-telechargement, 2009. Accessed on 03/06/2010. [6] Ph. Bootz & M. Frémiot, Passage, poème numérique, éléments d’esthétique et d’analyse. Les Cahiers du MIM n° 3, oct. 2009. Also available on http://www.labo-mim.org/site/index.php?cahier3bdc. Accessed on 03/06/2010. [7] A. Frey, C. Marie, L. Prod’homme, M. Timsit-Berthier, D. Schön & M. Besson, “Temporal Semiotic Units as Minimal Meaningful Units in Music? An Electrophysiological Approach”, Music Perception Vol. 26 issue 3, pp. 247-256, 2009

502