MasterPiece: interactive evolution in visual domain

Year: 2008 Authors: Artemis Moroni; Rafael Bocaletto Maiolla; Jônatas Manzolli

Core claim

Interactive evolutionary manipulation of predefined visual objects can produce aesthetically interesting compositions and support user-guided exploration of visual design.

Topics

interactive evolution, visual composition, evolutionary art, genetic algorithms, user-guided editing

Domains

genetic algorithms, genetic programming, chromosome representation, crossover and mutation, evolutionary computation, evolutionary art, abstract composition, visual design

Methods

interactive evaluation, crossover, mutation, object-based encoding, user editing

Media

two-dimensional images, predefined shapes, computer-generated compositions, visual interface

Paper text

The text below is the locally extracted OCR/Markdown version of the paper. Raw PDF files remain local and are not published here.

MasterPiece: interactive evolution in visual domain

Artemis Moroni, Rafael Bocaletto Maiolla Robotics and Computer Vision Division - Renato Archer Research Center Rod. Dom Pedro I, km 143,6 - 13069-901, Campinas, SP, Brazil Artemis.Moroni@cenpra.gov.br, rafaelmaiolla@gmail.com

Jônatas Manzolli Interdisciplinary Nucleus of Sound Studies - State University of Campinas 13091-970 - CP 6166 jonatas@nics.unicamp.br

Abstract

A research on interactive generation of visual composition is described here. This paper introduces MasterPiece, an evolutionary environment implemented as a tool for manipulating visual structures using genetic algorithms. MasterPiece allows to evolve visual compositions and also to genetically manipulate, edit, store and recover them, acting as a laboratory for experiences applied to the visual composition. Visual compositions are considered here as two-dimensional images generated by the computer by combining pre-defined shapes inspired in a Kandinsky artwork. Some visual results are presented.

Introduction

Art, music and design have been emerging from computers for many years. Evolutionary systems are a promising paradigm for such enterprise, recently growing in this direction. Richard Dawkins was one of the first to demonstrate how evolutions on a computer could be combined with the aesthetic preferences of a user to generate an almost infinite array of interesting forms [1]. He inspired other authors to explore evolutionary computation and to establish the area of evolutionary art [2, 3]. Since then, evolutionary algorithms have been applied to a wide range of creative design problems with such a great success that made some researchers speculate whether they modeled creativity in computers [4], although most commentators are very cautious and do not make such claims without some reservations [3].

In order to discover which possibilities could be used in this approach for an artistic production, objects inspired in Kandinsky’s Composition VIII [5] were programmed to find a way of automating the generation of two-dimension images using evolutionary techniques. This Kandinsky’s work was selected because 1) some graphical elements are easily programmable; 2) Kandinsky wrote two famous books on the theory of abstract expressionism. Artists make art, but seldom they try to explain why or what they make; 3) the authors of this paper appreciate this particular Kandinsky artwork, which is often present in all references. This brings up a curious aspect related to evolution: what makes some artworks more popular than others? They must have a kind of advantage, or a higher fitness, that makes them become more evident in some way.

The system developed by the authors, MasterPiece, explores evolutionary algorithms and genetic programming aiming to develop mechanisms to produce aesthetically interesting images. The user interface allows to evolve images and also to genetically manipulate, edit, store and recover them, acting as a laboratory for experiences applied to the visual composition. At last, such mechanisms, properly orchestrated, should result in an artificial artist. This is not less than the main objective of the area of artificial intelligence, perhaps even fuzzier.

409

Genetic Coding for Visual Compositions

There have been many studies made using different kinds of genotypes for evolutionary image synthesis and design. In biological systems, genotypes are normally made up of DNA [7]. In simulated evolutions there are many possible representations of genotypes, such as strings of binary digits, sets of procedural parameters, or symbolic expressions. Perhaps the most well known image synthesis technique using interactive evolution was explored by Sims [8], who used symbolic functions as genotypes, and simply evaluated the evolved functions at every pixel to express the genotypes into rendered image phenotypes. The phenotype is the individual itself, or the form that results from the developmental rules and the genotype. MasterPiece’s genotypes are described by a set of two-dimensional shapes named “objects”, shown below in Figure 1.

In MasterPiece, each individual is a code for the creation of a visual composition. Each composition is represented as a set of objects and has a chromosome associated with it that can be described as follows:

where ${\mathbf{C}}_{\mathbf{i}}$ denotes the i-th composition and $O_{(i,j)}$ denotes de j-th object of the i-th composition.

In this chromosome, each object has its specific attributes that are necessary to generate them and they are all randomly initialized in the first generation. Expression is the process by which the phenotype is generated from the genotype. In the MasterPiece environment, the expression of each object has a specific generating method. The background of a composition is treated as an object while the dimensions are pre-defined.

Figure 1: From left to right, some of the programmed objects: Sun, Target, Square, Square 2, Asterisk, Straight Line-Arcs, Mountain, Unequal, Triangle, Fence, Eclipse, Horn, Space, Triangle-Ball.

Crossover. Different from others, in the MasterPiece environment; the chromosomes may differ in length because the compositions can be made of different number of objects. For example, if ${\mathbf{C}}_{\mathbf{i}}$ and ${\mathbf{C}}_{\mathbf{j}}$ are two composition chromosomes, with length $n$ and $m$ , respectively, and objects ${\mathbf{O}}_{(i,p)}$ and ${\mathbf{O}}_{(j,q)}$ , $1\le p\le n$ and $1\le q\le m$ , then ${\mathbf{C}}_{\mathbf{i}}$ and ${\mathbf{C}}_{\mathbf{j}}$ can be described as:

During the crossover operation, two random numbers are generated for the cut, $r$ and $s$ , for example, such that $1\le r\le n$ and $1\le s\le m$ . Consequently, the length of the resulting chromosomes after the crossover can be even different from $n$ and $m$ . The resulting new compositions are:

Mutation. In most of the applications of the genetic algorithm, crossover is used about $70\%$ of the time to generate offspring; the remaining $30\%$ are simply clones of their parents. Mutation is then applied to the offspring, with a low probability of happening. In the MasterPiece environment the crossover is always

applied to pairs of chromosomes. The mutation rate can be determined by the user. Each object has a specific mutation operator. Once the object is selected for mutation, one of its characteristics is randomly chosen and modified. Figure 2 depicts the mutation applied to an instance of the Eclipse object.

Figure 2: Possible variations of an eclipse object.

MasterPiece composition rules are very simple. Each composition has a background and about 50 objects, which are randomly placed. Each object is an instance of a class in which particular attributes such as color, line width, angle, are randomly generated. Once created, transformations like rotation, scale and placement are applied to the object. The parameters for the transformations, like angle, size and position, are all randomly generated. Thus, it can be said that each object has intrinsic parameters and extrinsic parameters. The intrinsic parameters refer to the specific characteristics of each object and are instantiated when the objects are generated. The extrinsic parameters refer to the characteristics of the object with the composition and are instantiated when the object is inserted in the list of objects of the composition. Since the intrinsic parameters also contribute to the harmonious balance of the composition, both sets of parameters, intrinsic and extrinsic, interfere in the composition.

Evaluation and Discussion

Are there rules that guide aesthetical appreciation? Because if aesthetical appreciation would be ruled only by subjective opinion, it would not be possible to obtain (partially) automatic shapes of artistic production, with some aesthetical value, without a complete integration of the user with the machine. On the other hand, if general rules did not allow the maintenance of a set of free degrees of expression, there could be complete automation, in spite of the complexity of design. Since none of the extreme seems to suitably describe the artistic production process, one may conclude that there is room to automate the exploration of the free degrees of expression through a man-machine interaction, such as in the compliance of general rules. In short, the free degrees can be modeled such as optimizing problems of combinatory mathematics. The freedom of expression will be understood here as an exploratory search for the best combination of the free attributes among all possibilities. This is characterized by the existence of a huge number of possible solutions (in this case, visual compositions) or possible combinations of the free attributes.

After the proposition of a search space that contains the possible solutions, a search tool is applied to look for promising areas in the space, in which there are possible good solutions or combinations of free attributes with more aesthetical value than others. Among the factors that justify the choice of evolutionary computation techniques, is the fact that these algorithms apply population search techniques. But, independently of this, the search algorithms require the definition of an individual evaluation for each solution. The automation of the evaluation process requires that the machine be able to deterministically evaluate the aesthetical quality of each individual in the current cycle, or generation.

Instead of delegating this task to the machine, or to give the machine the evaluation capability, what is made here is to bring about an interaction with the artist/user, in such a way that automatic solutions are presented to the artist and that he/she evaluates the solutions according to his/her subjectivity. Because of this, and aiming to determine which would be the desirable aspects in the kind of compositions created by using MasterPiece, editing features were added to the environment. These features, roughly, allows the user to “push” the evolutionary process to an interesting direction. Figure 3 presents an automatic composition generated in MasterPiece and after editing into the environment.

411

Figure 3: Composition as it was generated and after editing in the MasterPiece environment.

It was noticed that, during evolution, desirable characteristics are easily lost because they are not part of the generative process. The next step is to build up a knowledge data base organized by type or class of objects. It is possible to observe that the most sensitive features that express better the aesthetical evolution are: position in two-dimensional space, size of the object and rotation in relation to the original position. Those aspects also have more affinity with the criteria mentioned by Kandinsky [8], for weight, balance and movement.

Conclusion

The MasterPiece system has been developed aiming to explore and discover the possibilities of evolutionary approach when applied to some kind of visual compositions. In spite of the conceptual space being limited to the possible variations of objects inspired in a Kandinsky artwork and their attributes, the number of possible combinations is huge. It seems advisable to break the problem into parts, e.g., for the treatment of balance, weight and movement. Other aspects to be treated are color schemes. But it was proved that the interaction with the artist/user may result into compositions having with aesthetic value.

Acknowledgements

We would like to thank PIBIC/CNPq program and CenPRA, for making this research possible. This research work is part of the AURAL project, supported by FAPESP process 05/56186-9.

References

[1] Dawkins, R.: The Blind Watchmaker. Penguin Books, London (1991) [2] Todd, S., Latham, W.: Evolutionary Art and Computers. Academic Press (1992) [3] Bentley, P., Corne, D.: Creative Evolutionary Systems. Morgan Kaufmann, San Francisco (2002) [4] Goldberg, D. E.: The Race, the Hurdle, and the Sweet Spot: Lessons from Genetic Algorithms for the Automation of Design Innovation and Creativity. In: Bentley, P. (ed.), Evolutionary Design by Computers, pp. 105-118. Morgan Kaufmann, San Francisco (1999) [5] http://lispsfmp.wordpress.com/2008/02/24/wassily-kandinsky/ [6] Holland, J. H. Adaptation in Natural and Artificial Systems. University of Michigan Press, (1975) [7] Sims, K.: Interactive Evolution of Equations for Procedural Models. The Visual Computer 9, 466-476 (1993) [8] Kandinsky, W. Point and Line to Plane. Dover, 1979. [9] Kandinsky, W. Concerning the Spiritual in Art. Dover, 1977.