World as Numbers: Living in an Algorithmic Culture
Year: 2016 Authors: Raine Koskimaa
Core claim
Algorithmic culture turns the world into a programmed environment, so mathematics education should use augmented reality and math-art methods to make algorithms visible and approachable.
Topics
algorithmic culture, mathematics education, augmented reality, virtual reality, design fiction
Domains
mathematical modeling, statistics, geometry, proof construction, math-art, critical design, design fiction, digital culture
Methods
historical argument, literature review, educational proposal
Media
augmented reality, virtual reality, digital games, physical computing
Paper text
The text below is the locally extracted OCR/Markdown version of the paper. Raw PDF files remain local and are not published here.
Bridges Finland Conference Proceedings
World as Numbers: Living in an Algorithmic Culture
Raine Koskimaa Department of Art and Culture Studies University of Jyväskylä Seminaarinkatu 15 Jyväskylä, 40014, Finland raine.koskimaa@jyu.fi
Abstract
There is a long tradition of trying to grasp the world around us in mathematical terms. From early man perceiving the motion of celestial bodies, to Pythagoras’ ‘celestial harmony’ and to Kepler’s and Newton’s laws of motion, calculations have provided ways to reduce the messy world of instances to a handful of mathematical formulae. Einstein’s Relativity Theory, and even more the quantum physics, complicated the situation, but still, even with random elements involved, the statistics could provide a model to understand the processes of the universe. When calculations grew ever more complex, and computers became necessary tools to deal with them, this lead to the idea of seeing the whole of the universe as a vast computer. As computers have become ubiquitous, they have provided a basis for the contemporary digital culture, where algorithmic processes govern all aspects of human life. Whereas the scientific models provided a way to understand and predict natural processes, in algorithmic culture we are building and constructing a programmed environment, in which an understanding of the principles of computer algorithms should be considered as a fundamental civic skill. Mathematics pedagogy could benefit from augmented reality applications combined with math-art approaches, to better cope with our algorithmic culture.
Mathematical World Explanations
There is a long tradition of trying to grasp the world around us in mathematical terms. The early man perceived the motion of celestial bodies, sun, moon, and stars. He detected regularities in their cycles and started to build world explanations. The ability to count the amount of sun rises and sets, and days contained in the cycles of the moon, gave man the first calendars, and the power to predict natural events. The celestial bodies were understood as divine powers, so the ability to comprehend the cyclical rhythms was also the beginning of religious thinking.
Much later, in Ancient Greece, Pythagoras developed his theory of musical harmonies based on numerical ratios between the pitches of sounds. In the context of the long history of understanding the world through cyclical periods, it was maybe not such a big leap to extend the Pythagorean theory of harmony to the prevailing model of the cosmos. With just a little of tweaking it was possible to see similar ratios between the distances of the planets – as they were estimated by that time – and to make the generalization that these same ratios governed all of the cosmos, where the harmony of the celestial spheres prevailed. It was not only that numbers could be used to define the form of the cosmos, but those numbers were also connected to the idea of beauty and grace. In this trajectory, it was Johannes Kepler’s role to formulate precise laws of planetary motion, and Isaac Newton’s universal laws of motion provided ways to reduce the messy world of instances to a handful of mathematical formulae.
Albert Einstein’s Relativity Theory, and even more the quantum physics, complicated the situation, but still, even with relativity and uncertainty included, the statistics could provide a model to understand the processes in the world of elementary particles. When calculations grew ever more complex, computers became necessary tools to deal with them. The development of quantum physics, especially,
Koskimaa
has led to a situation where the models employed by the scientists are so far removed from the everyday experience, in many cases so highly counter-intuitive and unimaginable for humans (even for scientists themselves), that the mathematical formulae have become the point of reference, and a kind of reality of their own. Inevitably, this kind of situation where the computing machines grind out the data as a closest approximation of what is happening in the quantum world, did lead to the idea that calculations performed by computers were not just modelling the universe, but rather imitating it, in effect to claim that the whole universe is a vast computer, ‘rechnender Raum’ of Konrad Zuse [1]. Another, more recent variation of this idea is Stephen Wolfram’s notion of the world as a cellular automaton [2], a more developed version of John H. Conway’s Game of Life. And then, a few years ago we could read a news piece titled “Physicists May Have Evidence Universe Is A Computer Simulation” (reported, for example, by Huffington Post UK on 11/10/2012). Later on, there were some reports to the contrary, but the case is still open. No matter the final verdict, it is significant that these kinds of ideas circulate in our culture, and they are taken seriously, at least to some extent. And, after all, it is not necessarily such a long way from the Pythagorean theory of the celestial harmony.
Virtual Realities as Imagined
The preceding brief overview aimed to show what a long a tradition there is to understanding the world in numbers, or, as numbers. That tradition is the framework within which the current constructions of virtual reality are set. From the 1960’s on, science fiction authors started to cultivate the idea that computers could be used to create immersive, and deceptive, environments for humans. The first such case, as far as I have managed to find, is Philip K. Dick’s early short story “The Defenders” (1953) where two groups of survivors from nuclear war, one American, one Russian, retreat both to their own underground bunkers and leave robots to continue the fight on the surface. The humans are totally blocked out of the surface Earth, and they only receive images and information provided by the robots. It turns out that as soon as the humans went underground, the robots jointly decided to end the war, as they could not see any point in fighting. Furthermore, they decided to keep humans blocked underground as long as it would take to make them evolve enough to be able to live peacefully together. To accomplish this, the robots, who seem to be run by a very smart artificial intelligence, feed the humans false information about the badly radioactive atmosphere on the ground. It is not a computer graphics based immersive virtual reality in the sense we know it today, but this is clearly a case of computer-run simulation made to replace the physical reality directly perceivable by the human senses. Later, in the short story “I have no mouth, and I must scream” (1967) by Harlan Ellison, there is very similar situation where again a group of human survivors of the nuclear war are living underground and being fully controlled by a computer AI called AM. From our current perspective, the central role of nuclear war in these stories may seem accidental, but it goes well to demonstrate Charlie Gere’s argument that the Cold War with its doctrine of Mutually Assured Destruction, provided the principal context in which our current digital culture developed [3].
It is from the beginning of the so-called cyberpunk science fiction that computer-based virtual realities have proliferated in popular culture. The first of these works was the novella “True Names” (1981) by Vernor Vinge. The story was influenced by the Colossal Cave Adventure (1975), Zork (1977), and MUD (1980), text-based computer games in which players try to find their way in an underground cave inhabited by trolls, dwarves and other monsters. All of these titles were strongly influenced by Dungeons and Dragons (1974), a fantasy-themed table-top role-playing game. In “True Names,” people are logging in to an online world called ‘The Other Plane.’ It is not described in detail what sort of interface is used, but it is explained how The Other Plane is based on the 0s and 1s of a microprocessor, and how it is more natural for human brain to perceive them as magical elements than as “20th Century atomistic concepts such as data structures.” As a significant detail in the story, there is a strong idea that a person’s ‘true self’ is revealed in the virtual reality. That is, the embodied existence of the everyday life is somehow limiting or conditioning humans, and only in virtual reality may the deeper truths be revealed.
World as Numbers: Living in an Algorithmic Culture
It is, however, William Gibson who is the best known, and most influential of the cyberpunk science fiction authors. Gibson’s novel Neuromancer (1984) and its two sequels are generally credited with defining the whole genre. Ever since Neuromancer the AI driven, computer-graphics based virtual realities have been commonplace in science fiction, and they have since found their way into the imagery of mainstream fiction as well. Gibson has said that he came up with the idea of his version of virtual reality, ‘The Matrix,’ after observing a player deeply immersed in a coin-op game in an arcade, and noticing the captivating, real-time feedback loop between the player’s actions and the events on screen, giving rise to the fictitious game world somewhere behind the physical screen. This turned into a literary description of virtual reality:
Cyberspace. A consensual hallucination experienced daily by billions of legitimate operators, in every nation, by children being taught mathematical concepts. … A graphic representation of data abstracted from banks of every computer in the human system. Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data. Like city lights, receding. (Gibson, Neuromancer, p. 69)
This is a very abstract, but simultaneously highly evocative description that manages to combine the nature of computer-based simulation both as an abstract data structure and a spatial visual environment. In Neuromancer, people are jacking in to the Matrix through connections plugged right into their brains, so it is, in fact, quite far from our VR’s based on audio-visual projections. But still, programmers, scientists, and engineers who were involved in the early development of the Internet have said in interviews how they were greatly influenced by Gibson’s powerful fiction. Gibson did not give his permission to make a movie out of the Neuromancer trilogy, as he claimed that the computer graphics and special effects technology were not advanced enough to realize his vision properly, but later on, the film Matrix by the Wachowsky’s in many ways took Gibson’s idea of virtual reality and turned it into an action movie, where they planted their own story.
There are many notable ideas in the Neuromancer, but I will take up only one of them here. The Matrix is not only a simulation of the online data structures, but there are also sentient AI’s in there, as well as lesser computer algorithms such as ‘Black Ice’ security software, making it a world of its own, with its own laws. Borrowing from “True Names,” and setting the trend for the whole genre to follow, the Matrix is a metaphysical ‘higher plane,’ or a portal to transcendence. The computer algorithms in the Matrix are playing roles of Voodoo deities, and the merger of the big AI’s at the end of the novel is bringing along something that is easily seen as a kind of ‘god.’ There are two possible explanations for this metaphysical move. As a contemporary culture, Western culture at least, is becoming increasingly secular; there are fewer and fewer spaces that offer a sense of the sacred. As a consequence, the transcendent, the sacred, is pushed into the realm of the virtual realities—in a quite ironic move, as it makes the computer, the most rational of all devices, into the seat of mythical, irrational and transcendent. Another explanation, maybe more relevant here, is the return to the Platonic ideas. In this case, we could understand the purely computation-based virtual realities as gateways to the eternal sphere of the mathematical ideas and truths, with no necessary recourse to any mysticism.
If Gibson’s novels stayed, despite their popularity, mostly within the science fiction fandom, The Matrix film trilogy took the notion of virtual reality and all-powerful artificial intelligence to much larger audiences. The frequent scenes in the film, where a steady flow of green digits filling the screen fades away to a gloomy cityscape, perfectly capture the notion of computational worlds rendered into landscapes more familiar to humans, as described in “True Names” and Neuromancer. This Matrix is as full of mythological characters and hints of transcendence as anything imagined by Gibson, and it perfectly captures the common threat of our computers making us slaves to their higher purposes, unintelligible to humans. The fluidity of the Matrix, the seamless morphing of the algorithms-com
Koskimaa
secret-agents, bullet-time effects and gravity-defying martial arts scenes, are all direct consequences of the Matrix being based on a vast algorithm itself, and always being subject to manipulation through just a slight change in its variables.
There is a similarity between the ideas of the universe as a computer, or a simulation, and the fictional representations of virtual realities inhabited by sentient AI’s. One could argue that we are dealing with an old phenomenon where humans have always understood the world through the lens of the latest tools: now that we are using computers in all our daily tasks, we are seeing everything in computer terms. A more nuanced version of that explanation is given by Erik Davis, who has written about what he calls techgnosis, how information in all its latest technological embodiments has always been closely linked to the revelation of the ultimate truths [4]. Every new development in the information technologies has been described as a step towards better perception of the world as it really is. This belief in ever-sharper vision of the world is often cloaked in mystical veils, as in the hermetic tradition, but plain and simple, isn’t that a more accurate understanding of the universe, what all scientist are striving for? Either the digital technology serves as our most developed metaphor to understand and explain the world around us, or, maybe, after all the millennia of misconceptions and dead ends, with the model of universe as a computer or computer simulation, we have finally arrived at the correct understanding and our digital tools have revealed us the true nature of the universe. And that nature is based on extreme simplification, reduced to the simple binary pair of yes/no, presence/absence, and emergent complexity being born out of computational algorithms.
The proliferation of fictional accounts of virtual realities took place simultaneously with the development of real experiments in virtual realities either in CAVE settings or employing VR goggles surprisingly similar to the Oculus Rift, Samsung Gear and Sony PS VR, and other devices of today. It is important to note the presence of the fictional imaginations of virtual reality, as they partly informed and influenced the design of the real-world technology development work.
Computational Culture
The digital virtual realities are fully based on algorithms built on digital code and they in many ways concretize the expectation of the clockwork-world, fully controllable and predictable. Their limitation, of course, is in that they immerse us only through visual and aural perception and in that they are rather enhanced versions of 3D cinema. More significant, in many ways, is the development of wireless communications and the Internet of Things, which is quickly evolving into something quite close to Teilhard de Chardin’s noosphere, a layer of intelligence surrounding the Earth [5]. Through this development, the physical world around us is increasingly permeated by digital technology, which makes it responsive and modifiable according to our wishes. And the point here is that the entirety of the Internet of Things is digital, its operations based on the elementary logical operations of microprocessors.
As computers have become ubiquitous, they have provided a basis for the contemporary digital culture, where algorithmic processes govern all aspects of human life. Whereas the scientific models provided a way to understand and predict natural processes, in algorithmic culture we are building and constructing our environment through programming. In such culture, an understanding of the principles of computer algorithms should be considered as a fundamental civic skill. The transformation of computers to pervasive and increasingly immersive media, however, has rendered the algorithmic layer opaque and invisible.
It is here that I want to introduce the notion of computational culture, highlighting the fundamental role of computational algorithms in all of our everyday lives. Whereas ‘digital culture’ somewhat misleadingly focuses attention to the level of ever-changing technological devices, ‘computational
World as Numbers: Living in an Algorithmic Culture
culture’ emphasizes the operational logic not dependent on specific technologies. One should be careful in not confusing computational culture with the idea of computationalism, the belief in that the human brain resembles a computer. There is a strong connection between these two strands of thinking, but the ubiquity of computational logic in the processes of our designed environment does not necessarily postulate that humans function, deep down, as computers. This is an important issue, but not the topic of this paper. David Golumbia, in his book The Cultural Logic of Computation [6] has gone through many of the facets of culture, looking at how the computational algorithms affect various aspects of life. Golumbia’s view is quite critical, when he writes about ‘spreadsheet management,’ computationalist governance and such issues. At the core of his argument lies the notion of striation through computations, how endless statistics are turned into profiles, categories, and groupings of people, which pretty much dictate the cultural logic. One of the contradictions here is that the majority of people are not able to comprehend the algorithms involved, and thus, the basis of much of the societal actions, but in many cases this ignorance instead works just to strengthen the power of these hidden rules.
Sherry Turkle has detected three ways to relate to computers [7]. The first is the mode ‘metaphysics,’ closely resembling science fiction author Arthur C. Clarke’s famous ‘law’ that any sufficiently advanced technology is indistinguishable from magic. This characterizes the prevailing stance towards computational culture: things just happen, and they must be somehow planned and intended, as there is a ‘higher’ algorithm behind it. In the second, ‘mastery’, mode one is able to control and program computers. There is a sense of power involved, even though the person’s mastery may be highly limited to a specific zone within a much larger system of control.
The third mode, ‘identity’, is about a situation where a person identifies with the algorithmic system and finds himself or herself as an active agent within that system. This does not necessarily require specific programming skills, but does require adequate understanding of the algorithmic processes and how they function. There are potential problems with the identification, for example, inability to distinguish between the real and virtual. But the full-fledged participation in the technology-saturated, algorithmically driven contemporary culture nevertheless in many cases encourages, even requires, one to adopt the mode of identification towards digital technologies. This, however, is hampered by the lower-than-needed numeracy and mathematical skills in much of the population. The lack in skills seems, if anything, to be getting even worse as witnessed by the recent news from Finland, where mathematics is an increasingly less popular subject in schools, and a wide range of university programs are suffering as a result of the shallow mathematical skills of the new classes.
It is the paradox of our age, that when mathematical thinking has become more a practical skill than ever, the attitudes towards mathematics learning are getting ever more reluctant. To some extent, that must be a pedagogical issue, and all the wonderful tools and approaches presented in the Bridges context, often employing concrete, hands-on solutions serving both the kinaesthetic and aesthetic senses, are doing good things in introducing more intuitive notions of mathematics and catering to a wider range of learners, badly needed in the contemporary algorithmic culture.
Math in Augmented Reality
What I would like to propose here, however, is to expand the repertoire of math-art approaches more towards the digital and the virtual. Mathematics educational games are one option to introduce experiential approaches to mathematics teaching. They do not involve hands-on contact with physical materials, but provide experiential practices through manipulation of virtual objects and environments. There is not necessarily such a fundamental difference between educational computer games and hands-on approaches, as argued by Kurt Squire [8]. Whereas digital games may be lacking in the concreteness of the manipulation, they bring in all the flexibility of simulation and make the exploration of the more
Koskimaa
complex models, especially, and their specific characteristics even more easy and engaging, thus helping the pupil to build a strong understanding of the problem in a way described by James Paul Gee as “performance before competence” in regards to educational gaming [9].
Elsewhere I have promoted the use of digital games employing visual paradoxes as their game mechanics in order to teach geometry and proof construction, all accompanied with a healthy dose of playful attitude [10]. Here, however, I would like to keep in line with my virtual reality focus, and propose an approach employing current, easy-to-use and relatively cheap, virtual reality, and especially, augmented reality technologies. I see huge potential in combining the mathematical art and hands-on approaches on display here, with the augmented layer providing all the computational and algorithmic capabilities of digital technology. That might be a way to make mathematics more approachable and alluring, in a word, ‘cool,’ for many of the 21st Century youth. The augmented reality mathematics exercises could also serve to make the mathematical concepts in nature, and algorithmic processes hidden in our network environment, visible, touchable and more concrete. There is a growing body of physical computing resources, and digital-material hybrid entertainment products, that could be tapped into, and combined with all the math-art traditions and the latest in the augmented reality technologies, to provide novel alternatives for mathematics education in the Matrix.
It is important to research the fictional and media representations of our technologies, in order to get a deeper insight on the positive expectations, but also on the preconceptions, prejudices, and fears people harbor, towards them. In the field of critical design, there is an interesting area of design fiction that uses fictional and narrative scenarios to envision, explain and raise questions about possible futures for design and the society, and I believe there are important lessons there to be learned also for educational technology development. In any case, we need to employ all available resources and possibilities when trying to address the widening gap between general mathematical competence and an increasingly computational contemporary culture.
References
[1] K. Zuse, Rechnender Raum. Braunschweig: Vieweg, 1969. [2] S. Wolfram, A New Kind of Science. Champaigne, IL: Wolfram Media Inc., 2002. [3] C. Gere, Digital Culture. London: Reaktion Books, 2002. [4] E. Davis, TechGnosis. Myth, Magic and Mysticism in the Age of Information. London: Serpents’s Tail, 2004. [5] P. T. de Chardin, The Human Phenomenon. English translation by S. Appleton-Weber. Brighton, England: Sussex Academic Press, 2003. [6] D. Golumbia, The Cultural Logic of Computation. Cambridge, MA & London, England: Harvard University Press, 2009. [7] S. Turkle. The Second Self. Computers and the Human Spirit. Twentieth Anniversary Edition. Cambridge, AM & London, England: The MIT Press, 2004. [8] K. Squire, “Changing the game: What happens when video games enter the classroom?,” Innovate: Journal of Online Education, 1(6). 2005. [9] D. P. Gee, “The new literacy studies: from ‘socially situated’ to the work,” Situated Literacies: Reading and Writing in Context, 2, 177-194. 2005. [10] K. Fenyvesi, R. Koskimaa & Z. Lavicza, “Experiential education of mathematics: Art and games for digital natives,” Kasvatus ja aika 9:1. 2015.