Alan N. Shapiro, Visiting Professor in Transdisciplinary Design, Folkwang University of the Arts, Essen, Germany

Blog and project archive about transdisciplinary design, media theory and creative coding

Creative Coding and Industry 4.0, by Alan N. Shapiro

Comments Off on Creative Coding and Industry 4.0, by Alan N. Shapiro

Creative Coding and Industry 4.0

Alan N. Shapiro

This is the English version of a lecture that I gave in German at the Braunschweig University of Art in January 2019.

Introduction

I am going to talk for about one hour, and then we will have questions and answers and some discussion. Let me tell you now a brief overview of what I am going to say. First, I will say something about: what is the second wave of digitalization or Industry 4.0 or the Fourth Industrial Revolution of media technologies that is now coming towards us? Second, I will explain: What is Creative Coding? Third, I will provide brief initial answers to the five questions which I have asked in the published public description of this lecture. The essence of what I want to say is that Creative Coding should become the successor to what has been called Critical Social Theory or Critical Media Theory in cultural studies at German universities. And I also essentially want to say that Creative Coding, as I envision it, offers an opportunity for German art and design universities to develop an area of study that is both and simultaneously theory and practice, rather than being strictly theory or strictly practice, as most subjects taught at art and design universities are, consistent with the founding principles of those educational institutions. Fourth, I will then come to the second half of my talk. In the second half, I will construct a step-by-step logical argument (it is, in fact, 8 steps) supporting why I think that making Creative Coding an important part of the educational curriculum is an appropriate response for art and design universities to the global situation of digitalization. At the end, I will present a conclusion about Industry 4.0 media technologies and what they mean for the careers of artists, designers, cultural scientists, and creators.

What is Digitalization?

The term “digitalization” – in common currency among politicians and in the media today in Germany (die Digitalisierung) – is confusing. The term “digitalization” mixes together media technologies of the past and of the future. In the 1960s, there were semiconductors, and mainframe and mini computers. There were procedural and functional programming languages. There was the command-line interface. There was the television of very few broadcasting channels. Since the 1980s, we have Personal Computers – Windows and the Mac – and the Graphical User Interface, object-oriented programming languages, and the expansion of television via cable and satellite transmission to hundreds of channels. Already in the 1990s, we had the Internet, web design, the computerized and robotics automation of office and factory work processes, worldwide connectivity, hypertext and hyperlinks, the idea of the information society, the idea of the collective intelligent hive mind, and utopian visions of the future of cyber-culture. Since the first decade of the twenty-first century, we have tablets, mobile phones and smart phones. We have social media platforms and the paradoxical appearance of intermedia online existence as both the explosion of millions of channels and the consolidation of influence and control in the hands of a few powerful monopolies. We have a paradigm shift within Artificial Intelligence research and development: advances in Machine Learning and Deep Learning and the increased importance of Big Data have led to the reality that AI is now a major force affecting our lives, society, and the economy. A reasonable pragmatic definition of Artificial Intelligence might be: software that learns from experience and goes beyond its initial programming. We have not yet understood the informatics paradigm shift from Object-Orientation to AI which is analogous to the earlier 1960s-to-1980s shift from Procedural to OO paradigms – this contemporary shift has a lot to do with migrating from engineering informatics to Creative Coding.

Industry 4.0 Media Technologies

Now the second wave of digitalization is upon us. A vast new range or Next Generation of advanced digital media technologies is emerging. My own preferred term for this second wave of digitalization is “Self-Aware Technologies.” In the book that I am currently writing, I use the terms Post-Capitalism and Technological Anarchism. But the future evolution of capitalism is not the subject of my talk today. I break down the Industry 4.0 technologies into nine areas: (1) Artificial Intelligence (AI), automated software processes, and self-learning and self-evolving algorithms; (2) Virtual Reality (VR) and Augmented Reality (AR); (3) 3D Printers and Additive Manufacturing; (4) Internet of Things and the Smart Home of the Future; (5) Self-Driving Cars and the future of mobility; (6) blockchains and other distributed ledger  transaction networked-databases, cryptocurrencies, smart contracts, and Decentralized Autonomous Organizations (DAOs); (7) virtual assistants like Siri and Alexa; (8) advances in biotechnology; and (9) digital-neurological interfaces.

 

The First Three Industrial Revolutions

What were the first three Industrial Revolutions? The First Industrial Revolution took place in the late eighteenth and early nineteenth centuries. Populations of large cities increased rapidly and exponentially. The steam engine was invented. The iron and textile industries grew, and railroad transportation networks were developed. Mechanical production in factories led to greater wealth and higher standards of living for some, but also to harsh living conditions for many. These poor conditions provoked the rise of the utopian socialist and revolutionary Marxist movements. The Second Industrial Revolution took place in the late nineteenth and early twentieth centuries. Industries such as steel, oil and electricity came to the forefront. Major inventions of the era included the telephone, the light bulb, the phonograph, and the automobile. The management strategies of the assembly line (Fordism) and Scientific Management (Taylorism) were applied to work processes. The Third Industrial Revolution, which began in the 1960s, is a synonym for the Digital Revolution. It involved the transition from analog electronic and mechanical devices to digital technologies.

What is Creative Coding?

What is Creative Coding? It is well known what informatics or computer science or programming or the writing of software code is: this is a technical discipline, an engineering profession, an established practice of learning and knowing “how to do it” which has as its goal-oriented mission to “get something to work,” to get your program up and running and bug-free. It is a purposive-rational activity, driven primarily by objectives like making money, implementing a cool new application, or the aesthetic fascination of engineering and all the technical details as ends in themselves. Technology universities train students in the skills of computer programming. Industries of all kinds employ programmers: banks, insurance companies, credit card companies, media conglomerates, airlines, automobile manufacturers, telecommunications providers – the list is endless. Every large organization has a massive database and transactions system and requires Information Technology expertise.

At a certain point in time, already in the 1960s with video art, artists started to make artworks and art installations which utilize technological means, or which even alter the media technology itself into an aesthetic or social commentary expression: New Media Art, digital art, electronic art, interactive art, generative art, software art, code art, Net art, Virtual Reality art, robotics art, cyborg art, Bio Art, ecosystems art, telepresence art, sound art. In the last fifteen years, artists and designers became increasingly interested in learning how to write software code. Books began to appear with titles like Java Programming for Artists. In addition, special Creative Coding development environments, distinctive toolkits for artists and creatives – like Processing, openFrameworks, Cinder, Max MSP and vvvv – began to appear.

So far, little or no challenge has been made by artists and cultural scientists to the essential nature or inherited understanding of what programming is. It has been taken for granted that programming is what it is, and that the project of Creative Coding coincides with the decision that the list of categories of people who should learn how to program should expand, and that now a whole new category of students was going to acquire those very same skills which students at engineering schools acquire. The idea that programming itself, as it is presently constituted, should be questioned, and with an eye towards changing it in its fundamentals – actively and in a participatory way, and not only theoretically and philosophically – by those with a commitment to the humanities, design, the arts and cultural studies – has never occurred to anyone.

Wikipedia defines Creative Coding as “a type of computer programming in which the goal is to create something expressive instead of something functional.” If this definition were accurate – and I think that it is not – then it would mean that artists and creatives would only be focused on the goal of their activity and not on the process, that they would have made a rather crude separation between form and content, between media and message, going against everything that art already knows about itself! If this narrow concentration on the goal were indeed true, it would mean that artists and creatives would not have learned the essential lesson of Critical Media Theory, the lesson of Marshall McLuhan, who said that the media is the message (McLuhan knew that because he was a Professor of Literature who invented media theory). Form is the essence of art, and the form of the software is decided by the software writers. There should be a deeper involvement in and reflection on informatics, and the reintroduction of the ambivalence of natural language and even poetry into software code.

Answering the Five Questions

Now to the five questions which I asked in the published public description of this lecture. Will university art and design education change as a result of the demands of the digital economy? The answer is yes. Art and design universities have potentially something unique which they can contribute to the digital economy which no other kind of higher educational institution can contribute. This future-oriented potential needs to be defined and developed. And the students who identify as creatives want to learn how to combine in their lives and careers how to both be creative and to make money in the digital economy.

What will be the impact on society and economy of the Fourth Industrial Revolution technologies such as Artificial Intelligence, Virtual Reality / Augmented Reality, Additive Manufacturing, Internet of Things, self-driving cars, blockchain, virtual assistants, advances in biotechnology, and digital-neurological interfaces? Answer: the situation right now of the Industry 4.0 media technologies is exactly that of a heterotopia, as this concept is defined by the French philosopher Michel Foucault. Heterotopia begins with the idea of the coming together of a “real” physical space and an “unreal” or mythical or imaginary (or, perhaps, unimaginable) virtual space. Foucault conceptualizes the heterotopic space as a “mixed, intermediate experience” between institutional everyday life and something that opens the possibility of a mirroring transformation that leads – in an undecidable way – towards either disciplinary dystopia or emancipatory utopia. As depicted most forcefully in the current science fiction TV series Black Mirror, the self-aware media technologies of Industry 4.0 can easily lead to the dystopia of totalitarian rule, universal surveillance, the end of privacy, and the twilight of thinking. On the other hand, the project of creative, alternative, non-mainstream digital media-and-technology heterotopic design can lead to the post-human utopia of radical decentralization, moral algorithms, the overcoming of material scarcity, beneficial partnerships between humans and non-human intelligences in the economy, and a situation where humans can live in fulfilling and creative ways. Utopia can avoid being totalitarian if it is internally self-critical and is not a “modernist” grand narrative in the sense that Jean-Francois Lyotard wrote about in his classic 1979 book The Post-Modern Condition.

We need to change what automation means. So far in the history of capitalism, automation has been coupled with bureaucratization. Automation should instead make society and commerce less bureaucratic, allowing more – even when this seems paradoxical – sensitivity to exceptions, and more flexibility with regards to specific circumstances.

Does the massive effect of digitalization on our lives call for a change in the science and study-curriculum of informatics itself? The answer is yes. The project of the digital humanities, as it has been defined so far, should be reversed. The social and technological force that is most affecting our lives is no longer ideology, as neo-Marxist Critical Social Theory believed, and the decisive force is no longer media, as post-modernist thinkers like Jean Baudrillard believed. It is informatics. It is the responsibility of cultural studies to consider informatics not only as a technical discipline, but rather in a trans-disciplinary way. Rather than taking informatics as it is and “applying” it to the humanities, I think that we need to change computer science with influence from the humanities. The impact of the humanities on reinventing computing, rather than just the impact of computing on the humanities.

How should art, design, creative, and cultural studies students deal with digital media? The students should be provided with information about what they can do with their careers, balancing creativity, ecologically-conscious morality, and worldly success. They should learn about entrepreneurism, sustainability, and the theory and practice of money: bitcoin, alternative currencies, open source and open data, the smart city, Creative Commons, copyright and copyleft, digital art attribution, and Maker/FabLab Culture, etc. In the new era of self-aware media technologies, it is not even a given any more that we know what money is. As creators make money, they will transform what money is. In Post-Capitalism, there should be many more creativity-to-capital conversion and transfer mechanisms.

And how should universities prepare students for the challenges they face? Knowledge or “theory” should be introduced both in a systematic way as a “long discourse,” and in a new way as small “nuggets” or “segments” of knowledge/theory, brought into immediate relation with a design project, artwork, film, computer game, or other cultural artefact. The goal is to have a hybrid of theory and practice, to be continuously on the border between the two. The practice of making films should be taught in a way that is integrated with the study of film theory and film history. Students need to develop a feeling for rigorousness and creativity in storytelling and narrative to go along with learning top-quality practical filmmaking skills. The practice of making websites should be taught in connection with gaining a sense of the social and technological context of the past, present and future of the web. How will society change, how will technology affect it, and how might the web adapt to these changes?

STEP-BY-STEP LOGICAL ARGUMENT

Step 1: German Critical Social Theory

What is the position about contemporary capitalism and media culture of German “Critical Theory”? The neo-Marxist Frankfurt School Critical Social Theory of Adorno, Horkheimer, Benjamin, Marcuse, Habermas, etc. that is taught again and again at German universities? (Where does Foucault fit into this is another question.) The Critical Theory position is essentially the position of CRITIQUE or opposition – “modernism” and capitalist society are to be criticized as standing in the way of “human emancipation.” To know cultural theory means to know that everything is dehumanizing, that “my perspective” as a critic is a privileged position from which one sees, as Marcuse said, while commenting on Hegel: that “the whole is the truth, and the whole is false.” The key concepts of this idea-system are: mass culture, ideology, commodification, fetishization, alienation, and reification. All these things, according to Critical Theory, are bad. Critical Theory also tends to have a view of technology that it is a tool of capitalist domination and control – although, ironically, this was not the position on technology of Benjamin or Marcuse.

Step 2: Germany Resists Baudrillard

The thinker who has gone beyond the position of the Frankfurt School of “Critical Social Theory” that is dominant in German universities is the French sociologist and philosopher Jean Baudrillard. Baudrillard claims that there has been a paradigm shift in capitalist society or culture from “modernism” to “post-modernism.” We need to understand the important consequences of this shift. The German philosophers and sociologists have refused to engage with Baudrillard, have RESISTED Baudrillard, and have misrepresented him. What they essentially do not like about Baudrillard is that he rejects the Hegelian-Marxist dialectic that claims that the contradictions within capitalism will lead to capitalism’s demise. They also do not like Baudrillard’s emphasis on Virtual Reality and Hyper-Reality, which, from their point of view, turns attention away from the hoped-for emergence of a critical consciousness or “working-class consciousness” or radical political subject who opposes the dominant system.

 

Step 3: What is the Advance of Baudrillard over German Critical Theory?

What is the important advance of Baudrillard over German Critical Theory? What is his perception of what is happening in post-modernism that they do not want to see? Strongly influenced by the founder of worldwide media theory Marshall McLuhan, Baudrillard develops his theory of simulation: a theory of media, image, and consumer culture. The key concepts are: simulation (processes), simulacra (cultural artefacts), virtuality, Virtual Reality, Hyper-Reality, Integral Reality, and “the models and codes … and images and rhetoric … precede the real.” Models and codes precede, determine, instantiate and rule over our everyday life existence. Simulation is a reversal of the existentialist formula of Jean-Paul Sartre that “existence precedes essence.” In post-modernism, the image-copy becomes more forceful than the original. The copy replaces the original. Aesthetic representation is over. The images become more powerful than the “reality” which they were alleged to represent. What was previously considered to be “reality” disappears under the power of the images.

Step 4: Who Were the Great Artists Who Put an End to Art?

Now let us ask the question: What kind of cultural activism do we want? Another way of asking this would be to call it an “artistic practice.” We want an artistic practice that deals with the essential cultural questions of our times. A practice that directly addresses what is going on in the media and capitalist culture.

In his essay “Der Ursprung des Kunstwerkes,” the German philosopher Martin Heidegger writes about the difference between art that is related to beauty and art that is related to truth. What is the difference between art that “makes something pretty” and art that is a critical or transformative commentary on the essential cultural questions of the times, including the media and technologies that we are using, and the informatics which now shapes our lives?

Who were the “great artists” who “put an end to art”? These were Marcel Duchamp and Andy Warhol and Banksy (for example). As the master of the “readymade,” Duchamp deconstructed the privileged aura of art’s exalted value as the special commodity-object of “creativity” by making banal objects such as the urinal into the subject of art. He metamorphosed art into pure insignificance, into nothingness, into a dramatic self-reflection on pure form.

Andy Warhol threw out the Critical Theory distinction between high culture and mass culture, instead ironically making consumerism and the advertising industry into the central construction sites of his work of shining a spotlight on the “artificial illusion” that is at the heart of media technology: the visual, coded, and serially reproductive quality of the image itself. Since the advertising industry has made aesthetics into its essential operation, it is naïve or ignorant to continue to promote aesthetics as the high-culture opposition to the supposed commercial nexus of the capitalist mass culture. Instead of being fascinated by the endless technical possibilities of the latest devices and tools, as most digital media practitioners are, Warhol considers the “radical illusion of the world” inherent in the technology itself, self-questioning that technology’s relation to the world. Warhol’s focus is on details of the media simulacra (cultural artefacts), identifying with them, and thereby wiping out the very figure of the artist and the subjective creative act.

Banksy fights the celebrity culture by having no known identity. His artworks are taken down by enterprising scavengers immediately after they appear, to be sold for huge sums of money, in an act of self-ironic commentary on the money system of art. Banksy’s street art interventions are beyond the duality of “public” and “private.” Media networks have so much entered our lives that space is no longer strictly a physical-geographical-architectural space.

Step 5: An Appendix of the Computer Industry?

The situation of Creative Coding, of software code and digital media today, considered from the point of view of cultural theory, is very similar to that of art, as described above, and to the possibility of “putting an end to art,” as this was practiced by Duchamp and Warhol and Banksy. So long as art remains the “art industry” – conceiving of itself as being in possession of the valuable commodity of “creativity” to be sold on the market – then art cannot connect with a truly relevant cultural theory that understands or changes our contemporary circumstances. So long as Creative Coding stays indistinguishable from the normal technical-engineering practice of writing software code, merely extending the possibilities and combinations of programming into new areas of art-and-design expression, then it remains an appendix of the computer industry, sharing the same promotional goals of Silicon Valley to increase the number of users engaged with these tools. What Creative Coding should instead do is to question the paradigm of Object-Oriented programming, taking coding in new directions of changing the design patterns of culture. Fortunately, this movement parallels transitions which are already taking place within computer science itself, towards Artificial Intelligence, Deep Learning, and greater autonomy for self-aware software objects.

Step 6: The Limitation of Baudrillard

But what is the limitation of Baudrillard? While some thinkers are still resisting his “post-modernist” insights about simulation and Hyper-Reality, Baudrillard has himself already, in many ways, become obsolete, as contemporary culture has now moved into a “hyper-modernist” situation. Although we have learned valuable philosophy and media theory lessons from him, the main defect of Baudrillard is that he nostalgically rejects all new media and new technologies as being complicit with Hyper-Reality. He does not want to understand the detailed ways that simulation gets implemented in the design patterns and codes of Object-Orientation and Virtual Reality. Who will be the Andy Warhol of Virtual Reality?

Step 7: Virtual Reality Art

We could look, for example, at the history of Virtual Reality art, especially in the period of its appearance and formation in the 1990s. We would not be interested in how these VR artists extended the possibilities of the programming media, in a way that would be indistinguishable from the standard enthusiasm of programmers. Instead we would be interested in how these Virtual Reality artists practiced New Media Art in the mode of a critical questioning and transformation of cultural patterns. The main examples are the artworks of Myron Krueger and Charlotte Davies. Krueger created interactive environments where the user, in an embodied way, or in a hybrid real-virtual way, moves around without any heavy-duty personal hardware like the “head-mounted display.” His installations use cameras and monitors to project the user’s body onto a screen, where she can manipulate graphic objects with her hands. The body’s movements become input for the software. The software synthesizes the body’s patterns into a reflection within the Virtual Reality. Charlotte Davies’ Osmose: to enter inside a tree and exit it through the leaves after having participated in its process of photosynthesis. Integrating three-dimensional images realized in software graphics with localized stereophonic sounds, Davies constructs twelve worlds in which one can immerse oneself and move around. She wants the participants to emerge from the virtual world to experience the “real world” in a fresh way, “reawakening their being-in-the-world.” Davies wants the user to go beyond the neat (Cartesian) boundaries between subject and object, between the user-agent and the world in which she acts.

Step 8: The “Processing” Creative Coding Environment

Now we switch to the practice of Processing, the development environment which we are using to become familiar with Creative Coding, while employing the pedagogical method of the double-learning of the technical patterns of software code and the philosophical-cultural patterns of the successive (past, present and future) historical phases of informatics. We become proficient as Processing creative coders, also gaining in our “cultural science” understanding at each step of the learning process. We proceed through 17 steps:

  • We understand that computer programming started historically with machine languages, then came assembly languages, then higher-level languages. In the first (procedural or functional) higher-level languages, there was a single main() function, which consisted of a single linear series of instructions sent to a machine (the processor).
  • Similar to the single main() function, coding in Processing begins with the setup() and draw() functions. In setup() is the code that is executed one single time at the startup of the application. The draw() function contains the code that handles what the software should do in response to user interaction events.
  • We learn how to do both direct and algorithmic visual drawing. Processing is primarily a tool for generating visual art and for interactive graphics.
  • We learn, in theory and in practice, about the event-driven model of Graphical User Interface programming.
  • We learn about constants and variables. We learn about the flow-control mechanisms of for(), while(), and do() loops, and if-then-else constructions.
  • We learn about the innovation of functions: delegating control to and deciding on the names for functions, and creating modular and reusable code.
  • We learned about the historical shift in the history of programming languages from the procedural to the Object-Oriented paradigm of coding: code and data, which were previously separate, became unified in the concept of the class.
  • What are Object-Oriented analysis, design, and modeling? How does Object-Orientation correspond to post-modernist cultural patterns in so far as it is a programming paradigm for modeling and simulation of “reality” and the world?
  • We learn essential Object-Oriented concepts like classes, objects, fields (attributes), methods, and messages (sent between objects). We learn the concepts of encapsulation, polymorphism, and inheritance. The class is the blueprint specification for the object, and the object is the real-time instantiation of one example of that class. An object both knows its internal data (the fields or attributes) and knows the actions or operations (methods) on itself. This makes it somewhat intelligent and autonomous.
  • We think about how the advance of the concepts of classes and objects is halfway towards the future paradigm of Artificial Intelligence and greater autonomy for self-aware software objects. The class entity already instantiates autonomous objects which have knowledge and control over themselves.
  • We learn the basics of programming animation, transitions, and transformations.
  • We learn the basics of programming with photography, video, and sound. How does Processing work with the Arduino electronic microcontroller computer? How does Processing control machines and hardware devices?
  • We learn about collection data structures like arrays, lists, sets, and maps.
  • What is the Object-Oriented Thought Process? What is the Thought Process of Artificial Intelligence and Deep Learning and “Big Data” or Data Science?
  • What is data visualization and data parsing and networked data? How does Processing work with data streams?
  • How can Processing integrate with filmmaking projects and with 3D Printers / Additive Manufacturing?
  • What are software design patterns, and what is their relationship to cultural design patterns?

Conclusion

What do Industry 4.0 media technologies mean for the careers of artists, designers, cultural scientists, and creators? To summarize, our goal is to discover the unity between cultural theory and an activist cultural-artistic practice. This will be Creative Coding 2.0 – Creative Coding as we want it to be! We started with (A) a Frankfurt School critique of the culture industry and “commodification” (Kommodifizierung, Verdinglichung). Then we (B) went beyond the Frankfurt School’s resistance to Baudrillard, accepting Baudrillard’s diagnosis of simulation, simulacra, and Hyper-Reality. Now (C) we go beyond Baudrillard to the insight that Virtual Reality can be transformed by a Creative Coding Practice, coding that is “philosophically and culturally informed” in a transdisciplinary way, taking as guiding examples the ways that Duchamp and Warhol and Banksy “put an end to art.”

The digital media technology of Virtual Reality/Augmented Reality is merely one example of where the new paradigm of writing software/cultural code will be developed. The construction sites for the projects of the Creative Coding movement are all 9 futuristic technologies of the Fourth Industrial Revolution: Artificial Intelligence, Virtual Reality/ Augmented Reality, 3D Printers/Additive Manufacturing, Self-Driving or autonomous cars, Internet of Things, blockchain, virtual assistants, advances in biotechnology, and digital-neurological interfaces.

Comments are closed.