Assignment: starting from the output of a machine learning system, add a “human touch” and produce a unique physical artwork – eg: painting, print, sculpture, art book, etc.
The field of Machine Learning (or Artificial Intelligence how it’s often referred to by mainstream media) is evolving rapidly, understanding neural networks and the underlying workings of the current models is beyond the scope of this course.
In this unit we are looking at the most common uses of ML through a series of art projects.
Computer vision: image detection, tracking and recognition
Detecting, recognizing, and tracking objects in digital images, in particular bodies and faces, is one of the most common applications of ML. Why is that?
How are these projects approaching this problematic technology?
Cheese by Christian Moeller (2003)
Insecurity camera by Silvia Ruzanka, Ben Chang, Dmitry Strakovsky (2003)
3x3x6 by Shu Lea Cheang (2019)
“3x3x6 spans four rooms of the Prigioni with an immersive, multidimensional installation. Departing from the architecture of the panopticon invented by Jeremy Bentham in the late eighteenth century, Cheang constructs the exhibition around a rotating and inverted surveillance tower: a 3-D camera surveillance system. From the moment the visitor elects to enter the exhibition they become implicated within the system—their faces are scanned and their image later modified. Here, gender and racial morphing become queer digital strategies to disrupt the tradition of colonial and anthropometric identification techniques. Connected to the Internet, 3x3x6 in turn allows visitors to send selfies and images to the exhibition system. The exhibition visitors are thus inside the total surveillance apparatus.”
– from Hyperallergic
Body, Movement, Language by Google & Bill T Jones (2019)
Data Organization
The Infinite Drum Machine by Kyle McDonald, Manny Tan, Yotam Mann, and friends at Google Creative Lab (2016)
A more scientific application of the same system Bird Sounds
Data Analysis and Forecast
White Collar Crime Risk Zones by Brian Clifton, Sam Lavigne and Francis Tseng (2017)
Text Generation
One of the fastest developing subfields. Used for chatbots.
Conversations with Bina48 by Stephanie Dinkins (2014 – Ongoing)
Voice Synthesis
Shell Song by Everest Pipkin (2020)
Machine Learning for Image Generation
For an initial contrast, let’s consider this generative 1974 plotter artwork by computer arts pioneer, Vera Molnár, below. How did she create this artwork? We might suppose there was something like a double-for-loop to create the main grid; another iterative loop to create the interior squares; and some randomness that determined whether or not to draw these interior squares, and if so, some additional randomness to govern the extent to which the positions of their vertices would be randomized. We can suppose that there were some parameters, specified by the artist, that controlled the amount of randomness, the dimensions of the grid, the various probabilities, etc.
As with ‘traditional’ generative art (e.g. Vera Molnár), artists using machine learning (ML) continue to develop programs that render an infinite variety of forms, and these forms are still characterized (or parameterized) by variables. What’s interesting about the use of ML in the arts, is that the values of these variables are no longer specified by the artist. Instead, the variables are now deduced indirectly from the training data that the artist provides. As Kyle McDonald has pointed out, machine learning is programming with examples, not instructions.
The use of ML typically means that the artists’ new variables control perceptually higher-order properties. (The parameter space, or number of possible variables, may also be significantly larger.) The artist’s job becomes one of selecting or creating training sets, and deftly controlling the values of the neural networks’ variables.
-from Golan Levin’s lecture
This installation by alumni Aman Tiwari and Gray Crawford uses gestural control to give the user a bodily awareness of this multidimensional visual space:
Art Breeder allows you to generate images by cross breeding images and manipulating many intuitive parameters
Image to image translation
Learning to See by Memo Atken (2017)
“An artificial neural network looks out onto the world, and tries to make sense of what it is seeing. But it can only see through the filter of what it already knows. Just like us. Because we too, see things not as they are, but as we are.”
Text To Image
As for August 2022 the most popular ML systems for image generation are text-to-image: DALL-E 2, Stable Diffusion, and midjourney. They are trained on image-caption sets scraped from the internet.
At the moment DALL-E2 seems to produce more realistic output while Midjourney seems to be more “creative” but their models are constantly updated so their differences are hard to pinpoint.
Reddit is a good place to find some of the best output: dalle2 – midjourney – stablediffusion
Dinosaurs (basically a style filter)
Pokemon (iconic subjects + recognizable styles)
Imaginary Sports (more creative interpretations)
Bear market (aka limits of language)
Figuring out the right text prompts to yield the best results is questionably referred as “prompt engineering”.
It’s a social, trial and error activity and there are many guides online. Some resources here at the bottom of this page.
More ML powered projects here: https://mlart.co/
Many of them are tech demos or commercial applications
Dataset ethics
Read Excavating AI by Kate Crawford and Trevor Paglen