Why Deep Learning Will Lead To New, Troublesome Art
by Jack Clark
KREUZBERG, BERLIN, 2017 — EXHIBITION LAUNCH OF “YES, COMPUTERS DO DREAM OF ELECTRIC SHEEP”:
Gallery-goers wander halls full of lifesize sculptures of sheep. Each sheep has a different, unique mutation, leading to one with six legs and another whose wool has been replaced with miniature spanners and hammers, the size of boardgame pieces, woven together in shining metal braids. Suddenly, the din of the house music is stalled and a trio of suited people stride in. One of them has a megaphone. “THIS IS AN ILLEGAL ART SHOW ,” they crackle. “THESE SHEEP WERE GENERATED USING PROPRIETARY DATA. WE’RE CONFISCATING THE SCULPTURES AND ALL GALLERY-GOERS WILL BE SUBJECTED TO A DATA AUDIT.” – fictional scenario, based on current AI research.
THE TROUBLE WITH MACHINE ART
The increasing sophistication of Deep Learning artificial intelligence techniques are going to lead to a new type of generative art. That’s going to be exciting for our culture, but may draw the ire of rights holders.
Artists have been working with computers since they were invented and have used techniques like procedural programming, cellular automata, and more to explore the new creative territories that computers let them access. New technologies coming out of the current AI boom will accelerate this.
Don’t believe me? Take a look at this recent paper from a group of researchers at German and US institutions: “A Neural Algorithm of Artistic Style” [1]
Researchers have figured out how to take the artistic style from one painting and apply it to a new image.
In the days after this research was published third-parties figured out how to implement the system and started generating their own images.
Third-parties are able to rapidly re-implement new discoveries in free software, accelerating the speed with which new artistic techniques find their way into culture.
They were able to do this because there’s a wealth of free software packages available for running deep learning algorithms ranging from Theano to Caffe to Torch, and more, and the researchers published their paper as open-access, so people could access it for free. That speaks to the overall speed of invention within AI, which is accelerating as more people enter the field and publish research, or free code.
[Edit: One day after this post was published someone posted an animation to Reddit [2] showing a generative system drawing the Eiffel Tower in a style reminiscent of Van Gogh.]
This shows how a neural network can “imagine” a never-before-seen image in a particular aesthetic style.
Where this gets complicated is the issue of copyright as deep learning systems need to be fed with vast amounts of data. Typically, that’s done through open access datasets compiled by academic researchers, or private stores of information amassed by companies like Google, Facebook, and others.
Individual artists have other needs, and my suspicion is that they’ll do what they’ve always done – hunt through the available imagery, pick the ones they like, and make great art out of the images. And, just as in the past, this will raise valid and complex questions about the originality of the generated work, just as it has done with the free-for-all collage art we see coming out of social platforms like Tumblr and Vine. That’s going to create conversations about fair use as people share their Neural Network recipes with others.
I chatted about this issue recently with @Samim and @graphific on the Ethical Machines [3] podcast. [4]
This is not an isolated incident: it follows Google outlining an earlier system called “DeepDream” [5] in a blog post in July. That system let you use the feature detectors from a trained neural network to enhance new images, applying the proclivities of the AI system to never-before-seen entities. The internet was rapidly flooded with pictures made using this technique. Google even published the code on GitHub [6]. And, inevitably, it led to websites like DeepDreamGenerator [7] where anyone — no neural network expertise required – can make their own images, leading to ghastly moments like this on peoples’ Facebook feeds. Hold on tight, things are about to get WEIRD.
New AI art techniques get mainstream very, very quickly.
[1] http://arxiv.org/pdf/1508.06576v1.pdf
[2] https://www.reddit.com/r/MachineLearning/comments/3iygt2/neural_art_in_action/
[3] http://ethicalmachines.com/
[4] https://soundcloud.com/samim/ethical-machines-ep2-jack-clark
[5] http://googleresearch.blogspot.com/2015/06/inceptionism-going-deeper-into-neural.html
[6] https://github.com/google/deepdream
[7] http://deepdreamgenerator.com/
Import AI 
Does art have a meaning without the artistic process of producing that art ? I don’t think so. This is why we pay so much to see the original in a museum, even though we can see a copy for free. The original painting is a relic of the artist’s experience, while he or she dabbed the brush with paint and drew those strokes. Even for non-visual art, the artist’s struggle to bring out that art work is a thing of beauty that we appreciate as spectators. This can be as simple as walking in a circle. It is why modern performance artists find an audience. The process of building an art work is a transformative experience. By watching the artist do this (or imagining him doing this), the audience can creep into his body and experience this transformation themselves.
When Keats said “a thing of beauty is a joy forever”, that thing of beauty is not divorced from the subjective experience of the artist who produced it.
This perspective of art as a continuous and interactive process, with human experience at the center of it, is sadly missing today. We have two ways of looking at machine produced art: either as an automatic process that is driven from data encoded into a mathematical model (such as a deep network), or as the result of an interactive process that produces results from data that is input by human users. If we take the second perspective, we will produce new artistic tools that can expand human imagination. Unfortunately, we researchers rarely take this viewpoint, as we are focussed on technical novelty and abandon the effort before we develop full-fledged tools that can be actually used by artists. The art-world, on the other hand, has a twisted economics which barely gives subsistence to any artist, and thus cannot create economic incentives to prompt the development of these tools. Thus, we are stuck with poorly developed artistic tools, even as technology advances tremendously.
Simpler tools like sand on a glass, or a computer graphics simulation of fluid dynamics, give greater joy to artists, than tweaking the numerical values of a neural network.
Can we expect these deep-learning filters to evolve into joyful artistic tool brushes ?
Yes, people are trying to train networks to generate images also. Theoretically, if you capture a time lapse of a canvas being drawn by an artist that style might be transferred to a network that draws step by step a new piece of “art”. A rudimentary example can be seen here: http://karpathy.github.io/assets/rnn/house_generate.gif
I am not sure about these networks generating art that is worthy of being called art but it can definitely fool people in the sense that it will be difficult for people to differentiate which art was machine generated and which was drawn by a person.
[…] passée – presque autant que pour le nouveau logo de Google [article à venir] – sur comment des systèmes d’apprentissage [Machine Learning] peuvent appliquer le style de tel ou tel …, généralement à partir d’une photographie. Donc, nous avons Gandalf peint par Picasso… […]