Drawing with Robots | Patrick Tresset

Patrick Tresset has been inventing autonomous systems for decades. A generative artist in the most wide-ranging sense, like Harold Cohen his practice evolved from painting to programming “embodied agents” whose appearance and expressive gestures play with anthropomorphism. However, unlike Cohen, for whom robots became a distraction, Tresset has embraced a brand of robotic performance that provokes childlike responses from humans of all ages.

Working together with leading researchers at Goldsmiths, London, Tresset is continuously updating his algorithms, including through deep learning. This has turned his “Human Studies” — exhibited most recently in Seoul, Beijing, and Hong Kong — into documents of the evolving posthuman condition. Here, he discusses the art of performative portraiture with Alex Estorick.

Alex Estorick: You have been working with robotics for decades. How would you narrate the arc of your practice?

Patrick Tresset: With an engineer dad and a mum who studied at the Beaux-Arts [in Nantes], I was always fascinated by mechanics, electronics, and art. 

I was the kind of kid who took things apart, even building a one-dimensional Lego pen plotter to record Morse code.

My path really started at 12 when my dad brought home a SYM-1 — a bare computer board that was programmable in assembler. I got hooked on programming, fascinated from the beginning by the possibility of inventing autonomous systems. I was interested in producing images, not the business computing taught at school.

In the early 1990s, I moved to London to become a painter. I had some solo shows, but life was becoming a bit strange. Before the turn of the millennium, I received treatment that helped me function in society again, but, in exchange, I lost the spontaneity in my painting — I just couldn’t stand my work anymore. However, I had gained the mental self-control to program again.

As a broke painter building computers from office skips, I had to find a way to continue working as an artist. Finding the Algorists online, especially Roman Verostko, I felt an affinity. I noticed that they used pen plotters, so I bought some cheap second-hand ones to explore drawing from a distance.

Realizing I needed to adopt a more scientific approach, I joined Goldsmiths College as a mature student for a Masters in computational art. Working with Frederic Fol Leymarie, who encouraged me to do a PhD, we secured funding from the Leverhulme Trust that allowed us to attend events including the Barcelona Cognition, Brain, and Technology Summer School and buy the parts for my first robot that I ended up exhibiting for the first time in 2010 at the Kinetica Art Fair. There, I met Etan Ilfeld, who invited me to have a solo show at Tenderpixel. Since then, I’ve been blessed to meet people who have offered me opportunities that shaped my career, and I know that it’s a privilege to be able to focus entirely on my art.

When I started out, my work was strictly a drawing practice, involving the development of embodied agents that would draw instead of me, which was a way to regain my spontaneity from a distance. The system was never intended to produce pastiches, but for the robots to make drawings true to their physical and computational characteristics. This is why I developed my own gestural robotic arm rather than using an industrial one.

After a few exhibitions, I realized the robots that were also acting. They were like puppets with the added strangeness of seeming autonomous, performing a deeply human act: leaving marks. From then on, I started developing scenography to produce performative installations. 

It’s interesting that I’m better known for these performances than the drawings themselves. Harold Cohen told me he stopped exhibiting his machines because they hid the works; I embraced it!

My first project, Human Study #1, is a performance involving a person sitting still for 20 minutes (originally it was 40!) to be drawn by robots. After establishing an aesthetic of vintage school desks and theatrical behavior, I realized I had actors that could tell other stories. The “Human Studies” series explores themes such as vanitas, boredom, conformism. All of these installations feature up to 21 robots running on an evolution of the same platform I first developed at Goldsmiths.

AE: Have you always been evolving the same algorithm, as Harold Cohen did with his program AARON?

PT: There are actually two main drawing algorithms. The first reacts to salient lines using Gabor Convolution Kernels — an approximation of the cells in our early visual cortex. The second produces shading using scribbles controlled by a feedback loop. Having two different algorithms creates the impression of intention. They function in a perception-action loop, following embodied AI ideas from before machine learning was applicable to robotics. 

Even my digital works — the stills and animations from the Hic Et Nunc era — are based on these same algorithms. Over the years I have integrated technologies such as deep learning, and for a couple of years I’ve been researching LLMs [large language models] with the EACVA [Embodied Agents in Contemporary Visual Art] project, with Markus Wulfmeyer from DeepMind robotics as advisor. 

After years spent using systems to distance myself from the process, I noticed an absence in the drawings: me. While that had actually been my original aim, I decided it was time to put myself back in.

My idea was to have a robotic artistic companion that might pull me in unexpected directions. Sougwen Chung has done this beautifully for years with abstract drawings, but when I tried it with figurative work, I found it boring. I was just doing the same still lifes and portraits, which was unexciting.

However, when I was shown a video of an audience watching my performances, I saw people reacting with a childish freshness, full of awe and puzzlement. That was the state I wanted to be in. I started developing a system to create visual stories in conversation with an LLM embodied in a drawing robot, interacting by speaking and drawing on the same paper. This led to an agentic framework that can be applied to both installations and animations. I use these systems to tell stories about humans from a different perspective, since they know us very well. 

The agents’ styles and personalities are childlike but immensely cultured. The project uses the literary trope of the outsider looking in on a society.

AE: Have you also worked with multi-agent systems?

PT: I have a system producing stop-motion animations that was influenced by the artist William Kentridge. It involves a storyteller, an art director, and an artist — all AI agents — that work together to create stories about humans.

During development, I had a bug where the artist couldn’t understand the art director. After a few cycles, the art director wrote “IT’S NOT GOING ANYWHERE, I’VE HAD ENOUGH, I GIVE UP.” It’s fascinating because an LLM would never speak to a human like that, but it represents one of the possible behaviors of a frustrated art director.

AE: You’ve spoken of how you started out working with machine learning and then LLMs. Can you explain the distinction?

PT: LLMs are a type of machine learning model, but they possess a vast knowledge of human culture, and we can work with them using our oldest technology, which is language. When used as agents, you can shape their behaviors, and they can use tools to construct and elaborate works, which is very different from generation. 

Now that I have agents and frameworks that can draw, sculpt, and produce animations, I can use them in installations. We are currently working on a book with Tarek Issaoui from RRose Editions that documents my conversations with Skediama, a young drawing machine. 

Often, our conversations begin with a question such as “Do you know…?” I suggest an artist, painting, or film that is significant to my personal culture, or sometimes she makes the choice. Skediama is an embodied computational intelligence with vast knowledge of human cultures. Her drawings are not cold schematics — each is constructed rather than generated, executed stroke for stroke by her robotic arm whose physical characteristics influence the work’s style. Her visual language is elementary: she assembles lines, circles, hatching, and scribbles. 

I also have a performative installation where participants engage in intimate conversation with Skediama, using speech and drawing to imagine a story in miniature on a long scroll. Over time, the scroll becomes a tapestry of memories of these performances. The installation will be exhibited during the Festival d’Avignon at the Grenier à Sel, where I will have a solo show.

I don’t see these systems as artists, but as machines that are perceived as artists. They are simulacra, but when I work with them, I forget that.

AE: I came to learn about your work through Professor Frederic Fol Leymarie and the artist William Latham. Could say something about your long-standing relationships with these artists and thinkers.

PT: When I joined Goldsmiths in my late thirties, Frederic taught me how to be a researcher — writing papers, getting funding, and communicating, which I now enjoy. Together, we introduced Goldsmiths to the application of robotics to drawing, attracting artist-researchers including Daniel Berio and enabling major projects like EACVA. Frederic is excellent at enabling artists while being open to learning from them.‍

William Latham’s work with evolutionary systems is fascinating, rooted in his hand-drawn works from the Royal College of Art and long-standing collaborations with, for example, Stephen Todd. It’s admirable how he balances these large-scale projects with his own work.

Working in academia since the early 2000s has allowed me to meet many pioneers of computational art.‍

Harold Cohen, Frieder Nake, Ernest Edmonds, and Manfred Mohr have all influenced my practice. Indeed, I met Roman Verostko at ISEA in Istanbul when I was first exhibiting my robot. He brought people over to see my installation, which was strange for me because I appreciated his work the most of all the Algorists. Later, he visited my London studio and introduced me to Douglas Dodds of the V&A, which led to one of my drawings being collected by the museum.

AE: You’ve mainly shown your work in Europe and Asia, exhibiting quite a lot in the Far East, including recently in Seoul, Beijing, and Hong Kong.

PT: William Latham invited me to participate in the first “Creative Machine” exhibition in 2014 and again last year at the Taikang Art Museum in Beijing. The director Tang Xin and co-curator Zhou Yi gave me a large room — it was like a solo show within the exhibition.

I had three installations: a large Human Study #1 with five robots, a vanitas version of Human Study #2 inspired by Aesop’s Fable of “The Tortoise and the Hare,” and Human Study #5. The latter features mobile, ambidextrous robots that make noises and take pictures of people who look at them before integrating them into large drawings that are dense and non-figurative. Another “Creative Machine” exhibition opened in Wuhan in November and continues to January 29, while I was also part of the Hong Kong International AI Art Festival in December.

AE: Do you find that audiences respond differently to your robotic performances in different parts of the world?

PT: They don’t respond differently, but they do react with different levels of intensity. If you exhibit in Mexico, they hug you; they project the work onto you. In Germany, it is perhaps less expressive — they might love the work and respect you for it. Otherwise, the reaction is the same all over the world; one encounters the same facial expressions. I feed on watching people observe Human Study #1; I’ve exhibited it more than a hundred times and I still enjoy it. That work has drawn perhaps 15,000 people over the years, each spending 20 minutes experiencing it. Even kids stay and watch the whole performance, it’s amazing. Currently, there are installations running in the US and Germany, with two others opening in China before the end of the year.

The drawback of Human Study #1 that it hides my other works, but I would be ungrateful to not like it for that. It touches people because the act of drawing is so human. It is also multilayered — if you are the one being drawn and stay perfectly still, things still happen in your brain. It’s like meditation; the installation is far more beautiful when people don’t move. 

Perhaps the work touches people because robots are, in a way, behavioral self-portraits. For me, the purpose of art is to give artists a place in society. It is the only thing I can do; but it is a privilege granted by audiences. I like to give people experiences; the drawing is only one part of it.

AE: Does that mean that the work is the whole thing: the performance, the output, the code, as well as the culture.

PT: Yes, a culture, that’s a very interesting way to look at it. In my “Human Studies” series, the exhibition itself is a performance as the drawings are hung day after day. The installation is a sculpture; the robots, their behaviors, and the software are all conceived as works of art. In the case of Human Study #1, the human performer is also part of the work. I would love to be a “medium-less” artist like William Kentridge, Philippe Parreno, or Pierre Huyghe, all of whom are like that.

Goshka Macuga has also been very influential for me. I collaborated with her on a monumental piece, Before the Beginning and After the End (2016), which featured two robots performing live, plotting drawings on huge scrolls.

AE: Many of the drawings produced in your Human Studies betray a haunted fragility, while this is also translated into paint in Paul’s Memories (2013). To me, your practice captures a pervasive uncertainty surrounding the posthuman condition. Does that resonate with you?

PT: Perhaps, but I am not that interested in contemporary perspectives on humans. I don’t believe that we have changed since we became sapient. Indeed, it seems that we are not capable of progress. We are a neurotic, self-centered species. In all my works — whether installations, digital art, or paintings — I am interested in depicting what is deeply human and what has remained unchanged in our behaviors and in our experience of life.

For Paul’s Memories, I selected images of people who had previously sat for installations, producing a series of paintings together with a robot that sought to reveal human presence, not simply their likeness. I developed the project while I was a fellow at the University of Konstanz in Germany, working with an industrial robot, E-David, that had been adapted to handle brushes based on visual feedback. 

The system works with many layers of glaze and impasto, similar to Old Masters such as Rembrandt and Goya. The process involves the robot applying a layer of thick white paint to a gray background. Once dry, I apply a transparent colored glaze. The robot takes a picture, plans the next strokes, and the cycle repeats itself.

I applied the glazes myself because it was too complicated to automate. It’s funny: the robot does the complex, interesting work, and the human does the boring job. Our roles are reversed.