Kyle Steinfeld Uses Machine Learning to Draw with Robots at Kresge Theatre

  Kyle Steinfeld discusses the future of machine learning in Kresge Theater on Monday, 9 April 2018. Photo by Christina Brown.

Kyle Steinfeld discusses the future of machine learning in Kresge Theater on Monday, 9 April 2018. Photo by Christina Brown.

By Chitika Vasudeva

While many people discuss machine learning (ML) from the standpoint of ethical and social concerns, Kyle Steinfeld claims he focuses on the opportunities. Steinfeld, Assistant Professor of Architecture at the University of California, Berkeley, delivered the last lecture of the 2018 SoA Spring Lecture Series on Monday, 9 April. Having earned a Master’s degree in Architecture from MIT and worked with several design firms including SOM, KPF, and DS+R, Steinfeld is interested in collaborative design technology platforms, design computation pedagogy, and bioclimatic design visualization. He is the co-author of Geometric Computations: Foundations for Design — a two-volume book that describes the mathematical and computational concepts that are central to the practical application of design computation in a manner tailored to the visual designer.

The potential impact of Steinfeld’s forthcoming book — and his work in general — was the focus of the Back2Front discussion preceding the lecture. The book’s strong and distinct pedagogical agenda proves interesting, as it encourages readers to think computationally about design. SoA Assistant Professor Daniel Cardoso Llach, a friend of Steinfeld’s, offered a clear understanding of data and its implications. Cardoso Llach suggested that rather than understand data as a complete, comprehensive, and fully computable entity floating in space, it should be thought of as something that is designed. Such an approach helps contextualize Steinfeld’s work, as does Mario Carpo’s book The Second Digital Turn (2017), which addresses how advancement in algorithms and statistics will change architecture in profound ways.

Steinfeld opened his talk by clarifying that he is “an architect, not a computer — nor a scientist who studies computers.” Addressing the widely-discussed ethical implications of ML, he expressed that he does not believe that “ML will find universal application to design, nor will use of ML be universally good,” but he does contend that machine learning can open up entirely new paradigms for design. Generative design, he proposed, entails a process of making (generating), iterating, and evaluating. It is essential to understand, however, that a machine’s worldview is based entirely on the data set it is trained on. Showing a series of images transforming from indistinguishable pixels to a banana, Steinfeld explained that having been trained to identify bananas, the program works to optimize, i.e. nudge the image towards what it has been taught to recognize as a banana, and create a “platonic banana.”

A highlight of the lecture was Steinfeld’s demonstration of the Sketch-RNN program developed by creative technologists David Ha, Jonas Jongejan, and Ian Johnson. Titled “‘Draw Together with a Neural Network,” the interactive web experiment generates several possible ways to complete a drawing started by the user. A project that came out of this program was a 2017 experiment by Nono Martinez Alonso called Pix2Pix. Using the software to draw flowers, Alonso pits his own artistic inclination against the machine’s instinct which is directed by his authorship. Steinfeld argued that this demonstrates the potential of ML to become a powerful tool for designers, contending that “creative misuse is as important as the use of the tool as intended."

Audience questions after the lecture dove deeper into the viability of ML as a design tool. While Steinfeld’s examples can be understood as potential tools for designers, the idea that they could have implications for architectural processes was not as easily accessible. Another issue was one of authorship. When asked to reflect on the amount of control a designer can have over a dataset, Steinfeld underlined the importance of media literacy. “Media matters,” he said, equating an improper understanding of the data structure to not knowing how to use control points in Adobe Illustrator, that is, lacking the kind of basic knowledge essential to operating a tool. While there may be some valid questions regarding the viability of ML as a design tool, Steinfeld’s talk was a fitting close to the spring lecture series. He left the audience wondering about the future of the profession and what their place in it might be.

Chitika Vasudeva is a third-year Bachelor of Architecture student in the Carnegie Mellon University School of Architecture.