Teaching AI to Collaborate, not Merely Create, Through Dance

Posted May 2, 2024

Two children are playing with a set of toys, each playing alone. That kind of play involves a somewhat limited set of interactions between the child and the toy. But what happens when the two children play together using the same toys?

“The actions are similar, but the choices and outcomes are very different because of the dynamic changes they’re making with the other person,” says Brian Magerko, Regents’ Professor in Georgia Tech’s School of Literature, Media, and Communication. “It’s a thing that humans do all the time, and computers don’t do with us at all.”

Welcome to the next frontier of artificial intelligence (AI) — not just generating but collaborating in real-time.

Magerko and his colleagues, Georgia Tech research scientist Milka Trajkova and Kennesaw State University Associate Professor of Dance Andrea Knowlton, are putting a collaborative AI system they’ve developed to the ultimate test: the world’s first collaborative AI dance performance.

Dance Partner

LuminAI is an interactive system that allows participants to engage in collaborative movement improvisation with an AI virtual dance partner projected on a nearby screen or wall. LuminAI analyzes participant movements and improvises responses informed by memories of past interactions with people. In other words, LuminAI learns how to dance by dancing with us.

The National Science Foundation-supported project began about 12 years ago in a lab and became an art installation and public demo. LuminAI has since moved into a different phase as a creative collaborator and education tool in a dance studio.

“We’re looking at the role LuminAI can play in dance education. As far as we’re aware, this is the first implemented version of an AI dancer in a dance studio,” says Trajkova, who was a professional ballet dancer before becoming a research scientist on the project.

To prepare LuminAI to collaborate with dancers, the research team started by studying pairs of improvisational dancers.

“We’re trying to understand how non-verbal, collaborative creativity occurs,” Knowlton says. “We start by trying to understand influencing factors that are perceived as contributing to improvisational success between two artists. Through that understanding, we applied those criteria to an AI system so it can have a similar experience with co-creative success.”

“We’re working on a creative arc,” adds Trajkova. “So instead of the AI agent just generating movements in response to the last thing that happened, we’re working to track and understand the dynamics of creative ideas across time as a continuous flow, rather than isolated instances of reaction.”

Students from Knowlton’s improvisational dance class at Kennesaw State spent two months of their spring semester working routinely with the LuminAI dancer and recording their impressions and experiences. One of the purposes the team discovered is that LuminAI serves as a third view for dancers and allows them to try ideas out with the system before trying it out with a partner.

The classroom experiment will culminate in a public performance on May 3 at Kennesaw State’s Marietta Dance Theater featuring the students performing with the LuminAI dancer. As far as the research team is aware the event is the world’s first collaborative AI dance performance.

While not all the dancers embraced having an AI collaborator, some of those who were skeptical at first left the experience more open to the possibility of collaborating with AI, Knowlton says. Regardless of their feelings toward working with AI, Knowlton says she believes the dancers gained valuable skills in working with specialized technology, especially as dance performances evolve to include more interactive media.

Refined Movement

So, what’s next for LuminAI? The project represents at least two possible paths for its learnings. The first includes continued exploration about how AI systems can be taught to cooperate and collaborate more like humans.

“With the advent of generative AI these past few years, it’s been really clear how great a need there is for this sort of social cognition,” says Magerko. “One of the things we’re going to be getting off the ground is sense-making with large language models. How do you collaborate with an AI system – rather than just making text or images, they’ll be able to make with us.”

The second involves the body movements LuminAI has been cataloging and analyzing over the years. Dance exemplifies highly refined motor skills, often exhibiting a level of detail surpassing that found in various athletic disciplines or physical therapy. While the tools designed to capture these intricate movements—through cameras and AI—are still nascent, the potential for harnessing this granular data is significant, Trajkova says.

That exploration begins on May 30 with a two-day summit being held at Georgia Tech to discuss its application for transforming performance athletics, with interdisciplinary participants in dance, computer vision, biomechanics, psychology, and human-computer interaction from Georgia Tech, Emory, KSU, Harvard, Royal Ballet in London, and Australian Ballet.

“It’s about understanding AI's role in augmenting training, promoting wellness as well as diving deep in decoding the artistry of human movements. How can we extract insights about the quality of athlete’s movements so we can help develop and enhance their own unique nuances?” Trajkova says.