Written by: The ANA Avatar XPRIZE Operations Team

The ANA Avatar XPRIZE Semifinals testing, held last month from September 10-18 in Miami, Florida, was an intriguing first look at what teams from around the world have been working on for the last two years while they compete in this bold competition. What we asked teams to accomplish went far beyond anything you might see in a standard robotics competition. We did not measure the teams on how much their robot could do, which is typical, but rather on how well their Avatar system serves to connect two people who are physically in different places.

While many of us have become used to connecting remotely during the COVID-19 pandemic, video conferencing provides only a subset of what we typically experience within our normal in-person communication. We believe that Avatars are the next phase of human interaction that will connect us in ways we never thought possible.

A robotic Avatar that truly represents the person operating it — that conveys our intentions, emotions, actually gives us a physical presence, and enables true interaction within a remote location — could bring a new level of communication to remote person to person connections. That is the ultimate goal of the ANA Avatar XPRIZE competition—to provide a way for people to connect over distances and surpass barriers to meaningful engagement, more similar to our real life interactions but through a revolutionary new medium.

“Being able to go beyond what we have today with telephones and video conferencing and adding in full embodiment, with movement and touch, takes the connection to a much more human level. [This competition is] actually about connecting people. It’s not about the robot! The robot should really recede into the background when it is done right.” — Ed Colgate, ANA Avatar XPRIZE Judge

Many technologies must be brought together for Avatars to take their place in our everyday connections, and teams from around the world have been actively inventing robotic solutions to enable this future. At the ANA Avatar Semifinals we were excited to witness some of these emerging solutions actively being demonstrated in public for the first time outside their research labs.

Photos by Andra Keay 

Images: Judge Bill Borgia with two of the many Avatar forms at the ANA Avatar XPRIZE Semifinals. Left: Team Touchlab. Right: Team SNU.  

While these Avatars took many robotic forms, they each had to provide the means for a Judge in the role of “Operator” to teleport their senses and feeling of physical being to a remote location to interact with another Judge, in the role of “Recipient,” in real time. An Avatar has to give an Operator a way to express themselves in as fully embodied a way as possible. 

Photos by Eric Liga 

Images: Two of the many solutions teams devised to allow an Operator to control a remote robotic Avatar. Left: A view of Team UNIST’s Operator Control System. Right: Judges had to adapt to a wide range of Avatar control systems during Semifinals.

Recipients were just as important as a role as an Operator, as the two Judges each had scoring mechanisms based on specifics on their own experiences and how they felt about their interaction with each other. Was there a shared experience during the test runs and did they really feel as if the other person was there with them? Judge experiences were critical in determining a true measure of interaction. 

Photos by Eric Liga, Dilip Patel

Images: Recipient Judges establishing a rapport with an Operator Judge via the Avatar. Left: Team Roboy. Right: Team Avatar-Hubo.

One challenge we faced in designing Semifinals testing was how to truly evaluate the Avatars systems as a whole for how well they connected the Operator and Recipient Judges. Determining evaluation criteria and how to score for this round of testing required a great deal of thought and discussion. The scoring had to include not only objective measures (such as “Could the Operator see and hear what was being said in the remote location?” and “Could the Avatar hold and lift a beverage container?”) but importantly the subtler, subjective criteria (such as “Was the Operator Judge able to express their emotions to the Recipient through the Avatar?” and “Did the Recipient Judge feel a sense of shared experience with the remote Operator?”)

Over a year’s worth of discussions with input from our esteemed Judging panel, as well as industry and subject matter experts, led to the design of three scenarios that the competing teams’ Avatar systems were required to perform. These scenarios comprised simple tasks, but were based on real world situations in which future Avatars are expected to play starring roles.

The scenarios included a cooperative puzzle task, a celebration between two colleagues to mark the end of a successful business deal, and a visit to a remote museum of antiquities. The teams’ Avatars had to enable the Operator to select and pick up objects and feel their texture and weight, see and converse with the Recipient, move around the testing space and interact in a natural conversational manner. Judges assessed not only the Avatar’s ability to facilitate this interaction, but also how much the two Judges really felt a connection with each other.

“In a competition like this, it is easy to get lost in the technical details. Did the Avatar move? Did the Avatar place the puzzle piece? It is important that we remember this is really about how to bring two people who are at a distance into a shared physical space. And we need to focus on interacting with that person and getting a sense of feeling like I am really together with this person.

Once the technical stuff works, you need to focus on how do you build that connection? How do you enable that interaction? — Cori Lathan, ANA Avatar XPRIZE Judge

These scoring criteria coupled with a trio of distinct scenarios enabled us to evaluate the Avatars in a completely different manner than robots are commonly tested. This is one of the few competitions in which robots have been evaluated through a subjective measure, ensuring that the human interactions these systems are designed for are closely addressed.

Teams were given a two hour slot for their testing session. The first hour was set aside to train the Operator Judge on the Avatar control system they would be using. The second hour was for the performance of the three scenarios. Teams underwent this testing process  on two separate days. Two testing days ensured that a technical problem on one day did not preclude them from testing. The best run from each scenario on either testing day was used to determine the total score. The overall team scores of our advancing teams were high, showing great initial progress towards the goal of having Avatars connecting humans to humans in new and meaningful ways in the near future. The top 15 ranked teams now advance to the Finals to be held in Fall 2022.

“Being able to address more social aspects of human interaction is an excellent focus of this prize and has been a key driver to helping roboticists think about robots in a different way.” — Peggy Wu, ANA Avatar XPRIZE Judge

The ANA Avatar Finals will push teams even further along towards creating the Avatars that will populate our future and we can’t wait to see how teams will rise to meet these ambitious goals. 

Click through for the overall ranking and more info about Finalist Teams.