Transforming Boston Dynamics’ Robot Dog into a Charming ChatGPT Tour Guide

We witnessed Spot’s physical capabilities of running, jumping, and dancing, but now, an astounding development allows him to talk. A recent video posted by Boston Dynamics unveils Spot, the robot dog, wearing a top hat, mustache, and googly eyes, engaging in conversation with staff members in a charming British accent, leading them on a tour of Boston Dynamics’ facilities.

“Shall we commence our journey?” Spot inquires. “Our first destination is the charging stations, where Spot robots rest and recharge. Please follow me, gentlemen.” As seen in the demonstration, Spot can answer questions and even simulate speech by opening its “mouth.”

To enable Spot’s “talking” abilities, Boston Dynamics leveraged OpenAI’s ChatGPT API and open-source large language models (LLM) for meticulous training. The robot was then equipped with a speaker for text-to-speech functionality, and its gripper was modified to mimic speech, just like a puppet’s mouth.

According to Matt Klingensmith, the principal software engineer at Boston Dynamics, Spot was given a succinct script for each room within their facilities. The robot then combined the script with inputs from its gripper and body cameras, extracting more information about its surroundings before generating a response. Boston Dynamics states that Spot employs Visual Question Answering models to describe images and provide answers.

Spot assumes various personas during the video, including a sophisticated butler, a 1920s archaeologist, a teenager, and even a Shakespearean time traveler. It can even adopt a sarcastic personality, as demonstrated when asked to create a haiku: “Generator hums low in a room devoid of joy. Much like my soul.”

While experimenting with Spot as a tour guide, Boston Dynamics encountered intriguing surprises. When asked about its “parents,” Spot autonomously led the team to older Spot models displayed in the company’s office. However, there were instances where the language models generated false information, suggesting that Stretch, a robot designed for box movement, had a purpose in yoga.

“We are enthusiastic about further exploring the combination of artificial intelligence and robotics,” expresses Klingensmith in a blog post on Boston Dynamics’ website. “LLMs can contribute cultural context, common knowledge, and adaptability that prove valuable for various robotic tasks. For instance, conversing with a robot to assign it a task reduces the learning curve associated with using these systems.”