At the start of the new year, Facebook CEO (and new dad) Mark Zuckerberg, announced a “personal challenge” to spend 2016 working on a robot “butler” to help out at home. The news comes at a time of big innovations in AI, with Toyota, Google, and others unveiling plans to invest heavily in AI and robotics this year.

But what is really needed when it comes to a robot butler?

Carnegie Mellon, a global leader in robotics, has their own version, HERB, the Home Exploring Robot Butler, who’s already 11 years old. HERB’s real-world experience can shed some light on how Zuckerberg might want to move forward in his development of an AI butler. Its creator, Siddhartha Srinivasa, who’s an associate professor at Carnegie Mellon University’s Robotics Institute, offers some insight into what he’s learned from his experience with the dynamics of robot-human interactions.

Srinivasa, who’s spent time researching how to get robots to interact with the physical world, said that Zuckerberg’s plan sounds like a “souped up Siri.” While it’s important to have more intelligence in household appliances, he said, it’s important to see how the intelligence translates in the physical realm.

Srinivasa’s 80-year-old grandmother in India, wants more than something “to tell her the toast is ready. What she needs,” said Srinivasa, is something “that could actually bring the toast to her.”

“For the technical interaction to be effective, it needs to be intelligent,” he said, “but simply having awareness is insufficient.”

Working with the elderly and people with disabilities has shown Srinivasa how important it is to have something that “is a caregiver more than a Siri. What’s critical for their day to day living is something that can do something for them.”

Srinivasa looks at the issue of hardware. While he says it’s encouraging that the price of hardware is becoming more affordable, there’s still a need for “precise” robotic arms and other hardware. Right now, there are “the big bulky arms that build our cars,” he said, “but I can’t put it in my house or in my grandma’s house. We need precise robot arms that can work with people.”

To accomplish this, we need an AI that can work in unpredictable, “cluttered,” real-world situations. Rather than intelligent arms, what we have now are merely “puppets.” CMU is working to address these kinds of issues.

“We are developing not just the AI for intelligent robot assistants,” said Srinivasa, “but the physical hardware that can enable intelligence to interact physically with the world.”

An example? CMU’s HERB, funded by the National Science Foundation, was born in 2005 to help people in their homes. CMU has since started a new assistive care center called “Center for Assistive Robotics for Everyday Living” (CARE) which “works with caregivers to put the algorithms for perception, for machine learning, for control, out there in the real world with patients.”

“Building herb is much like having a child,” said Srinivasa. “You have to actually experience it to see how hard it is.”

So, what can Facebook learn from HERB? Here are a couple takeaways:

  1. Deploy your system in the real world. According to Srinivasa, in order to build an intelligent system, you must first build a physical system that can go out and do something in the real world. “The hardware design influenced our AI design, and AI design influenced our hardware design. It was a harmonious, positive, feedback loop that resulted in HERB. The real world is the best teacher for our experiences.”
  2. Have a big vision. Srinivasa says that having a big goal is the key to success. Carnegie Mellon’s goal was to build a system for assistive care in people’s homes. “Having that goal meant we focused all our decisions, both in Siri and in algorithms, towards that goal. If we want to really go after something, everyone becomes excited and motivated.”

“People like Mark [Zuckerberg] have the ability to create large new initiatives,” said Srinivasa. “But we need initiatives that are not just focused on profit, but on making the world a better place.”

“Making a souped-up Siri is a first step but it’s not going to solve any of our society’s problems,” said Srinivasa. “What people expect from leaders like Mark [Zuckerberg] are these visionary ideas. He’s taking a step toward addressing the problem, but I’m looking forward to seeing greater vision and greater direction from him.”

Ultimately, though, people like Zuckerberg have the chance to make a big impact. “If he can excite kids to work on robotics, that’s a huge win for us.”