The robots who ask for help: Why bots need people as much as we need them

Robots that ask people for assistance when they get stuck show how modern bots could surpass their limitations and play a useful role in hotels, hospitals and schools.

Professor Manuela Veloso greets a CoBot outside an office at CMU
Image: CMU

In movies robots often transcend human abilities, capable of acting smarter and faster than their fleshy creators.

But in reality there are tasks that most people find unthinkably simple, such as walking on two legs or picking up items, which bots find a real struggle.

Not wanting the field of robotics to be defined by its limitations, professor Manuela Veloso, an expert in AI and robotics, has devised a simple way of leaping these hurdles. If the robot can't do it, then get a human to help.

To this end, Veloso and her colleagues have built robots that can ask for assistance, the aptly named CoBots (short for Collaborative Robots). At various points during the day these bots can be seen trundling the corridors in the building where Veloso works at Carneige Mellon University (CMU).

At a time when robots in the office may still seem a long way off, these wheeled bots actually play an - admittedly limited - role in the running of the university, guiding visitors to Veloso's office and driving themselves across the building to fetch items or deliver messages.

The CoBots stand ready to work
Image: CMU

While impressive, what allows these bots to function in a busy human workplace isn't a step change in their capabilities but the ability to seek aid from people when stuck.

Upon reaching an elevator, a bot on the way to Veloso's office will ask the nearest human to call the lift or if a person is blocking their path the machine will ask them, politely but repeatedly, to move out of the way. If there are no people around then the machine will fire off an email asking for advice.

"We call this a symbiotic autonomy approach, where they rely on humans around them in their environment and will ask for help," said Veloso.

"I'm very determined to not let the limitations stop the research on AI and robotics."

Humans interact with via the bots' touchscreen face, whose interface flashes up messages and allows people to schedule tasks via speech or drop-down menus. Jobs can also be queued using a web app.

The bots even get to join in on the office fun. Last week marked the fourth time the CoBots have delivered Halloween candy throughout the building. The task is more than an excuse for spreading cheer, providing Veloso and her students with an opportunity to test how the robots perform at planning a trip to each office and memorising which offices were closed and need to be revisited.

The robots don't just chat to people, but also to each other, wirelessly communicating to co-ordinate tasks they have been given and share useful updates - not only building maps but also information, such as an office door that another bot is en-route to being shut. This group knowledge can extend to web-sourced concepts, such as 'Where is coffee most likely kept?', which may in future aid bots in understanding what is meant by requests that assume a lot of knowledge, such as 'Get me a cup of coffee'.

So if the bots are so capable, why aren't they a fixture of the modern workplace?

Their absence has more to do with a lack of business interest than the readiness of the technology, says Veloso.

"What is really between us and having such robots is having some company to make it happen.

"I don't think that it's a research problem because we currently have shown that it can happen."

These collaborative bots could be useful in any large public building - hospitals, schools, hotels and supermarkets - she said, whirring back and forth showing people to where they need to go or fetching objects.

"These robots actually fill in for various things that a person may need help with."

Years of traversing the hallways of Veloso's block at CMU have allowed the wheeled bots to become intimately familiar with its layout - mapping the building's interior using depth-sensing cameras and laser scanners.

In fact, the process of generating these scans - creating a 3D model of a building's corridors and mapping them to a floor plan showing each room's location - can be done in a relatively short space of time.

Within a few days of the CoBots being deployed inside the Center for Urban Science and Progress at New York University the bots knew the layout well enough to navigate.

"Conceptually there was no difficulty at all, as soon as you have the floor plans the robot was able to navigate."

However, the bots can capture far more than an office's layout. During their journeys around the CMU building the CoBots have travelled more than 1000km and have recorded data on the building's wifi signals, noise levels and temperature.

"We have the most accurate data about these buildings in terms of wi-fi signal strength and temperature and that can help with planning a much better use of resources. The amount of data they can generate can be of great use for humans."

The future is collaborative

Robotics is a field made up of many different disciplines, each advancing at a different rate. Fellow CMU academic and AI expert Andrew Moore recently highlighted this differing pace of progress, citing how improvements to robotic manipulation had not kept up with those to computer vision.

Given the likelihood that bots will always have limits to their knowledge and abilities that will restrict what they can do on their own, Veloso foresees a world where robots recognise what they can't do and ask humans for help. This co-operative approach could play a role in every potentially difficult area where robots are beginning to help out, from driving cars to disaster cleanup.

"I think it will happen that when it's a very snowy day an autonomous car may say as it's leaving a garage 'I cannot handle this scenario with all this snow'," she said.

In this world where humans and bots collaborate to carry out tasks, the challenge for researchers then becomes how to represent all the things that a bot can't do, she added.

"Just as humans like you and I are not able to do everything and don't know about everything, robots will always have limitations.

"The thing would be to continue developing algorithms in which the robots themselves are useful but capable of asking for help.

"The research is more of an understanding that humans will eventually co-exist and be in the same space, sharing tasks with the robots. So there will be dogs, cats, humans and robots."

Breaking the language barrier

Given that collaborative robots rely on both people and machines being able to share information, Veloso says that future research will focus on clearer bot-person communication.

"We have a lot of research to do on a better human-robot interface," said Veloso.

"The robot can move around but we cannot really communicate with them asking them 'What did you do yesterday?' or 'How long did it take you to go from here to there?'. There is very little ability so far to really query the robot, asking 'What are you going to do next?'."

Veloso isn't talking about letting people make an emotional connection with machines, rather finding a way to reconcile the very different way that bots and people represent the world.

"It's bridging the gap between natural language - between how a human would eventually talk with the robot - and the internal representation of the robot.

"The robot lives in a world of sensory data, of maps and x-y locations, of times, of currents and voltages. It's a whole other representation and to have that be translated into English in a way that a human understands. It is almost like asking a car 'How does your engine work?' and expecting it to generate an explanation."

Yet in spite of the limitations of CoBots today, Veloso believes the bots are sophisticated enough to help people outside the lab and admits to being disappointed they aren't yet a fixture in the wider world.

"People need to realise that it's not as much science fiction as they think it's. They talk about autonomous cars but I think about robots in the human environment, not on Mars, not on roads but in our buildings, helping people.

"Every time I go somewhere and enter these big hotels I say 'Where are the robots?'."