Two years ago, Google bought some of the most advanced robotics firms in the world.

Now two patents filed by Google have shed light on what the technology giant may be planning to do with its bots.

In the first patent, Google describes a computer system that could manage a fleet of robots inside a warehouse. Warehouse automation would fit with the type of firms owned by Google. Among the seven robotics companies Google purchased are outfits that specialise in technologies suited to loading and unloading goods. For example, Redwood Robotics, with its goal of building electromechanical arms that can operate safely around people, and Industrial Perception, with its expertise in computer-vision systems.

The patent shows a warehouse with an array of robots carrying out various tasks, co-ordinated by the central system. As an example, the patent includes a sketch, shown above, picturing how the system could manage the bots and get them to unload a truck.

The image shows a wheeled bot with a robotic arm unloading goods from a truck onto an autonomous flat-backed vehicle, which takes the items to another robotic arm for sorting. A robotic forklift then takes pallets of items to shelves in the warehouse. The patent suggests a similar system could also be used to load goods from shelves onto trucks.

Google’s patent states that the bots would relay regular updates on their location and current activity back to the central control system. Using this information, the system would be able to predict where at least some of these bots would be and what they would be doing in the future. The system could use this knowledge — combined with regularly updated information on where materials and goods were in the warehouse — to co-ordinate the robots’ actions and movements, enabling the machines to work together to sort and move objects.

Google references the bots employing computer vision, lifting and gripping technologies to move the objects within the warehouse. Bots would also be able to sync their movements and avoid collisions by performing what Google calls a “visual handshake”, for instance, one bot using its cameras to spot tags on a second bot. Doing so would supply the bot and the central system with information about the bots’ relative positions. The process is the subject of the second patent. Both patents were published yesterday.

Fellow technology giant Amazon has already begun stepping up the robotic automation of its warehouses. A growing number of Amazon’s fulfilment centers are partially automated, with centers using Kiva robots to carry shelves of products to human workers, who then pick the items to be shipped.

A more difficult problem that Amazon has been wrestling with is how to get robots to reliably recognise and pick up items from shelves. To that end, the firm last year set up the Amazon Picking Challenge (APC), a contest where robotics researchers compete for a $25,000 prize for designing the best picking bot.

The inaugural contest highlighted just how difficult bots find such tasks, with about half of the teams competing failing to score a single point and the bots working far more slowly than humans.

In spite of the setback, firms across the world are working on improving the capabilities of robotic grippers. Earlier this year, Google hooked 14 robotic arms to a deep learning neural network, which was able to teach the arms to pick up small and irregular-shaped objects, such as a cup, tape dispenser and toy dolphin. The network learned how to pick up these items after being fed video footage of more than 800,000 attempts by the arms to grasp these objects. Their performance still wasn’t perfect, with a failure rate of about 18-percent.

However, with continued technical progress, automation engineers have predicted that in the near future robots will perform a variety of warehouse roles currently carried out by humans.

More on robotics…