New Improved Policy for Multi-Robot Coordination

New Improved Policy for Multi-Robot Coordination

12 August 2015, MIT – To achieve a common goal with a team of multiple robots is very difficult because the human world is full of so much uncertainty. Specifically, robots deal with three kinds of uncertainty, related to sensors, outcomes, and communications. There should be some policy to deal with these uncertainties and that decide the work for each robot and its execution method in the real world.

The MIT researchers came with new and more efficient policy that provides better collaboration with respect to previous policies. The researchers presented a paper on “Policy Search for Multi-Robot Coordination under Uncertainty” in RSS 2015 and demonstrated it on a team of three PR2 robots to serve the beers and meds to the customers. The paper, which was named a Best Paper Finalist, was co-authored by Duke University professor and former CSAIL postdoc George Konidaris, MIT graduate students Ariel Anders and Gabriel Cruz, MIT professors Jonathan How and Leslie Kaelbling, and lead author Chris Amato, a former CSAIL postdoc who is now a professor at the University of New Hampshire.

CSAIL team presented a new system of three robots that can work together to deliver items quickly, accurately and, perhaps most importantly, in unpredictable environments. The team says its models could extend to a variety of other applications, including hospitals, disaster situations, and even restaurants and bars.

To demonstrate their approach, the CSAIL researchers converted their lab into a miniature “bar” that included a PR2 robot “bartender” and two four-wheeled Turtlebot robots that would go into the different offices and ask the human participants for drink orders. The Turtlebots then reasoned about which orders were required in the different rooms and when other robots may have delivered drinks, in order to search most efficiently for new orders and deliver the items to the spaces.

The team’s techniques reflect state-of-the-art planning algorithms that allow groups of robots to perform tasks given little more than a high-level description of the general problem to be solved. And the researchers were ultimately able to develop the first planning approach to demonstrate optimized solutions for all three types of uncertainty.

Policy for Multi-Robot Coordination

Their key insight was to program the robots to view tasks much like humans do. As humans, we don’t have to think about every single footstep we take; through experience, such actions become second nature. With this in mind, the team programmed the robots to perform a series of “macro-actions” that each include multiple steps.

For example, when the waiter robot moves from the room to the bar, it must be prepared for several possible situations: The bartender may be serving another robot; it may not be ready to serve; or it may not be observable by the robot at all.

“You’d like to be able to just tell one robot to go to the first room and one to get the beverage without having to walk them through every move in the process,” Anders says. “This method folds in that level of flexibility.”

The team’s macro-action approach, dubbed “MacDec-POMDPs,” builds on previous planning models that are referred to as “decentralized partially observable Markov decision processes,” or Dec-POMDPs.

“These processes have traditionally been too complex to scale to the real world,” says Karl Tuyls, a professor of computer science at the University of Liverpool. “The MIT team’s approach makes it possible to plan actions at a much higher level, which allows them to apply it to an actual multi-robot setting.”

The findings suggest that such methods could soon be applied to even larger, more complex domains. Amato and his collaborators are currently testing the planning algorithms in larger simulated search-and-rescue problems with the Lincoln Lab, as well as imaging and damage assessment on the International Space Station.

“Almost all real-world problems have some form of uncertainty baked into them,” says Amato. “As a result, there is a huge range of areas where these planning approaches could be of help.”


  • Explore further: Multi-Robot Handling Under Uncertainty
  • Reference: “Policy Search for Multi-Robot Coordination under Uncertainty” Christopher Amato, George Konidarisy, Ariel Andersz, Gabriel Cruzz, Jonathan P. Howx and Leslie P. Kaelblingz; RSS 2015. [Paper]
  • Source: MIT 
  • Image: MIT

# Robotics Labs in MIT


Leave a Reply

Your email address will not be published. Required fields are marked *