Thursday, July 21, 2016

How Team Delft won the APC 2016 (1): System Overview

When we decided to participate in the Amazon Picking Challenge, the first decision was to a) gather the best team possible, b) design the best possible system, to win. We would choose the best technologies and components available to build a reliable and efficient solution, as industry requires.
Team Delft has gathered academic researchers and industry professionals,
but overall passionate roboticists!
To make the baseline design decisions for our solution we analyzed the application requirements, studied the results from the previous edition of APC, and considered the expertise we had in our team. We aimed for a fast and robust system, capable of detecting failed picks and try them again. For example, instead of complex maneuvers to reach occluded products, our robot would simply move blocking objects to other bins.

Our solution was based on the sense-plan-act paradigm. This allows for optimal motions at the price of more precise sensing information. The loop for the Picking challenge proceeds as follows. First the order of the picks is decided to maximize final scoring and guarantee the fastest first pick possible (this proved key to win!). Then, for each item to pick the robot moves to scan the bin with the first target item with the 3D camera to: a) locate it, b) get the collision information to move inside the bin. Then the grasp planner computes the grasping strategy and grasp pose, and a motion plan is generated to approach, and retreat from the bin with the item. Following the motion is executed, including gripper configuration and activation on the way. State is checked to confirm a successful pick. If so, the robot moves to place the item in the tote, using a simple logic to drop the item in pre-defined locations without crashing fragile products. The flow for the Stowing Challenge is similar.

For the control software we used ROS-Industrial. We had extensive experience in the team using ROS-I for industrial applications. The use of ROS proved mandatory. ROS allowed us to quickly adapt, test and integrate extant software components, and develop the new ones that we needed  (e.g. a grasp planner for our hybrid gripper, or the application executive/coordination component).

We describe below our basic design approach and core decisions for each general issue n the APC challenge: robot, perception, grasping and gripper, motion and task planning. In following posts we will cover more details of our robotic system in each specific area.

After preliminary analysis, we concluded that maximum reachability in the workspace was a critical requirement for the competition. We chose a configuration with a SIA20F Motoman robot arm by Yaskawa mounted on a rail. The total 8 degrees of freedom allowed the system to reach all the bins with enough maneuverability to pick the objects from any location inside.

We decided to use two industrial stereo camera's with rgb overlay camera's. One camera for detection of objects in the tote and one on the robot gripper to scan the bins. Detection went in two steps. For object recognition we used a deep learning neural network based on Faster RCNN. Next we performed localization using an implementation of super4PCS to do a global optimization of the pose estimation. After this the estimation is refined using ICP.

Grasping and gripper
We developed a hybrid gripper customized to handle all the 39 products in the competition. It incorporates both suction, with 1dof for more grasping flexibility, and a pinch mechanism for the products difficult to suck. We designed an algorithm that auto generates grasp locations on the objects based on their position, geometry and environment constraints. Object-specific heuristics were also included after testing.

To make the robot motions as fast as possible, yet accurate enough for our sense-plan-act approach, we divided the problem into the motions outside the shelf and inside its bins. Outside there is no need for dynamic path planning: the environment is static and known a priori. We created a database of collision-free trajectories between the relevant fixed locations, e.g. image-capture and approach locations in front of the shelf’s bin and over the tote. To move inside the bins we simplified the dynamic path planning problem by using cartesian trajectories. To implement our solution we developed customized ROS-services on top of the ROS framework for motion planning MoveIt!

Task planning and coordination
To control the execution of our sense-plan-act overarching loop we used a state machine approach implemented with ROS-Smach.
We implemented a dynamic task-planning to decide in which order to pick the items and where to place them. It coded the competition scoring rules in logic rules, averaged with heuristics for the difficulty of picking the different items, also depending on the bin cluttering.

More details about Team Delft robot coming soon!

Twitter: @teamdelft_apc

Sunday, July 3, 2016

Team Delft wins the Amazon Picking Challenge 2016

Team Delft wins the Amazon Picking Challenge 2016

We are happy and proud to announce that Team Delft has won both the Picking and the Stowing competitions in the Amazon Picking Challenge. The competition was really intense (just check the results), and we have a lot of fun at RoboCup2016 with the rest of the APC teams.
Team Delft celebrates victory. So many people... try to find the robot!

The stowing challenge was easier than expected, with high scores, and an outstanding performance of Team Delft robot, that stowed correctly 11 products in less than half the max time allowed!

The pick task finals proved to be nerve wrecking. Both Team Delft and our colleagues in team PFN collected 105 points. According to the competition rules, the tie breaker would be the time to successfully complete the first pick; a ‘photo finish’!  The jury maintained the nervous expectancy till the very last minute in the awards ceremony, when they announced Team Delft . NimbRo Picking ended third.

Several news and tech sites have made echo of it:
For more details about the technology behind our robot success, keep alert to this blog in the upcoming days!