How Team Delft won the APC 2016 (1): System Overview

When we decided to participate in the Amazon Picking Challenge, the first decision was to a) gather the best team possible, b) design the best possible system, to win. We would choose the best technologies and components available to build a reliable and efficient solution, as industry requires.

Team Delft has gathered academic researchers and industry professionals,
but overall passionate roboticists! 

To make the baseline design decisions for our solution we analyzed the application requirements, studied the results from the previous edition of APC, and considered the expertise we had in our team. We aimed for a fast and robust system, capable of detecting failed picks and try them again. For example, instead of complex maneuvers to reach occluded products, our robot would simply move blocking objects to other bins.

Our solution was based on the sense-plan-act paradigm. This allows for optimal motions at the price of more precise sensing information. The loop for the Picking challenge proceeds as follows. First the order of the picks is decided to maximize final scoring and guarantee the fastest first pick possible (this proved key to win!). Then, for each item to pick the robot moves to scan the bin with the first target item with the 3D camera to: a) locate it, b) get the collision information to move inside the bin. Then the grasp planner computes the grasping strategy and grasp pose, and a motion plan is generated to approach, and retreat from the bin with the item. Following the motion is executed, including gripper configuration and activation on the way. State is checked to confirm a successful pick. If so, the robot moves to place the item in the tote, using a simple logic to drop the item in pre-defined locations without crashing fragile products. The flow for the Stowing Challenge is similar.

For the control software we used ROS-Industrial. We had extensive experience in the team using ROS-I for industrial applications. The use of ROS proved mandatory. ROS allowed us to quickly adapt, test and integrate extant software components, and develop the new ones that we needed  (e.g. a grasp planner for our hybrid gripper, or the application executive/coordination component).

We describe below our basic design approach and core decisions for each general issue n the APC challenge: robot, perception, grasping and gripper, motion and task planning. In following posts we will cover more details of our robotic system in each specific area.

Robot

After preliminary analysis, we concluded that maximum reachability in the workspace was a critical requirement for the competition. We chose a configuration with a SIA20F Motoman robot arm by Yaskawa mounted on a rail. The total 8 degrees of freedom allowed the system to reach all the bins with enough maneuverability to pick the objects from any location inside.

Vision

We decided to use two industrial stereo camera's with rgb overlay camera's. One camera for detection of objects in the tote and one on the robot gripper to scan the bins. Detection went in two steps. For object recognition we used a deep learning neural network based on Faster RCNN. Next we performed localization using an implementation of super4PCS to do a global optimization of the pose estimation. After this the estimation is refined using ICP.

Grasping and gripper

We developed a hybrid gripper customized to handle all the 39 products in the competition. It incorporates both suction, with 1dof for more grasping flexibility, and a pinch mechanism for the products difficult to suck. We designed an algorithm that auto generates grasp locations on the objects based on their position, geometry and environment constraints. Object-specific heuristics were also included after testing.

Motion

To make the robot motions as fast as possible, yet accurate enough for our sense-plan-act approach, we divided the problem into the motions outside the shelf and inside its bins. Outside there is no need for dynamic path planning: the environment is static and known a priori. We created a database of collision-free trajectories between the relevant fixed locations, e.g. image-capture and approach locations in front of the shelf’s bin and over the tote. To move inside the bins we simplified the dynamic path planning problem by using cartesian trajectories. To implement our solution we developed customized ROS-services on top of the ROS framework for motion planning MoveIt!

Task planning and coordination

To control the execution of our sense-plan-act overarching loop we used a state machine approach implemented with ROS-Smach.

We implemented a dynamic task-planning to decide in which order to pick the items and where to place them. It coded the competition scoring rules in logic rules, averaged with heuristics for the difficulty of picking the different items, also depending on the bin cluttering.

More details about Team Delft robot coming soon!

Twitter: @teamdelft_apc