I recently participated in the Agile Robotics for Industrial Automation Competition held by the National Institute of Standards and Technology. After spending the last several months developing a working control system for the provided environment, I will now discuss in more detail the design of the final system, as well as possible future improvements.
The software, which is available here, is written in C++, using the ROS package format and the CMake build system. ROS provides communication between modular processes called nodes. The packages each implement separate nodes for world representation, robot controllers, and collision object detection. The control system uses the MoveIt Motion Planning Framework for calculating collision-free motion trajectories for the robot to execute within its environment.
The ARIA Competition focused on order fulfillment in a factory setting - picking and placing parts into shipment containers for delivery. Given a high-fidelity simulation environment of the shipment area and an industrial robot, the KUKA IIWA14 Robot, I was tasked with developing a system that could autonomously detect order parts in the environment, move the robot to correctly place them into a shipment box based on received order specifications, and submit the parts for processing. The simulation provides a realistic interface for the IIWA14 robot where I had access to joint motor commands. Using the MoveIt framework, I was able to create smooth motion executions to perform the order fulfillment task.
The two main entities to track in the environment are the parts and the robot. As the robot performs actions, grabbing and moving parts, filling shipment boxes, and fulfilling orders, the world representation must keep track of object relationships so that the robot has a valid perception of the environment. Camera sensors provide the perception input, and this allows the system to detect and track the various parts. Certain abstractions help keep track of the robot and part relationships. All parts belong in a “container,” whether they are the bins, the shipment box, or the robot’s gripper. As the robot grabs and moves parts around, it modifies the parts and their containers to maintain a valid representation of the environment.
The world_state package implements the world representation. This is an abstraction to help the robot reason about its environment, so the control system must update the world representation when any actions are taken that have an effect on the world. A client interface provides a simple and clean way to communicate this, abstracting the details of ROS communication and implementation, focusing just on higher level intentions.
Robot Control System
The control system takes the available low-level interface of robot motor commands, and coordinates them into a higher level action controller. The main goal of fulfilling a shipment order can be broken down into individual tasks (move a part to the box). These tasks can be further broken down into actions (grab this part, move to the box, etc…). The control system takes these action commands and converts them into physical robot motions to achieve the desired outcome.
The system was designed with modularity and extensibility in mind. The ActionManager class receives action commands, interprets them into more specific robot motions, and passes them to the ControllerServer for execution. The ControllerServer class then receives these requests and must determine which robot controller to send them to. The abstraction between The ControllerServer and the specific robot controller allows easy interchangeability for various types of robots. This way, the ActionManager does not need to worry about how the robot will actually execute its commands, or even what specific type of robot is in use.
In this case, the simulation uses the KUKA IIWA14 Robot, and the IIWA14Controller class implements the specific executions necessary to carry out the motion requests generated in the ActionManager.
The robot motions are implemented using the MoveIt motion planning framework. In order to calculate feasible motions in the environment, the system needs a representation of all possible collision objects.
MoveIt keeps track of collision objects in its planning scene, and everything in the environment (bins, shelves, conveyor belt, etc…) is saved in the planning scene using the predefined object dimensions and transforms. The load_scene package computes the world transforms of all collision objects and sends them to the MoveIt planning scene. Then, the motion planning algorithms can detect environment objects and compute trajectories around them.
For more complicated tasks - emergency orders, long term reasoning, and action verification - simple action planning is not enough. AI Planning languages such as Planning Domain Definition Language can be used to create better plans, given a thorough representation of the environment. Instead of a custom WorldState class, a formalized ontology of the working environment and the possible relationships, as discussed in Kootbally et al. 2017, along with a PDDL problem definition and planner can provide a robust method for planning complex goals, verifying task successes, and replanning in case of failures.