Offline Feature-Based Reinforcement Learning with Preprocessed Image Inputs for Liquid Pouring Control

Abstract

A method for the creation of a liquid pouring controller is proposed, based on experimental data gathered from a small number of experiments. In a laboratory configuration, a UR5 robot arm equipped with a camera near the end effector holds a container. The camera captures the liquid pouring from the container as the robot adjusts its turning angles to achieve a specific pouring target volume. The proposed controller applies image analysis in a preprocessing stage to determine the liquid volume pouring from the container at each frame. This calculated volume, in conjunction with an estimated target volume in the receiving container, serves as input for a policy that computes the necessary turning angles for precise liquid pouring. The data received on the physical system is used as Monte-Carlo episodes for training an artificial neural network using a policy gradient method. Experiments with the proposed method are conducted using a simple simulation. Convergence proves to be fast and the achieved policy is independent of initial and goal volumes.