Virtual Test Beds for Image-Based Control Simulations Using Blender

Abstract

Process systems engineering research often utilizes virtual testbeds consisting of physicsbasedprocess models. As machine learning and image processing become more relevant sensingframeworks for control, it becomes important to address how process systems engineers can researchthe development of control and analysis frameworks that utilize images of physical processes.One method for achieving this is to develop experimental systems; another is to use software thatintegrates the visualization of systems, as well as modeling of the physics, such as three-dimensionalgraphics software. The prior work in our group analyzed image-based control for the small-scaleexample of level in a tank and hinted at some of its potential extensions, using Blender as the graphicssoftware and programming the physics of the tank level via the Python programming interface. Thepresent work focuses on exploring more practical applications of image-based control. Specifically, inthis work, we first utilize Blender to demonstrate how a process like zinc flotation, where imagesof the froth can play a key role in assessing the quality of the process, can be modeled in graphicssoftware through the integration of visualization and programming of the process physics. Then, wedemonstrate the use of Blender for testing image-based controllers applied to two other processes:(1) control of the stochastic motion of a nanorod as a precursor simulation toward image-based controlof colloidal self-assembly using a virtual testbed; and (2) controller updates based on environmentrecognition to modify the controller behavior in the presence of different levels of sunlight to reducethe impacts of environmental disturbances on the controller performance. Throughout, we discussboth the setup used in Blender for these systems, as well as some of the features when utilizingBlender for such simulations, including highlighting cases where non-physical parameters of thegraphics software would need to be assumed or tuned to the needs of a given process for the testbedsimulation. These studies highlight benefits and limitations of this framework as a testbed forimage-based controllers and discuss how it can be used to derive insights on image-based controlfunctionality without the development of an experimental testbed.