The Multipurpose Autonomous Agent Project: Experiential Learning for Engineering Assistive Artificial Intelligence

Abstract

Assistive Technologies (AT) and Artificial Intelligence (AI) that support humans in decision making and in difficult or dangerous tasks are in high demand. We created a two-semester capstone project, for undergraduate seniors, providing the opportunity to build an assistive AI algorithm implemented on a skid-steer rover platform. By the end of the program, students created a system with the potential for assisting humans in dangerous indoor situations such as: gas leaks, bomb threats, fires, and active shooters. Our unique approach allowed the skid-steer rovers to autonomously navigate indoor areas never before encountered or previously mapped. Students used deep behavioral cloning techniques coupled with deep reinforcement learning to train the rovers for speed, steering control, and cornering. Outfitted with nothing more than a depth-sensing optical camera, an inexpensive autopilot, and an onboard, assistive NVIDIA Jetson Xavier NX computer, the rover quickly scanned and oriented to a new environment and then located objects of interest. The students’ final product demonstrated impressive abilities and skills demanded by industry in developing AT and AI platforms for mission-critical applications. Herein we share our approach, technology stack, experiences, and artifacts produced by our students at the end of the project.