Seg-CURL: Segmented Contrastive Unsupervised Reinforcement Learning for Sim-to-Real in Visual Robotic Manipulation

1Citations
Citations of this article
6Readers
Mendeley users who have this article in their library.

This article is free to access.

Abstract

Training image-based reinforcement learning (RL) agents are sample-inefficient, limiting their effectiveness in real-world manipulation tasks. Sim2Real, which involves training in simulations and transferring to the real world, effectively reduces the dependence on real data. However, the performance of the transferred agent degrades due to the visual difference between the two environments. This research presents a low-cost segmentation-driven unsupervised RL framework (Seg-CURL) to solve the Sim2Real problem. We transform the input RGB views to the proposed semantic segmentation-based canonical domain. Our method incorporates two levels of Sim2Real: task-level Sim2Real, which transfers the RL agent to the real world, and observation-level Sim2Real, which transfers the simulated U-nets to segment real-world scenes. Specifically, we first train contrastive unsupervised RL(CURL) with segmented images in the simulation environment. Next, we employ two U-Nets to segment robotic hand-view and side-view images during real robot control. These U-Net are pre-trained with synthetic RGB and segmentation masks in the simulation environment and fine-tuned with only 20 real images. We evaluate the robustness of the proposed framework in both simulation and real environments. Seg-CURL is robust to the texture, lighting, shadow, and camera position gap. Finally, our algorithm is tested on a real Baxter robot with a dark hand-view in the cube lifting task with a success rate of 16/20 in zero-shot transfer.

Cite

CITATION STYLE

APA

Xu, B., Hassan, T., & Hussain, I. (2023). Seg-CURL: Segmented Contrastive Unsupervised Reinforcement Learning for Sim-to-Real in Visual Robotic Manipulation. IEEE Access, 11, 50195–50204. https://doi.org/10.1109/ACCESS.2023.3278208

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free