Guided Reinforcement Learning with Vision Feedback

Abstract

This thesis investigates how to improve reinforcement learning (RL) for contact-rich robotic assembly by combining probabilistic trajectory guidance with visual representation learning. Building on Kernelized Guided Reinforcement Learning (KGRL), where Kernelized Movement Primitives (KMP) encode demonstrations as probabilistic trajectory priors and guide policy learning through uncertainty-aware null space actions, we introduce Vision-KGRL, which augments this framework with compact visual features learned via task-specific Variational Autoencoders (VAEs). The approach is evaluated on peg insertion, gear meshing, and nut threading tasks in the NVIDIA Isaac Lab Factory and Forge environments, showing that visual augmentation preserves the faster convergence of KGRL over standard RL, with stronger gains in higher-dimensional action spaces. Systematic ablations across observation modalities (proprioception, force, vision) and action spaces (4D vs 6D) highlight the complementary roles of trajectory guidance and learned visual features, while results further show that KMP-guided policies significantly reduce interaction forces, leading to smoother and more stable behavior. Overall, Vision-KGRL provides a data-efficient and robust solution for contact-rich manipulation, combining learned visual representations with trajectory priors to improve both learning performance and execution safety.