AI Security & Vision Lab: Reinforcement Learning for Robotics

I am currently working as a programmer at the AI Security & Vision Lab under the supervision of Dr. Krishna Roy at New Mexico Tech. As the primary programmer, my role involves the research, development, and implementation of reinforcement learning algorithms for robotics, with a focus on utilizing neuromorphic hardware.

Project Overview

Our initial objective was to develop a method for teaching a bipedal robot to walk while leveraging memristor-based electronic structures. To this end, I began by programming a custom simulation environment and researching bipedal dynamics for control theory. After initial research, I proposed a shift towards machine learning over traditional control theory. This decision was driven by two key factors:

1. They dynamics of bipedal robots are highly complex and difficult to model accurately for traditional control algorithms.
2. Memristors, being inherently efficient at performing complex matrix operations within cross-bar arrays, are well-suited for supporting machine learning models.

Given this, we pivoted to using reinforcement learning to enable the robot to learn to walk autonomously. This approach aimed to offload computational workload onto the memristor arrays, utilizing their analog capabilities.

Current focus

While the project’s focus has shifted from physical analog hardware implementation to algorithmic design, we continue to explore the principles of analog hardware design. Our current research primarily involves comparing spiking neural networks (SNNs) with traditional artificial neural networks (ANNs). This comparison aims to understand the advantages and applications of both approaches in the conetext of reinforcement learning for robotics.

We are currently preparing a manuscript detailing our findings for publication.