Train an SO-101 Robot From Sim-to-Real With NVIDIA Isaac

Workshop 1

Getting Started with Jetson Nano - SO-101 Robot

Getting Started:

• What is the Jetson Nano?
• Features and Applications
• Setting up the Jetson Nano

Machine Vision:

• Image Classification
• Object Detection

Robotics Motion

Basic Motion with the Jetson Nano

Basic Motion:

• Mobile Robotics
• Jetbot Platform Assembly
• Basic Motion Jupyter Notebook
• Moving Forward to place, Backward to pick, Left, and Right
• Behavior-Based Programming & Pseudocode
• Python Programming
• Basic Motion Challenges
• Basic Motion Interactive Control

Workshop 2

Teleoperation with Jetson Nano

Teleoperation:

• What is Teleoperation?
• What is the Internet of Things (IoT)?
• Jetson Nano and IoT
• Teleoperation Configuration
• Teleoperation Jupyter Notebook
• Teleoperation Practice Challenges
• Configuring Remote Camera Feed

Cybersecurity:

• Cybersecurity Threats in Robotics
• Understanding AIoT
• Defense in Depth

GPIO Programming

GPIO (LED & Bumper) on Jetson Nano

GPIO (General Purpose Input & Output):

• Introduction to GPIO on Jetson Nano
• GPIO Software Setup
• Introduction to LEDs
• Controlling LEDs
• Introduction to Bumper Switches
• Bumper Switches
• Python Programming
• GPIO Practice Challenges

Workshop 3

Collision Avoidance with Jetson Nano

Collision Avoidance:

• What is Collision Avoidance?
• Collision Avoidance on Jetbot
• Virtual Safety Bubble
• Collision Avoidance Jupyter Notebook
• Obstacle Data Collection

Training the Road Follower Model:

• Path Location Follower: RESnet-18
• Object Following Video Feed (TensorRT)
• Object Following Model Optimization
• Troubleshooting Object Following
• Object Following Practice Challenges

AI Navigation

Pick and Play Following with Jetson Nano

Pick and Play Following:

• What is Pick and Play Following?
• Pick and Play Following on Jetbot
• Path Following Jupyter Notebook
• Path Data Collection

Training the Neural Network

• Convolutional Neural Networks (CNNs): RESnet-18
• Training the RESnet-18 model
• Model Optimization for Jetson Nano
• Running Pick and Play Following
• Troubleshooting Pick and Play Following
• Pick and Play Following Practice Challenges

Workshop 4

Road Following + Collision Avoidance with Jetson Nano

Pick and Play Following + Collision Avoidance:

• What is Pick and Play Following with Collision Avoidance?
• Pick and Play Following with Collision Avoidance Jupyter Notebook
• Pick and Play Following with Collision Avoidance Practice Challenges

Autonomous Racing

Autonomous Racing with Jetson Nano

• Introduction to Reinforcement Learning
• Components of Reinforcement Learning
• Training the Reinforcement Learning Agent
• Real-World Applications
• Reinforcement Learning Game
• Reinforcement Learning with the Jetbot
• Autonomous Racing with Reinforcement Learning

Workshop 5

Autonomous Racing with Jetson Nano

• Introduction to Reinforcement Learning
• Components of Reinforcement Learning
• Training the Reinforcement Learning Agent
• Real-World Applications
• Reinforcement Learning Game
• Reinforcement Learning with the Jetbot
• Autonomous Racing with Reinforcement Learning