It turns out I wasn't saving PyTorch models correctly. You really need to save the optimizer state as well as the state of the current weights of your deep neural network. This is critical to getting deep Q learning and actor critic agents training on complex environments that may require multiple sessions spread over time. Starter code for this video is here: https://gist.github.com/philtabor/c26386e9b875b57313529338c905ecd5 Learn how to turn deep reinforcement learning papers into code: Get instant access to all my courses, including the new Hindsight Experience Replay course, with my subscription service. $24.99 a month gives you instant access to 36 hours of instructional content plus access to future updates, added weekly. Discounts available for Udemy students (enrolled longer than 30 days). Just send an email to sales@neuralnet.ai https://www.neuralnet.ai/courses Or, pickup my Udemy courses here: Deep Q Learning: https://www.udemy.com/course/deep-q-learning-from-paper-to-code/?couponCode=DQN-JUNE-22 Actor Critic Methods: https://www.udemy.com/course/actor-critic-methods-from-paper-to-code-with-pytorch/?couponCode=AC-JUNE-22 Curiosity Driven Deep Reinforcement Learning https://www.udemy.com/course/curiosity-driven-deep-reinforcement-learning/?couponCode=ICM-JUNE-22 Natural Language Processing from First Principles: https://www.udemy.com/course/natural-language-processing-from-first-principles/?couponCode=NLP1-JUNE-22 Just getting started in deep reinforcement learning? Check out my intro level course through Manning Publications. Reinforcement Learning Fundamentals https://www.manning.com/livevideo/reinforcement-learning-in-motion Here are some books / courses I recommend (affiliate links): Grokking Deep Learning in Motion: https://bit.ly/3fXHy8W Grokking Deep Learning: https://bit.ly/3yJ14gT Grokking Deep Reinforcement Learning: https://bit.ly/2VNAXql Come hang out on Discord here: https://discord.gg/Zr4VCdv Website: https://www.neuralnet.ai Github: https://github.com/philtabor Twitter: https://twitter.com/MLWithPhil