01394nas a2200193   4500000000100000008004100001260002100042100001900063700002400082700002400106700002000130700002000150700002300170700002000193700001800213245007500231856004600306520084800352       2023                            d  aMontreal, Canada1 aSamuel Kessler1 aMateusz Ostaszewski1 aMichał Bortkiewicz1 aMateusz Żarski1 aMaciej Wołczyk1 aJack Parker-Holder1 aStephen Roberts1 aPiotr Miłoś00aThe Effectiveness of World Models for Continual Reinforcement Learning  uhttps://lifelong-ml.cc/online_proceedings3 a<p>World models power some of the most efficient reinforcement learning algorithms. In this work, we showcase that they can be harnessed for continual learning – a situation when the agent faces changing environments. World models typically employ a replay buffer for training, which can be naturally extended to continual learning. We systematically study how different selective experience replay methods affect performance, forgetting, and transfer. We also provide recommendations regarding various modeling options for using world models. The best set of choices is called Continual-Dreamer, it is task-agnostic and utilizes the world model for continual exploration. Continual-Dreamer is sample efficient and outperforms state-of-the-art task-agnostic continual reinforcement learning methods on Minigrid and Minihack benchmarks.</p>