Who would you pick to win in a head-to-head competition — a state-of-the-art AI agent or a mouse?
Isaac Kauvar, a Wu Tsai Neurosciences Institute interdisciplinary postdoctoral scholar, and Chris Doyle, a machine learning researcher at Stanford, decided to pit them against each other to find out.
Working in the lab of Nick Haber, an assistant professor at the Stanford Graduate School of Education, Kauvar and Doyle designed a simple task based on their longtime interest in a skill set that animals naturally excel at exploring and adapting to their surroundings.
Kauvar put a mouse in a small empty box and similarly put a simulated AI agent in an empty 3D virtual arena.
Then, he placed a red ball in both environments. Kauvar measured to see which would be the quicker to explore the new object.
The test showed that the mouse quickly approached the ball and repeatedly interacted with it for several minutes.
But the AI agent didn’t seem to notice it. “That wasn’t expected,” said Kauvar. “Already, we realized that even with a state-of-the-art algorithm, there were performance gaps.”
The scholars pondered: Could they use such seemingly simple animal behaviors as inspiration to improve AI systems?
That question catalyzed Kauvar, Doyle, graduate students Linqi Zhou, and Haber to design a new training method called curious replay.
It programs AI agents to self-reflect about the most novel and exciting things they recently encountered. Adding curious replay was all needed for the AI agent to approach and engage with the red ball much faster.
Plus, it dramatically improved performance on a game based on Minecraft called Crafter. The results of this project, currently published on the preprint service arXiv, will be presented at the International Conference on Machine Learning on July 25.
Learning Through Curiosity
It may seem like curiosity offers only intellectual benefits, but it’s crucial to our survival, both in…