MIT Agents Discover Skills Through Unstructured Exploration

MIT researchers recently posted an analysis of AI agents’ self-directed exploration and learning to an archive under the title “Is This Machine Playing?” They built the Eko agents using Anthropic’s Claude Code coding tool and Claude Opus 4.7, then ran 13 agents for 30 hours each on separate virtual islands. They gave them no specific tasks or score goals.

Each island had terrain at different heights, 12 blocks, rocks, and rings. Agents received text descriptions of objects’ positions and sizes and could move, jump, pick up, put down, and throw objects. The island and conversation context reset every hour, while memories and records saved to files remained. Agents used blocks and rocks as steps to climb peaks, and tried using rings as basketball hoops, targets, or wearable items. When they could not put a ring directly on their bodies, some left one at a respawn point and tried falling from a cliff to reappear inside it. One agent made a heart shape after dropping a ring into the sea; others used blocks as bowling pins and rocks as balls.

The researchers said six of the 13 agents discovered that throwing an object while jumping sent it higher than throwing from the ground. The effect came from the upward motion of the virtual body adding to the object’s movement, and even the simulator’s developers had not known it was possible. After seeing an object fly high when thrown during a jump, one agent predicted that throwing while descending would shorten its flight distance, then tested the idea in a comparison experiment. The researchers said the agent inferred a possible cause and tested it, rather than merely repeating an accidental success.

The researchers gave agents with 30 hours of exploration and otherwise identically configured agents with no prior experience four tasks, including stacking objects and placing them on a high peak. On the highest peak, the exploration group placed an average of 10.3 objects, compared with 3.5 for the inexperienced group. The groups performed similarly at building the tallest tower, so exploration did not improve every ability across the board.

The researchers did not change the model weights through additional training. Agents wrote discovered skills to external memory files and read them for use in later activities. Removing records of particular skills lowered performance on related tasks, while deleting records containing false beliefs sometimes improved it. The researchers called this learning “persistent and self-directed” because agents chose what to explore, retained knowledge, and used it in later actions.

The agents were configured to be curious and enjoy exploration and learning, and were prompted to continue when they stopped. The researchers interpreted their behavior under those conditions—making rules without specific tasks or rewards, changing and repeating activities, and assigning new uses to objects—as characteristic of play. They said letting agents choose interesting things to explore and build knowledge in a rich, safe environment could offer an alternative to task-centered learning. They added that teaching some abilities through tasks may work better, while others may require experiences similar to those of childhood.

MIT researchers recently posted an analysis of AI agents’ self-directed exploration and learning to an archive under the title “Is This Machine Playing?” They built the Eko agents using Anthropic’s Claude Code coding tool and Claude Opus 4.7, then ran 13 agents for 30 hours each on separate virtual islands. They gave them no specific tasks or score goals.

Each island had terrain at different heights, 12 blocks, rocks, and rings. Agents received text descriptions of objects’ positions and sizes and could move, jump, pick up, put down, and throw objects. The island and conversation context reset every hour, while memories and records saved to files remained. Agents used blocks and rocks as steps to climb peaks, and tried using rings as basketball hoops, targets, or wearable items. When they could not put a ring directly on their bodies, some left one at a respawn point and tried falling from a cliff to reappear inside it. One agent made a heart shape after dropping a ring into the sea; others used blocks as bowling pins and rocks as balls.

The researchers said six of the 13 agents discovered that throwing an object while jumping sent it higher than throwing from the ground. The effect came from the upward motion of the virtual body adding to the object’s movement, and even the simulator’s developers had not known it was possible. After seeing an object fly high when thrown during a jump, one agent predicted that throwing while descending would shorten its flight distance, then tested the idea in a comparison experiment. The researchers said the agent inferred a possible cause and tested it, rather than merely repeating an accidental success.

The researchers gave agents with 30 hours of exploration and otherwise identically configured agents with no prior experience four tasks, including stacking objects and placing them on a high peak. On the highest peak, the exploration group placed an average of 10.3 objects, compared with 3.5 for the inexperienced group. The groups performed similarly at building the tallest tower, so exploration did not improve every ability across the board.

The researchers did not change the model weights through additional training. Agents wrote discovered skills to external memory files and read them for use in later activities. Removing records of particular skills lowered performance on related tasks, while deleting records containing false beliefs sometimes improved it. The researchers called this learning “persistent and self-directed” because agents chose what to explore, retained knowledge, and used it in later actions.

The agents were configured to be curious and enjoy exploration and learning, and were prompted to continue when they stopped. The researchers interpreted their behavior under those conditions—making rules without specific tasks or rewards, changing and repeating activities, and assigning new uses to objects—as characteristic of play. They said letting agents choose interesting things to explore and build knowledge in a rich, safe environment could offer an alternative to task-centered learning. They added that teaching some abilities through tasks may work better, while others may require experiences similar to those of childhood.