PyRUA-DataFarm Generates Robot Training Data

A research team introduced PyRUA-DataFarm, a system that uses a coding agent to generate robot training demonstrations in simulation, aiming to reduce the cost of collecting data through teleoperation. The agent takes a data request and one calibration episode from a robot in its target scene, then builds the simulated scene, an expert controller (software that carries out the task) and a success judge. A simulator provides actions, while a video model uses real images to render each simulated rollout as photographic video across synchronized camera views. The resulting data pairs those views with robot actions so a policy can be trained and deployed on a real robot. Independent judges check the controller and each rendered episode before accepting it.

On five real-robot tasks, policies trained only on DataFarm data achieved 93 successes in 100 trials, compared with 84 for policies trained on the same number of teleoperated episodes. With an unseen tablecloth and coloured lighting, the DataFarm-trained policies recorded 87 successes, while teleoperation-trained policies recorded 46. The paper says adapting a task is fully automatic and averages $21 in API usage per task, paid once, plus $5 for a render contract. Each additional episode costs an average of $0.80 for simulation, rendering and review, without using a robot or operator.

The researchers estimate that producing one hour of three-view data for a task costs an average of $107, including those expenses. They compare this with a $117 teleoperation quote in China and $290 in labour alone in North America. The paper was published on November 4 and submitted to Hugging Face Papers by DAGroup-PKU; its authors describe DataFarm as a way to create deployable robot training data at lower cost than teleoperation.

A research team introduced PyRUA-DataFarm, a system that uses a coding agent to generate robot training demonstrations in simulation, aiming to reduce the cost of collecting data through teleoperation. The agent takes a data request and one calibration episode from a robot in its target scene, then builds the simulated scene, an expert controller (software that carries out the task) and a success judge. A simulator provides actions, while a video model uses real images to render each simulated rollout as photographic video across synchronized camera views. The resulting data pairs those views with robot actions so a policy can be trained and deployed on a real robot. Independent judges check the controller and each rendered episode before accepting it.

On five real-robot tasks, policies trained only on DataFarm data achieved 93 successes in 100 trials, compared with 84 for policies trained on the same number of teleoperated episodes. With an unseen tablecloth and coloured lighting, the DataFarm-trained policies recorded 87 successes, while teleoperation-trained policies recorded 46. The paper says adapting a task is fully automatic and averages $21 in API usage per task, paid once, plus $5 for a render contract. Each additional episode costs an average of $0.80 for simulation, rendering and review, without using a robot or operator.

The researchers estimate that producing one hour of three-view data for a task costs an average of $107, including those expenses. They compare this with a $117 teleoperation quote in China and $290 in labour alone in North America. The paper was published on November 4 and submitted to Hugging Face Papers by DAGroup-PKU; its authors describe DataFarm as a way to create deployable robot training data at lower cost than teleoperation.