KAIST Researchers Introduce ReTrace Recovery Framework

KAIST AI researchers introduced ReTrace, a framework for evaluating how computer-use agents recover from errors, in a paper published on October 20. The study examines agents that use keyboard and mouse actions to automate digital workflows, and asks whether they can recover when actions go wrong during long tasks. The authors say errors can accumulate and actions can leave unintended side effects, making recovery important for reliability over extended workflows.

Existing approaches have limitations, the researchers say: resetting an environment is infeasible in proprietary settings, replaying actions is too costly over long tasks, and API-based rollback is limited to web environments. ReTrace formalizes “transition-consistent recovery”: after recovery, repeating the same downstream actions from the recovered state and the target state must produce identical outcomes. The framework tests this through goal-oriented action replay after controlled error injection.

Using ReTrace, the researchers conducted what they describe as the first systematic study of autonomous, model-based recovery. They report that ideal recovery can reveal models’ latent task-completion capabilities, current agents fail to use their recovery capabilities effectively, and accurate recovery produces measurable end-to-end gains. The paper presents recovery as a fundamental capability for reliable computer-use agents. The authors made the code and dataset available on GitHub.

KAIST AI researchers introduced ReTrace, a framework for evaluating how computer-use agents recover from errors, in a paper published on October 20. The study examines agents that use keyboard and mouse actions to automate digital workflows, and asks whether they can recover when actions go wrong during long tasks. The authors say errors can accumulate and actions can leave unintended side effects, making recovery important for reliability over extended workflows.

Existing approaches have limitations, the researchers say: resetting an environment is infeasible in proprietary settings, replaying actions is too costly over long tasks, and API-based rollback is limited to web environments. ReTrace formalizes “transition-consistent recovery”: after recovery, repeating the same downstream actions from the recovered state and the target state must produce identical outcomes. The framework tests this through goal-oriented action replay after controlled error injection.

Using ReTrace, the researchers conducted what they describe as the first systematic study of autonomous, model-based recovery. They report that ideal recovery can reveal models’ latent task-completion capabilities, current agents fail to use their recovery capabilities effectively, and accurate recovery produces measurable end-to-end gains. The paper presents recovery as a fundamental capability for reliable computer-use agents. The authors made the code and dataset available on GitHub.