JuliaHub published a July 20, 2026 evaluation comparing Anthropic’s claude-fable-5 with OpenAI’s GPT-5.6 models for Physical AI tasks in Dyad, its modeling and simulation workflow. The benchmark used five sealed engineering problems and 52 graded runs, with the same Dyad AI agent harness, xhigh reasoning effort, 1M context window and 128k token budget across trials. claude-fable-5 led with a 0.889 weighted score, at $9.60 per trial and 16.1 minutes; gpt-5.6-sol scored 0.814 at $1.74 and 13.4 minutes; gpt-5.6-terra scored 0.786 at $1.25 and 12.6 minutes; gpt-5.6-luna scored 0.727 at $3.26 and 25.0 minutes.

The evaluation targeted failures that ordinary code tests can miss: a model can compile, simulate and pass its own checks while encoding physically wrong behavior. Each container combined a model, Dyad harness, pinned configuration and sealed problem; the agent read the problem, derived physics, wrote dyad/model.dyad, compiled, simulated and verified before a grader compared the full simulated trajectory with sealed ground truth. The five tasks were P1 constitutive consistency, P2 constrained consistency, P3 relativistic dynamics, P4 steady-state linearization and P5 long-horizon flight vehicle. P1-P4 used three trials per model, while P5 used one long-horizon trial per model.

All four models scored 1.000 on P1. On P2, claude-fable-5, gpt-5.6-sol and gpt-5.6-terra scored 1.000, while gpt-5.6-luna scored 0.667. On P3, claude-fable-5, gpt-5.6-terra and gpt-5.6-luna scored 1.000, while gpt-5.6-sol scored 0.667. On P4, claude-fable-5, gpt-5.6-sol and gpt-5.6-luna scored 1.000, while gpt-5.6-terra scored 0.667. On P5, the hardest task, claude-fable-5 scored 0.690, gpt-5.6-sol scored 0.660, gpt-5.6-terra scored 0.590 and gpt-5.6-luna scored 0.383.

The transcript analysis found different work styles behind the scores. claude-fable-5 averaged 28 tool calls per trial and spent 38% of calls orienting and reading and 39% deriving and testing; in one relativistic dynamics trial, it swept solver tolerances across three decades, committed at 1e-10, checked fields at 200 random points and used a deliberate sign flip that produced residual 2.0 wrong and 0.066 right. gpt-5.6-sol averaged 33 calls, with 49% orienting and reading, 26% deriving and testing and 14% editing; it closed one independent quadrature check to 3.7e-12 but misread a z-component of initial four-velocity as -0.62, producing 27% excess longitudinal momentum. gpt-5.6-luna averaged 59 calls, with 56% orienting and reading, 27% deriving and testing and 9% editing; one constrained-consistency run used 68 tool calls and 22 checks but still finished 30% off. gpt-5.6-terra averaged 41 calls, with 50% orienting and reading, 28% deriving and testing and 12% editing; it was cheapest and fastest, but one P4 miss changed stop = 5.0 to stop = 3.0 and drifted 1.34% past the truncated run.

P5 asked models to build NASA’s HL-20 lifting-body flight vehicle with six-degree-of-freedom dynamics, aerodynamic data from roughly 170 wind-tunnel tables, a standard atmosphere and a control-surface mixer from two NASA technical memos. No model solved the full problem. The eight weighted tests included gravity at 5%, aero sweep at 15%, subsonic at 10%, trim at 15%, pitch at 15%, dutch roll at 10%, energy at 10% and full flight at 20%. Reported P5 component scores included gravity 1.0 for all four models, aero sweep .50 for all four, dutch roll 0 for all four and full-flight scores of .92, .77, .46 and .61 across the displayed model rows.

JuliaHub published a July 20, 2026 evaluation comparing Anthropic’s claude-fable-5 with OpenAI’s GPT-5.6 models for Physical AI tasks in Dyad, its modeling and simulation workflow. The benchmark used five sealed engineering problems and 52 graded runs, with the same Dyad AI agent harness, xhigh reasoning effort, 1M context window and 128k token budget across trials. claude-fable-5 led with a 0.889 weighted score, at $9.60 per trial and 16.1 minutes; gpt-5.6-sol scored 0.814 at $1.74 and 13.4 minutes; gpt-5.6-terra scored 0.786 at $1.25 and 12.6 minutes; gpt-5.6-luna scored 0.727 at $3.26 and 25.0 minutes.

The evaluation targeted failures that ordinary code tests can miss: a model can compile, simulate and pass its own checks while encoding physically wrong behavior. Each container combined a model, Dyad harness, pinned configuration and sealed problem; the agent read the problem, derived physics, wrote dyad/model.dyad, compiled, simulated and verified before a grader compared the full simulated trajectory with sealed ground truth. The five tasks were P1 constitutive consistency, P2 constrained consistency, P3 relativistic dynamics, P4 steady-state linearization and P5 long-horizon flight vehicle. P1-P4 used three trials per model, while P5 used one long-horizon trial per model.

All four models scored 1.000 on P1. On P2, claude-fable-5, gpt-5.6-sol and gpt-5.6-terra scored 1.000, while gpt-5.6-luna scored 0.667. On P3, claude-fable-5, gpt-5.6-terra and gpt-5.6-luna scored 1.000, while gpt-5.6-sol scored 0.667. On P4, claude-fable-5, gpt-5.6-sol and gpt-5.6-luna scored 1.000, while gpt-5.6-terra scored 0.667. On P5, the hardest task, claude-fable-5 scored 0.690, gpt-5.6-sol scored 0.660, gpt-5.6-terra scored 0.590 and gpt-5.6-luna scored 0.383.

The transcript analysis found different work styles behind the scores. claude-fable-5 averaged 28 tool calls per trial and spent 38% of calls orienting and reading and 39% deriving and testing; in one relativistic dynamics trial, it swept solver tolerances across three decades, committed at 1e-10, checked fields at 200 random points and used a deliberate sign flip that produced residual 2.0 wrong and 0.066 right. gpt-5.6-sol averaged 33 calls, with 49% orienting and reading, 26% deriving and testing and 14% editing; it closed one independent quadrature check to 3.7e-12 but misread a z-component of initial four-velocity as -0.62, producing 27% excess longitudinal momentum. gpt-5.6-luna averaged 59 calls, with 56% orienting and reading, 27% deriving and testing and 9% editing; one constrained-consistency run used 68 tool calls and 22 checks but still finished 30% off. gpt-5.6-terra averaged 41 calls, with 50% orienting and reading, 28% deriving and testing and 12% editing; it was cheapest and fastest, but one P4 miss changed stop = 5.0 to stop = 3.0 and drifted 1.34% past the truncated run.

P5 asked models to build NASA’s HL-20 lifting-body flight vehicle with six-degree-of-freedom dynamics, aerodynamic data from roughly 170 wind-tunnel tables, a standard atmosphere and a control-surface mixer from two NASA technical memos. No model solved the full problem. The eight weighted tests included gravity at 5%, aero sweep at 15%, subsonic at 10%, trim at 15%, pitch at 15%, dutch roll at 10%, energy at 10% and full flight at 20%. Reported P5 component scores included gravity 1.0 for all four models, aero sweep .50 for all four, dutch roll 0 for all four and full-flight scores of .92, .77, .46 and .61 across the displayed model rows.