LLM Agents Struggle With Backend Code Generation Constraints
- •LLM agents exhibit significant constraint decay when handling complex, multi-file backend code generation tasks.
- •Assertion pass rates drop by 30 points as structural requirements increase across 100 total generation tasks.
- •Data-layer errors, specifically ORM violations and incorrect queries, are the leading causes of agent failure.
Researchers Francesco Dente, Dario Satriani, and Paolo Papotti released a study on May 7, 2026, documenting the 'constraint decay' phenomenon in LLM agents tasked with backend code generation. While agents perform well on simple tasks, their effectiveness drops when forced to adhere to strict structural constraints like architectural patterns and database mappings. The authors evaluated agent performance across 80 greenfield generation tasks and 20 feature-implementation tasks involving eight different web frameworks.
The study shows that agent assertion pass rates decline by 30 points on average when moving from baseline specifications to fully specified structural requirements, with some configurations falling near zero. Performance varies significantly by environment: agents struggle in convention-heavy frameworks like Django or FastAPI compared to minimal, explicit ones like Flask.
Detailed error analysis identified data-layer defects, such as incorrect query composition and ORM (object-relational mapping, a technique connecting code to databases) runtime violations, as the primary reasons for failure. These findings suggest that balancing functional requirements with rigid structural rules remains a major hurdle for current autonomous coding agents.