Operational Impact of LLM Use in Software Development
- •Software teams using LLMs saw a 50% increase in defect rates per developer.
- •System throughput for AI-adopting teams decreased by an estimated 71-80% in development pipelines.
- •Faros.ai data shows that lead times for shipping features to production increased nearly 5x.
Operational data from Faros.ai, a software development telemetry firm, suggests that while Large Language Models (LLMs) improve individual developer productivity, they negatively impact overall organizational throughput and software quality. A March report analyzing 22,000 developers across 4,000 teams found that teams using AI in their development process experienced a 50% increase in defect rates per developer. System-level flow also suffered, with lead times for moving features to production increasing by nearly 5x, resulting in an estimated 71-80% decline in system throughput.
The analysis applied Little’s Law (a queuing theory formula relating the number of items in a system to arrival rate and wait time) to calculate the impact on software delivery. By comparing developer contexts and daily task volumes, the data indicates that AI adoption creates a bottleneck, significantly increasing work-in-progress (WIP) while slowing the actual delivery of finished features to production. High-performing engineering organizations showed no protection against these downstream deteriorations, suggesting the trend is systemic rather than specific to lower-maturity teams.
The report notes that deployment frequency fell by 11% among AI-using teams. The author argues that LLMs are tools, not intelligent agents, and that common workflows—such as using AI to generate first drafts—often shift the intellectual burden downstream, leading to higher costs for identifying and fixing defects. Instead of viewing AI as a replacement for foundational work, the author suggests that the structural understanding gained during manual drafting is essential for maintaining code quality. The article concludes that these technologies, due to their inherent unreliability, fail to provide a stable foundation for software development, with current usage patterns actively destroying enterprise value.