Review Surveys Large Language Model Challenges

Researchers P. Treleaven, Edward Sharkey and Sara Craciun published a review of large language model (LLM) developments in Royal Society Open Science in 2026, examining models’ computational strengths and weaknesses as research expands and developers seek to match LLMs to applications. The review builds on Sharkey and Treleaven’s 2025 work, “Optimising large language models: taxonomy and techniques.”

The authors identify four challenges. Training and deploying large models can be costly and slow, with energy consumption also a concern. Models can struggle with very long-range dependencies and context longer than their processing window, such as keeping a long document coherent; the review connects this limit to the lack of continual learning.

The review says it remains difficult to interpret why a model gives a particular output, and that models may amplify biases in their training data. It also describes challenges in fine-tuning for specific tasks without overfitting or losing generality, and in making inference efficient enough for real-time applications. Inference latency means the delay before a response is completed.

The authors describe the review as covering architectural, optimization and interpretability developments. It does not provide a systematic benchmark comparison, exhaustive implementation details or empirical performance evaluation, topics the authors say are covered extensively elsewhere. They say they intend to update the review as the technology changes and invite readers to suggest additions.

Researchers P. Treleaven, Edward Sharkey and Sara Craciun published a review of large language model (LLM) developments in Royal Society Open Science in 2026, examining models’ computational strengths and weaknesses as research expands and developers seek to match LLMs to applications. The review builds on Sharkey and Treleaven’s 2025 work, “Optimising large language models: taxonomy and techniques.”

The authors identify four challenges. Training and deploying large models can be costly and slow, with energy consumption also a concern. Models can struggle with very long-range dependencies and context longer than their processing window, such as keeping a long document coherent; the review connects this limit to the lack of continual learning.

The review says it remains difficult to interpret why a model gives a particular output, and that models may amplify biases in their training data. It also describes challenges in fine-tuning for specific tasks without overfitting or losing generality, and in making inference efficient enough for real-time applications. Inference latency means the delay before a response is completed.

The authors describe the review as covering architectural, optimization and interpretability developments. It does not provide a systematic benchmark comparison, exhaustive implementation details or empirical performance evaluation, topics the authors say are covered extensively elsewhere. They say they intend to update the review as the technology changes and invite readers to suggest additions.