Kiarash Rezaei and colleagues introduced HuGLEN, a new evaluation pipeline for assessing large language models (LLMs) used in optical network automation. The system employs an LLM-as-a-judge (an AI model evaluating another model's output) combined with expert human ratings to create scalable and reproducible performance rankings. Researchers measured candidate models using a quality efficiency score (QES) to balance performance against inference costs.

Testing the pipeline on explainable artificial intelligence models for optical network quality of transmission estimation, researchers identified a medium-sized model with 12B parameters as the top performer. This specific model achieved the highest QES, demonstrating an optimal balance between output quality and operational efficiency. The framework successfully reduces the manual labeling workload for network operators.

Kiarash Rezaei and colleagues introduced HuGLEN, a new evaluation pipeline for assessing large language models (LLMs) used in optical network automation. The system employs an LLM-as-a-judge (an AI model evaluating another model's output) combined with expert human ratings to create scalable and reproducible performance rankings. Researchers measured candidate models using a quality efficiency score (QES) to balance performance against inference costs.

Testing the pipeline on explainable artificial intelligence models for optical network quality of transmission estimation, researchers identified a medium-sized model with 12B parameters as the top performer. This specific model achieved the highest QES, demonstrating an optimal balance between output quality and operational efficiency. The framework successfully reduces the manual labeling workload for network operators.