Anna Villarreal, a developer, conducted a benchmark study to evaluate the performance of various LLMs on an NVIDIA Jetson Nano, a compact single-board computer, using the Ollama local deployment tool. The primary objective was to find a model capable of generating flashcards and quizzes from text without crashing the hardware. Due to the limited memory of the Jetson Nano, a swap file was implemented to prevent system failure during testing. The accuracy of each model was measured against a baseline quiz derived from the OSI model consisting of 10 specific questions.

The testing results indicate that qwen2.5:3b-instruct achieved a 100% accuracy rate across q4_K_M, q5_K_M, and q8_0 quantization levels. In contrast, llama3.2:3b-instruct performed at 40% accuracy with q2_K and 90% accuracy with q4_K_M, q5_K_M, and q8_0. The mistral:7b-instruct model reached 100% accuracy with q4_K_M, 80% with q2_K, and 80% with q5_K_M. The qwen3.5:2b model failed to produce output, resulting in 0% accuracy, as it did not fit within the GPU constraints of the device. Certain test configurations were omitted from the final dataset because they exceeded the hardware resources of the Jetson Nano.

The author notes that while quantization—the process of reducing the precision of model weights to save memory—impacts output quality, the specific use case of quiz generation heavily influences performance outcomes. Based on these findings, the author concluded that qwen2.5:3b-instruct is the most effective choice for their local quiz generation application. The findings emphasize the challenges of running generative models on resource-constrained edge hardware, where balancing model size and precision is necessary to maintain operational stability. Future work involves transitioning the local application from using Llama models to the Qwen family based on these specific performance metrics.

Anna Villarreal, a developer, conducted a benchmark study to evaluate the performance of various LLMs on an NVIDIA Jetson Nano, a compact single-board computer, using the Ollama local deployment tool. The primary objective was to find a model capable of generating flashcards and quizzes from text without crashing the hardware. Due to the limited memory of the Jetson Nano, a swap file was implemented to prevent system failure during testing. The accuracy of each model was measured against a baseline quiz derived from the OSI model consisting of 10 specific questions.

The testing results indicate that qwen2.5:3b-instruct achieved a 100% accuracy rate across q4_K_M, q5_K_M, and q8_0 quantization levels. In contrast, llama3.2:3b-instruct performed at 40% accuracy with q2_K and 90% accuracy with q4_K_M, q5_K_M, and q8_0. The mistral:7b-instruct model reached 100% accuracy with q4_K_M, 80% with q2_K, and 80% with q5_K_M. The qwen3.5:2b model failed to produce output, resulting in 0% accuracy, as it did not fit within the GPU constraints of the device. Certain test configurations were omitted from the final dataset because they exceeded the hardware resources of the Jetson Nano.

The author notes that while quantization—the process of reducing the precision of model weights to save memory—impacts output quality, the specific use case of quiz generation heavily influences performance outcomes. Based on these findings, the author concluded that qwen2.5:3b-instruct is the most effective choice for their local quiz generation application. The findings emphasize the challenges of running generative models on resource-constrained edge hardware, where balancing model size and precision is necessary to maintain operational stability. Future work involves transitioning the local application from using Llama models to the Qwen family based on these specific performance metrics.