Developers increasingly favor local AI tools to avoid cloud API billing and data privacy risks associated with sending proprietary codebases to external servers. The zerikai_memory project addresses these needs by routing queries between cloud and local modes, with local processing handled by Ollama. In a recent analysis conducted on hardware with an Intel i7-12700 CPU, 32GB of RAM, and an NVIDIA RTX 3050 GPU (8GB VRAM), two local models were benchmarked: mistral:7b (v0.3) and ornith:9b-dense. The study focused on both inference latency and synthesis quality when interacting with a local ChromaDB workspace index.

Latency benchmarks revealed that mistral:7b, which fits within 8GB of VRAM, achieved a mean response time of 6.14s compared to 13.39s for ornith:9b. The latter experienced higher latency due to shared memory overflow during cold starts. However, in live queries, ornith:9b demonstrated superior reliability. When asked about specific technical concurrency implementations not present in the codebase index, ornith:9b correctly identified the information gap, whereas mistral:7b provided a confident but incorrect answer by conflating background scan logic with query-time operations.

The research highlights that model performance is heavily dependent on the quality of retrieved context. Successful indexing requires a precise workflow: updating the .memignore file, running the embedding-docstring tool to enrich documentation, performing a workspace scan, and finally querying. Skipping docstring enrichment results in sparse vector data, which degrades the performance of any model. To prevent GPU saturation during local brief generation, the project implemented a global asyncio semaphore (a concurrency control mechanism) to gate concurrent Ollama calls. This update ensures that local mode remains stable on limited VRAM configurations by limiting parallel processing during intensive tasks like background brief synthesis.

Developers increasingly favor local AI tools to avoid cloud API billing and data privacy risks associated with sending proprietary codebases to external servers. The zerikai_memory project addresses these needs by routing queries between cloud and local modes, with local processing handled by Ollama. In a recent analysis conducted on hardware with an Intel i7-12700 CPU, 32GB of RAM, and an NVIDIA RTX 3050 GPU (8GB VRAM), two local models were benchmarked: mistral:7b (v0.3) and ornith:9b-dense. The study focused on both inference latency and synthesis quality when interacting with a local ChromaDB workspace index.

Latency benchmarks revealed that mistral:7b, which fits within 8GB of VRAM, achieved a mean response time of 6.14s compared to 13.39s for ornith:9b. The latter experienced higher latency due to shared memory overflow during cold starts. However, in live queries, ornith:9b demonstrated superior reliability. When asked about specific technical concurrency implementations not present in the codebase index, ornith:9b correctly identified the information gap, whereas mistral:7b provided a confident but incorrect answer by conflating background scan logic with query-time operations.

The research highlights that model performance is heavily dependent on the quality of retrieved context. Successful indexing requires a precise workflow: updating the .memignore file, running the embedding-docstring tool to enrich documentation, performing a workspace scan, and finally querying. Skipping docstring enrichment results in sparse vector data, which degrades the performance of any model. To prevent GPU saturation during local brief generation, the project implemented a global asyncio semaphore (a concurrency control mechanism) to gate concurrent Ollama calls. This update ensures that local mode remains stable on limited VRAM configurations by limiting parallel processing during intensive tasks like background brief synthesis.