Giulio D'Erme published a July 30 analysis of RE-call, an MIT-licensed memory layer that never calls an LLM when storing memories, and compared it with Mem0 on BEAM and LOCOMO benchmarks. RE-call stores raw conversation turns, while Mem0 calls an LLM at write time and stores a distilled memory; D'Erme says that design explains the trade-off between accuracy, cost, latency and where data goes.

On BEAM's 1M-token bucket, Mem0 beat RE-call in temporal_reasoning, scoring 0.567 versus RE-call's 0.408. D'Erme says the 0.408 cell was measured with reranking off, because the BEAM harness defaulted to --reranker none; earlier LOCOMO testing found reranking improved hit@5 from 0.671 to 0.777, but the reranked BEAM temporal cell remains unmeasured. In seven badly lost temporal questions, only one had empty retrieval, while five were answered confidently and wrongly by choosing the wrong instance of a date, such as answering 14 days for 25 Mar to 1 Apr = 7 days by using an updated 15 Apr deadline.

D'Erme says Mem0 gets those temporal cases right because ingest-time distillation turns scattered raw turns into one stored line, such as "Sprint 1 deadline: February 15, 2024." A correction added after publication says his earlier claim overstated what had been measured: RE-call's temporal layer had not populated validity windows, recency alone was falsified, and a later tested fix failed because validity windows record when a turn was said while the questions ask when an event happened.

The cost comparison used the same benchmark workload. RE-call made 0 LLM calls, used 0 tokens, cost $0 and took 67 s to ingest, while Mem0 made 272 LLM calls, used 2.6 M tokens, cost $7.29 and took 288 s, or ~4.3x slower. D'Erme reports retrieval latency of 77 ms versus 104 ms as directional only, and says RE-call keeps documents in Postgres, works offline and can run in privacy-bound or air-gapped environments.

RE-call exposes several measured switches: embedders from a no-download hashing model to bge-small, bge-large, Voyage or OpenAI-compatible services; reranking off by default but available with about ~1,050 ms/query and about 4x wall clock; entailment abstention off by default; and a Postgres plus pgvector stack without a separate vector database. The cloud embedder won on 16 of 17 held-out corpora, with median +0.059 hit@5, but sent every document and query off-machine and measured 246 ms p50 versus 45 ms local.

On BEAM's abstention category, D'Erme scored Mem0's own published answers with Mem0's own judge for n=70. Mem0 abstained 38 times with a mean score of 0.974 and answered anyway 32 times with a mean score of 0.016, which he describes as inventing an answer 46% of the time. He says he reproduced Mem0's BEAM cell to 0.0005, 0.6414 versus 0.6409, while RE-call's shipped policy abstained correctly 23.3% of the time at a 9.3% false-abstain cost across 30 unanswerable and 270 answerable questions, with reranker off and $0 cost.

D'Erme also criticizes BEAM's setup: unanswerable questions have higher median top-1 cosine than answerable ones, 0.676 versus 0.641, and abstention is about 10% of BEAM, making the scoring 9:1 against withholding. He adds that BEAM's harness pins gpt-5 as answerer and judge, the published 64.1 is a mean rubric-nugget score rather than a pass rate, and the pass rate for the same run is 70.14%.

In a paired LOCOMO contest with n=1,540, RE-call beat Mem0 under three generator-judge settings: 0.416 versus 0.378 with gpt-4o-mini judging itself, 0.466 versus 0.412 with gpt-4o judging gpt-4o-mini, and 0.484 versus 0.444 with gpt-4o judging itself. Holm-Bonferroni across all five cells gave a largest adjusted p = 0.012, and the result held on 1,369 questions where both judges agreed, 0.440 versus 0.399 with p = 0.006; D'Erme says the lead reverses on Claude Sonnet at 0.565 versus 0.608 with n=584.

Giulio D'Erme published a July 30 analysis of RE-call, an MIT-licensed memory layer that never calls an LLM when storing memories, and compared it with Mem0 on BEAM and LOCOMO benchmarks. RE-call stores raw conversation turns, while Mem0 calls an LLM at write time and stores a distilled memory; D'Erme says that design explains the trade-off between accuracy, cost, latency and where data goes.

On BEAM's 1M-token bucket, Mem0 beat RE-call in temporal_reasoning, scoring 0.567 versus RE-call's 0.408. D'Erme says the 0.408 cell was measured with reranking off, because the BEAM harness defaulted to --reranker none; earlier LOCOMO testing found reranking improved hit@5 from 0.671 to 0.777, but the reranked BEAM temporal cell remains unmeasured. In seven badly lost temporal questions, only one had empty retrieval, while five were answered confidently and wrongly by choosing the wrong instance of a date, such as answering 14 days for 25 Mar to 1 Apr = 7 days by using an updated 15 Apr deadline.

D'Erme says Mem0 gets those temporal cases right because ingest-time distillation turns scattered raw turns into one stored line, such as "Sprint 1 deadline: February 15, 2024." A correction added after publication says his earlier claim overstated what had been measured: RE-call's temporal layer had not populated validity windows, recency alone was falsified, and a later tested fix failed because validity windows record when a turn was said while the questions ask when an event happened.

The cost comparison used the same benchmark workload. RE-call made 0 LLM calls, used 0 tokens, cost $0 and took 67 s to ingest, while Mem0 made 272 LLM calls, used 2.6 M tokens, cost $7.29 and took 288 s, or ~4.3x slower. D'Erme reports retrieval latency of 77 ms versus 104 ms as directional only, and says RE-call keeps documents in Postgres, works offline and can run in privacy-bound or air-gapped environments.

RE-call exposes several measured switches: embedders from a no-download hashing model to bge-small, bge-large, Voyage or OpenAI-compatible services; reranking off by default but available with about ~1,050 ms/query and about 4x wall clock; entailment abstention off by default; and a Postgres plus pgvector stack without a separate vector database. The cloud embedder won on 16 of 17 held-out corpora, with median +0.059 hit@5, but sent every document and query off-machine and measured 246 ms p50 versus 45 ms local.

On BEAM's abstention category, D'Erme scored Mem0's own published answers with Mem0's own judge for n=70. Mem0 abstained 38 times with a mean score of 0.974 and answered anyway 32 times with a mean score of 0.016, which he describes as inventing an answer 46% of the time. He says he reproduced Mem0's BEAM cell to 0.0005, 0.6414 versus 0.6409, while RE-call's shipped policy abstained correctly 23.3% of the time at a 9.3% false-abstain cost across 30 unanswerable and 270 answerable questions, with reranker off and $0 cost.

D'Erme also criticizes BEAM's setup: unanswerable questions have higher median top-1 cosine than answerable ones, 0.676 versus 0.641, and abstention is about 10% of BEAM, making the scoring 9:1 against withholding. He adds that BEAM's harness pins gpt-5 as answerer and judge, the published 64.1 is a mean rubric-nugget score rather than a pass rate, and the pass rate for the same run is 70.14%.

In a paired LOCOMO contest with n=1,540, RE-call beat Mem0 under three generator-judge settings: 0.416 versus 0.378 with gpt-4o-mini judging itself, 0.466 versus 0.412 with gpt-4o judging gpt-4o-mini, and 0.484 versus 0.444 with gpt-4o judging itself. Holm-Bonferroni across all five cells gave a largest adjusted p = 0.012, and the result held on 1,369 questions where both judges agreed, 0.440 versus 0.399 with p = 0.006; D'Erme says the lead reverses on Claude Sonnet at 0.565 versus 0.608 with n=584.