Researchers Propose MAEMs for Model Interpretability

Researchers submitted a paper to arXiv on November 5, 2026, proposing Maximally-Activating Example Metamodels (MAEMs), language models that turn a direction in a subject model’s residual stream into text examples that strongly activate it. The researchers train MAEMs with a label-free reinforcement-learning objective: maximize the cosine similarity between the supplied direction and activations from generated text. The approach aims to find examples beyond the contents of a scanned text corpus.

In experiments, a Qwen3.6-27B MAEM inverted held-out activations better than corpus search and a natural-language autoencoder (NLA). Against corpus search using 10M tokens, it performed better across a majority of held-out languages and domains, and it activated 89% of held-out sparse autoencoder (SAE) features.

The MAEM identified concepts behind steering vectors about as well as NLAs and recovered the payloads of LoRA backdoors more often than corpus search. It also verbalized some model workspace inner monologue. The researchers say MAEM examples can be checked by running them through the subject model in a forward pass, making them a grounded alternative for scaling metamodel training. The paper is listed under Machine Learning and Artificial Intelligence on arXiv; its authors are Celeste De Schamphelaere, Juan J. Vazquez, Ari Spiesberger, Tomáš Gavenčiak, Gavin Leech, Euan Ong and Jiahai Feng.

The paper's central method uses generated text as an interpretable example of a model activation: the generated text is tested against the original model to see whether it activates the supplied residual-stream direction. The reported findings cover held-out activations, languages, domains and SAE features, as well as steering vectors, backdoor payloads and model workspace text.

Researchers submitted a paper to arXiv on November 5, 2026, proposing Maximally-Activating Example Metamodels (MAEMs), language models that turn a direction in a subject model’s residual stream into text examples that strongly activate it. The researchers train MAEMs with a label-free reinforcement-learning objective: maximize the cosine similarity between the supplied direction and activations from generated text. The approach aims to find examples beyond the contents of a scanned text corpus.

In experiments, a Qwen3.6-27B MAEM inverted held-out activations better than corpus search and a natural-language autoencoder (NLA). Against corpus search using 10M tokens, it performed better across a majority of held-out languages and domains, and it activated 89% of held-out sparse autoencoder (SAE) features.

The MAEM identified concepts behind steering vectors about as well as NLAs and recovered the payloads of LoRA backdoors more often than corpus search. It also verbalized some model workspace inner monologue. The researchers say MAEM examples can be checked by running them through the subject model in a forward pass, making them a grounded alternative for scaling metamodel training. The paper is listed under Machine Learning and Artificial Intelligence on arXiv; its authors are Celeste De Schamphelaere, Juan J. Vazquez, Ari Spiesberger, Tomáš Gavenčiak, Gavin Leech, Euan Ong and Jiahai Feng.

The paper's central method uses generated text as an interpretable example of a model activation: the generated text is tested against the original model to see whether it activates the supplied residual-stream direction. The reported findings cover held-out activations, languages, domains and SAE features, as well as steering vectors, backdoor payloads and model workspace text.