Researchers from the AI Safety Initiative will be presenting their novel work at the International Conference on Learning Representations and the IEEE/CVF Conference on Computer Vision and Pattern Recognition in 2025. Their papers involve pioneering approaches to understanding and controlling large language models and diffusion models through advanced interpretability techniques.
The team’s work on contrastive activation engineering (CAE), led by Yixiong Hao, explores a new paradigm for steering LLM outputs through targeted modifications to internal representations. Unlike traditional fine-tuning methods that require significant computational resources, CAE can be applied at inference time with minimal overhead.
“We’ve made significant progress in understanding the capabilities and limitations of CAE techniques,” notes Hao. “Our research reveals that while CAE is effective for in-distribution contexts, it has clear boundaries that practitioners need to be aware of.”
The team’s findings include important insights about the practical implementation of CAE, such as the optimal number of samples needed for generating effective steering vectors and the susceptibility of these vectors to adversarial inputs. They also discovered that while steering can impact model perplexity, larger models demonstrate greater resilience to steering-induced degradation.
In a parallel research track, Stepan Shabalin’s work on scaling sparse autoencoder circuits alongside researchers at Google Deepmind provides groundbreaking insights into in-context learning mechanisms. By adapting sparse feature circuits methodology to work with the much larger Gemma-1 2B model, Shabalin’s team has identified specific features that encode task knowledge and can causally induce task execution zero-shot.
“We’ve been able to demonstrate that task vectors in large language models can be approximated by a sparse sum of autoencoder latents,” explains Shabalin. “This gives us a deeper understanding of how models recognize and execute tasks based on context.”
A third paper, co-authored by Shabalin, Hao, and Ayush Panda, extends interpretability techniques to large text-to-image diffusion models. Their research applies Sparse Autoencoders (SAEs) and Inference-Time Decomposition of Activations (ITDA) to Flux 1, a state-of-the-art diffusion model.
“By developing an automated interpretation pipeline for vision models, we’ve been able to extract semantically meaningful features,” says Panda. “Our results show that SAEs and IDTAs - a technique expanded on in forthcoming work - outperform MLP neurons on interpretability metrics.”
The team demonstrated practical applications of their research by using SAE features to steer image generation through activation addition, opening new possibilities for controlled content generation.
Parv Mahajan, Collaborative Initiative Lead of the AI Safety Initiative, emphasized the significance of this work. “These papers represent important advances in our ability to understand and control the behavior of increasingly complex AI systems. As these models become more powerful and widely deployed, interpretability research like this becomes essential for ensuring their safe and beneficial use.”
The research team will present their findings at dedicated workshops during ICLR and CVPR, providing opportunities for collaboration with other researchers in the field. Their work represents AISI’s missions to make frontier AI systems more transparent, controllable, and aligned with human values.
