Mahdi Soltanolkotabi - Interpreting Generative Models for Better Steering: Generation to Verifiable

Value of the Information & Strength of the Argument

The talk provides valuable insights into the internal dynamics of diffusion models, showing that SAEs can reveal when different semantic features (layout vs. style) emerge. This is a significant contribution to interpretability research, which has largely focused on language models. The proposed steering method, Aphina, is a practical application of these insights, offering a compute-efficient alternative to existing approaches. The argumentation is solid: the speaker motivates the problem with concrete examples, explains the methodology clearly, and supports claims with experimental results. The comparison with proprietary models’ reasoning traces is speculative but well-flagged. The main limitation is that the talk presents work in progress, and some technical details are glossed over.

Scientific Rigor, Source Quality, Title Accuracy

The talk is scientifically rigorous, presenting original research with a clear methodology. The speaker references standard interpretability tools (SAEs, J-Lens from Anthropic) and existing literature on counting issues. The quality of sources is high, as the work is presented at an IPAM workshop, a reputable institution. The title accurately reflects the content, covering both interpretation and steering. The talk does not cite specific papers, but the description links to the workshop page, which likely contains further references. The speaker’s claims are appropriately hedged, and the experimental setup is described in sufficient detail to assess validity.

The title accurately reflects the content: the talk covers interpreting generative models (diffusion models) via sparse autoencoders and using these insights for steering, including a method for verifiable reasoning (counting).

8/10

The talk presents original research with a clear methodology, including training sparse autoencoders on diffusion models and proposing a novel steering mechanism. The claims are supported by experimental results on custom and existing datasets. However, the talk is a presentation of ongoing work, and some details are omitted, limiting full reproducibility. The speaker is a recognized researcher, and the venue (IPAM) is reputable.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 01:00Introduction: motivation on visual reasoning failures in generative models.
• 05:00Overview of using sparse autoencoders (SAEs) on diffusion models.
• 10:00Key finding: layout emerges early in diffusion, style emerges later.
• 15:00Demonstration of manipulating SAE features to move objects in the scene.
• 20:00Introduction of the counting problem and limitations of prompt tuning.
• 25:00Presentation of the Aphina steering method using object detectors.
• 30:00Explanation of control prompts and feedback mechanisms in Aphina.
• 35:00Experimental results on custom dataset showing accuracy improvements.
• 40:00Discussion of proprietary models' reasoning traces and speculation on their methods.
• 42:30Conclusion and mention of future work on RLVR for broader visual reasoning.

• Foundations of Interpretability Workshop — Workshop page where the talk was recorded, providing context and potentially further references.

• Foundations of Interpretability Workshop — The workshop context supports the relevance and validity of the research.