Scientific Understanding of Foundation Models
Foundation models have transformed AI across language, vision, science, and multimodal reasoning — but we still lack a systematic scientific understanding of how they represent knowledge, generalize, reason, and align with human intent. This workshop brings together researchers committed to building that understanding.
About the Workshop
Moving from empirical scaling phenomena toward predictive science for foundation models.
Despite the extraordinary capabilities of modern foundation models, our scientific understanding of these systems remains remarkably shallow. We can observe that scaling works — but we cannot yet predict when capabilities will emerge, why certain representations form, or how reasoning behavior arises from training dynamics.
This workshop aims to catalyze a shift from capability demonstration to mechanistic explanation. We seek to identify laws, invariants, causal mechanisms, and rigorous evaluation methodologies that can make foundation models more controllable, reliable, and interpretable.
By bringing together researchers from theory, empirical ML, interpretability, optimization, evaluation, and scientific methodology, we aim to lay groundwork for a genuine science of foundation models — one that offers not just post-hoc explanations, but predictive understanding.
Motivating Questions
- 1What principles govern the emergence of capabilities in large models?
- 2How do representation geometry and training dynamics shape reasoning behavior?
- 3What are the limits of scaling laws — and what comes after them?
- 4How can we build predictive theories of generalization in overparameterized regimes?
- 5What scientific tools are needed to study foundation models as complex adaptive systems?
Topics
The workshop is organized around three major pillars, each addressing a critical dimension of scientific understanding.
Theory
- Scaling laws and breakdown regimes
- Emergence and phase transitions in capability
- Implicit bias of optimization and representation geometry
- Information-theoretic perspectives on in-context learning
- Expressivity vs. compression trade-offs
Controlled Studies & Empirical Understanding
- Representation collapse, specialization, and disentanglement
- Emergent reasoning behavior in LLMs
- Probing and causal interventions
- Robustness and systematic failure modes
- Training dynamics and representation evolution
Evaluation & Scientific Methodology
- Designing principled evaluation benchmarks
- Capability measurement vs. superficial performance
- Reproducibility and experimental design for large-model studies
- Predictive evaluation for safety and alignment
- Stress testing and adversarial robustness
We especially encourage submissions that bridge theory and empirical observation — work that uses formal tools to explain experimental findings, or controlled experiments designed to test theoretical predictions.
Call for Papers
We invite original contributions that advance the scientific understanding of foundation models — spanning theory, controlled empirical studies, and rigorous evaluation methodology.
We welcome work that goes beyond benchmark improvements to offer mechanistic insight, theoretical grounding, or methodological innovation. Negative results, careful reproductions, and position papers that articulate open problems are valued.
Full Papers
Up to 8 pagesOriginal research contributions presenting substantial theoretical, empirical, or methodological results.
Short Papers
Up to 4 pagesPreliminary findings, negative results, position papers, and focused contributions that advance the workshop's scientific goals.
Review Process
- All submissions undergo double-blind peer review.
- Each submission receives at least two expert reviews.
- Top-scoring submissions will be selected for spotlight talks.
- All accepted papers will be presented as posters during the workshop.
Key Dates
- Submission DeadlineJune 23, 2026
- Author NotificationJuly 24, 2026
- Camera-Ready DeadlineTBA
- Workshop DateOctober 9, 2026
All deadlines are 11:59 PM AoE (Anywhere on Earth).
Invited Speakers
Our invited speakers bring deep expertise spanning theoretical foundations, empirical methodology, and large-scale model evaluation.
Jikai Jin
Research on theoretical foundations and empirical phenomena in large neural networks, including representation learning, scaling behavior, training dynamics, and emergent capabilities.
Ludwig Schmidt
Research on robustness, dataset quality, evaluation methodology, and rigorous measurement — developing systematic approaches to evaluating and understanding large-scale models.
Surya Ganguli
Andrew Gordon Wilson
Zhiyuan Li
Valentina Pyatkin
Mohammad Shoeybi
Additional invited speakers to be announced. Check back for updates.
Workshop Format & Schedule
A full-day program designed to balance deep technical talks with open discussion and community engagement.
Program Components
Opening Remarks
Welcome and framing of the workshop's scientific goals.
Invited Talks
In-depth invited presentations on scaling, emergence, post-training, and alignment.
Poster Sessions
Two dedicated poster sessions during coffee breaks to discuss accepted work.
Panel Discussions
Two thematic panels on predictive understanding and controlled studies.
Contributed Spotlights
Top submissions presented as contributed spotlight talks.
Closing Remarks & Awards
Summary, best paper awards, and next steps for the community.
Panel Discussion Themes
- “Are scaling laws enough for predictive understanding?”
- “Controlled Studies: science or engineering tool?”
Schedule Overview
Schedule is tentative and subject to change. All times are in local conference time.
Organizers
The workshop is organized by a team of researchers spanning theory, empirical machine learning, interpretability, and evaluation methodology.
Hanlin Zhang
Natalie Abreu
Yizhou Liu
Yizhong Wang
Sham Kakade
Kaiyue Wen
Sewon Min
Alex Damian
Diversity & Inclusion
We are committed to fostering an inclusive, welcoming, and respectful environment for all participants. The scientific questions at the heart of this workshop benefit from diverse perspectives — across geography, career stage, research tradition, gender, and background.
We will provide remote access options to support participation from researchers who cannot attend in person. If you require specific accessibility accommodations, please contact us and we will do our best to help.