Jiajun Song

Starting as an applied statistician, I have developed a keen interest in challenging inference problems in the real world. There is rarely a clean setup, and novel and complex issues consistently emerge in the data, such as spill-over spatial effects, feedback loop in the system, network interference and so on.

Meanwhile, in an era when more people come to foundation models for their excellence at prediction, obtaining interpretability and effective model inference poses a formidable challenge.

My current interests lie in

Interpretability of large language models: uncovering hidden geometry and compositional mechanisms (e.g., induction heads) inside transformers.
Reinforcement learning for LLM post-training: policy optimization and reward design for RLVR, including mitigating multi-reward conflicts.
Reliability and evaluation of LLMs: hallucination mitigation via multi-agent self-check, theory-of-mind benchmarks, and the divergence between human- and LLM-generated tasks.

Posts

Mar 26, 2021	Investigating Mortality Rates from Cardiovascular Pediatric Surgery in STS Public Reporting
Feb 21, 2021	Assessing the Racial Disparities in New York City Policing
Jan 2, 2021	Analyzing the effect of early adversities in the life expectancy of wild baboons
Dec 22, 2018	a distill-style blog post

Publications

PNAS

Out-of-distribution generalization via composition: a lens through induction heads in transformers

Jiajun Song, Zhuoyan Xu, and Yiqiao Zhong

Proceedings of the National Academy of Sciences, 2025

arXiv Blog Code
Nat. Mach. Intell.

Proposing and solving olympiad geometry with guided tree search

Chi Zhang, Jiajun Song, Siyu Li, and 5 more authors

Nature Machine Intelligence, 2026

arXiv

HDSR

Uncovering hidden geometry in Transformers via disentangling position and context

Jiajun Song, and Yiqiao Zhong

Harvard Data Science Review, 2026

arXiv Bib Blog Code

@article{song2023uncovering,
  title = {Uncovering hidden geometry in Transformers via disentangling position and context},
  author = {Song, Jiajun and Zhong, Yiqiao},
  journal = {Harvard Data Science Review},
  volume = {8},
  number = {2},
  year = {2026},
}

ACL

MARCH: Multi-Agent Reinforced Check for Hallucination

Zhuo Li, Yupeng Zhang, Pengyu Cheng, and 8 more authors

In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, 2026

arXiv Bib

@inproceedings{li2026march,
  title = {MARCH: Multi-Agent Reinforced Check for Hallucination},
  author = {Li, Zhuo and Zhang, Yupeng and Cheng, Pengyu and Song, Jiajun and Zhou, Mengyu and Li, Hao and Hu, Shujie and Qin, Yu and Zhao, Erchao and Jiang, Xiaoxi and Jiang, Guanjun},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
}

AAAI

Mind the gap: The divergence between human and llm-generated tasks

Yi-Long Lu, Jiajun Song, Chunhui Zhang, and 1 more author

In Proceedings of the AAAI Conference on Artificial Intelligence, 2026

Bib

@inproceedings{lu2026mind,
  title = {Mind the gap: The divergence between human and llm-generated tasks},
  author = {Lu, Yi-Long and Song, Jiajun and Zhang, Chunhui and Wang, Wei},
  booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
  volume = {40},
  number = {3},
  pages = {1964--1972},
  year = {2026},
}