Jiajun Song

Starting as an applied statistician, I have developed a keen interest in challenging inference problems in the real world. There is rarely a clean setup, and novel and complex issues consistently emerge in the data, such as spill-over spatial effects, feedback loop in the system, network interference and so on.

Meanwhile, in an era when more people come to foundation models for their excellence at prediction, obtaining interpretability and effective model inference poses a formidable challenge.

My current interests lie in

(Causal) Inference framework in complex settings, such as networks or correlated data.
Analyzing foundation models with statistical methods, such as high-dimensional statistics.

Posts

Mar 26, 2021	Investigating Mortality Rates from Cardiovascular Pediatric Surgery in STS Public Reporting
Feb 21, 2021	Assessing the Racial Disparities in New York City Policing
Jan 2, 2021	Analyzing the effect of early adversities in the life expectancy of wild baboons
Dec 22, 2018	a distill-style blog post

Publications

PNAS

Out-of-distribution generalization via composition: a lens through induction heads in transformers

Jiajun Song, Zhuoyan Xu, and Yiqiao Zhong

Proceedings of the National Academy of Sciences, 2025

arXiv Blog Code
Preprint

Proposing and solving olympiad geometry with guided tree search

Chi Zhang, Jiajun Song, Siyu Li, and 5 more authors

arXiv preprint arXiv:2412.10673, 2024

Preprint

Uncovering hidden geometry in Transformers via disentangling position and context

Jiajun Song, and Yiqiao Zhong

arXiv preprint arXiv:2310.04861, 2023

arXiv Bib Blog Code

@article{song2023uncovering,
  title = {Uncovering hidden geometry in Transformers via disentangling position and context},
  author = {Song, Jiajun and Zhong, Yiqiao},
  journal = {arXiv preprint arXiv:2310.04861},
  year = {2023},
}