Taosong Fang

I am a Researcher at Dots Studio, Xiaohongshu, where I focus on LLM pretraining and backbone and infrastructure co-design. Beyond these areas, my previous research also covered LLM post-training and ML System for LLM training, inference, and agentic workflows.

I received my master’s degree in 2026 from the Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, co-advised by Prof. Hongyu Lin and Prof. Yaojie Lu. I earned my bachelor’s degree from the School of Software at Dalian University of Technology in 2023.

Education

Institute of Software, Chinese Academy of Sciences (ISCAS), 2023 – 2026
M.S., Chinese Information Processing Laboratory
Advisors: Prof. Hongyu Lin and Prof. Yaojie Lu
Dalian University of Technology (DUT), 2019 – 2023
B.E., School of Software

Work Experience

Xiaohongshu, Dots Studio, Researcher, Jun. 2026 – Present
Qwen, Summer Intern, May 2025 – Oct. 2025
Mentors: Yuqiong Liu and Kai Dang
Microsoft, Research Intern, Dec. 2023 – May 2025
Transitioned from the DeepSpeed team to the Microsoft Superintelligence team
Mentor: Dr. Zhen Zheng
Alibaba Cloud PAI, Research Intern, Mar. 2023 – Dec. 2023
Mentors: Tianyou Guo, Xiafei Qiu, and Dr. Zhen Zheng

Selected Publications

* Co-first authors · † Corresponding author

arXiv’26

DoSA: Dots Sparse Attention

Fanxu Meng^* and Taosong Fang^*†

arXiv preprint arXiv:2610.13990, 2026

Abs arXiv PDF Code

Lossless sparse attention that skips provably unnecessary indexer computation while preserving exact token selection, accelerating sparse-attention indexing and reducing time to first token.
EMNLP’26

ZServe: An Adaptive Framework for Optimizing Large Language Models Serving in Heterogeneous Infrastructures

Taosong Fang, Zhengzhao Ma, Jiawei Chen, Yushen Yang, Kaijie Zhou, Jia Zheng, Xianpei Han, Le Sun, Hongyu Lin, and Yaojie Lu^†

In EMNLP System Demonstrations (to appear), 2026

Abs Website

Adaptive LLM serving across heterogeneous hardware, combining resource-aware configuration with runtime switching between latency- and throughput-oriented serving modes.
MLSys’26

FlashAgents: Accelerating Multi-Agent LLM Systems via Streaming Prefill Overlap

Taosong Fang, Zhen Zheng^†, Zhengzhao Ma, Yaojie Lu^†, Hongyu Lin, Xianpei Han, and Le Sun

In Proceedings of Machine Learning and Systems (MLSys), 2026

Abs HTML PDF

Overlaps upstream token generation with downstream incremental prefill and shares prefixes across concurrent requests to accelerate multi-agent LLM workflows.
MLSys’26

BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching

Zhen Zheng^†, Xin Ji, Taosong Fang, Fanghao Zhou, Chuanjie Liu, and Gang Peng

In Proceedings of Machine Learning and Systems (MLSys), 2026

Abs arXiv Code

Improves batched LLM inference through global prefix sharing, request reordering, and memory-centric token batching to increase GPU utilization and throughput.