publications

2026

1. arXiv’26

   DoSA: Dots Sparse Attention

   Fanxu Meng^* and Taosong Fang^*†

   arXiv preprint arXiv:2610.13990, 2026

   Abs arXiv PDF Code

   Lossless sparse attention that skips provably unnecessary indexer computation while preserving exact token selection, accelerating sparse-attention indexing and reducing time to first token.

2. EMNLP’26

   ZServe: An Adaptive Framework for Optimizing Large Language Models Serving in Heterogeneous Infrastructures

   Taosong Fang, Zhengzhao Ma, Jiawei Chen, Yushen Yang, Kaijie Zhou, Jia Zheng, Xianpei Han, Le Sun, Hongyu Lin, and Yaojie Lu^†

   In EMNLP System Demonstrations (to appear), 2026

   Abs Website

   Adaptive LLM serving across heterogeneous hardware, combining resource-aware configuration with runtime switching between latency- and throughput-oriented serving modes.

3. MLSys’26

   FlashAgents: Accelerating Multi-Agent LLM Systems via Streaming Prefill Overlap

   Taosong Fang, Zhen Zheng^†, Zhengzhao Ma, Yaojie Lu^†, Hongyu Lin, Xianpei Han, and Le Sun

   In Proceedings of Machine Learning and Systems (MLSys), 2026

   Abs HTML PDF

   Overlaps upstream token generation with downstream incremental prefill and shares prefixes across concurrent requests to accelerate multi-agent LLM workflows.

4. MLSys’26

   BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching

   Zhen Zheng^†, Xin Ji, Taosong Fang, Fanghao Zhou, Chuanjie Liu, and Gang Peng

   In Proceedings of Machine Learning and Systems (MLSys), 2026

   Abs arXiv Code

   Improves batched LLM inference through global prefix sharing, request reordering, and memory-centric token batching to increase GPU utilization and throughput.