Literature Review Consultation Speech
Speaking time: Aim for four to five minutes at a measured pace. Rehearse once with a timer.
Use: Read the English speech naturally. The Chinese section underneath is a memory aid, not additional spoken text.
Status: The comparison is a proposed study. No new experiment has been completed.
English speech
Hello. While preparing for graduate job applications, I studied vLLM and SGLang. vLLM serves large language models to users whose requests change over time. I therefore asked: which CUDA Graph mode should vLLM use for different workloads?
Each inference step runs many GPU kernels. A kernel is one GPU computation; a launch is the CPU submitting it. A CUDA Graph records several operations and replays them with one launch, reducing repeated CPU work. But graphs need compatible batches and cost time and GPU memory to capture. MPK, or Mirage Persistent Kernel, tackles launch overhead by organizing work into one persistent mega-kernel. It is not a vLLM graph mode.
Of my six sources, five are papers and one is vLLM documentation. GraCE [1] explains launch costs and selective graph use. FlashInfer [2] handles dynamic attention scheduling. PagedAttention [3] explains changing requests and KV-cache needs. PyTorch 2 [4] discusses full and partial computation graphs, distinct from CUDA Graphs. The vLLM documentation [5] describes its modes. MPK [6] presents the mega-kernel alternative.
My draft moves from the launch problem to graph trade-offs and changing LLM requests. It then compares the studies, identifies a gap, and proposes a study. The reviewed sources evaluate different systems and conditions, so they do not directly show which vLLM mode suits each workload.
I propose keeping the model, GPU, vLLM version, and attention backend fixed, while varying batch stability, request load, and mode. I would compare latency, memory, and capture costs to develop a mode-selection guide. I have not run this experiment.
My teacher appreciated the literature and citations. I will revise the conclusion to summarize the gap before stating the research question, improve the MPK transition, and correct terminology and referencing. I would welcome feedback on whether my gap now leads clearly to my proposed study.
中文记忆线索
- 动机： 准备校招时学vLLM和SGLang，看到vLLM有不同graph模式，想知道动态请求下怎么选。
- 概念： vLLM服务多个请求；kernel是GPU的一项工作；CPU launch是把工作提交给GPU；CUDA Graph记录多个操作，用一次graph launch重放。MPK则把工作组织成一个持续运行的mega-kernel。
- 来源： GraCE讲launch开销；FlashInfer讲动态attention调度；PagedAttention讲请求和KV-cache变化；PyTorch 2讲编译图；vLLM文档讲实际模式；MPK讲另一条技术路线。
- 文章逻辑： 为什么有开销 → graph有什么收益与成本 → LLM请求为什么动态 → 现有工作研究什么 → 为什么仍无法直接选择vLLM模式。
- gap和目标： 缺少相同条件下、随batch稳定性和请求负载变化的模式比较；拟议控制实验，形成配置建议；尚未做实验。
- 老师反馈： 先归纳gap再提出研究问题；改MPK过渡与结尾衔接，修缩写、作者名和参考文献格式。
If the teacher asks for the gap in one sentence
The reviewed studies do not directly show which vLLM CUDA Graph mode works best for which workload under the same experimental conditions.
Sources in the same order as my draft
[1] GraCE
[2] FlashInfer
[3] PagedAttention
[4] PyTorch 2
[5] vLLM CUDA Graphs documentation, v0.21.0
[6] MPK