Tutorial roadmap
The update plan for four modules and 29 chapters
A note first
Thanks for following AIInfraGuide. I have a day job, so I write and update the tutorials on evenings and weekends. Updates will not be fast, but I try to make every article thorough and correct.
If the project helps you, a Star on GitHub is appreciated, and so is sharing it with friends who are interested in AI Infra. That support is what keeps me writing.
If you have suggestions or topics you want covered, tell me through a GitHub issue or a comment on the WeChat account.
Getting the latest articles
New articles go out first on Zhihu and the WeChat account "AI Infra Guide", and reach GitHub and this site afterwards. Follow one of those if you want to read them as soon as they are published. Both are in Chinese.
Plan overview
The tutorials are split into 4 modules that cover the AI Infra stack from the prerequisites to inference optimization. Here are the planned chapters for each module:
Module 1: Prerequisites
6 chapters- 1 Chapter 1: Programming language basics
- 2 Chapter 2: Math basics
- 3 Chapter 3: The Transformer architecture in detail
- 4 Chapter 4: PyTorch
- 5 Chapter 5: GPU hardware overview
- 6 Chapter 6: Collective communication basics
Module 2: CUDA programming and kernel optimization
8 chapters- 1 Chapter 1: Getting started with CUDA
- 2 Chapter 2: CUDA performance basics
- 3 Chapter 3: Classic kernels - Reduce
- 4 Chapter 4: Classic kernels - GEMM
- 5 Chapter 5: Classic kernels - Softmax and kernel fusion
- 6 Chapter 6: Attention kernels
- 7 Chapter 7: AI compilers
- 8 Chapter 8: Profiling tools
Module 3: Distributed training
7 chapters- 1 Chapter 1: Distributed training overview
- 2 Chapter 2: Data parallelism
- 3 Chapter 3: The ZeRO family
- 4 Chapter 4: Tensor and sequence parallelism
- 5 Chapter 5: Pipeline parallelism
- 6 Chapter 6: 3D parallelism and hybrid strategies
- 7 Chapter 7: Training frameworks in practice
Module 4: Inference optimization
12 chapters- 1 Chapter 1: LLM inference basics
- 2 Chapter 2: Inference engine internals
- 3 Chapter 3: Reading the nano-vllm source
- 4 Chapter 4: Quantization
- 5 Chapter 5: Speculative decoding
- 6 Chapter 6: Distributed inference and serving large models
- 7 Chapter 7: Prefill/decode disaggregation
- 8 Chapter 8: Production serving features
- 9 Chapter 9: Profiling and benchmarking
- 10 Chapter 10: Production deployment and operations
- 11 Chapter 11: Choosing optimizations and an end-to-end project
- 12 Chapter 12: On-device inference
Update schedule
- How often: about one or two articles a week, depending on the topic and how busy work is
- Order: module by module, starting with the prerequisites and CUDA modules
- Style: every article explains the idea in plain words before the terminology, with code examples and diagrams
- Feedback: chapters that many readers ask for may be moved up