Alibaba Group Holding AI Infra first round

Fundamentals

1. Walk through Dispatch and Combine in Expert Parallelism.
2. What are the architectural traits of DeepSeek-series models?
3. In Tensor Parallelism, how many AllReduce ops are needed, and how do you derive that?
4. How do you compute KV Cache GPU-memory size?
5. Which recent vLLM features are worth watching?

Coding

6. Hand-write MQA and GQA.