Skip to main content
T2 AI unicorns MiniMax Internship Round 2

MiniMax AI Infra internship second round

Real MiniMax AI Infra internship second-round interview questions covering training optimization and high-performance computing

Training optimization High-performance computing

Project questions

  1. Share an internship project, focusing on the hardest technical problem you hit and the optimization results you got

Theory

  1. What state does each DeepSpeed ZeRO stage (Stage-1 / Stage-2 / Stage-3) shard? How do they progress in communication volume and memory savings?
  2. Which parallelism schemes have you used for large-model training? What are the design ideas and fit conditions for data, tensor, and pipeline parallelism?
  3. To train a ~70B model, how do you roughly estimate memory needed on one GPU?
  4. What is the core idea of LoRA? Why does low-rank factorization cut the number of updated parameters so much?
  5. In LoRA, why are down-projection A and up-projection B initialized differently? What is that for?
  6. Besides ZeRO, what other large-model training optimizations do you know?

Coding

  1. Implement sliding-window maximum