Skip to main content
General question bank

AI Infra mixed interview question bank (7)

Real AI Infra interview questions covering inference optimization and training optimization

Inference optimization Training optimization

Basics

  1. What are the mainstream acceleration approaches for large-model training and inference?
  2. Basic ideas of multi-node multi-GPU distributed training, and common frameworks
  3. Do you know DeepSpeed? What are its core features?
  4. How do you improve training efficiency along time and resource axes?
  5. If training is abnormally slow, what do you check?
  6. Communication bottlenecks in multi-node multi-GPU training, and how to optimize them
  7. How does gradient accumulation work?
  8. Common model quantization methods and inference acceleration schemes
  9. Have you optimized large-model training or inference performance on GPUs in practice?
  10. What can cause loss oscillation during training?