Skip to main content
General question bank

AI Infra mixed interview question bank (4)

Real AI Infra interview questions covering operator optimization and high-performance computing

Operator optimization High-performance computing

Project experience

  1. Deep dive into your project

Basics

  1. Have you done kernel-level optimization, for example with CuTe DSL or hand-written CUDA for operator fusion? Describe what you did.
  2. When fusing kernels, which approach do you usually prefer?
  3. Have you seen performance drop after operator fusion? Why?
  4. When writing CUDA, have you looked at low-level details such as Warp Specialization on Hopper and how it works underneath?
  5. When deploying or training large models, have you used low-level debugging tools? How do you locate and fix an NCCL Timeout at thousand-GPU scale?
  6. Have you optimized MoE models for RL settings?
  7. Before shipping an inference service, what performance work did you do?
  8. Have you tried using an Agent to auto-generate CUDA kernels? How did that work?