Skip to main content
General question bank

AI Infra mixed interview question bank (5)

Real AI Infra interview questions covering training optimization and high-performance computing

Training optimization High-performance computing

Project experience

  1. Deep dive into your project

Basics

  1. Explain Python’s GIL and how it affects multithreading
  2. What IPC mechanisms exist between Python processes?
  3. How do you compute All-Reduce communication cost? Derive it.
  4. How do tensor.view and tensor.contiguous relate, and how do they differ?
  5. In tensor parallelism, how does column-first sharding differ from row-first sharding?

Coding

  1. Implement integer power of a number
  2. Implement a Graph Fusion algorithm