Skip to main content
T0 Big tech ByteDance

ByteDance AI Infra (2)

ByteDance AI Infra interview questions covering inference optimization and high-performance computing

Inference optimization High-performance computing

Inference optimization

  1. How is PagedAttention implemented in vLLM, and what motivated the design?
  2. How does KV Cache work? What targeted optimizations exist?
  3. What are the core technical points of FlashAttention?

High-performance computing

  1. In large-model inference, what matters for system resource scheduling and concurrency?
  2. How do you combine multiple techniques to accelerate inference?