Skip to main content
T0 Big tech Baidu Internship Round 1

Baidu AI Infra internship first round (3)

Real Baidu AI Infra internship first-round interview questions covering inference and operator optimization

Inference optimization Operator optimization

Distributed training

  1. Explain what TP, PP, and DP mean and how each runs
  2. How do you choose a parallel strategy based on TP vs PP communication cost?

Quantization

  1. Explain differences among per-tensor, per-channel, and group-wise quantization granularities
  2. Accuracy differences across quantization methods, and why group-wise can further cut quantization error
  3. Compute-cost comparison across quantization methods and optimization techniques
  4. How do you handle outliers during quantization?
  5. Introduce the core ideas of GPTQ, AWQ, and SmoothQuant
  6. Principle and implementation of KV Cache quantization

Attention and inference optimization

  1. What is FlashAttention’s acceleration principle?
  2. Main differences between FlashAttention v1 and v2
  3. Bc-block tiling strategy in FlashAttention, and how 1-loop FlashAttention is implemented
  4. Design idea of PagedAttention
  5. Differences between large-model prefill and decoding stages, and why they differ
  6. What problems does FlashAttention have in decoding? How does FlashDecoding improve it?

Operator implementation

  1. Concrete implementation of RMSNorm
  2. General approach and methodology for CUDA Kernel optimization
  3. Given Conv2D input [C=64, W=64, H=128], 3x3 kernel, output [C=128, W=64, H=128], compute parameter count and FLOPs
  4. What methods exist for operator optimization on CPU (e.g. AVX-512)?

C++ basics

  1. Categories of smart pointers and their use cases

Coding

  1. Implement a CUDA LayerNorm Kernel