Skip to main content
T5 Others Honor New grad Round 2

Honor AI Infra campus hire round 2

Honor AI Infra campus-hire round-2 interview questions covering inference optimization

Inference optimization

Honor AI Infra campus-hire round-2 topics.

Inference optimization

  1. Main performance bottlenecks when running LLM inference on NPU
  2. Matmul optimization strategies for Prefill vs Decoding
  3. In blocked compute, how do you keep data contiguous in cache
  4. How well does Ascend NPU architecture fit Transformer-style models

Scenario questions

  1. Roughly how much memory does a quantized large model (e.g. INT8/INT4) use at runtime