Skip to main content
T0 Big tech ByteDance Internship Round 1

ByteDance AI Infra internship first round (4)

ByteDance AI Infra internship first-round interview questions covering training optimization and high-performance computing

Training optimization High-performance computing

Fundamentals

  1. Explain DeepSpeed’s core design philosophy and main features.
  2. Why choose ZeRO Stage 3? Derive the formula for how much GPU memory it saves.
  3. Besides DeepSpeed, which distributed training acceleration frameworks have you used? How do they differ?
  4. What are the main differences between BLIP and BLIP-2? What architecture do mainstream multimodal models use today?
  5. What is the design idea behind Q-Former, and what does it do?

Training optimization

  1. What is the technical path for content understanding? If you train a chart-to-story model, how should you construct the data?

Project experience

  1. Introduce the core projects you worked on.

Coding

  1. Implement integer square root (LeetCode 69), and discuss possible optimizations.