Quiz: AI Cluster Orchestration & Service Delivery (Part 10)

The core scheduling-plane fork for an AI fleet is…

cron vs systemdSlurm (HPC gang scheduling) vs Kubernetes + batch scheduler vs a hybrid of bothmanual queues vs spreadsheetsone vendor's cloud console

On a rack-scale machine, the atomic unit the scheduler should allocate is…

the individual GPUthe NVLink domain (the rack) — placement across its boundary is an ~18× bandwidth cliffthe CPU socketthe availability zone

The 'golden stack' discipline means…

buying gold-tier supportone pinned, attested driver+CUDA+NCCL+firmware tuple fleet-wide — drift is a reliability eventalways running latest versionsletting each team pick their stack

Why does the guide call raw GPU utilization a metric that 'lies'?

vendors misreport ita GPU can show 100% busy while stalled on communication or recomputing lost work — goodput measures what actually advanced the jobit can't be measuredit excludes idle nodes

The two latency targets that bind an inference SLO are…

ping and jitterTTFT (prefill-bound) and TPOT/ITL (decode-bound)DNS and TLS timep50 and p50 again

GPU sharing across tenant classes should follow which spectrum?

everyone shares everythingwhole-GPU for tightly-coupled training → hardware-partitioned (MIG) → time-sliced, matched to coupling and isolation needsno sharing, everrandom assignment

The value-stack ladder an operator chooses a rung on is…

bronze, silver, goldbare-metal → GPUaaS → managed platform → serverless tokenson-prem vs cloudCPU → GPU → ASIC

Bare-metal lifecycle at fleet scale should be driven…

by runbooks a human followsdeclaratively, as infrastructure-as-code — the fleet's desired state is versioned and convergedby the vendor's field techsmanually per node

The parallelism-framework standardization choice (FSDP2 / DeepSpeed / Megatron-Core) matters because…

they license differentlyit decides which parallelism forms are first-class and how your checkpoint formats and topology mapping workonly one runs on NVIDIAit sets the Python version

Training-corpus provenance tiers, per the guide, run…

big, medium, smallfully licensed → lawfully-acquired-and-transformed → scraped — each with different legal and commercial exposurepublic vs privatetext vs image

Check your understanding: Software, Orchestration & Service Delivery