Performance and GPU Foundations for AI Systems

Calculate latency, throughput, utilization, and in-flight concurrency from workload timing data, using Little’s Law where applicable.

Apply Amdahl’s Law to estimate the maximum benefit of a proposed optimization.

Use arithmetic intensity and a roofline model to determine whether an AI workload is compute-bound or memory-bandwidth-bound.

Trace how a CUDA thread block executes as warps and accesses registers, shared memory, caches, and high-bandwidth device memory.