A university consortium in Cape Town plans a training run on rented 80 GB GPUs. See exactly how much memory each GPU needs for weights, gradients and optimizer state, then shard them with ZeRO and FSDP and keep a pipeline busy with micro-batches. Then try plain data parallel for 70B, shard across too few GPUs, or send one micro-batch. This lesson needs JavaScript.