tail latency

Tail latency in GPU collective operations is the slowdown caused when the slowest node, link, or process sets the pace for an entire operation, forcing every other GPU to wait before training can proc