You want to make sure that every GPU, every CPU, every LPU get the data at the same time so they can compute at the same time and then exchange data at the same time and do it over and over and over again. 

If you don't have a purpose-built network, if you take an off-the-shelf network, they didn't really care about synchronizations for... 

distribute computing workloads, that network may create delays. And delays means that the compute engines will not be synchronized here, and one will need to wait for another. 

And when one waits for another, essentially, your AI factor becomes a seraphone. Actually, you are not able to generate tokens. So that's why there is a heavy investment in making sure that everything is synchronized.