byteflow
https://arxiv.org/pdf/2310.03742.pdf
Novel low-diameter network topologies such as Slim Fly (SF)
offer significant cost and power advantages over the established Fat Tree, Clos, or Dragonfly. To spearhead the adoption
of low-diameter networks, we design, implement, deploy, and
evaluate the first real-world SF installation. We focus on deployment, management, and operational aspects of our test
cluster with 200 servers and carefully analyze performance.
We demonstrate techniques for simple cabling and cabling
validation as well as a novel high-performance routing architecture for InfiniBand-based low-diameter topologies. Our
real-world benchmarks show SF’s strong performance for
many modern workloads such as deep neural network training, graph analytics, or linear algebra kernels. SF outperforms
non-blocking Fat Trees in scalability while offering comparable or better performance and lower cost for large network
sizes. Our work can facilitate deploying SF while the associated (open-source)1
routing architecture is fully portable and
applicable to accelerate any low-diameter interconnect.
offer significant cost and power advantages over the established Fat Tree, Clos, or Dragonfly. To spearhead the adoption
of low-diameter networks, we design, implement, deploy, and
evaluate the first real-world SF installation. We focus on deployment, management, and operational aspects of our test
cluster with 200 servers and carefully analyze performance.
We demonstrate techniques for simple cabling and cabling
validation as well as a novel high-performance routing architecture for InfiniBand-based low-diameter topologies. Our
real-world benchmarks show SF’s strong performance for
many modern workloads such as deep neural network training, graph analytics, or linear algebra kernels. SF outperforms
non-blocking Fat Trees in scalability while offering comparable or better performance and lower cost for large network
sizes. Our work can facilitate deploying SF while the associated (open-source)1
routing architecture is fully portable and
applicable to accelerate any low-diameter interconnect.
Since the problem has been identified, we have retrofitted all InfiniBand cables between compute nodes and switches with ferrite beads. This effectively reduced the frequency of these events, but did not fully solve the problem as it was initially expected. We are working together with Atos to find a final solution. In the meantime, if your job ends unexpectedly with this or a similar error, despite using the suggested workaround below,
😁2
byteflow
https://developer.nvidia.com/blog/one-giant-superchip-for-llms-recommenders-and-gnns-introducing-nvidia-gh200-nvl32
из интересного, прямое охлаждение водой и мезанины нвлинка
https://www.youtube.com/live/Xoji3cEDl2Y
Если вы живёте в идеальном мире, стоит глянуть
Если вы живёте в идеальном мире, стоит глянуть
YouTube
AI/ML Data Center Design - Part 1
Petr Lapukhov joins Jeff Doyle and Jeff Tantsura to discuss the finer points of AI/ML Data Center design.
😁1