Системное администрирование.

Neural Networks for Beginners. Part Zero. Overview

Every time you say “Thank you” to a neural network, you trigger a pipeline that multiplies hundreds of matrices with billions of elements, burning as much electricity as an LED lamp in a few seconds.This is the first article in a small series dedicated to networks for AI/ML clusters and HPC.In this series, we will touch upon the principles of how models work and are trained, parallelization, DMA and RDMA technologies, network topologies, InfiniBand and RoCE, and we’ll also philosophize on the topic of general and special solutions.Specifically in this article, we will figure out what a neural network is, how it works, how it is trained, and most importantly, why it needs hundreds of expensive GPU cards and some kind of special network.The refrain of today’s story is: there is no magic in neural networks—it’s just a multitude of simple operations on numbers performed on computers with special chips. There is no magic in how they work, nor in the infrastructure on which they run.Let’s dive in!Table of Contents

продолжить чтение

• Оставлено в
  • ai   
  • backpropagation   
  • infiniband   
  • llm   
  • ml   
  • mlp   
  • neural networks   
  • roce   
  • transformers   
  • искусственный интеллект   
  • Сетевые технологии   
  • Системное администрирование