Train Large Language Models Faster - Parallelism Deep Dive: Accelerating Large Language Model Training with Parallelism Techniques

Paulo Dichone

€82.99

Video Jun 2025 8hrs 50mins 1st Edition

Paulo Dichone

€82.99

Video Jun 2025 8hrs 50mins 1st Edition

What do you get with a video?

Download this video in MP4 format

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

View table of contents

Download Code

Key benefits

Learn data parallelism, model parallelism, hybrid, pipeline, and tensor parallelism strategies
Implement strategies in real-world scenarios, from small model training on MNIST to advanced parallelism with multiple GPUs
Master checkpointing, storage, and failure management in distributed LLM training

Description

This course offers an in-depth exploration of parallelism in Large Language Model (LLM) training. Beginning with foundational IT concepts like cloud computing, GPUs, and network communication, the course introduces various parallelism techniques such as data parallelism, model parallelism, hybrid approaches, and pipeline parallelism, explaining their benefits and trade-offs. You’ll then apply these strategies in hands-on demos using real-world datasets like MNIST and WikiText. As you progress, you’ll work on true parallelism with multiple GPUs through platforms like Runpod.io, and dive into essential topics such as fault tolerance, scalability, and checkpointing strategies. These lessons ensure your training systems are resilient and optimized for large-scale machine learning workflows. With insights into GPU architectures and advanced tools like DeepSpeed, you'll be equipped to handle the complexities of training massive models efficiently. Whether you're an AI researcher or a data scientist, this course provides the knowledge and practical experience needed to accelerate LLM training and build scalable, efficient AI systems. Through a combination of theoretical lessons and hands-on applications, you’ll master parallelism techniques and become proficient in building and optimizing high-performance LLM training pipelines.

Who is this book for?

This course is designed for AI/ML engineers, researchers, and data scientists interested in optimizing Large Language Model (LLM) training. It’s ideal for professionals who want to deepen their understanding of deep learning infrastructure and parallel computing. A basic knowledge of machine learning, deep learning, and Python programming is required, along with familiarity with GPUs and cloud computing concepts. Whether you're working with large-scale machine learning models or seeking to enhance your parallelism skills, this course is tailored for you.

What you will learn

Understand various parallelism strategies used in LLM training, including data, model, hybrid, pipeline, and tensor parallelism
Implement data parallelism techniques with real-world datasets like MNIST and WikiText
Set up and use Runpod.io for multi-GPU distributed training in LLMs
Evaluate and apply fault tolerance strategies for distributed model training
Optimize large-scale LLM training workflows using GPU architectures and cloud computing
Implement checkpointing strategies for scaling LLM training systems

What do you get with a video?