Refresh Your Home For Fall
Buy New
To see product details, add this item to your cart.
Ships from: Amazon.com
Sold by: Amazon.com
To see product details, add this item to your cart. You can always remove it later.
Shipper / Seller
Amazon.com
Amazon.com
Shipper / Seller
Amazon.com
Returns
FREE 30-day refund/replacement
FREE 30-day refund/replacement
Quick refund
Usually issued within 24 hours. See exceptions
FREE return
At least one free return option available.
Convenient dropoff
At any of our 50,000 US locations.
See return policy
Payment
Secure transaction
Your transaction is secure
We work hard to protect your security and privacy. Our payment security system encrypts your information during transmission. We don’t share your credit card details with third-party sellers, and we don’t sell your information to others. Learn more
Gift options
Available at checkout
Available at checkout This item is a gift. Change
At checkout, you can add a custom message, a gift receipt for easy returns and have the item gift-wrapped
To see product details, add this item to your cart. You can always remove it later.
FREE Returns
Access codes and supplements are not guaranteed with used items.
Added to

Sorry, there was a problem.

There was an error retrieving your Wish Lists. Please try again.

Sorry, there was a problem.

List unavailable.
Kindle app logo image

Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet, or computer - no Kindle device required.

Read instantly on your browser with Kindle for Web.

Using your mobile phone camera - scan the code below and download the Kindle app.

QR code to download the Kindle App

  • Build a Large Language Model (From Scratch)
  • Why Build LLMs From Scratch?
  • 5 VIDEOS

Follow the author

Get new release updates & improved recommendations
Something went wrong. Please try your request again later.

Build a Large Language Model (From Scratch)

4.5 out of 5 stars (611)

Purchase options and add-ons

How to implement LLM attention mechanisms and GPT-style transformers.

In
Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You’ll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.

Build a Large Language Model (from Scratch) teaches you how to:
  • Plan and code all the parts of an LLM
  • Prepare a dataset suitable for LLM training
  • Fine-tune LLMs for text classification and with your own data
  • Use human feedback to ensure your LLM follows instructions
  • Load pretrained weights into an LLM

Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you’ll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant.

About the technology

Physicist Richard P. Feynman reportedly said, “I don’t understand anything I can’t build.” Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning.

About the book

Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself!

What's inside
  • Plan and code an LLM comparable to GPT-2
  • Load pretrained weights
  • Construct a complete training pipeline
  • Fine-tune your LLM for text classification
  • Develop LLMs that follow human instructions

About the reader

Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs.

About the author

Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.

The technical editor on this book was
David Caswell.

Table of Contents

1 Understanding large language models
2 Working with text data
3 Coding attention mechanisms
4 Implementing a GPT model from scratch to generate text
5 Pretraining on unlabeled data
6 Fine-tuning for classification
7 Fine-tuning to follow instructions
A Introduction to PyTorch
B References and further reading
C Exercise solutions
D Adding bells and whistles to the training loop
E Parameter-efficient fine-tuning with LoRA

Frequently bought together

This item: Build a Large Language Model (From Scratch)
$49.24
Get it as soon as Sunday, Sep 20
In Stock
Ships from and sold by Amazon.com.
+
$47.82
Get it as soon as Sunday, Sep 20
In Stock
Ships from and sold by Amazon.com.
+
$52.40
Get it as soon as Sunday, Sep 20
In Stock
Ships from and sold by Amazon.com.
Total price: $00
To see our price, add these items to your cart.
Details
Added to Cart
Choose items to buy together.

Customers also bought or read

Loading...

From the Publisher

Build LLM (From Scratch) header

right quote

“The most understandable and comprehensive explanation of language models yet! Its unique and practical teaching style achieves a level of understanding you can’t get any other way.”

Cameron Wolfe, Senior Scientist, Netflix

middle quote

“Sebastian combines deep knowledge with practical engineering skills and a knack for making complex ideas simple. This is the guide you need!”

Chip Huyen, author of Designing Machine Learning Systems and AI Engineering

left quote

“Definitive, up-to-date coverage. Highly recommended!”

Dr. Vahid Mirjalili, Senior Data Scientist, FM Global

about the book

why this book?

Build a Large Language Model (From Scratch) offers a practical, hands-on approach to understanding and constructing large language models (LLMs) from the ground up.

By guiding you through each stage—from data preparation and coding attention mechanisms to pretraining and fine-tuning—this book demystifies the inner workings of LLMs using Python and PyTorch.

Ideal for developers and machine learning enthusiasts, it empowers you to build a functional GPT-style model on a standard laptop, fostering a deeper comprehension of generative AI technologies.

about manning

about Manning

Manning helps developers and tech professionals stay ahead in a fast-moving industry with expert-led books, videos, and projects. Learning never stops, but it’s hard to keep up, so we focus on content that’s practical, clear, and trusted. As an independent publisher, we adapt quickly, from pioneering early-access books to offering DRM-free eBooks. Our series, like "In Action" and "In a Month of Lunches", reflect a commitment to making complex topics accessible.

LLMs in Production: From language models to successful products
AI Agents in Action
Natural Language Processing in Action, Second Edition
Effective Conversational AI: Chatbots that work
Data Analysis with LLMs: Text, tables, images and sound (In Action)
Causal AI
Customer Reviews
4.5 out of 5 stars 36
4.0 out of 5 stars 51
4.8 out of 5 stars 8
4.8 out of 5 stars 7
4.8 out of 5 stars 7
4.4 out of 5 stars 14
Level of proficiency Intermediate Intermediate Intermediate Intermediate Intermediate Advanced
About the reader For data scientists and ML engineers. For intermediate Python programmers. For intermediate Python programmers. For developers, engineers, and product managers. For data scientists and data analysts. For data scientists and machine learning engineers.
Special features Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant.
Pages 456 344 688 328 232 520

Editorial Reviews

Review

The most comprehensive book I've seen on building LLMs. Highly recommended! -- Raul Ciotescu, CTO, Netzinkubator Software

A clear, hands-on guide that empowers readers to build their own models and explore the cutting edge of AI. -- Guillermo Alcántara, Project manager, PepsiCo Global

Must-have resource for quickly getting up to speed on LLMs. Whether you're new to the field or looking to deepen your knowledge, it’s the perfect guide. -- Walter Reade, Staff Developer Relations Engineer, Kaggle/Google

A fantastic resource for diving into LLMs—a must-read for anyone eager to get hands-on! -- Dr. Vahid Mirjalili, Senior Data Scientist, FM Global

From the Back Cover

From the back cover:

Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you'll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you'll really understand it because you built it yourself!

About the reader:

Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs.

Product details

  • Publisher ‏ : ‎ Manning
  • Publication date ‏ : ‎ October 29, 2024
  • Language ‏ : ‎ English
  • Print length ‏ : ‎ 368 pages
  • ISBN-10 ‏ : ‎ 1633437167
  • ISBN-13 ‏ : ‎ 978-1633437166
  • Item Weight ‏ : ‎ 1.35 pounds
  • Dimensions ‏ : ‎ 7.38 x 0.7 x 9.25 inches
  • Best Sellers Rank: #5,175 in Books (See Top 100 in Books)
  • Customer Reviews:
    4.5 out of 5 stars (611)

About the author

Follow authors to get new release updates, plus improved recommendations.
Sebastian Raschka
Brief content visible, double tap to read full content.
Full content visible, double tap to read brief content.

Sebastian Raschka, PhD is an LLM Research Engineer with over a decade of experience in artificial intelligence. His work bridges academia and industry, including roles as senior engineering staff at an AI company and a statistics professor.

As an independent researcher and industry expert, Sebastian collaborates with companies on AI solutions and serves on the Open Source Advisory Board at University of Wisconsin–Madison.

Sebastian specializes in LLMs and the development of high-performance AI systems, with a deep focus on practical, code-driven implementations.