What this book covers
Chapter 1, Introduction to Generative AI: Drawing Data from Models, sets the stage for understanding how AI models, like those behind Midjourney, are reshaping fields beyond art—ranging from natural language processing to medical diagnostics and game-playing mastery. You’ll explore the fundamental differences between discriminative and generative models, the rules of probability that underpin them, and why generative models present unique challenges. This chapter aims to offer you a solid grasp of the foundations that power today’s most talked-about AI systems.
Chapter 2, Building Blocks of Deep Neural Networks, takes a step back to explore the foundational principles that make modern generative AI possible. You will walk through the essential components, from perceptrons to transformers, activation functions, and optimization algorithms. You’ll also gain insight into how different design choices impact model performance and why certain approaches have become dominant. By the end of this chapter, you’ll have a deeper appreciation for the mechanics behind neural networks and a strong foundation for tackling more advanced topics later in the book.
Chapter 3, The Rise of Methods for Text Generation, introduces concepts and techniques related to the task of text generation. It includes details related to the very basics of language generation using deep learning models starting right from different methods/techniques for representing text in vector space to different architectural choices and decoding mechanisms to achieve high quality outputs. This chapter also sets the foundation for more complex text generation methods covered in the subsequent chapter.
Chapter 4, NLP 2.0: Using Transformers to Generate Text, covers the latest and greatest in the NLP domain, with primary focus on text generation capabilities of some of the state-of-the-art architectures based on transformers and the like. The chapter also covers how transformers and architectures (like GPT-x) have revolutionized the language generation and NLP domain in general.
Chapter 5, LLM Foundations, explores the foundational aspects of LLMs, which have emerged as transformative forces in AI in just a few short years. Building on NLP concepts discussed in previous chapter, this chapter dives into what distinguishes LLMs from earlier models. It includes a recap of transformer architectures, insights into LLM training setups, and an exploration of instruction tuning and RLHF through hands-on exercises to solidify understanding.
Chapter 6, Open-Source LLMs, introduces some of the leading open-source LLMs, including Falcon, LLaMA, and Dolly, and discusses publicly available datasets and benchmarks that help evaluate their performance. While proprietary models like GPT-4 keep key details under wraps, open-source alternatives provide researchers and developers with the tools to experiment, analyze, and innovate outside corporate labs. After this chapter, you’ll know how open-source models enable broader participation in AI research.
Chapter 7, Prompt Engineering, goes into the evolving field of prompt engineering, which bridges the gap between human intention and machine understanding by transforming task instructions into natural language. The chapter explores core concepts like the fundamentals of prompt design, various types of prompts (zero-shot, few-shot, chain of thought, ReAct, and more), and tasks such as summarization and translation. It also covers advanced techniques, including Tree of Thought and Voting/Self-Consistency, along with applications in cross-domain applications, and discussions on challenges, limitations, and defensive strategies against prompt attacks provide a comprehensive understanding of this transformative technique.
Chapter 8, LLM Toolbox, moves beyond basic prompt interactions and explores the tools that turn LLMs into fully functional systems. You’ll learn how to integrate AI with external data sources, store and retrieve contextual information using vector databases, and create specialized AI agents that can execute tasks dynamically. This chapter also introduces LangChain, walks through building a simple LLM-powered application, and demonstrates how to construct more advanced systems using LangGraph.
Chapter 9, LLM Optimization Techniques, focuses on optimizing transformer-based architectures to balance performance with efficiency. It covers the motivations for optimization, techniques for improving training, finetuning and inference, and emerging trends in AI. Topics include pretraining strategies like data efficiency, quantization, and efficient architectures, fine-tuning methods such as PEFT and LoRa, and inference enhancements like offloading and sharding. The chapter also explores emerging areas like MaMBa, RWKV, specialized hardware, and small language models, with applications extending beyond LLMs to other deep learning domains.
Chapter 10, Emerging Applications in Generative AI, explores the cutting-edge advancements shaping the next generation of AI. You will dive into emerging trends, including new techniques for text generation, reinforcement learning for alignment, and model distillation for efficiency. You’ll also explore novel approaches to detecting hallucinations, multimodal AI capable of generating language and images, and the rise of agentic models.
Chapter 11, Neural Networks Using VAEs, introduces Variational Autoencoders (VAEs), a powerful approach to generating complex, real-world images. This chapter breaks down how neural networks create low-dimensional representations, how variational methods enable efficient sampling, and how techniques like the reparameterization trick and Inverse Autoregressive Flow (IAF) refine model outputs. You’ll also implement VAEs in PyTorch, gaining hands-on experience with one of the most versatile generative models.
Chapter 12, Image Generation with GANs, introduces Generative Adversarial Networks (GANs) as powerful deep learning architectures for generative modeling. Starting with the building blocks of GANs and other key fundamental concepts, this chapter covers a number of GAN architectures and how they are used to generate high resolution images from random noise.
Chapter 13, Style Transfer with GANs, focuses upon a creative application of generative modeling, particularly GANs, called style transfer. Applications such as transforming black and white images to colored, aerial maps to Google-maps like outputs, background removal are all made possible using style transfer. In this chapter, we cover a number of paired and un-paired architectures, such as Pix2Pix and CycleGAN.
Chapter 14, Deepfakes with GANs, introduces an interesting and controversial application of generative models (with focus on GANs) called deepfakes. The chapter includes details about basic building blocks for deepfakes such as features, different modes of operations along with a number of key architectures to develop your own deepfake pipelines. The chapter includes a number of hands-on examples to generate fake photos and videos based on the concepts covered.
Chapter 15, Diffusion Models and AI Art, show you how diffusion models work, how they compare to other image-generation techniques, and how Stable Diffusion combines VAEs with denoising steps for efficient image creation. Through hands-on exercises with the Hugging Face pipeline, you’ll see how user prompts are tokenized, encoded, and transformed into AI-generated images.
