AI VIETNAM
All-in-One Course
(TA Session)
Building LLMs application with
LangChain
Extra Class: LLMs
Dinh-Thang Duong – TA
Nguyen-Thuan Duong – TA
Year 2024
AI VIETNAM
All-in-One Course
(TA Session) Objectives
In this lecture, we will discuss about:
1. What is LLMs in Production?
2. What is Langchain?
3. Basic components of LangChain.
4. How to use LangChain to deploy an
API?
5. How to use LangChain to deploy a
RAG application?
2
AI VIETNAM
All-in-One Course
(TA Session)
Outline
Ø Introduction
Ø LangChain
Ø RAG with LangChain
Ø Question
3
AI VIETNAM
All-in-One Course
(TA Session)
Introduction
4
AI VIETNAM
All-in-One Course
(TA Session) Introduction
v Getting Started
5
AI VIETNAM
All-in-One Course
(TA Session) Introduction
v LLMs size over time
6
[Link]
AI VIETNAM
All-in-One Course
(TA Session) Introduction
v LLMs Applications
7
[Link]
AI VIETNAM
All-in-One Course
(TA Session) Introduction
v Getting Started
Overview of LLMs in Production
8
[Link]
AI VIETNAM
All-in-One Course
(TA Session)
LangChain
9
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v Introduction
LangChain: A framework for developing applications
powered by large language models (LLMs). LangChain
simplifies every stage of the LLM application lifecycle:
Development, Productionization, Deployment.
10
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain usecases
11
[Link]
AI VIETNAM
All-in-One Course
(TA Session)
LangChain
v Introduction
12
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v Introduction
Development: Build your applications using
LangChain's open-source building blocks
and components. Hit the ground running
using third-party integrations and Templates.
13
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v Introduction
LangServe helps developers deploy LangChain
runnables and chains as a REST API. This library is
integrated with FastAPI and uses pydantic for data
validation.
14
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v Introduction
LangSmith: used to inspect, monitor and evaluate
your chains, so that you can continuously optimize
and deploy with confidence (productionization).
15
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangSmith
16
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain Components
17
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain Components
LLM (Large Language Model): The core AI model
responsible for processing natural language and
generating outputs.
18
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain Components
Prompts: Template or structures for crafting
natural language prompts that can be fed into the
LLM.
19
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain Components
Document Loaders and Utils: Tools and utilities
for loading and managing documents or data
sources that can be used as inputs or references
for the LLM.
20
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain Components
Agents: Software agents or programs that can
interact with the LLM and other tools or services to
perform specific tasks or actions.
21
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain Components
22
[Link]
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain Components
Chains: These are sequences or combinations of
calls to the LLM and other components, allowing
for the creation of more complex applications or
workflows.
23
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: How to use
24
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Create an LLM (provider)
25
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Installation
26
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Create an LLM
LangChain LLM
GPT from OpenAI LLM from HuggingFace
27
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Using ChatGPT
28
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Using ChatGPT
API_KEY is required
Using invoke() method to input string to the model.
29
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Create an LLM
LangChain LLM
GPT from OpenAI LLM from HuggingFace
30
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Using LLM from HuggingFace
31
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Create an LLM from HuggingFace
32
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Create an LLM from HuggingFace
33
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Create an LLM from HuggingFace
34
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Create an LLM from HuggingFace
35
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Prompt Template
When using LLM, prompt is very important:
prompt = “hello how are you?”
36
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Prompt Template
In LangChain, we use PromptTemplate: Predefined recipes
for generating prompts for language models.
37
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Prompt Template
In LangChain, we use PromptTemplate: Predefined
recipes for generating prompts for language models.
A template may include:
• Instructions.
• Few-shot examples.
• Specific context and questions appropriate for a
given task.
38
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: List of template
39
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: PromptTemplate
40
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: ChatPromptTemplate
ChatPromptTemplate: A list of chat messages (conversation). 41
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Use PromptTemplate with LLM
Using previous model:
42
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Use PromptTemplate with LLM
Using invoke() method to get the response.
43
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: LLMChain
In practice, we may combine multiple tools, LLMs, modules to solve a task 44
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: LLMChain
In LangChain, we can build complex usage of LLMs by
chaining them using LLMChain.
Chains refer to sequences of calls - whether to an LLM, a
tool, or a data preprocessing step.
45
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: LLMChain
An example of LLMChain including Prompt Template and LLM
46
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Using LLMChain with LLM and PromptTemplate
47
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: Using LLMChain with LLM and PromptTemplate
48
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: OutputParser
User Input LLM “A string”
Sometimes, we might want LLMs to
output a specific data structure.
OutputParser
49
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: OutputParser
Output parsers are classes that help structure language
model responses. There are two main methods an output
parser must implement:
• “Get format instructions”: A method which returns a string
containing instructions for how the output of a language
model should be formatted.
• “Parse”: A method which takes in a string (assumed to be
the response from a language model) and parses it into
some structure.
50
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: OutputParser to JSON
Pydantic Model is a way of defining data
structures with type annotations, ensuring
that the data adheres to a specified format
and type (Data Validator).
51
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: OutputParser to JSON
52
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: OutputParser to JSON
53
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: OutputParser to JSON
We can connect OutputParser to LLMChain
54
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: OutputParser to JSON
We can connect OutputParser to LLMChain
55
[Link]
AI VIETNAM
All-in-One Course
(TA Session) LangChain
v LangChain: OutputParser to JSON
56
AI VIETNAM
All-in-One Course
(TA Session)
RAG with LangChain
57
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Introduction
Description: Serve a LLMs RAG application as an API that receive a simple question and return the response of
the LLMs (utilizing context retrieved from a vector database).
58
[Link]
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Introduction
Our app’s pipeline
59
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Source code structure
- data_source/: Folder containing corpus for retrieval.
- src/base/: Folder containing code to inititalize LLM.
- src/rag/: Folder containing code for RAG.
- [Link]: Python file containing codes for FastAPI app
initialization.
- [Link]: File containing packages version
information to run the soruce code.
60
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 1: Download corpus
61
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 1: Download corpus
Update [Link]:
62
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 1: Download corpus
Update [Link]:
63
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 2: Build the load LLM function
Update src/base/llm_model.py:
64
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 2: Build the load LLM function
Update src/base/llm_model.py:
65
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 3: Build RAG source code
Update src/rag/[Link]:
66
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 3: Build RAG source code
Update src/rag/offline_rag.py:
67
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 3: Build RAG source code
Update src/rag/offline_rag.py:
68
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 3: Build RAG source code
Update src/rag/[Link]:
69
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 3: Build RAG source code
Update src/rag/[Link]:
70
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 3: Build RAG source code
Update src/rag/file_loader.py:
71
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 3: Build RAG source code
Update src/rag/file_loader.py:
72
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 3: Build RAG source code
Update src/rag/file_loader.py:
73
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 3: Build RAG source code
Update src/rag/file_loader.py:
74
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 4: Build API
Update src/[Link]:
75
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 4: Build API
Update src/[Link]:
76
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 4: Build API
Update src/[Link]:
77
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 4: Build API
Update src/[Link]:
78
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 5: Deploy and test API
uvicorn [Link]:app --host "[Link]" --port 5000 --reload
79
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 5: Deploy and test API
80
AI VIETNAM
All-in-One Course
(TA Session) RAG with LangChain
v Step 5: Deploy and test API
81
AI VIETNAM
All-in-One Course
(TA Session)
Quiz
82
AI VIETNAM
All-in-One Course
(TA Session) Summary
In this lecture, we have discussed:
1. What is LLMs in production?
1. How is it differ from LLMs in research?
2. What are some challenges when deploying LLMs in production?
2. Basics of LangChain
1. The key concept of LangChain.
2. Basic components of LangChain: Prompt Template, LLM Chain, Chat History, Document Loader…
3. How to build an API using LangChain.
4. How to build a RAG application using LangChain.
1. Retrieve and answer questions related to Academic Paper.
83
AI VIETNAM
All-in-One Course
(TA Session) Question
?
84
85
