Building a RAG Application

Last Updated : 23 Sep, 2026

Retrieval-Augmented Generation (RAG) combines information retrieval with text generation to produce answers using relevant external information. A retriever finds useful documents from a knowledge base and a generator uses those documents along with the user's query to generate an answer.

Building a Customer Help Bot

We will build a RAG-based chatbot that answers customer queries using a small knowledge base.

The workflow is:

User Query -> Document Retrieval -> Context Creation -> Answer Generation

For example:

When a user asks "How can I track my order?", the retriever finds the relevant information from the knowledge base and passes it to the language model to generate the answer.

Note: The Amazon-related information used in this example is illustrative and is included only to demonstrate the RAG workflow. Actual policies and features may change.

Steps to Build a RAG Application

Step 1: Install Required Libraries

We use Sentence Transformers to create embeddings, FAISS for similarity search and Transformers to load the text-generation model.

Python 
pip install sentence-transformers faiss-cpu transformers torch

Step 2: Import Libraries

Python 
from sentence_transformers import SentenceTransformer
import numpy as np
import faiss
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

Step 3: Create a Knowledge Base

The knowledge base contains documents with information that the chatbot can use to answer questions.

Python 
documents = [
    "To track your Amazon order, log into your account, go to 'Your Orders,' and click 'Track Package' for real-time updates.",
    "Amazon's return policy allows most items to be returned within 30 days of delivery for a full refund, provided they are in new condition with original packaging and accessories.",
    "To return an Amazon order, initiate a return through 'Your Orders,' ship the item back, and receive a refund once processed.",
    "To contact Amazon customer service, use the 'Help' section on the website or app to chat, call, or email support.",
    "Amazon Prime members receive free two-day shipping, exclusive deals, and access to Prime Video and Music.",
    "If your Amazon package is delayed, check the estimated delivery date in 'Your Orders' or contact customer service for assistance.",
    "To cancel an Amazon order, go to 'Your Orders,' select the order, and click 'Cancel Items' if it hasn't shipped yet.",
    "To purchase an Amazon gift card, visit the Amazon website, navigate to 'Gift Cards,' select a design and amount, add to cart, and complete the purchase at checkout; the gift card can be redeemed for eligible products.",
    "To update your Amazon payment method, go to 'Your Account,' select 'Your Payments,' and add or edit your card details.",
    "To log into your Amazon account, go to the Amazon website or app, click 'Sign In,' and enter your email or phone number and password."
]

Step 4: Generate Document Embeddings

We convert each document into a numerical vector called an embedding. These embeddings capture the semantic meaning of the documents, allowing us to find documents that are similar to a user's query.

Python 
embedding_model = SentenceTransformer("all-MiniLM-L6-v2")

doc_embeddings = embedding_model.encode(
    documents,
    normalize_embeddings=True
).astype("float32")

Here, normalize_embeddings=True normalizes the vectors so that their inner product can be used as cosine similarity.

Step 5: Create a FAISS Index

FAISS is used to efficiently search for documents with embeddings similar to the user's query.

Python 
dimension = doc_embeddings.shape[1]

index = faiss.IndexFlatIP(dimension)
index.add(doc_embeddings)

IndexFlatIP uses inner-product similarity. Since the document embeddings are normalized, this is equivalent to cosine similarity.

Step 6: Load the Generative Model

  • We use FLAN-T5 to generate an answer from the retrieved context.
  • The current Transformers API can load the model directly using AutoTokenizer and AutoModelForSeq2SeqLM.
Python 
model_name = "google/flan-t5-base"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

flan-t5-base is larger than flan-t5-small and generally produces more complete answers, although it requires more memory and may run more slowly.

Step 7: Retrieve Relevant Documents & Generate an Answer

The following function performs the main RAG process:

  1. Converts the query into an embedding.
  2. Searches FAISS for similar documents.
  3. Keeps only documents above the similarity threshold.
  4. Combines the retrieved documents into context.
  5. Sends the context and query to FLAN-T5.
  6. Generates a concise answer.
Python 
def rag_answer(query, top_k=2, threshold=0.3):
    query_embedding = embedding_model.encode(
        [query],
        normalize_embeddings=True
    ).astype("float32")

    distances, indices = index.search(query_embedding, top_k)

    if distances[0][0] < threshold:
        return None, "Sorry, I couldn't find a relevant document to answer your question."

    retrieved_docs = [
        documents[i]
        for i in indices[0]
        if i != -1
    ]

    context = "\n".join(retrieved_docs)

    prompt = f"""
Answer the question using only the information in the context.

Context:
{context}

Question:
{query}

Answer:
"""

    inputs = tokenizer(
        prompt,
        return_tensors="pt",
        truncation=True
    )

    outputs = model.generate(
        **inputs,
        max_new_tokens=50,
        do_sample=False,
        num_beams=4
    )

    response = tokenizer.decode(
        outputs[0],
        skip_special_tokens=True
    ).strip()

    return context, response

The threshold prevents documents with very low similarity from being included in the context. num_beams=4 uses beam search to improve the quality of the generated response.

Step 8: Run the Q&A Bot

Now, we can create an interactive loop that accepts questions from the user.

Python 
def run_qa_bot():
    print("Welcome to the RAG Q&A Bot! Ask a question or type 'exit' to quit.")

    while True:
        query = input("User: ")

        if query.lower() == "exit":
            print("Goodbye!")
            break

        if not query.strip():
            print("Please enter a valid question.")
            continue

        context, answer = rag_answer(query)

        print(f"HelpBot: {answer}\n")


run_qa_bot()

Output:

Screenshot-2026-09-14-092411

You can download the complete source code from here.

Applications

  • Customer Support: Answer questions using product manuals, FAQs and support documents.
  • Document Question Answering: Retrieve information from reports, PDFs and other documents.
  • Knowledge Assistants: Provide answers based on an organization's internal knowledge base.
  • Research Tools: Retrieve relevant information from large collections of documents.
  • Enterprise Search: Combine semantic search with natural-language answers.

Challenges

  • Retrieval quality: If the wrong documents are retrieved, the generated answer may also be incorrect.
  • Data quality: Outdated or incorrect documents can lead to poor answers.
  • Latency: Retrieval and generation add processing time.
  • Context limits: Passing too much retrieved information can make generation less effective.
  • System complexity: A complete RAG system requires both a retrieval component and a generative model.
Comment

Explore