Building Robust, Secure LLM
and Agentic AI Workflows
APRIL 2025
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2025. All rights reserved.
Shivay Lamba, Developer Evangelist @ Couchbase
Docker Captain
CNCF Ambassador
@howdevelop on X
APRIL 2025
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2025. All rights reserved.
Scan this qr code to get some special swag at our booth
Booth number: 26
APRIL 2025
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2025. All rights reserved.
The Dynamic AI Shift in last 4 years
2022 2023 2024 2025
Vectors, RAG,
Launch of
Rise of LLMs Multi-modal AI Agents
ChatGPT
LLMs
• 30 November - • Launch of open • Vector search • AI Agents
Launch of and closed • Rise of RAG • MCP, tool
ChatGPT source LLMs • More powerful calling
LLMs with
reasoning
capabilities
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 4
A quick recap on
AI Agents
5
Enterprises are Excited about AI & Agents
They can perform complex, autonomous tasks that save time, reduce costs, and drive
innovation.
?
Why Customers Care Surging Interest & Investment
• Enhanced customer experience • Devs: 3X since Chat GPT launch
• Innovation and agility • Executives: 96% agree its a significant
• Increased efficiency opportunity in the next three years
• Improved decision-making • Past year, 2X in spending on AI and ML efforts
• Cost reduction • Fastest of spend across all tech categories
• Scalability
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 6
But, What are AI Agents?
• Autonomous
• Automate complexity • Reasoning logic
Goal What it Does
• Interact and iterate • Perform many tasks
• Natural language • Multiple tools / data
End User Benefit How it Works
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 7
A Simple Example: Travel Planner Agent
Planning, orchestrating and iterating to perform multiple tasks
Jane Trip Agent
"I'm looking for a trip from SFO to Tokyo in April for the Cherry Blossom Festival."
"Great! Do you have specific dates or preferences for your trip?" task
task
task
"April 10th to April 20th, and I love exploring cultural sites and nature."
"I found a flight from SFO to Tokyo on April 10th for $700, returning on April 20th for $680. Would
you like to book these flights?" task
task
task
"Yes, please book them."
"Done! Your flights are booked. I recommend Ueno Park and Shinjuku Gyoen National Garden for
cherry blossoms, and exploring the Edo-Tokyo Museum for a cultural experience. Would you like task
task
task
me to book a hotel near these sites?"
"Yes, that would be perfect."
"I found a hotel in central Tokyo close to these for $150 per night. Shall I book it?" task
task
task
"Yes, please."
"Your hotel is booked. Would you like a rental car as well?" task
task
task
"No, public transport should be fine."
"Great! Your itinerary includes visits to Ueno Park, Shinjuku Gyoen National Garden, and the
Edo-Tokyo Museum. Estimated costs are $200 for activities. Is there anything else I can assist you
task
task
with?" task
"No, that's all. Thank you!"
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 8
How Does an Agent Work?
It uses tools and prompts to interact between users and
Agent LLMS, completing multiple tasks using LLM reasoning
• Tools
○ Define data sources and processes
○ Action data interaction
Agent Agent • Prompt Templates
tools Memory and ○ Predefined by developer
Logging ○ Suggest tools to used
• LLMS
○ Analyzing and reason based on prompts and data
○ Generating natural language responses
Prompt
• Agent memory and logging
templates
○ Track user session, tool & data interaction, to
maintain guardrails, versioned and
change-controlled over time
■ That’s a lot of new JSON data
○ Allows for improving agent over time
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 9
Building robust
Agentic
Workflows
10
Agents and Workflows
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 11
What is an Agentic Workflow?
An agentic workflow offers an iterative and collaborative
model, transforming interactions with LLMs into a series of
manageable, refinable steps. This approach enables
continuous improvement and adaptation throughout the
task completion process. A system is more “agentic” the
more an LLM decides how the system can behave.
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 12
Implementing agentic workflows with LangGraph
LangGraph is an open-source framework designed for
creating agent and multi-agent applications.
While LangChain facilitates the creation of Directed Acyclic
Graphs (DAGs) for linear workflows, LangGraph advances this
by allowing the incorporation of cycles thus enabling agent
orchestration
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 13
Components for LangGraph
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 14
A Typical Agentic Architecture with LangGraph
Agent Loop Backends
RDBMS
Prompts
LLM Vector DB
Tools
NoSQL
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 15
A Production-Ready Typical Agentic Architecture…
Agent Loop Backends
RDBMS
Prompts
LLM Vector DB
Tools
Tools
Tools NoSQL
Tools
Tools
Tools
Tools
Tools
Tools
Tools
Tools
Volume & Variety of Tools Tools
Tools
Tools
Tools
Tools
Tools
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 16
A Production-Ready Typical Agentic Architecture…
Prompt Selection
Prompts
Prompts
Agent Loop Prompts Backends
Prompts
Prompts
Prompts
Prompts RDBMS
Prompts
LLM Vector DB
Tools
Tools
Tools NoSQL
Tools
Tools
Tools
Tools
Tools
Tools
Tools
Tools
Tools
Tool Authentication & Selection Tools
Tools
Tools
Tools
Tools
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 17
A Production-Ready Typical Agentic Architecture…
Agent Loop Backends
RDBMS
Prompts
LLM Vector DB
Tools
NoSQL
Monitor Alert Optimize
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 18
What AGENTC Brings to the Table
Dynamic Prompt & Tools Selection &
Agent Loop Versioning
Database as a Backend
Tools
Tools Tools
Tools Tools
Tools Tools
Prompts Tools
Placeholder Tools SQL
… Tools Tools Vector DB NoSQL
Tools Query
Tools
LLM Tools Tools
Tools Tools
Tools Tools
Tools
Placeholder … Tools Tools
Tools Tools Logs
Tools Tools
Prompts Tools
Tools
Tools
Tools
Logs
Persistence
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 19
Let’s look at a demo!
20
Implementing Zero Trust Security for
LLM/Agentic Workflows
21
Recent security incidents involving large language models
The Truffle security team used the open-source tool
TruffleHog to scan 400 TB of December 2024 data
(covering 2.67 billion web pages from 47.5 million hosts) in
Common Crawl, a crawler database. The scan results
showed that approximately 11,908 valid DeepSeek API
keys, credentials, and authentication tokens were
hard-coded into a large number of Web pages.
Malicious user bvk uploaded two malicious python packages
(deepseek and deepseekai) to PyPI for the first time.
OmniGPT
data sold on
BreachForums
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 22
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 23
genai.owasp.org
Zero Trust for LLMs means, no implicit
trust — anywhere.
Every prompt, every model, every output is verified, monitored,
and logged.
25
Implementing Security for LLMs
i. Prompt Injection:
○ Disguise malicious inputs as legitimate prompts, manipulating
generative AI systems GenAI) into leaking sensitive data, spreading
misinformation, or worse
○ Mitigation: Use tools like Rebuff (heuristic + LLM-based detection) or
Garak (vulnerability scanning)
Implementing Security for LLMs
i. Prompt Injection:
ii. Runtime Security:
○ Monitor API traffic for anomalies (e.g., BurpGPT)
○ Perform sophisticated technical tasks; such as evaluating
cryptographic integrity of custom libraries or even detect zero-days.
Implementing Security for LLMs
i. Prompt Injection:
ii. Runtime Security:
iii. Model Artifacts:
○ Sign models cryptographically;
○ Ensure that model artifacts are always signed before getting pushed to
OCI compliant registries.
○ Use federated learning for sensitive data
Implementing Security for LLMs
i. Prompt Injection:
ii. Runtime Security:
iii. Model Artifacts:
iv. Auth:
○ Implement OAuth2/OIDC with fine-grained RBAC (e.g., Azure API
Management)
○ Fine Grained Authorization - Relationship-based access control
ReBAC) is particularly well-suited for fine-grained authorization in
Retrieval-Augmented Generation RAG
Implementing Security for LLMs
i. Prompt Injection:
ii. Runtime Security:
iii. Model Artifacts:
iv. Auth
v. Personally Identifiable Information PII Protection:
○ Use differential privacy or tokenization (e.g., CalypsoAI)
○ Use the open source library LLM Guard for run-time security checks in
conjunction with Langfuse to Anonymize data and trace the application
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 31
Capella AI Services | Building Blocks for Agentic Applications
Build agentic RAG applications with control over structured and unstructured data,
simplified workflows, and AI models in a single, secure platform
Agent ● Metadata for tool, prompt, LLM interactions ● Multi agent system collaboration
Catalog ● Security, Governance and reuse ● Debug, traceability and explainability
Model Unstructured Vectorization AI
Service Data Service Service Functions
Models and data together Chunking, convert to JSON, Automatic embedding, storing and AI extension to SQL
vectorize indexing of ingested data
LLMs, SLMs, VLMs, Guard and Call LLMs with SQL queries
Hallucination models Single source of record for Flexible use cases across hybrid
vectorized structured and search, composite and columnar Sentiment, summarization,
Semantic and Conversational unstructured data translation, entity extraction, etc
caching
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 32
Slide Deck
33
Any questions?
Do check out our booth! No. 25
We are running contests and giving out some cool swag!
34
Getting Started with Couchbase is Easy
Community Couchbase Discord Capella Free Tier
Channel
Thank you!
[email protected] @howdevelop
36
