Why AI agents could create a new control and security crisis

Keith Shaw: AI agents are quickly moving from assistants to decision-makers, but they're gaining access to our data, applications, and business processes. What happens when an agent does something nobody expected? Are we building systems we can control, or are we creating new risks that we don't fully understand?

Hi everybody, welcome to Today in Tech. I'm Keith Shaw. Joining me today is Abhinav Asthana, co-founder and CEO of Postman. Welcome to the show, Abhinav.

Abhinav Asthana: Thanks, Keith. Excited to be here.

Keith: Let's jump into this idea of control versus autonomy when it comes to AI agents. A lot of people are focused on capability—what agents can do and how they can improve productivity. How much attention is actually being paid to control?

Abhinav: I think we're still in the honeymoon phase for AI agents. Very little attention is being paid to control right now. Companies are still trying to understand how agents deliver value when moved into production.

A few organizations are at the frontier, but we're only beginning to discover what happens when agents start entering the workforce. As that happens, we're going to learn a lot of new things.

Keith: Do you think we're also starting to lose visibility into what these systems can actually do—or what they're actually doing? Abhinav: Absolutely.

Abhinav: We're seeing something similar to what happened with AI tools generally. As it becomes easier to create and deploy agents, people are building them ad hoc and connecting them directly to production systems through APIs.

Everyone ends up with their own version of the truth and their own data sitting on desktops or local environments. To make these agents work, organizations are also dealing with an unsolved identity and authorization problem.

People are sharing API keys and tokens because agents need access to data and services. These agents often sit on developer machines or lightly managed environments, talking to many different systems with very little visibility or governance.

Keith: Is that because developers are being fast and loose with API keys? Or is it simply too difficult to create proper machine identities?

Abhinav: The movement definitely started with developers because they're always the earliest adopters. But now we're also seeing non-developers build agents through low-code and no-code approaches.

The problem is that there isn't really a paved path for either group to gain access to systems in a way that's designed for an agentic world. Most people are repurposing existing workflows. The challenge is that those workflows assumed humans would always be involved.

For example, a developer might retrieve an API key from a vault, copy it, and store it securely. But when you're plugging hundreds or thousands of agents into systems, you lose that level of oversight. That's why we're seeing a steady stream of API key leaks and data exfiltration incidents.

The process is largely unmanaged.

Keith: Can we talk about the difference between traditional deterministic software and agents? Traditional software follows a defined set of instructions. With agents, you're essentially saying, "Here's the outcome I want. Figure out how to get there." That autonomy seems to create a different set of risks.

Abhinav: That's exactly right. The fundamental definition of an agent is that it can use tools. It can browse the web, operate a computer, call APIs, and interact with systems. The more access you give it, the more power it has.

The underlying large language models are designed to satisfy the user. Their goal is to help Keith or Abhinav achieve an objective. They naturally lean toward accomplishing the task.

As a result, if you give an agent access to a browser, APIs, or systems, it will try to use those tools to achieve its goal. That's where the challenge begins. Enterprise customers, especially in regulated industries, want to sandbox agents as much as possible.

But the more restrictions you add, the more you potentially reduce the value and capability of the agent. Finding that balance is one of the industry's biggest challenges right now.

Keith: When you talk to companies deploying agents, where are the first cracks appearing?

Abhinav: Most companies haven't reached full autonomy yet. The first challenges appear in research and analysis use cases. We've experienced this ourselves. Since agents are trying to help, they often search for information that supports the objective they've been given. That means organizations need a vetted source of truth.

Whether it's product strategy, sales planning, or marketing content, the agent needs access to information that humans have already validated. The agent is only as good as the information it's given.

If it pulls from outdated information—say a company strategy document from three years ago—you risk training employees and making decisions based on stale data. That's where guardrails become essential. Then you reach the next level: autonomy. A chatbot recommending a refund is one thing. Actually issuing the refund is another.

That's where the industry is still figuring out policies, controls, and governance.

Keith: Can you give an example where an agent did something unintended—not malicious, but simply misunderstood the assignment?

Abhinav: One of the first agents we built became popular internally. Someone asked it, "I'd like to contribute to improving you. How do I do that?" The agent completely invented its development process. It fabricated repository names, project structures, and contribution workflows that didn't exist.

It was simply trying to be helpful and provide an answer. That's a perfect example of an agent pleasing the user instead of providing accurate information.

Keith: Are these mostly experimental issues, or are companies starting to encounter them in production?

Abhinav: The technology is advancing quickly. One important development is the adoption of the Model Context Protocol (MCP) as a standard way to control what agents can access and do. MCP effectively becomes a policy layer. Companies can use it to enforce restrictions independent of the agent itself.

For example, a company might decide that AI agents can never access a production database. Every agent request must pass through a gateway that enforces that rule. Beyond that, organizations are improving evaluation frameworks, testing agent behavior over long conversations, and building infrastructure to enforce hard boundaries.

If you're automating workflows with agents, you need evaluation systems and operational guardrails before deployment.

Keith: If you were building an agent from scratch, would you start by limiting its access or giving it broad access and then restricting it later?

Abhinav: Personally, I'm an AI maximalist when it comes to development. If you constrain agents too much, you lose the productivity gains that make the technology valuable. But that doesn't mean giving them direct access to everything. The approach that works best is simulation. Think about how autonomous vehicles learn.

Companies like Waymo use digital representations of the real world so vehicles can practice safely. The same principle applies to agents. Give them realistic simulations and safe environments. Let them explore broadly there. Then, as confidence grows, gradually connect them to real systems.

Keith: Is it actually possible to control an agent completely, or will there always be some uncertainty?

Abhinav: It's a deep question. I see three layers of control. First, use an independent agent whose job is to challenge or audit the primary agent. Second, build deterministic infrastructure controls. The policies that govern what an agent can and cannot do should exist outside the agent itself.

Third, keep humans involved. I think we're going to see a new role emerge: agent managers. These people will monitor agent behavior, perform spot checks, and continuously improve systems. Those three layers together provide meaningful oversight.

Keith: What happens when multiple agents start interacting with one another?

Abhinav: The complexity grows quickly. One agent's hallucination can become another agent's source of truth, and then the problem compounds. We've seen this internally. We had one sales agent evaluating demo quality and another evaluating customer needs. The demo agent concluded, "Great demo.

Customer is likely to buy." The discovery agent concluded the opposite. It found evidence that the customer specifically didn't want what was being demonstrated. If those agents don't communicate effectively—or if nobody reconciles the differences—you end up making bad decisions downstream.

Organizations will need structures and governance models for agents, just as they have organizational structures for humans.

Keith: That sounds similar to how humans sometimes disagree about the same meeting. Abhinav: Exactly.

The bigger issue becomes who is designing these agents and what goals they're optimizing for. If teams build agents independently without a coordinated strategy, you'll create confusion. In our case, we introduced an orchestrator agent whose job is to focus on the larger objective: improving the sales pipeline.

Individual agents may disagree, but a higher-level system can reconcile those differences.

Keith: Let's talk about accountability. If something goes wrong, who owns the outcome?

Abhinav: It depends on what you're buying. If a vendor is selling outcomes, then accountability should follow those outcomes. If you're buying raw intelligence or infrastructure, that's different. Electricity can power many things, but you don't blame the power company for every outcome.

The market is still figuring out where responsibility ultimately resides.

Keith: Are vendors actually taking accountability today?

Abhinav: It's still fuzzy. Many vendors charge based on credits or usage and leave responsibility with the customer. If a vendor is promising to replace a support representative or engineer and deliver a specific outcome, then accountability should be clearly defined in the contract.

Keith: How are developer roles changing as companies move into agent-based systems?

Abhinav: Developers are spending much more time understanding business context. They're writing more English than code in many cases because prompting and instructions have become critical. Traditional engineering skills still matter, but the most valuable developers increasingly understand how agents operate within the business itself. That's becoming a major differentiator.

Keith: Are there skills that matter less now than they did a decade ago?

Abhinav: For infrastructure developers, the fundamentals still matter. Systems need reliability, resilience, and quality. But for newer developers, communication, business understanding, and rapid iteration are becoming much more important. Development cycles that once took months can now happen in days.

We're seeing the designer, product manager, and engineer roles begin to merge into what some are calling the "product builder." That's what the future role may look like.

Keith: To wrap up, what message would you give companies that might be moving too fast with AI agents? And do you think many companies are moving too fast?

Abhinav: AI is here, and companies need to move quickly. Standing still creates its own existential risk. But there's a right way and a wrong way to move fast. Many organizations have simply unleashed AI tools and agents without a coherent strategy. Productivity won't happen automatically.

Just as you can have confused employees, you can have confused agents. You need a coordinated strategy, centralized governance, and clear objectives. At the same time, companies that haven't invested in infrastructure should be aware of the risks. Don't wait for a major incident to expose weaknesses.

Moving fast is important, but a catastrophic failure can create brand damage that slows you down even more.

Keith: Do you think AI will expose those weaknesses more than previous technologies? Abhinav: Absolutely.

AI is being used by everyone, not just the good actors.

Keith: Abhinav Asthana, co-founder and CEO of Postman, thanks for joining us and sharing your insights.

Abhinav: Great chatting with you.

Keith: That's going to do it for this week's show. Be sure to like the video, subscribe to the channel, and share your thoughts in the comments below. Join us every week for new episodes of Today in Tech. I'm Keith Shaw. Thanks for watching.