human-in-the-loop

[jleva12]

Human Review/Approval Step for Agent Tool Execution

Problem Statement

When using AI agents with tool execution capabilities, we currently don't have a way to intercept the tool execution and have a human review before allowing the action to proceed. I need a mechanism to:

1. Pause the agent's execution before the tool is called
2. Display what the agent is about to do to a human user
3. Present an approval UI for the human to accept or reject
4. Continue execution only after approval (or abort if rejected)

Current Limitations

• We can't rely on prompt-based solutions as they're not reliable for this use case
• Need an architectural solution that intercepts tool execution at the system level
• Want to avoid simply terminating the entire run - we need to gracefully pause and resume

Desired Implementation

I'm looking for a pattern/architecture that would allow:

• Agent begins a run and determines it needs to use a tool
• System intercepts the tool call before execution
• Human user receives a notification/message showing the intended action
• A UI component displays the details and provides Accept/Reject buttons
• On Accept: tool executes and agent continues normally
• On Reject: agent is informed the tool call was rejected and can choose alternate approaches

Questions

• Is this possible with the current agent architecture?

• Are there examples of similar implementations?

• What's the recommended approach for intercepting tool calls?

• What's the best way to design the human approval UI flow?

• How should we handle timeouts if a human doesn't respond promptly?

• human-in-the-loop

[rm-openai]

Thanks for the question - please search the closed issues for human in the loop for some ideas, and followup here if you still have questions!

[Cayden96]

Potential Ideas:

1. Embed a UI confirmation flow in the function tool itself, though could become excessive—maybe use an agent to decide when user confirmation is needed? This might be an independent agent that gets called as part of the function, rather than via handoff/agent-as-tool.
2. Ensure that an informative message is returned by the tool that captures confirmation/rejection, so the original agent knows how to handle failure due to user rejection (e.g. asking the user how to proceed or trying a different approach). Ideally you would also capture why a rejection occurred, so the agent doesn't retry the same thing (or conversely, does if rejection is temporary—such as a timeout).

I imagine the flow to be something like: ... -> Function Tool call -> trigger for UI confirmation flow/agent -> Accept=continue; Reject=return error message -> Original agent see's either success or failure output.

Hopefully this helps.

[github-actions]

This issue is stale because it has been open for 7 days with no activity.

[devv-shayan]

is there any update on this human in the loop feature @rm-openai

[vladkolotvin]

The "human-in-the-loop" mechanism would also solve the issue of long-running tool executions. This would enable moving execution outside the main agent loop and processing resource-intensive operations asynchronously. If an application instance restarts, the system could restore state and continue execution after receiving the result, significantly improving reliability when working with resource-intensive tasks.

[yanmxa]

Another way to implement the review/approval step is by using the agent hook on_tool_start. Currently, the tool call is missing parameters needed for the hook, and I'm working on adding them here: #252. Once it's merged, one approach would be to insert the pre-check (human-in-the-loop) step within the hook.