{"id":114188,"date":"2026-03-16T13:30:00","date_gmt":"2026-03-16T20:30:00","guid":{"rendered":"https:\/\/2.zoppoz.workers.dev:443\/https\/developer.nvidia.com\/blog\/?p=114188"},"modified":"2026-04-16T10:15:40","modified_gmt":"2026-04-16T17:15:40","slug":"scaling-autonomous-ai-agents-and-workloads-with-nvidia-dgx-spark","status":"publish","type":"post","link":"https:\/\/2.zoppoz.workers.dev:443\/https\/developer.nvidia.com\/blog\/scaling-autonomous-ai-agents-and-workloads-with-nvidia-dgx-spark\/","title":{"rendered":"Scaling Autonomous AI Agents and Workloads with NVIDIA DGX Spark"},"content":{"rendered":"\n

Autonomous AI agents<\/a> are driving the next wave of AI innovation. These agents must often manage long-running tasks that use multiple communication channels and background subprocesses simultaneously to explore options, test solutions, and generate optimal results. This places extreme demands on local compute.<\/p>\n\n\n\n

NVIDIA DGX Spark<\/a> provides the performance necessary for autonomous agents to execute these complex workflows efficiently and locally. Now with NVIDIA NemoClaw<\/a>, part of the NVIDIA Agent Toolkit, it installs the NVIDIA OpenShell runtime\u2014a secure environment for running autonomous agents, and open source models like NVIDIA Nemotron<\/a>.<\/p>\n\n\n\n

This post discusses several important aspects of system capabilities and performance that are necessary to power always-on autonomous agents and explains why NVIDIA DGX Spark is an ideal desktop platform for autonomous AI.<\/p>\n\n\n\n

Inference for autonomous AI agents<\/strong> <\/i><\/a><\/h2>\n\n\n\n

Agentic tools often need to process massive context windows. OpenClaw, for example, is an AI agent runtime that requires these large context windows to comprehend requests and environments, and to think through the best approach to a problem. <\/p>\n\n\n\n

Prompt processing (prefill) throughput can be thought of as the reading comprehension phase of inference and can easily become a bottleneck with a slow GPU. It\u2019s common to see autonomous agents easily using contexts of 30K-120K tokens (100K tokens is equivalent to reading Harry Potter and the Philosopher’s Stone<\/em>), with some agents processing 250K tokens for complex requests. <\/p>\n\n\n\n

Table 1 shows how a potential agent or subagent performs with a large context window, (128K\/1K of ISL\/OSL). <\/p>\n\n\n\n

Model <\/strong><\/td>End-to-end  latency <\/strong>
(s)<\/strong><\/td>		Prompt processing latency<\/strong>
(s) <\/strong><\/td>		Prompt processing throughput <\/strong>
(tok\/s)<\/strong><\/td>		Token generation throughput <\/strong>
(tok\/s)<\/strong><\/td><\/tr>
NVIDIA Nemotron 3 Super 120B  NVFP4 with TensorRT LLM <\/td>99<\/td>44<\/td>2,855<\/td>18<\/td><\/tr>
Qwen3.5 35B A3B FP8 with vLLM <\/td>73 <\/td>41 <\/td>3,080 <\/td>35.75 <\/td><\/tr>
Qwen3 Coder Next 80B FP8 with vLLM <\/td>89 <\/td>54 <\/td>2,390 <\/td>28.95 <\/td><\/tr><\/tbody><\/table>
Table 1. Performance representative of 128K tokens input prompt and response of 1K tokens, at batch size 1 <\/em><\/em><\/figcaption><\/figure>\n\n\n\n

 <\/em>When moving from a single subagent to multiple subagents, simultaneous workloads must scale without impacting performance significantly. NVIDIA DGX Spark effectively handles high concurrency in this scenario.<\/p>\n\n\n\n

Thanks to the power of the NVIDIA Grace Blackwell Superchip<\/a>, the GPU can parallelize multiple subagents. Two, four, or even eight subagents concurrently working through requests can make use of the strong concurrency capabilities in DGX Spark. <\/p>\n\n\n\n

With support from frameworks that handle concurrency well (such as NVIDIA TensorRT LLM<\/a>, vLLM, and SGLang), multiagent workloads run smoothly on NVIDIA DGX Spark. For tasks with 32K ISL of 1K OSL, completing four times as many tasks requires only 2.6x more time, while prompt processing throughput increases by about 3x (Table 2).<\/p>\n\n\n\n

NVIDIA DGX Spark is an ideal platform for OpenClaw development. With NVIDIA OpenShell, you can run autonomous, self-evolving agents more safely. Get started running OpenClaw locally on NVIDIA DGX Spark.<\/a><\/p>\n\n\n\n

Concurrency<\/strong>
(# of simultaneous tasks) <\/strong><\/td>		End-to-end latency <\/strong>
(s)<\/strong><\/td>		Median TTFT <\/strong>
(s)<\/strong><\/td>		Prompt processing throughput <\/strong>
(tok\/s)<\/strong><\/td>		Token generation throughput <\/strong>
(tok\/s)<\/strong><\/td><\/tr>
<\/td>Lower is better<\/strong><\/td>Higher is better<\/strong><\/td><\/tr>
1 <\/td>35<\/td>9 <\/td>3,261 <\/td>38 <\/td><\/tr>
2 <\/td>54 <\/td>12 <\/td>5,363 <\/td>47 <\/td><\/tr>
4 <\/td>91 <\/td>15 <\/td>9,616 <\/td>53 <\/td><\/tr><\/tbody><\/table>
Table 2. Performance representative of Qwen3 Coder Next in FP8 in vLLM for a 32K tokens input prompt and response of 1K tokens at different concurrency levels<\/em><\/em><\/figcaption><\/figure>\n\n\n\n

Scale inference and fine-tuning on up to four NVIDIA DGX Spark nodes<\/strong><\/i><\/a><\/h2>\n\n\n\n

Larger models and multiple subagents require more memory to load and execute. Until now, NVIDIA DGX Spark has supported scaling up to two nodes, increasing the available memory from 128 GB on one node to 256 GB on two nodes. This capability has now been increased to up to four DGX Spark nodes. <\/p>\n\n\n\n

DGX Spark also now supports several execution topologies, each tailored to different goals through the low latency of RoCE communication enabled by ConnectX-7 NICs<\/a>. <\/p>\n\n\n\n