Together AI ThunderAgent launched at AI Native Conf as an open-source system that treats agentic workflows as LLM programs rather than isolated API calls. The result: 1.5–3.6x throughput improvements in serving, 1.8–3.9x gains in reinforcement learning rollout, and up to 4.2x disk memory savings. By abstracting agent workflows at the program level, ThunderAgent optimizes resource allocation in ways that single-request serving frameworks cannot. For teams running agents at scale, this is infrastructure that was badly needed.

But there is a foundational layer that ThunderAgent, like every inference optimization, still depends on getting right — memory.

Together AI ThunderAgent: What Everyone's Getting Right (And Missing)

ThunderAgent gets infrastructure efficiency right. Program-aware serving means the system understands the full agentic workflow — not just individual LLM calls — and can allocate compute, schedule requests, and manage resources at the workflow level. This eliminates redundant computation and dramatically improves throughput for multi-step agent tasks.

What ThunderAgent does not address is what happens across workflows. Each agentic program executes faster, but it still starts without knowledge of previous executions. The agent that analyzed ten thousand data points yesterday runs the same exploratory queries today. ThunderAgent optimizes how fast agents run, not how much they retain.

Other inference optimization frameworks — vLLM, TensorRT-LLM, SGLang — share this same architectural constraint. They make agents faster; none of them make agents wiser.

The MemU Agentic Memory Framework: Persistent Intelligence for High-Throughput Agents

The MemU Agentic Memory Framework adds the missing layer. Where ThunderAgent manages how fast agents execute, MemU manages what agents carry forward.

Consider a research pipeline running hundreds of agent invocations per hour. With MemU, each invocation starts with knowledge of what previous runs discovered — which sources were valuable, which queries returned noise, what patterns emerged. Without MemU, every invocation explores blindly, wasting the throughput ThunderAgent worked hard to deliver.

The MemU Agentic Memory Framework provides:

• Drop-in integration: A simple API that works alongside any serving infrastructure — ThunderAgent, vLLM, or custom deployment. Add memory read/write calls; your high-throughput agents gain cumulative intelligence.
• Dual-mode retrieval: Semantic search for finding relevant past executions plus a structured memory graph for tracking causal relationships between agent decisions and outcomes. Not just embeddings — actual execution genealogy.
• Cross-execution persistence: Memory survives across agent runs, scaling tiers, and infrastructure changes. Throughput improvements compound when agents stop repeating work they have already done.

Speed without memory is redundant execution at higher throughput. The MemU Agentic Memory Framework ensures that faster agents are also smarter agents — every execution builds on the last.

Retrieval operates across 10,000+ memory entries with sub-100ms latency, adding negligible overhead to the throughput gains ThunderAgent delivers.

Head-to-Head: ThunderAgent Alone vs. ThunderAgent + MemU

ThunderAgent alone: Agentic workflows execute 3.6x faster. But faster execution of redundant work is still wasted compute. The thousandth data analysis run carries the same starting context as the first — it just gets there quicker.

ThunderAgent + MemU Agentic Memory Framework: Each fast execution reads from persistent memory. The agent skips exploration it has already completed, focuses on genuinely new territory, and writes discoveries back for future runs. Throughput multiplied by memory equals exponential productivity gains.

RL rollout intelligence: ThunderAgent delivers 3.9x improvement in RL rollout throughput. With MemU, those rollouts accumulate strategic memory — which exploration paths were dead ends, which reward signals were reliable. Faster rollouts plus persistent strategy memory means RL pipelines that genuinely converge rather than repeatedly rediscovering the same territory.

Empowering ThunderAgent: Better Together

MemU does not replace ThunderAgent — it makes ThunderAgent dramatically more productive:

• Research pipelines: ThunderAgent serves hundreds of parallel agent requests efficiently; MemU ensures each request builds on findings from previous runs rather than starting from scratch.
• Customer interaction agents: ThunderAgent handles high-concurrency customer conversations; MemU gives each conversation access to the full history of customer preferences and past interactions.
• Code generation at scale: ThunderAgent powers fast multi-step coding agents; MemU remembers project conventions, past bugs, and architectural decisions so generated code improves with every session.

Get Started with MemU

Add persistent memory to your high-throughput agent infrastructure in minutes. The MemU Agentic Memory Framework works with any serving layer — one API, zero lock-in, immediate intelligence gains. Visit memu.pro to explore the Agentic Memory Framework API, or check out the GitHub repository to start building agents that remember.

Tags: Together AI, ThunderAgent, agentic throughput, AI inference optimization, agentic memory, LLM memory, MemU AI