October 10, 2026

Build agent memory with NVIDIA NeMo Agent Toolkit and Amazon S3 Vectors

← AI for Your Work

WebCrimp analysis and practical explanation. AI-assisted; reviewed before publication.

Quick Summary

This guide explains how to implement Amazon S3 Vectors as a custom memory provider within the NVIDIA NeMo Agent Toolkit (NAT) for multi-agent AI systems. It covers the architecture of NAT's memory module, the implementation of a custom MemoryEditor plugin backed by S3 Vectors, and deployment on Amazon EKS. The approach enables persistent, scalable, and semantically retrievable agent memory using vector databases, supporting multi-agent collaboration through shared long-term memory.

What Happened?

The implementation involves three main steps: first, creating an Amazon S3 Vectors infrastructure by provisioning a vector bucket and index optimized for high-dimensional embeddings (1024 dimensions matching Amazon Titan Text Embeddings V2); second, implementing a custom MemoryEditor plugin (S3VectorsMemoryEditor) that integrates with NAT's memory subsystem to generate embeddings, store vectors, and perform semantic searches; third, configuring NAT's YAML configuration to use the custom memory provider and enabling the auto_memory_agent workflow for seamless memory capture and retrieval across agents.

Why It Matters

Persistent memory is foundational for production multi-agent systems because it enables agents to retain context, learn from past interactions, and coordinate across sessions. Amazon S3 Vectors provide semantic retrieval, rich metadata filtering, strong write consistency, and elastic scaling to billions of vectors—capabilities essential for multi-agent environments where agents must share and reference historical knowledge. Using NAT's extensible memory architecture allows teams to plug in custom backends like S3 Vectors while maintaining compatibility with other frameworks such as Strands Agents, LangChain, and CrewAI.

Who Can Use It?

AI/ML · Software Development · DevOps · Cloud · Data Analytics · Finance · Business · Accounting · Research · Operations

How Can I Use This?

If you work in AI/ML, Software Development, DevOps, Cloud, Data Analytics, Finance, Business, Accounting, Research, Operations

  • To deploy this solution, set up an AWS account with S3 Vectors and EKS permissions, install NVIDIA NeMo Agent Toolkit v1.6+, and ensure Python 3.11+. Create an S3 Vectors bucket and index with 1024 dimensions, then implement the S3VectorsMemoryConfig and S3VectorsMemoryEditor classes as shown in the code examples. Register the plugin via the builder in your NAT configuration, enable the auto_memory_agent workflow with appropriate memory and function definitions, and test the integration by having agents store and retrieve memories using the memory client. For the investment research use case, configure agents to pull shared findings and team-wide knowledge through metadata filters (e.g., team_id, agent_id) to avoid redundant computations and improve reasoning coherence.

AI Tip

When implementing custom memory providers, always validate that your embedding model outputs compatible dimensionality with the target vector database—here, 1024 dimensions matched Titan Text Embeddings V2—to ensure efficient indexing and accurate similarity search.

AI Trick

Use the MemoryItem metadata fields strategically: populate user_id, agent_id, team_id, and confidence scores to enable granular filtering and traceability in multi-agent contexts, which reduces hallucination and improves task alignment.

Example Workflow

  1. Step 1: Provision S3 Vectors infrastructure (create bucket and index with 1024 dims, cosine distance metric, non-filterable content metadata). Step 2: Implement the S3VectorsMemoryConfig and S3VectorsMemoryEditor classes, embedding text via Amazon Titan Text Embeddings V2, storing vectors with scoped metadata, and exposing add_items, search, and remove_items methods. Step 3: Register the plugin with NAT's builder and update config.yml to point to the custom memory provider. Step 4: Deploy the NAT agent workflow with auto_memory_agent enabled, linking memory storage to agent functions (add_memory, get_memory, save_user_messages_to_memory). Step 5: Test by having agents interact, verify memory persistence across sessions, and refine filters for performance.

Copy This Prompt

Implement Amazon S3 Vectors as a custom memory provider for the NVIDIA NeMo Agent Toolkit. Create a S3VectorsMemoryConfig class extending MemoryBaseConfig with vector_bucket, index_name, and aws_region. Implement an S3VectorsMemoryEditor that generates embeddings using Amazon Titan Text Embeddings V2, stores vectors in the specified S3 Vectors index, and supports semantic search with filters on agent_id, memory_type, ticker, team_id, and is_shared. Register the plugin via the builder and integrate it into a NAT agent workflow using the auto_memory_agent wrapper. Ensure the implementation handles metadata truncation (limit content to 1024 chars) and respects per-tenant isolation. Example use case: multi-agent investment research where agents share long-term memory indexed under a team_id.

Recommended Tools

  • boto3
  • Python 3.11+
  • NVIDIA NeMo Agent Toolkit
  • Amazon EKS
  • Docker
  • kubectl
  • AWS CLI / credentials

Check official documentation, availability and pricing before using these suggestions.

Difficulty

Intermediate

Estimated Time Saved

Not measured; validate with your own workflow.

Source

AWS AI
Build agent memory with NVIDIA NeMo Agent Toolkit and Amazon S3 Vectors
Original publication date: 2026-10-01T17:34:56+00:00