20. Exact vs semantic retrieval

The most common mistake in memory systems is using semantic retrieval for everything. Semantic retrieval is the right tool for "find documents conceptually related to X". It is the wrong tool for "what is this user's timezone preference". Using the wrong retrieval mode for a query type is a bug, not a modelling choice.

Figure 18: Retrieval mode decision matrix. Key-value lookups → exact match. Concept search → semantic. Episode recall → hybrid. The table below shows specific examples.

Here's how to choose the right retrieval mode for each query type:

Query type	Retrieval mode	Why
"What is the refund policy?"	Semantic (vector)	Concept-based lookup, no exact key
"What is this user's timezone?"	Exact (DynamoDB)	Key-value lookup: pref_key = 'timezone'
"Show me past invoices for Acme Corp"	Exact + filter	Filter by customer_id = 'acme', no ranking
"How did we solve X last time?"	Hybrid (vector + lexical)	Episodic memory: concept + keyword match
"What are the approval rules?"	Exact (DynamoDB)	Policy lookup: policy_key = 'approval_threshold'

A practical rule: if you know the key, use exact retrieval. If you're searching for a concept, use semantic retrieval.

21. AgentCore Memory SDK integration

AWS AgentCore Memory is a managed service that provides a memory API for agents. Instead of building the memory manager yourself, you can use the AgentCore SDK to handle storage, retrieval, and scope enforcement. The SDK abstracts away Aurora, DynamoDB, and S3 — you get a simple interface for create, search, and delete operations.

Figure 19: AgentCore Memory SDK architecture. The SDK sits between your agent code and managed storage. Scope enforcement, deduplication, and GDPR deletion are handled automatically.

Here's the basic usage pattern:

Python — AgentCore Memory SDK:

import boto3
from agentcore import MemoryClient

# Initialize AgentCore Memory client
memory_client = MemoryClient(
    region_name="eu-west-1",
    memory_id="mem-abc123"  # Created via AWS Console or CloudFormation
)

# Store a fact
memory_client.create_memory(
    session_id="session-xyz",
    memory_type="FACT",
    content="Acme Corp's production database is in eu-west-1",
    metadata={
        "subject": "customer:acme-corp",
        "predicate": "infrastructure",
        "confidence": 0.92
    },
    scope={
        "tenant_id": "tenant-abc",
        "user_id": "user-123"
    }
)

# Retrieve relevant memories
results = memory_client.search_memories(
    session_id="session-xyz",
    query="Where is Acme's database?",
    memory_types=["FACT", "EPISODE"],
    scope={
        "tenant_id": "tenant-abc",
        "user_id": "user-123"
    },
    max_results=5
)

for memory in results:
    print(f"[{memory.confidence:.2f}] {memory.content}")

# Delete user data (GDPR)
memory_client.delete_user_memories(
    tenant_id="tenant-abc",
    user_id="user-123"
)

AgentCore Memory handles the underlying Aurora, DynamoDB, and S3 infrastructure. You get:

• Managed storage and scaling
• Built-in scope enforcement (RLS)
• Automatic deduplication
• GDPR-compliant deletion
• CloudWatch metrics out of the box

Trade-off: you lose control over the exact storage layout and cannot tune the hybrid retrieval logic. For most teams this is the right trade-off — you get a production-ready memory system without operating it yourself.

22. Bedrock Prompt Caching — economics

Prompt caching reduces input token costs when the prefix of your prompt stays stable across requests. For agents with long system prompts (2000+ tokens of instructions, tool schemas, policies), caching can reduce input costs by 90% or more. The key is structuring your prompt so the stable content comes first.

Figure 20: Bedrock prompt caching economics. The static prefix (system + policy + preferences) is cached; the dynamic suffix (facts + context + user message) changes each turn. Cache hits reduce input costs by 80-90%.

Here's the cost breakdown for different caching scenarios:

Scenario	Input tokens	Cached tokens	Cost (Claude 3.5 Sonnet)	Savings
No caching	5,000	0	$0.015	—
Static prefix cached (3000 tokens)	2,000	3,000	$0.003	80%
Cache miss (write new cache)	5,000	0	$0.019 (write penalty)	−27%

Cache hit rates in production memory systems:

• Policy + preferences only: 95–98% cache hit rate. These rarely change during a session.
• Policy + preferences + top-5 facts: 60–75% cache hit rate. Facts change as the conversation progresses.
• Full conversation history: 10–20% cache hit rate. The prefix changes on every turn — caching is ineffective.

For Claude 3.5 Sonnet: if the static prefix is reused ≥2 times, caching is net positive. For agents with multi-turn conversations, this threshold is always met.

23. Observability — instrument the loop

Production memory systems need instrumentation at every step: retrieval latency, cache hit rate, gate rejection rate, token costs, and turn-level traces. Without observability, you cannot debug issues, optimize costs, or prove compliance. The diagram below shows the three observability layers: metrics, traces, and replay logs.

Figure 21: Observability stack. CloudWatch metrics for dashboards and alerts. X-Ray traces for debugging latency. Athena queries against S3 trace logs for replay and audit.

Here's how to instrument each component:

Python — CloudWatch custom metrics:

import boto3
from datetime import datetime

cloudwatch = boto3.client("cloudwatch")

def put_memory_metrics(tenant_id: str, metrics: dict):
    cloudwatch.put_metric_data(
        Namespace="MemorySystem",
        MetricData=[
            {
                "MetricName": "CacheHitRate",
                "Dimensions": [{"Name": "TenantId", "Value": tenant_id}],
                "Value": metrics["cache_hit_rate"],
                "Unit": "Percent",
                "Timestamp": datetime.utcnow()
            },
            {
                "MetricName": "GateRejectionRate",
                "Dimensions": [{"Name": "TenantId", "Value": tenant_id}],
                "Value": metrics["gate_rejection_rate"],
                "Unit": "Percent",
                "Timestamp": datetime.utcnow()
            },
            {
                "MetricName": "HybridRetrievalLatency",
                "Dimensions": [{"Name": "TenantId", "Value": tenant_id}],
                "Value": metrics["retrieval_latency_ms"],
                "Unit": "Milliseconds",
                "Timestamp": datetime.utcnow()
            },
            {
                "MetricName": "TokenCostPerTurn",
                "Dimensions": [{"Name": "TenantId", "Value": tenant_id}],
                "Value": metrics["token_cost_usd"],
                "Unit": "None",
                "Timestamp": datetime.utcnow()
            },
            {
                "MetricName": "PromotedFactsPerTurn",
                "Dimensions": [{"Name": "TenantId", "Value": tenant_id}],
                "Value": metrics["promoted_facts"],
                "Unit": "Count",
                "Timestamp": datetime.utcnow()
            }
        ]
    )

X-Ray distributed tracing

Python — X-Ray tracing for Bedrock calls:

from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.ext.boto_patcher import patch_all

patch_all()  # Patch boto3 clients

@xray_recorder.capture("memory_retrieval")
async def retrieve_with_tracing(query: str, tenant_id: str, user_id: str):
    xray_recorder.put_annotation("tenant_id", tenant_id)
    xray_recorder.put_metadata("query", query)

    with xray_recorder.capture("dynamo_preferences"):
        preferences = await get_preferences(tenant_id, user_id)

    with xray_recorder.capture("aurora_hybrid_search"):
        facts = await hybrid_search_facts(query, tenant_id, user_id)

    return MemoryBundle(preferences=preferences, facts=facts)

X-Ray traces show you:

• Which component is slow (Aurora? DynamoDB? Bedrock?)
• Where retries are happening
• Which tenants are seeing errors
• End-to-end latency for each turn

Athena replay from S3 trace logs

SQL — Athena query for trace replay:

-- Find all turns where the agent hallucinated (no grounding facts retrieved)
SELECT run_id, turn_index, user_msg, agent_response, facts_retrieved
  FROM memory_traces
 WHERE facts_retrieved = 0
   AND agent_response LIKE '%according to%'  -- claimed grounding
   AND year = '2026' AND month = '06'
 ORDER BY timestamp DESC
 LIMIT 100;

24. Cost optimization

Memory systems add infrastructure costs: Bedrock tokens, Aurora ACUs, DynamoDB reads, OpenSearch OCUs, and S3 trace storage. The good news is each cost driver has a clear lever. The diagram below shows where the AWS bill goes and how to reduce each component.

Figure 22: Cost breakdown for a typical memory system at 10k turns/day. Bedrock tokens dominate — prompt caching is the biggest lever. Aurora and OpenSearch scale with traffic; S3 scales with retention.

Here are the most effective optimizations:

Optimization	Impact	Implementation
Bedrock prompt caching	80–90% input token cost reduction	Structure prompt with static prefix. Cache system + policy + preferences.
Embed at write, not read	Eliminates per-query embedding cost	Compute embedding once during promotion, store in Aurora.
Use Nova Micro for extraction	5–10× cheaper than Sonnet	Extract facts with Nova Micro ($0.15/MTok). Use Sonnet only for user-facing inference.
ElastiCache in front of DynamoDB	80% reduction in DynamoDB read costs	Cache preferences in Redis with TTL 3600s. DynamoDB fallback on miss.
Aurora Serverless v2 scaling	Pay only for active ACUs	Set min=0.5, max=8 ACU. Scales to zero during idle periods.
S3 Lifecycle: 30d → Glacier	90% reduction in trace log storage cost	S3 Lifecycle policy: 30d Standard → 90d Intelligent Tiering → 365d Glacier IR → expire.
Top-K cutoff	Reduces prompt size by 30–40%	Retrieve top-20 facts, filter by fused_score >= 0.4, return top-5.

Cost breakdown for 10k turns/day

Component                      Daily cost    Monthly cost
────────────────────────────────────────────────────────────
Bedrock (Sonnet, cached)       $15           $450
Bedrock (Nova Micro extract)   $3            $90
Aurora Serverless v2 (2 ACU)   $4            $120
DynamoDB (on-demand)           $2            $60
ElastiCache Redis (t4g.small)  $1            $30
OpenSearch Serverless (4 OCU)  $10           $300
Kinesis + S3 (trace logs)      $2            $60
────────────────────────────────────────────────────────────
Total                          $37           $1,110

Per-turn cost: $0.0037

25. Memory in multi-agent scenarios

When you have multiple agents operating on behalf of the same user, memory becomes a shared resource. The challenge is deciding what each agent can see: should Agent B see facts that Agent A promoted? Should agents be able to hand off working context to each other? The diagram below shows three patterns for multi-agent memory coordination.

Figure 23: Multi-agent memory patterns. Shared memory for non-conflicting agents. Agent-scoped for private working memory. Coordinated handoff for explicit context transfer.

Here are the three main patterns:

1. Shared memory — all agents read from the same store

Default mode. All agents scoped to the same (tenant_id, user_id) see the same facts and preferences. Works well when agents are specialised (one handles scheduling, one handles billing) but don't conflict.

Python — Shared memory access:

# Agent A (scheduling) promotes a fact
memory.promote(
    FactCandidate(
        content="User prefers meetings after 2pm",
        subject="user:preferences",
        predicate="scheduling",
        confidence=0.9
    ),
    tenant_id="tenant-abc",
    user_id="user-123",
    agent_id="agent-scheduling"
)

# Agent B (email) retrieves the same fact
facts = memory.retrieve(
    query="What are the user's scheduling preferences?",
    tenant_id="tenant-abc",
    user_id="user-123",
    agent_id="agent-email"  # different agent, same user
)
# Result: sees the fact promoted by Agent A

2. Agent-scoped memory — each agent has private working memory

Add agent_id to the scope. Facts promoted by Agent A are only visible to Agent A unless explicitly shared.

SQL — Agent-scoped retrieval:

SELECT * FROM fact_memory
 WHERE tenant_id = :tenant_id
   AND user_id   = :user_id
   AND (agent_id IS NULL OR agent_id = :agent_id)  -- agent-scoped OR shared
   AND status    = 'active'
 ORDER BY embedding <=> :query_embedding
 LIMIT 5;

3. Coordinated memory with handoff

Agent A completes a task and explicitly hands off working memory to Agent B. Implemented as an episodic memory record with metadata.handoff_to = "agent-B".

Python — Memory handoff between agents:

# Agent A completes task and hands off
memory.promote(
    EpisodeCandidate(
        task="Drafted proposal for Acme Corp",
        outcome="Proposal saved to S3, awaiting approval",
        reusable_pattern="proposal_generation",
        metadata={
            "handoff_to": "agent-approval",
            "context": {
                "s3_key": "proposals/acme-2026-06.pdf",
                "amount_eur": 50000
            }
        }
    ),
    tenant_id="tenant-abc",
    user_id="user-123",
    agent_id="agent-drafter"
)

# Agent B retrieves handoff episode
episodes = memory.retrieve_episodes(
    query="pending approvals",
    tenant_id="tenant-abc",
    user_id="user-123",
    agent_id="agent-approval",
    filter_metadata={"handoff_to": "agent-approval"}
)

26. Evaluating the memory system

Evaluation is harder for memory systems than for RAG because the system's behaviour evolves over time as memory accumulates. Standard offline metrics (recall@K, MRR) are necessary but not sufficient. You also need to test promotion precision, scope isolation, supersession correctness, and replay fidelity.

Figure 24: Evaluation harness for memory systems. Offline metrics measure retrieval quality. Online metrics measure production behaviour. Invariant tests verify scope isolation and supersession.

Here's the complete evaluation framework:

Offline metrics

Metric	What it measures	How to compute
Recall@5	% of relevant facts in top-5 results	Human-labeled ground truth on 100 queries
MRR (Mean Reciprocal Rank)	Rank of first relevant result	1/rank_of_first_relevant_fact, averaged
Precision@K	% of retrieved facts that are relevant	relevant_in_topK / K
Gate acceptance rate	% of candidates promoted to durable memory	promoted / (promoted + rejected)
Dedup hit rate	% of promotions that hit existing fact	reconfirmed / total_promotions

Online evaluation — what actually matters

Python — Evaluation tests:

import pytest
from memory_manager import MemoryManager

@pytest.mark.asyncio
async def test_continuity_across_sessions():
    """Agent should remember facts from previous session"""
    manager = MemoryManager(...)

    # Session 1: User states preference
    await manager.promote(
        FactCandidate(content="I prefer JSON output", confidence=0.95),
        tenant_id="test-tenant", user_id="test-user", run_id="run-1"
    )

    # Session 2: 24 hours later
    results = await manager.retrieve(
        query="How should I format the response?",
        tenant_id="test-tenant", user_id="test-user"
    )

    assert any("JSON" in f.content for f in results.facts)

@pytest.mark.asyncio
async def test_policy_always_retrieved():
    """Policy memory must be retrieved exhaustively, not ranked"""
    manager = MemoryManager(...)

    # Insert 3 policies
    policies = ["refund_threshold", "data_residency", "tone_guardrail"]
    for p in policies:
        await manager.store_policy(tenant_id="test-tenant", policy_key=p, ...)

    # Retrieve with unrelated query
    results = await manager.retrieve(
        query="What is the weather?",  # unrelated query
        tenant_id="test-tenant", user_id="test-user"
    )

    # All 3 policies must be present
    assert len(results.policies) == 3

@pytest.mark.asyncio
async def test_gdpr_deletion_cascade():
    """User deletion must cascade across all stores"""
    manager = MemoryManager(...)

    # Create facts + preferences
    await manager.promote(FactCandidate(...), tenant_id="t", user_id="u", ...)
    await manager.store_preference(tenant_id="t", user_id="u", ...)

    # Delete user
    await manager.gdpr_delete_user(tenant_id="t", user_id="u")

    # Verify nothing is retrievable
    results = await manager.retrieve(query="anything", tenant_id="t", user_id="u")
    assert len(results.facts) == 0
    assert len(results.preferences) == 0

27. Production checklist

Before deploying a memory system to production, validate these operational requirements. The diagram below shows the five production-readiness gates: scope enforcement, observability, cost controls, correctness, and compliance. All five gates must pass — a failure in any one can cause data leaks, runaway costs, or incorrect behaviour.

Figure 25: Production-readiness gates. All five categories must pass before shipping. The table below lists specific requirements and how to verify each one.

Validate each requirement before deploying:

Category	Requirement	How to verify
Scope enforcement	Every query filters by tenant_id BEFORE ranking	Inspect SQL execution plans
	RLS enabled on all Aurora tables	SELECT * FROM pg_policies;
	No cross-tenant data leakage in test suite	Run multi-tenant isolation tests
Observability	CloudWatch metrics: CacheHitRate, GateRejectionRate	Check CloudWatch dashboard
	X-Ray tracing on all Bedrock calls	Open X-Ray console, filter by service
	Trace log to S3 with Parquet format	Query with Athena
Cost controls	Bedrock prompt caching enabled	Check cacheReadInputTokens > 0 in responses
	S3 Lifecycle: 30d → Glacier	Verify S3 bucket lifecycle policy
Correctness	Policy and preference never use similarity search	Code review: no ORDER BY embedding on these tables
	Promotion gate rejects confidence < 0.5	Check gate rejection logs
	Supersession marks old facts as superseded_by	Manual test: update a fact, verify old row excluded
Compliance	GDPR deletion cascades across Aurora, DynamoDB, S3	Run test_gdpr_deletion_cascade
	Provenance: every fact has source_run_id	SQL: SELECT COUNT(*) FROM fact_memory WHERE source_run_id IS NULL; → 0
Resilience	Cascade fallback: hybrid → vector → lexical → LIKE	Simulate Aurora downtime, verify fallback
	Dead-letter queue for failed promotions	Check SQS DLQ for messages

Conclusion: Models are shared. The memory system is yours.

Much of the current talk about "AI memory" is retrieval plus larger prompts with better branding. If your system only needs semantic lookup over documents, basic RAG is sufficient. If it must maintain continuity across sessions, precisely recall structured information, govern reuse across real privacy boundaries, and learn what to retain over time — you have a memory system design problem. A vector store alone does not solve it.

On AWS you have all the components natively: Aurora Serverless v2 with pgvector for hybrid retrieval and ACID transactions; DynamoDB for exact-match policy and preferences; OpenSearch Serverless for lexical recall; S3 + Kinesis for the trace log; AgentCore Memory for a managed memory API; Amazon Bedrock with prompt caching for inference. The pieces are there. The architecture connecting them is what this article describes.

Models are shared. The memory system is yours — because it reflects decisions only your team could make: what is remembered, what is forgotten, who can see it, and what that implies on the next turn.