How to Evaluate and Connect a Memory MCP Server
Evaluate a memory MCP server with a practical connection, persistence, permissions, and recovery checklist for Claude Code, Cursor, and custom agents.
A memory MCP connection is ready for a pilot when the intended clients can discover the required tools, access the correct records, observe updates and respect permissions after restarting. A successful connection alone proves neither durable storage nor safe shared use. This guide separates connection setup from the lifecycle tests that make it useful.
Key takeaways
- MCP standardizes communication; the selected server and application provide persistence, authorization and concurrency behavior.
- stdio launches a subprocess. A proxy may forward to a shared backend, but stdio itself is neither a network endpoint nor a security sandbox.
- Verify the actual client, server, proxy and protocol versions. Modern discovery and legacy initialization are different contracts.
- Use confirmed record identities, revisions and observable outcomes to test sharing, updates, conflicts and recovery.
Choose the component and connection path
| Component | What it supplies | Integration work to verify |
|---|---|---|
| Reference MCP memory server | A local graph of entities, relations and observations persisted in JSONL. | Client setup, filesystem permissions, backup and safe concurrent access. |
| Mem0, selected edition | Readable memory records, metadata and retrieval indexes; optional graph capabilities. | The chosen MCP integration, authentication, extraction behavior and hosted versus OSS differences. |
| Letta and a selected adapter | Stateful agents with memory blocks and archival retrieval. | Whether an adapter exposes the required memory operations; consuming MCP tools is not the same as serving them. |
| PostgreSQL with pgvector | Relational storage and vector retrieval. | An application or MCP wrapper, permission checks, memory semantics and database operations. |
| AKB | Organizational documents, tables, files, relationships and retrieval through documented interfaces. | The backend, supported client connection, identities, vault access and operational configuration. |
These components operate at different abstraction levels. A database can underlie a memory service; an agent runtime can use a separate knowledge layer. Evaluate a complete configuration rather than ranking isolated components as interchangeable MCP servers.
The reference memory server searches entity names, types and observation text. Do not infer arbitrary relation-text search, database transactions or production hardening from its graph representation. Mem0's OSS graph documentation describes optional graph context alongside memory retrieval; hosted features and authentication must be checked separately.
Understand the transport and protocol version
With stdio, the client launches a subprocess and exchanges protocol messages through its standard streams. Shared state requires a common backend or a storage design explicitly safe for multiple processes. AKB's documented akb-mcp proxy is one way to connect a stdio client to an existing backend; the proxy does not replace that service.
Streamable HTTP exposes an MCP endpoint to compatible network clients. Multiple clients reaching an endpoint still need appropriate identities, authorization and supported write semantics. Neither transport ensures that two clients use identical prompts or return identical answers.
The 2026-07-28 MCP specification uses per-request protocol metadata and describes compatibility with earlier connection-scoped initialization. Follow the contract supported by the exact client/server pair; do not copy a legacy initialize sequence into a modern-only configuration. Discover the capabilities actually advertised instead of assuming all servers expose every tool, resource or prompt feature.
MCP's HTTP authorization specification defines a transport-level authorization flow. It does not replace the application's decisions about who may read or modify a vault or record. stdio uses a different credential arrangement and does not sandbox the launched process. Keep credentials out of shared configuration examples and logs.
Connect and verify two clients
- Record the environment. Capture client versions, server release, proxy version, supported protocol revision, transport and authentication method. Follow the AKB release instructions for an AKB deployment; do not invent package names, ports, environment variables or configuration paths.
- Verify discovery. Confirm the required tools and input/output schemas are available through the selected protocol generation. Check schema failures separately from credential failures.
- Use an isolated test space. Give each test identity only its intended access. Record the backend and vault or namespace so a client cannot silently target a different store.
- Confirm a write. Store a distinctive, non-sensitive record through client A. Capture its identity and version or other documented success evidence.
- Read through client B. Retrieve the same record under an authorized identity. Test a direct lookup and indexed search separately; they can have different visibility timing.
- Restart and update. Restart both clients and the server with the intended persistent storage. Verify the record survives, update it, then measure when the new state appears through each read path.
- Test restrictions and contention. Check an unauthorized identity, expired or revoked credentials, and two writes from the same base state. Record actual errors and conflict behavior.
- Test recovery. Export or back up the required data, restore into an isolated environment, and repeat retrieval and permission checks.
Use a small acceptance worksheet, then expand it
As a hypothetical planning example, 6 scenarios x 2 client types x 2 identity profiles yields 24 cases. Use one authorized and one unauthorized profile; the expected result depends on the scenario and permission. Pairwise concurrency tests and broader security coverage are additional work. This count is neither a benchmark result nor a certification.
| Scenario | Expected evidence |
|---|---|
| Discovery | Required schemas discovered for permitted access; unsupported versions rejected according to the documented contract. |
| Write and retrieve | Authorized write confirmed and the correct identity/version retrieved; unauthorized operations do not expose or change protected state. |
| Update visibility | Direct-read and indexed-search lag recorded separately, with no confusion between an old result and a lost write. |
| Restart persistence | The expected durable records survive server and client restart; unauthorized access remains denied. |
| Expiry and revocation | Access changes take effect within the agreed policy across content, snippets, metadata and cached results. |
| Failure and retry | Timeout outcomes are reconciled and retries do not silently duplicate or overwrite accepted work. |
Measure cases meeting their predefined expected outcome over attempted eligible cases. A correctly denied unauthorized request is a successful security test. Record unauthorized disclosures over unauthorized attempts separately; zero observed disclosures applies only to the tested scope. Report p50/p95 latency, error rates and freshness under representative load rather than treating 24 cases as a performance study.
A response timeout can occur after a write succeeds. Inspect current state or use documented idempotency and conditional-write mechanisms before retrying. Add a pairwise test in which two authorized writers submit conflicting changes against the same revision.
For a defined period, include infrastructure, storage, extraction, embedding, retrieval and operator effort in the declared cost boundary. Include failed and retried work within those categories without double counting. Divide by unique successful logical operations if reporting unit cost; zero successes makes that ratio undefined.
Keep recovery and correction mechanisms distinct
Use supported mutation tools for record corrections. Do not hand-edit a live JSONL store while writers are active. If manual maintenance is necessary, stop writers and follow a documented, backed-up procedure.
PostgreSQL PITR requires a suitable base backup and an unbroken required sequence of archived WAL, with a tested restore procedure. It restores cluster state, not an individual memory as a simple undo operation. The presence of ordinary WAL alone is insufficient.
AKB document revision history is separate from database and file recovery. Do not assume every table or file operation is preserved as a Git document revision. Audit coverage depends on enabled components and access paths. Verify what the selected release records, how events are retained and who can inspect them.
Core storage and retrieval do not imply bundled ingestion or curation. Confirm optional Collector/Gardener packaging and use supported interfaces. People using UI/API-based applications and agents using MCP should be tested against the same intended resource and permission model.
Check deployment and license boundaries
Local storage alone does not prove offline operation. Inspect model, embedding, identity, telemetry, package and client dependencies. Managed hosting transfers only the responsibilities specified in the agreement.
Review the deployed release's AKB LICENSE. Its source-available BSL 1.1 terms include a conditional grant below 100 aggregated Named Seats as defined there; the threshold and qualifying third-party services require separate commercial licensing. Check core software, adapters and hosted offerings independently for every candidate.
Frequently asked questions
Will switching models erase stored memory?
Not inherently if the external backend and retention policy remain unchanged. Verify that the new client can authenticate, use the tools and interpret the records correctly.
Can the system run completely offline?
Only if all necessary request and operational dependencies can function locally. Test the full path with external network access unavailable; local database storage alone is not enough.
How can we reduce incorrect memory writes?
Use evidence-linked proposals, appropriate review, scoped writes and recoverable changes. Treat retrieved content as untrusted data. These controls reduce risk; they do not eliminate hallucinations.
Do vector stores contain only embeddings?
Not necessarily. Many also store readable payloads and metadata, while others refer to records held elsewhere. Inspect the selected backend and application model.
When should we move beyond a reference server?
When the tested durability, access, concurrency, recovery or support requirements exceed what the reference configuration provides. Team size alone does not determine readiness.
Explore persistent shared knowledge
Evaluate AKB's supported interfaces, permission scopes and operational requirements for your people and agents.
