> ## Documentation Index
> Fetch the complete documentation index at: https://supermemory-capy-add-llmstxt-summary-and.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Adding a provider

> Register your own memory implementation so MemoryBench can score and compare it

A "provider" in MemoryBench is any memory or RAG system that can ingest sessions and answer a search query — Supermemory, Mem0, and Zep ship as built-in providers, and **your own memory implementation is just another provider**. Once it's registered, it runs through the exact same pipeline and gets scored on the exact same footing as everything else.

There are two ways to do this: let the skill generate the provider for you, or write it by hand.

## The fast path: the MemoryBench skill

MemoryBench ships a Claude Code skill (`benchmark-context`) that automates the whole flow — from reading your code to a finished comparison report — without you writing any MemoryBench-specific code yourself.

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
# Run from your project root, not from inside memorybench
/memorybench
```

It walks through 7 phases:

| Phase | What happens |
| - | - |
| **1. Setup** | Clones `memorybench` into `./memorybench` and installs dependencies |
| **2. Discovery** | An agent reads your memory code to find its init, ingest, and search methods |
| **3. Code generation** | Generates a provider adapter implementing the `Provider` interface, adapted to your code |
| **4. Registration** | Registers the provider in the framework's types and config |
| **5. Configuration** | Asks for your API keys (your provider, any comparison providers, and a judge model) and writes `.env.local` |
| **6. Validation** | Runs one question end-to-end to confirm ingest/search actually work before committing to a full run |
| **7. Benchmark execution** | Runs the full comparison and reports accuracy, latency, and context-token results side by side |

Before it does anything, it asks 5 quick questions: your provider's name, where your memory code lives, which [benchmark](/memorybench/extend-benchmark) to run, which providers to compare against (Supermemory, Mem0, Zep, or the no-API-key `filesystem`/`rag` baselines), and how many questions to sample (5 for a quick check, 20 for a real signal, or the full set).

<Note>
  Run it from **your project's root**, not from inside `memorybench` — the skill clones the framework as a subdirectory and analyzes your code via relative paths.
</Note>

## Doing it by hand

If you'd rather write the adapter yourself, every provider implements the same interface (`src/types/provider.ts`):

```typescript theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
interface Provider {
  name: string
  prompts?: ProviderPrompts
  initialize(config: ProviderConfig): Promise<void>
  ingest(sessions: UnifiedSession[], options: IngestOptions): Promise<IngestResult>
  awaitIndexing(result: IngestResult, containerTag: string): Promise<void>
  search(query: string, options: SearchOptions): Promise<unknown[]>
  clear(containerTag: string): Promise<void>
}
```

| Method | Responsibility |
| - | - |
| `initialize()` | Set up your client with an API key / config |
| `ingest()` | Push benchmark sessions into your system, return the resulting document IDs |
| `awaitIndexing()` | Block until those documents are actually searchable — a no-op if your system indexes synchronously, a poll loop with backoff if it's async |
| `search()` | Run a query, return results in whatever shape your system returns them |
| `clear()` | Delete everything under a `containerTag`, so runs don't bleed into each other |

Steps:

1. Create `src/providers/myprovider/index.ts` implementing `Provider`
2. Register it in `src/providers/index.ts`
3. Add the name to `ProviderName` in `src/types/provider.ts`
4. Add its config (API key, base URL, etc.) in `src/utils/config.ts`
5. Optionally override `prompts` (`ProviderPrompts`) if your search results need custom formatting before they're handed to the answering LLM or the judge

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
bun run src/index.ts test -p myprovider -b locomo -q question_1   # one question, fast sanity check
bun run src/index.ts run -p myprovider -b locomo                  # full run
bun run src/index.ts compare -p myprovider,supermemory,mem0 -b locomo -l 20
```

<Note>
  Full interface, including async-indexing and custom-prompt examples: [`src/providers/README.md`](https://github.com/supermemoryai/memorybench/blob/main/src/providers/README.md) in the repo.
</Note>

***

## Next

<CardGroup cols={2}>
  <Card title="Building a benchmark" icon="https://mintcdn.com/supermemory-capy-add-llmstxt-summary-and/LIMkcglt81IfjBVR/icons/hugeicons/test-tube-01.svg?fit=max&auto=format&n=LIMkcglt81IfjBVR&q=85&s=6ea50f02549a743d9edd079782db4ee6" href="/memorybench/extend-benchmark" width="24" height="24" data-path="icons/hugeicons/test-tube-01.svg">
    Run your provider against a dataset built for your own use case.
  </Card>

  <Card title="Measuring results" icon="https://mintcdn.com/supermemory-capy-add-llmstxt-summary-and/LIMkcglt81IfjBVR/icons/hugeicons/dashboard-speed-01.svg?fit=max&auto=format&n=LIMkcglt81IfjBVR&q=85&s=ca7e4c67e6d89c86368671af77ebd95d" href="/memorybench/memscore" width="24" height="24" data-path="icons/hugeicons/dashboard-speed-01.svg">
    What the report actually tells you.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.