@mcpjam/sdk weekly downloads statistics

@mcpjam/sdk

Use the MCPJam SDK to write unit tests and evals for your MCP server.

Installation

Compatible with your favorite testing framework like Jest and Vitest

Quick Start

Unit Test

Test the individual parts, request response flow of your MCP server. MCP unit tests are deterministic.

MCP evals

Test that an LLM correctly understands how to use your MCP server. Evals are non-deterministic and multiple runs are needed.

Heads up: renames in 1.11. TestAgent → HostRunner, EvalAgent → HostExecutor, .prompt() → .run(), Host.addServer() → Host.requireServer(). No deprecation aliases. See the changelog for the codemod.

Host + HostRuntime: bring your own runtime

Host is the portable host-configuration spec — the same object the MCPJam Inspector uses to drive its Playground and eval suites. You can build one in your own code and run evals against it, so the inspector and your CI exercise the same host behavior (style, model, MCP profile, sandbox/permission policy, OpenAI-Apps compat, tool-visibility policy, etc.).

HostRuntime.run() is stateless across turns. Prior PromptResults accumulate in runtime.getPromptHistory() for inspection but are NOT auto-replayed. Multi-turn continuity stays explicit via PromptOptions.context:

Static specs use HostRunner directly. When you've already resolved the tool set and don't need live-binding to a manager, skip the runtime and pass host straight into HostRunner:

HostRunner snapshots the host once at construction; post-construction mutations to the original Host do NOT affect the runner. HostRuntime snapshots on every .run() so live mutations to the bound Host take effect on the next iteration.

One-shot sugar:

Eval reporting (Stage 5)

When the inspector backend advertises the evalsHostConfig capability at GET /sdk/v1/info, the eval reporter additionally sends { hostConfig, hostConfigHash } so the persisted run row in the MCPJam UI shows the exact host you ran with. Source order:

1. iteration.hostSnapshot (per-iteration capture from HostRuntime) 2. executor.getHostSnapshot?.() (fallback for executors that don't expose per-iteration snapshots) 3. MCPJamReportingConfig.host (explicit override, compatibility path)

Pass-1 only sends a run-level host config when all iteration snapshots canonicalize to the same hash. If you mutate the bound Host between iterations, the reporter omits the run-level field (per-iteration wire support is a later stage). Old backends without the capability are unaffected.

@mcpjam/sdk/host-config/internal

The internal subpath exposes the canonicalizer, hasher, normalizer, and policy resolvers (canonicalizeHostConfigV2, computeHostConfigHashV2, normalizeSdkEvalHostConfigForWire, extractHostExecutionPolicy, resolveOpenAiCompatForHostConfig, …) that the MCPJam backend and the inspector server both import. It is first-party-only, not semver-stable — external consumers should use the Host facade. The MCPJam backend convex/lib/hostConfigV2.ts imports the canonicalizer directly from this subpath so there is exactly one source of truth shared between the SDK and the persisted hostConfigs rows.

OAuth Conformance

Test that your MCP server's OAuth implementation works across all registration methods and protocol versions.

Or use the CLI:

MCP Apps Conformance

Validate the server-side MCP Apps surface your server exposes through tools and ui:// resources.

Or use the CLI:

The current runner validates tool metadata, ui:// resource discovery, resources/read, HTML payload shape, and _meta.ui metadata such as csp, permissions, domain, and prefersBorder.

It does not yet validate full host-side SEP-1865 behavior such as ui/initialize, sandbox proxy behavior, or host notification ordering.

API Reference

MCPClientManager

Manages connections to one or more MCP servers.

Retry policy is SDK-owned and disabled by default. To enable transient retries for connection and read-style manager operations, pass a manager-level policy:

The manager applies that policy to connectToServer() when called directly and to read/diagnostic methods such as listTools, listResources, readResource, listPrompts, getPrompt, pingServer, and task reads. Read retries wrap the full connect plus RPC operation, so a single retry budget covers reconnect and the follow-on request together.

Tool execution stays single-shot unless you opt in explicitly at the call site:

withEphemeralClient(), probeMcpServer(), and runServerDoctor() also accept the same retryPolicy shape. Retry classifiers are intentionally conservative in v1: transient network/connect/reset/DNS/timeout failures and HTTP 408, 425, 429, and 5xx are retryable; auth, validation, and method-unavailable errors are not.

HostRunner

Runs LLM prompts with MCP tool access.

No provider key? Prefix the model with mcpjam/ and MCPJam runs it on your organization's credits, with an MCPJam API key as the only secret:

Hosted models are mcpjam/anthropic/claude-* and mcpjam/openai/gpt-5*. An EvalSuite releases its leases when it finishes; if you build runs by hand, call releaseMcpjamModelLeases() when you are done.

stopWhen does not skip tool execution. It controls whether the prompt loop continues after the current step completes, and HostRunner also applies stepCountIs(maxSteps) as a safety guard.

timeout bounds prompt runtime. number and totalMs cap the full prompt, stepMs caps each step, and chunkMs is accepted for parity but mainly matters in streaming flows. The runtime creates an internal abort signal, so tools can stop early if their implementation respects the provided abortSignal.

stopAfterToolCall is intended for evals that only care about tool selection and arguments. The targeted tool is short-circuited with a stub result, and the PromptResult still includes the tool name and args. If multiple tools are emitted in the same step, non-target siblings may still execute before the loop stops.

Supported providers: openai, anthropic, azure, google, mistral, deepseek, ollama, openrouter, xai…

Description from the npm listing.