llm-connect weekly downloads statistics

llm-connect

Let the users of a web app bring their own LLM: pick a provider and a model from a catalog, paste a key (or point at Ollama / LM Studio), test the connection, save, then call the model through the AI SDK straight from the browser.

• Framework-free core: provider registry, createModel / loadModel, testConnection, and a settings store you can drive from any UI. llm-connect/react binds it to React; Svelte reads it as a store.
• Five providers built in: OpenAI, Anthropic (and Anthropic-compatible APIs), Google, Mistral, and every provider of the generic @ai-sdk/openai-compatible SDK (DeepSeek, Z.ai, Groq…). Each SDK loads only when used.
• The catalog comes from llm-models, built live in the browser (liveCatalog) with a static models.json as fallback, and a recommended model per provider.
• The key never sits in clear text: secureStorage encrypts it with AES-GCM under a non-extractable key, createKeyVault keeps the keys of several providers behind a passphrase, and a custom endpoint must be https (plain http only on this machine).
• Each provider's request rules: reasoning effort per provider and model, what each model accepts (sampling, prefill, tools, web search), Claude's adaptive thinking, stateless OpenAI Responses calls, headers that pass a browser's CORS preflight, and errors classified without the key.
• Speech to text: transcribe with the provider's own transcription model (OpenAI, Gemini, Mistral, Groq), or on the device with NVIDIA Parakeet or Canary (llm-connect/nemo) or Whisper (llm-connect/whisper), all in one shape and timed to the word.

Install

npm install llm-connect llm-models ai @ai-sdk/openai @ai-sdk/anthropic @ai-sdk/google @ai-sdk/mistral @ai-sdk/openai-compatible

ai (v7), the @ai-sdk/* packages (v4) and @ai-sdk/openai-compatible (v3) are optional peers: install the ones your users can pick. react (18+) is only needed for llm-connect/react.

The store builds the catalog in the browser when settings open (see The catalog). Write the fallback at build time:

npx llm-models catalog --npm callable --require openai,anthropic,google,mistral --out public/models.json

Call a model

import { generateText } from "ai";
import { loadModel } from "llm-connect";

const model = await loadModel(settings); // imports @ai-sdk/anthropic on first use
const { text } = await generateText({ model, prompt: "Hello" });

settings is what the store saves (LlmSettings):

Field
apiKey, model	The key and the catalog model id
providerId, providerName, providerNpm, providerApi	The catalog provider: its SDK package and base URL
customEndpoint, customModel	An OpenAI-compatible server (Ollama, LM Studio, vLLM…); wins over the provider
headers	Extra request headers
language	Code of the language answers are written in ("fr")

Both take options as a second argument:

Option	Default
fetch	globalThis.fetch	The fetch every request of the model goes through: route them through your own wrapper (logging, a proxy, a network policy)
strictHeaders	false	Send only the headers the provider's API defines (plus content-type and accept); the settings' headers are dropped

createModel(settings) does the same synchronously, for providers you registered yourself:

import { createModel, registerProvider, fromSDK } from "llm-connect";
import { register as registerAnthropic } from "llm-connect/providers/anthropic";
import { createOpenAI } from "@ai-sdk/openai";

registerAnthropic(); // or "llm-connect/providers/all" for the five
registerProvider("openai", fromSDK(createOpenAI));
registerProvider("my-llm", (config) => myLanguageModel(config.apiKey, config.model));

The registry is shared by every copy of llm-connect loaded in the page, so a library and the app that resolve different installs still see the same providers.

What the built-in providers take care of:

• A custom endpoint goes through the OpenAI SDK with Chat Completions; the official OpenAI URL keeps the Responses API.
• Every request goes through guardedFetch, which removes the user-agent header the AI SDK adds (no provider's CORS preflight allows it, and Firefox sends it).
• Anthropic's own API gets the anthropic-dangerous-direct-browser-access header. An Anthropic-compatible provider (MiniMax…) gets its own base URL, so its key never goes to Anthropic, and not that header, which its preflight does not allow. Its base URL works with or without the trailing /v1 (anthropicBaseUrl): gateways document it without, for Anthropic's own clients, as Z.ai does with https://api.z.ai/api/anthropic.
• A gateway that answers an error with HTTP 200 and a JSON body (Z.ai: {"code":500,"msg":"404 NOT_FOUND","success":false}) gets the status the body stands for (maskedErrorStatus), so the call fails with the provider's message instead of a parse error or a stream that ends without a finish.
• The generic SDK refuses a provider without URL, asks for the usage chunk at the end of a stream (stream_options.include_usage), and turns structured output (json_schema) into json_object plus the schema in a system message, for APIs such as GLM.

guardedFetch(kind, { fetch, strictHeaders }) is exported for providers you register yourself: pass it as their fetch.

Call options

What a call may carry depends on the provider and the model. callOptions builds the provider options for the model the settings point to, to spread into streamText or generateText:

import { streamText } from "ai";
import { callOptions, callProblem, loadModel, resolveCapabilities, providerKindOf } from "llm-connect";…