@cascade-flow/ai-sdk weekly downloads statistics

@cascade-flow/ai-sdk

Drop-in replacement for Vercel AI SDK's generateText with automatic checkpoint support. Each LLM call in the agentic loop is wrapped in a checkpoint, so on retry the loop continues from where it left off.

Installation

bun add @cascade-flow/ai-sdk ai

Usage

Just import generateText from @cascade-flow/ai-sdk instead of ai. All AI SDK options work as expected:

import { defineStep } from "@cascade-flow/workflow";
import { generateText } from "@cascade-flow/ai-sdk";
import { openai } from "@ai-sdk/openai";
import { tool, stepCountIs } from "ai";
import { z } from "zod";

const getWeather = tool({
  description: "Get weather for a location",
  parameters: z.object({ location: z.string() }),
  execute: async ({ location }) => {
    return { location, temperature: 72, conditions: "sunny" };
  },
});

export const step = defineStep({
  name: "weather-agent",
  maxRetries: 3,
  fn: async ({ ctx }) => {
    const result = await generateText({
      model: openai("gpt-4"),
      system: "You are a helpful weather assistant.",
      prompt: ctx.input.query,
      tools: { getWeather },
      stopWhen: stepCountIs(5),
      checkpoint: ctx.checkpoint,
      // All AI SDK options work: temperature, maxTokens, etc.
    });

    return {
      response: result.text,
      stepsExecuted: result.totalSteps,
    };
  },
});

How It Works

The function implements an agentic loop where LLM calls and tool executions are checkpointed separately:

// Loop continues until stopWhen conditions are met
for (let stepIndex = 0; stepIndex < MAX_ITERATIONS; stepIndex++) {
  // 1. Checkpoint the LLM call only (no tool execution)
  const llmResult = await checkpoint("agent-llm", async () => {
    return generateText({ model, messages, tools: toolSchemas, stopWhen });
  });

  if (llmResult.finishReason !== "tool-calls") {
    return llmResult;
  }

  // 2. Execute all tools in parallel, each in its own checkpoint
  const toolPromises = llmResult.toolCalls.map((toolCall) =>
    checkpoint(`agent-tool-${toolCall.toolCallId}`, async () => ({
      toolCallId: toolCall.toolCallId,
      toolName: toolCall.toolName,
      result: await tools[toolCall.toolName].execute(toolCall.args),
    }))
  );
  const toolResults = await awaitCheckpoints(toolPromises);

  // 3. Build messages and continue
  messages = [...messages, assistantMessage, toolResultMessage];
}

Key Design: Separate LLM and Tool Checkpoints

The LLM call and tool executions are in separate checkpoints. This means:

• LLM calls are expensive - if a tool fails, we don't want to re-run the LLM
• Tool failures restart from the tool - the cached LLM result is reused
• Each tool is independently checkpointed - fine-grained retry control
• Tools run in parallel - uses awaitCheckpoints for concurrent execution

On Retry

1. Cached LLM checkpoint returns instantly (no LLM call)
2. Cached tool checkpoints return instantly
3. Only failed/new steps make actual calls

API Reference

generateText(options)

Drop-in replacement for AI SDK's generateText. All AI SDK options are supported and passed through.

Additional option:

• checkpointName - Checkpoint name prefix (default: "agent-llm"). Use unique names if calling multiple times in the same step.

Returns: GenerateTextResult extended with:

• totalSteps - Total number of steps executed

Without Checkpointing

When used outside a CascadeFlow step (no checkpoint context), the function works like regular generateText:

import { stepCountIs } from "ai";

// Works without checkpointing (useful for testing)
const result = await generateText({
  model: openai("gpt-4"),
  prompt: "Hello",
  stopWhen: stepCountIs(5),
});

See Also

• AI SDK Documentation
• CascadeFlow Workflow Package
• CascadeFlow Checkpoints