Backend Integration with BlockNote AI

The most common (and recommended) setup to integrate BlockNote AI with an LLM is to have BlockNote AI call your backend, which then calls an LLM of your choice using the Vercel AI SDK. This page explains the default setup, but also provides several alternative approaches.

Default setup (Vercel AI SDK)

The example below closely follows the basic example from the Vercel AI SDK for Next.js. The only difference is that we're retrieving the BlockNote tools from the request body and using the toolDefinitionsToToolSet function to convert them to AI SDK tools. We also forward the serialized document state (selection, cursor, block IDs) that BlockNote adds to every user message by calling injectDocumentStateMessages. The LLM will now be able to invoke these tools to make modifications to the BlockNote document as requested by the user. The tool calls are forwarded to the client application where they're handled automatically by the AI Extension.

import { openai } from "@ai-sdk/openai";
import { convertToModelMessages, streamText } from "ai";
import {
  aiDocumentFormats,
  injectDocumentStateMessages,
  toolDefinitionsToToolSet,
} from "@blocknote/xl-ai/server";

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages, toolDefinitions } = await req.json();

  const result = streamText({
    model: openai("gpt-4.1"), // see https://ai-sdk.dev/docs/foundations/providers-and-models
    system: aiDocumentFormats.html.systemPrompt,
    messages: convertToModelMessages(injectDocumentStateMessages(messages)),
    tools: toolDefinitionsToToolSet(toolDefinitions),
    toolChoice: "required",
  });

  return result.toUIMessageStreamResponse();
}

Different javascript frameworks will have a very similar setup. For example, see our Hono example.

If your backend is in another language, you're unable to use Vercel AI SDK, or you can't setup a backend at all - there are several alternatives to integrate BlockNote AI:

Data Stream Protocol

BlockNote AI expects your backend to respond with Server-Sent Events (SSE) data streams according to the Data Stream Protocol. This protocol specified by the Vercel AI SDK. You can use this information to develop custom backends for your use case. For example, to provide compatible API endpoints that are implemented in a different language such as Python.

• Here's an example using Python FastAPI as a backend (note this open issue).

Custom transport

Instead of modifying your backend to support the Data Stream Protocol, you can also implement a custom transport layer in the client application. The transport layer determines how AI SDK requests are sent to your backend to retrieve an LLM response.

ClientSideTransport

BlockNote AI also provides a ClientSideTransport class that can be used to connect directly to LLMs without routing through a backend. To use this transport, create a Vercel AI SDK Provider and LanguageModel directly on the client. Then, use this to instantiate the transport and pass it to the AI Extension.

The example below uses the OpenAI Compatible provider, but you can use any provider / model you want.

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { ClientSideTransport } from "@blocknote/xl-ai";

const model = createOpenAICompatible({
  apiKey: 'your-api-key',
  baseURL: 'https://your-provider',
})('model-id');

// ...
 createAIExtension({
    transport: new ClientSideTransport({
        model,
    }),
}),
// ...

With a proxy server

It's likely you cannot call your LLM provider directly in your client application using ClientSideTransport, because you need to hide API keys or prevent CORS issues. For this reason, you can use a proxy server to route requests to your LLM provider. This proxy server can then inject your API keys and forward the request to your LLM provider.

BlockNote AI provides a fetchViaProxy function that can be used to create a fetch function that routes requests through a proxy server (the example below uses Groq as the LLM provider):

import { createGroq } from "@ai-sdk/groq";
import { fetchViaProxy } from "@blocknote/xl-ai";

const model = createGroq({
  fetch: fetchViaProxy(
    (url) => `${BASE_URL}/proxy?provider=groq&url=${encodeURIComponent(url)}`,
  ),
  apiKey: "fake-api-key", // the API key is not used as it's actually added in the proxy server
})("llama-3.3-70b-versatile");

• See a full example of how to use ClientSideTransport with a proxy server.

Advanced patterns

You can connect BlockNote AI features with more advanced AI pipelines. You can integrate concepts like Agents, RAG (Retrieval-Augmented Generation), multi-step LLM calls. Make sure you always expose the BlockNote tools (as passed via the toolDefinitions in the request body) to the LLM and forward invocations to the client.

• By default, BlockNote AI sends the entire LLM chat history to the backend. See the server persistence example for a pattern where the backend stores chat and only the latest message is sent to the backend.