Moss - Real-time Semantic Search for Conversational AI

Browser SDK / Inference MossInference takes OpenAI’s chat.completions.create arguments plus indexes, and returns OpenAI’s shapes from Moss Inference, which searches your indexes in the cloud, so the page loads no index. It works like the JavaScript client, and this page covers what differs in the browser.

Requires @moss-dev/moss-web 1.7.0 or later, installed as in Install.

Create a client

import { MossInference } from "@moss-dev/moss-web";

const client = new MossInference("your_project_id", "your_project_key");

new MossInference(projectId, projectKey, options) takes your project id and key from the Moss portal. CORS is open, so the page can call Moss Inference directly, but a project key in page code is visible to every visitor and can also create and delete your indexes. Keep direct calls to prototypes, local demos and internal tools. On a public page, pass null for both and a baseURL path served by a route of yours that adds them. Build one client and reuse it for every call. It has nothing to close.

Get a reply

const completion = await client.chat.completions.create({
  messages: [{ role: "user", content: "Do you ship to Canada?" }],
  indexes: ["faq"],
});
document.querySelector("#answer")!.textContent = completion.choices[0].message.content;

completion.requestId is the response’s X-Moss-Request-Id, and so is stream.requestId on a stream. Include it when you report a problem. Both are undefined when the response has none.

Stream a reply

const answer = document.querySelector("#answer")!;

const stream = await client.chat.completions.create({
  messages: [{ role: "user", content: "Do you ship to Canada?" }],
  indexes: ["faq"],
  stream: true,
});
document.querySelector("#stop")!.addEventListener("click", () => stream.controller.abort());
for await (const chunk of stream) {
  answer.textContent += chunk.choices[0]?.delta.content ?? "";
}

To stop early, leave the loop or call stream.controller.abort(), as the stop button does: either closes the connection, which stops the model, and neither throws. Only the last chunk carries usage, and it has no choices, which is why the loop reads choices[0]?..

Errors

Every failure other than an abort throws InferenceError, from the for await loop once a stream has started. status is the HTTP status of a refused request, type names the failure, and message describes it. errorCode is the error code Moss Inference sent, such as "rate_limit_exceeded", requestId is the response’s X-Moss-Request-Id, and retryAfter is the wait in milliseconds that a Retry-After header asks for. A field the failure lacks is undefined. A call without baseURL gets no retryAfter, since Moss Inference does not expose Retry-After to the page. The SDK sets these types itself, with status undefined:

`type`	Cause
`invalid_request_error`	An argument the SDK rejects before sending, such as an empty key, an index name that is empty or holds a comma, a `null` id and key without `baseURL`, or a `baseURL` that is not a path on the page’s origin.
`connection_error`	Your route or Moss Inference could not be reached, the browser blocked the call, for example over CORS, or the connection broke.
`timeout`	A time limit ran out, such as `timeout` or `idleTimeout`.
`incomplete_stream`	The stream ended before `[DONE]`.

Every other error reports what Moss Inference sent. See Errors, and Limits for a 429.

Tool calls

tools, tool_choice and parallel_tool_calls take OpenAI’s shapes, and your code runs each call. After the loop, stream.finalChatCompletion() joins every chunk into the completion that a call without stream: true returns. Push its message onto messages as it is, then one tool message per call, and call again, as in Tool calls:

import type { ChatCompletionMessageParam, ChatCompletionTool } from "@moss-dev/moss-web";
import { orderStatus } from "./orders";

const tools: ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "order_status",
      parameters: { type: "object", properties: { id: { type: "string" } }, required: ["id"] },
    },
  },
];
const messages: ChatCompletionMessageParam[] = [{ role: "user", content: "Where is order A1?" }];

const stream = await client.chat.completions.create({ messages, tools, stream: true });
for await (const chunk of stream) {
  answer.textContent += chunk.choices[0]?.delta.content ?? "";
}
const { message } = (await stream.finalChatCompletion()).choices[0];
messages.push(message);
for (const call of message.tool_calls ?? []) {
  const status = await orderStatus(JSON.parse(call.function.arguments).id);
  messages.push({ role: "tool", tool_call_id: call.id, content: status });
}
const reply = await client.chat.completions.create({ messages, tools });

orderStatus is your own function, and it returns text. finalChatCompletion() also works without the for await loop, and a second call returns the same completion. It rejects with what the loop would throw, or with the signal’s reason after an abort.

MCP servers

Add MCP entries in your server route, never in page code. Steps arrive in chunk.moss_tool_step and completion.moss_tool_steps, and MCP failures throw InferenceError with status, type, errorCode and retryAfter, as in MCP servers.

Keep the project key on your server

A public page calls Moss Inference through a route like this one, which adds the key on your server so it never reaches the page:

// Serves POST /moss/v1/chat/completions. Every call spends your project's quota,
// so the route answers only a visitor your own session check accepts.
import { isSignedIn } from "./session";

const ENDPOINT = "https://inference.moss.dev/v1/chat/completions";
const INDEXES = ["faq"];
const FORWARD = ["content-type", "accept"];
const REPLY = ["content-type", "retry-after", "x-moss-request-id"];

export async function POST(request: Request): Promise<Response> {
  if (!(await isSignedIn(request))) {
    return Response.json({ detail: "Sign in first." }, { status: 401 });
  }
  const { MOSS_PROJECT_ID, MOSS_PROJECT_KEY } = process.env;
  const indexes = request.headers.get("x-moss-indexes")?.split(",") ?? INDEXES;
  if (!indexes.every((name) => INDEXES.includes(name))) {
    return Response.json({ detail: "This page cannot search that index." }, { status: 403 });
  }
  const headers = new Headers();
  for (const name of FORWARD) {
    const value = request.headers.get(name);
    if (value !== null) headers.set(name, value);
  }
  headers.set("X-Moss-Indexes", indexes.join(","));
  headers.set("Authorization", `Bearer ${MOSS_PROJECT_ID}:${MOSS_PROJECT_KEY}`);
  const reply = await fetch(ENDPOINT, {
    method: "POST",
    headers,
    body: await request.text(),
    signal: request.signal,
    redirect: "manual",
  });
  const replyHeaders = [...reply.headers].filter(([name]) => REPLY.includes(name));
  return new Response(reply.body, { status: reply.status, headers: replyHeaders });
}

Every visitor can set any header, so the route forwards only the ones it names, refuses an index outside its list with 403, and always sends X-Moss-Indexes, because a call without it searches every index of the project. retrieve, cache, an empty indexes list and apiKey in page code therefore do not reach Moss Inference, and a route that wants them sets X-Moss-Retrieve, X-Moss-Cache or X-Moss-Provider-Key itself. With null credentials the client sends no Authorization header, so one you pass in headers, such as the visitor’s session token, reaches the route.

Options

new MossInference(projectId, projectKey, options?):

Option	Default	Notes
`baseURL`	none	A path on the page’s origin, such as `/moss/v1`, with no query or fragment. Each call resolves it against the page, so a client built during server rendering works in the browser. Without it, calls go to `https://inference.moss.dev/v1`. Required when the id and key are `null`.
`apiKey`	none	Your own OpenAI, Anthropic or Google key, for a vendor `model` such as `gpt-4o-mini`. See Models. It is sent as the `X-Moss-Provider-Key` header, never in the body. In page code every visitor can read it, like a project key. An empty key throws `InferenceError`.
`timeout`	none	Milliseconds for a whole call, the stream included. `null` turns it off.
`idleTimeout`	`120_000`	Milliseconds to wait for the reply to start, and for each pause in it. `null` turns it off.
`maxRetries`	`2`	Retries before the reply starts. `0` turns them off.
`headers`	none	Headers sent with every call.

chat.completions.create(params, options?) takes OpenAI’s fields and the Moss fields in params, and settings for this call only in options:

Option	Default	Notes
`model`	`"moss"`	The Moss model, or a vendor model with `apiKey`. See Models.
`messages`	required	OpenAI’s message list.
`stream`	`false`	`true` resolves to a `ChatCompletionStream` of chunks, otherwise to one `ChatCompletion`.
`indexes`	every index of the project	Names of the indexes to search, at most 32. Required when the project has more than 32. An empty list skips the search, as `retrieve: false` does, except through a server route, which sends its own list.
`cache`	`true`	Moss can answer an identical request from its reply cache. `false` always asks the model.
`retrieve`	`true`	`false` skips the search, and the model answers from the messages alone.
Other OpenAI fields	none	`temperature`, `max_tokens`, `tools` and the rest go to the model. An `apiKey` or `api_key` field throws.
`options.apiKey`	the client’s	Your provider key for this call, in place of the client’s.
`options.timeout`, `options.idleTimeout`	the client’s	This call’s limits, in place of the client’s. `null` turns one off.
`options.maxRetries`	the client’s	This call’s retries, in place of the client’s.
`options.headers`	none	Headers for this call. They add to the client’s or replace them by name.
`options.signal`	none	An `AbortSignal` that cancels the call.

Python

JavaScript

Swift (iOS)

Android

Elixir

Browser

C

Inference

Create a client

Get a reply

Stream a reply

Errors

Tool calls

MCP servers

Keep the project key on your server

Options

​Create a client

​Get a reply

​Stream a reply

​Errors

​Tool calls

​MCP servers

​Keep the project key on your server

​Options

Create a client

Get a reply

Stream a reply

Errors

Tool calls

MCP servers

Keep the project key on your server

Options