Moss - Real-time Semantic Search for Conversational AI

C SDK / Inference MossInference is the C client for Moss Inference, a drop-in for OpenAI’s chat completions. moss_inference_chat_completions_create takes an OpenAI request as JSON, searches your indexes in the cloud, adds what it finds to the prompt, and hands back the reply as OpenAI’s JSON.

Requires libmoss 0.28.0 or later, installed as in Install. A libmoss.h with these functions defines MOSS_INFERENCE_API_VERSION. From version 2, the calls keep every tool call field and report MCP tool steps.

Create a client

#include "libmoss.h"
#include <stdio.h>

MossInference *client = NULL;
const char *options =
    "{\"project_id\": \"your-project-id\","
    " \"project_key\": \"your-project-key\"}";
if (moss_inference_new(options, &client) != OK) {
    fprintf(stderr, "%s\n", moss_last_error());
    return 1;
}

The project id and key come from the Moss portal. Build one client and reuse it for every call, from any thread. Free it with moss_inference_free once no call on it is running.

Get a reply

With NULL for the callback, out_json gets the reply as the JSON of one OpenAI chat.completion:

char *completion = NULL;
const char *request =
    "{\"indexes\": [\"faq\", \"policies\"],"
    " \"messages\": [{\"role\": \"user\", \"content\": \"Do you ship to Canada?\"}]}";
if (moss_inference_chat_completions_create(client, request, NULL, NULL, NULL,
                                           &completion) == OK) {
    printf("%s\n", completion);
    moss_free_string(completion);
} else {
    fprintf(stderr, "%s\n", moss_last_error());
}

The text is in choices[0].message.content, which is null when the reply only calls tools. The completion’s request_id is the response’s X-Moss-Request-Id. Include it when you report a problem. It is null when the response has none.

Stream a reply

static int32_t print_chunk(const char *chunk_json, void *user_data) {
    (void)user_data;
    printf("%s\n", chunk_json);
    return 0; // any other value stops the stream
}

const char *request =
    "{\"indexes\": [\"faq\", \"policies\"],"
    " \"messages\": [{\"role\": \"user\", \"content\": \"Do you ship to Canada?\"}]}";
if (moss_inference_chat_completions_create(client, request, NULL, print_chunk, NULL,
                                           NULL) != OK) {
    fprintf(stderr, "%s\n", moss_last_error());
}

The call blocks until the reply ends, and hands the callback each chunk as the JSON text of one OpenAI chat.completion.chunk, with the new text in choices[0].delta.content when choices is not empty. To stop early, return a value other than 0 from the callback: the call closes the connection, which stops the model, and returns OK. Only the last chunk carries usage, and it has no choices, so check choices before reading choices[0].

Errors

A failed call returns a negative MossResult, a streaming one after the chunks the callback already got. Right after it, on the same thread, moss_last_error() holds the message, and moss_last_error_detail() a JSON object in which status is the HTTP status of a refused request, type names the failure, code is the error code Moss Inference sent, such as "rate_limit_exceeded", request_id is the response’s X-Moss-Request-Id, and retry_after_ms is the wait in milliseconds that a Retry-After header asks for. A field the failure lacks is null. The SDK sets these itself:

`type`	Returns	Cause
none, the detail is `NULL`	`ERR_NULL_POINTER`, `ERR_INVALID_ARG`	An argument the SDK rejects before sending, such as an empty key, an index name that is empty or holds a comma, a `NULL` client or request, or no callback and no `out_json`.
`connection_error`	`ERR_CLOUD`	Moss Inference could not be reached, or the connection broke.
`timeout`	`ERR_CLOUD`	A time limit ran out, such as `timeout_ms` or `idle_timeout_ms`.
`incomplete_stream`	`ERR_CLOUD`	The stream ended before `[DONE]`.

Every other error reports what Moss Inference sent. The call returns ERR_UNAUTHORIZED for authentication_error and permission_error, ERR_INVALID_ARG for invalid_request_error, ERR_DESERIALIZATION for a stream event that is not a JSON object, and ERR_CLOUD for the rest. See Errors, and Limits for a 429.

Tool calls

tools, tool_choice and parallel_tool_calls go in the request JSON as in OpenAI’s API, and your code runs the tools. Pass a callback to stream the reply and out_json to get it joined, in the same call:

const char *request =
    "{\"messages\": [{\"role\": \"user\", \"content\": \"Where is order A1?\"}],"
    " \"tools\": [{\"type\": \"function\", \"function\": {\"name\": \"lookup_order\","
    " \"parameters\": {\"type\": \"object\","
    " \"properties\": {\"id\": {\"type\": \"string\"}}}}}]}";
char *completion = NULL;
if (moss_inference_chat_completions_create(client, request, NULL, print_chunk, NULL,
                                           &completion) == OK) {
    // With your JSON library, run each call in choices[0].message.tool_calls.
    moss_free_string(completion);
} else {
    fprintf(stderr, "%s\n", moss_last_error());
}

print_chunk is the callback from Stream a reply. The joined message keeps every field of each call, with "" for an id, name or arguments the model left out. Append it to messages as it is, then one tool message per call, and call again with the same tools, as in Tool calls.

MCP servers

A tools entry of type "mcp" names an MCP server whose tools Moss Inference runs for the model within the same call, so you send no tool messages for them:

const char *request =
    "{\"messages\": [{\"role\": \"user\", \"content\": \"Where is order A1?\"}],"
    " \"tools\": [{\"type\": \"mcp\", \"server_label\": \"crm\","
    " \"server_url\": \"https://mcp.example.com/mcp\","
    " \"headers\": {\"Authorization\": \"Bearer your-crm-token\"}}]}";
char *completion = NULL;
if (moss_inference_chat_completions_create(client, request, NULL, print_chunk, NULL,
                                           &completion) == OK) {
    printf("%s\n", completion);
    moss_free_string(completion);
} else {
    fprintf(stderr, "%s\n", moss_last_error());
}

Steps arrive in each chunk’s moss_tool_step and the completion’s moss_tool_steps, and MCP failures return a negative MossResult, with status, type, code and retry_after_ms in moss_last_error_detail(), as in MCP servers.

Memory and the chunk callback

Pointer	Owner	Release
`MossInference *` from `moss_inference_new`	You	`moss_inference_free`, once no call on it is running. `NULL` is a no-op.
`*out_json` from `moss_inference_chat_completions_create`	You	`moss_free_string`. It is set only when the call returns `OK`.
`chunk_json` in the callback	`libmoss`	Nothing. It is valid until the callback returns, so copy it to keep it.
`moss_last_error()`, `moss_last_error_detail()`	`libmoss`	Nothing. They are valid until the next `libmoss` call on the same thread.
`options_json`, `request_json`	You	Your own. They only need to live until the call returns.

The callback runs on the thread that made the call, once per chunk, and never after the call returns. libmoss never reads or frees user_data.
The next chunk is read only after the callback returns, so a slow callback holds up the stream, and its time counts toward timeout_ms.
A streaming call can pass out_json too, and gets the reply as one chat.completion. When the callback stops the stream, that reply ends with the chunk it stopped at.

Options

typedef int32_t (*MossInferenceChunkFn)(const char *chunk_json, void *user_data);
MossResult moss_inference_new(const char *options_json, MossInference **out);
void moss_inference_free(MossInference *client);
MossResult moss_inference_chat_completions_create(
    const MossInference *client, const char *request_json, const char *options_json,
    MossInferenceChunkFn on_chunk, void *user_data, char **out_json);

options_json of moss_inference_new:

Key	Default	Notes
`project_id`, `project_key`	required	Your project id and key.
`api_key`	none	Your own OpenAI, Anthropic or Google key, for a vendor `model` such as `gpt-4o-mini`. See Models. It is sent as the `X-Moss-Provider-Key` header, never in the body. An empty key returns `ERR_INVALID_ARG`.
`timeout_ms`	none	Milliseconds for a whole call, the stream included. `0` turns it off.
`idle_timeout_ms`	`120000`	Milliseconds to wait for the reply to start, and for each pause in it. `0` turns it off.
`max_retries`	`2`	Retries before the reply starts. `0` turns them off.

options_json of moss_inference_chat_completions_create, which may be NULL:

Key	Default	Notes
`api_key`	the client’s	Your provider key for this call, in place of the client’s.
`timeout_ms`, `idle_timeout_ms`	the client’s	This call’s limits, in place of the client’s. `0` turns one off.
`max_retries`	the client’s	This call’s retries, in place of the client’s.

request_json of moss_inference_chat_completions_create:

Key	Default	Notes
`model`	`"moss"`	The Moss model, or a vendor model with `api_key`. See Models.
`messages`	required	OpenAI messages. A reply’s `message` can go in too.
`indexes`	every index of the project	Names of the indexes to search, at most 32. Required when the project has more than 32. An empty list skips the search, as `"retrieve": false` does.
`retrieve`	`true`	`false` skips the search, and the model answers from the messages alone.
`cache`	`true`	Moss can answer an identical request from its reply cache. `false` always asks the model.
`tools`, `tool_choice`, `parallel_tool_calls`	none	OpenAI’s tool fields. A `tools` entry can also name an MCP server. See Tool calls and MCP servers.
Other OpenAI fields	none	`temperature`, `max_tokens` and the rest go to the model. `stream` is always `true`.

Python

JavaScript

Swift (iOS)

Android

Elixir

Browser

C

Inference

Create a client

Get a reply

Stream a reply

Errors

Tool calls

MCP servers

Memory and the chunk callback

Options

​Create a client

​Get a reply

​Stream a reply

​Errors

​Tool calls

​MCP servers

​Memory and the chunk callback

​Options

Create a client

Get a reply

Stream a reply

Errors

Tool calls

MCP servers

Memory and the chunk callback

Options