> ## Documentation Index
> Fetch the complete documentation index at: https://docs.moss.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference

> Chat completions that answer from your indexes, in OpenAI's shape, with the Moss Swift SDK.

[Swift SDK](https://docs.moss.dev/docs/reference/swift/api.md) / Inference

`MossInference` is a Swift client for
[Moss Inference](https://docs.moss.dev/docs/api-reference/v1/inference/chat-completions.md), which searches your indexes in
the cloud and answers in OpenAI's chat completions shape. `chat.completions.create` returns the
whole reply, and `createStreaming` streams it.

> [!NOTE]
>
> Requires the `Moss` Swift package 0.12.0 or later, installed as in [Install](https://docs.moss.dev/docs/reference/swift/api.md#install).

## Create a client

```swift theme={null}
import Moss

let client = try MossInference(
  projectId: "your_project_id",
  projectKey: "your_project_key"
)
```

The project id and key come from the [Moss portal](https://portal.usemoss.dev).
Build one client and reuse it for every call. The initializer throws
[`InferenceError`](#errors) for an argument it refuses.

## Get a reply

```swift theme={null}
let completion = try await client.chat.completions.create(
  messages: [.system("Answer from the FAQ."), .user("Do you ship to Canada?")],
  indexes: ["faq", "policies"]
)
print(completion.choices.first?.message.content ?? "")
```

`completion.requestId` is the response's `X-Moss-Request-Id`, and so is `stream.requestId` on a
stream. Include it when you report a problem. Both are `nil` when the response has none.

## Stream a reply

```swift theme={null}
let stream = try await client.chat.completions.createStreaming(
  messages: [.user("Do you ship to Canada?")],
  indexes: ["faq"]
)
for try await chunk in stream {
  print(chunk.choices.first?.delta.content ?? "", terminator: "")
}
```

To stop early, call `stream.cancel()`: it closes the connection, which stops the
model, and ends the loop without an error.

Only the last chunk carries `usage`, and it has no choices, which is why the loop reads `choices.first`.

## Errors

A failed call throws `InferenceError`, from the `for try await` loop once a
stream has started. `status` is the HTTP status of a refused request, `type`
names the failure, and `message` describes it. `code` is the error code Moss
Inference sent, such as `"rate_limit_exceeded"`, `requestId` is the response's
`X-Moss-Request-Id`, and `retryAfter` is the wait in seconds that a
`Retry-After` header asks for. A field the failure lacks is `nil`.

The SDK sets these types itself, with `status` `nil`:

| `type` | Cause |
| - | - |
| `invalid_request_error` | An argument the SDK rejects before sending, such as an empty key, or an index name that is empty or holds a comma. |
| `connection_error` | Moss Inference could not be reached, or the connection broke. |
| `timeout` | A time limit ran out, such as `timeout` or `idleTimeout`. |
| `incomplete_stream` | The stream ended before `[DONE]`. |

Every other error reports what Moss Inference sent. See
[Errors](https://docs.moss.dev/docs/api-reference/v1/inference/chat-completions.md#errors), and
[Limits](https://docs.moss.dev/docs/api-reference/v1/inference/chat-completions.md#limits) for a `429`.

## Tool calls

Pass your functions in `tools`. When the model calls them, append its message unchanged with
`ChatMessage(message)`, then one `.tool(callId:content:)` per call, and call again, as in
[Tool calls](https://docs.moss.dev/docs/api-reference/v1/inference/chat-completions.md#tool-calls):

```swift theme={null}
let tools: [Tool] = [
  .function(
    name: "lookup_order",
    description: "Find an order by its id.",
    parameters: ["type": "object", "properties": ["id": ["type": "string"]], "required": ["id"]]
  )
]
var messages: [ChatMessage] = [.user("Where is order A1?")]
let message = try await client.chat.completions.create(messages: messages, tools: tools)
  .choices[0].message
messages.append(ChatMessage(message))
for call in message.toolCalls ?? [] {
  let output = try await lookupOrder(call.function.arguments) // your code, returns a String
  messages.append(.tool(callId: call.id, content: output))
}
let answer = try await client.chat.completions.create(messages: messages, tools: tools)
print(answer.choices.first?.message.content ?? "")
```

`ChatMessage(message)` keeps every field of each tool call. `createStreaming` takes the same
`tools` and streams each call in pieces, in `delta.toolCalls`. `create` returns the calls joined,
so use it for a reply you send back.

## MCP servers

With an `McpServer` in `tools`, Moss Inference calls that server's tools itself, in the same call:

```swift theme={null}
let crm = McpServer(
  serverLabel: "crm",
  serverURL: "https://mcp.example.com/mcp",
  headers: ["Authorization": "Bearer \(crmToken)"]
)
let stream = try await client.chat.completions.createStreaming(
  messages: [.user("Where is order A1?")],
  tools: [.mcp(crm)]
)
for try await chunk in stream {
  if let step = chunk.toolStep {
    print(step.type, step.serverLabel, step.name)
  }
  print(chunk.choices.first?.delta.content ?? "", terminator: "")
}
```

`McpServer` also takes `allowedTools` and `transport`, `.streamableHTTP` or `.sse`. Steps arrive
in `chunk.toolStep` and `completion.toolSteps`, with camelCase fields such as `isError`, and MCP
failures throw `InferenceError` with `status`, `type`, `code` and `retryAfter`, as in
[MCP servers](https://docs.moss.dev/docs/api-reference/v1/inference/chat-completions.md#mcp-servers).

## Options

`MossInference(...)`:

| Option | Default | Notes |
| - | - | - |
| `projectId`, `projectKey` | required | Your project id and key. |
| `apiKey` | none | Your own OpenAI, Anthropic or Google key, for a vendor `model` such as `gpt-4o-mini`. See [Models](https://docs.moss.dev/docs/api-reference/v1/inference/chat-completions.md#models). It is sent as the `X-Moss-Provider-Key` header, never in the body. An empty key throws `InferenceError`. |
| `timeout` | none | Seconds for a whole call, the stream included. `nil` or `.infinity` turns it off. |
| `idleTimeout` | `120` | Seconds to wait for the reply to start, and for each pause in it. `nil` or `.infinity` turns it off. |
| `maxRetries` | `2` | Retries before the reply starts. `0` turns them off. |

`create` and `createStreaming` take the same labeled arguments, in this order:

| Option | Default | Notes |
| - | - | - |
| `model` | `"moss"` | The Moss model, or a vendor model with `apiKey`. See [Models](https://docs.moss.dev/docs/api-reference/v1/inference/chat-completions.md#models). |
| `messages` | required | `[ChatMessage]`. `.system`, `.user` and `.assistant` take the text, `.tool(callId:content:)` a tool's result, `ChatMessage(message)` a reply to send back, and `.raw` any other message as a JSON object. |
| `tools` | `[]` | `[Tool]`: `.function(name:description:parameters:strict:)` for a function you run, `.mcp(server)` for an [MCP server](#mcp-servers), or `.raw(object)` for any other entry. See [Tool calls](#tool-calls). |
| `toolChoice` | `nil` | `.auto`, `.required`, `.function(name)`, `.never` (sent as `none`), or `.raw(object)` for any other object. |
| `parallelToolCalls` | `nil` | Whether the model may call several tools at once. |
| `indexes` | every index of the project | Names of the indexes to search, at most 32. Required when the project has more than 32. An empty list skips the search, as `retrieve: false` does. |
| `retrieve` | `true` | `false` skips the search, and the model answers from the messages alone. |
| `cache` | `true` | Moss can answer an identical request from its reply cache. `false` always asks the model. |
| `apiKey` | the client's | Your provider key for this call, in place of the client's. |
| `timeout`, `idleTimeout` | the client's | This call's limits, in place of the client's. `.infinity` turns one off, and `nil` keeps the client's. |
| `maxRetries` | the client's | This call's retries, in place of the client's. |
| `extra` | `[:]` | Other OpenAI body fields as `[String: JSONValue]`, such as `["temperature": 0.2]`. The call's own `model`, `messages` and any `tools`, `toolChoice` or `parallelToolCalls` it sets win over the same keys here, and `stream` is always `true`. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.