Moss - Real-time Semantic Search for Conversational AI

Six steps from a project key to a streamed reply that answers from your data. Run the steps in order, and every block runs as written once the two environment variables are set.

Get your project id and key

In the Moss portal, copy the project id, then open API Keys and create a key. Inference bills from credits with no included allowance, so add credits on the Billing page first (see Pricing). Export the id and key:

export MOSS_PROJECT_ID=your_project_id
export MOSS_PROJECT_KEY=your_project_key

Install a client

curl needs no install until step 5, which creates the index with the Python or JavaScript SDK.

pip install "moss>=1.22.0"

npm install @moss-js/moss@^1.22.0

pip install openai

Get a response

curl https://inference.moss.dev/v1/responses \
  -H "Authorization: Bearer $MOSS_PROJECT_ID:$MOSS_PROJECT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "moss/fastlane-1",
    "instructions": "You are the support agent for Acme.",
    "input": "Do you ship to Canada?"
  }'

import asyncio
import os

from moss import MossInference

client = MossInference(os.environ["MOSS_PROJECT_ID"], os.environ["MOSS_PROJECT_KEY"])


async def main():
    response = await client.responses.create(
        instructions="You are the support agent for Acme.",
        input="Do you ship to Canada?",
    )
    print(response.output_text)


asyncio.run(main())

import { MossInference } from '@moss-js/moss'

const client = new MossInference(process.env.MOSS_PROJECT_ID!, process.env.MOSS_PROJECT_KEY!)
const response = await client.responses.create({
  instructions: 'You are the support agent for Acme.',
  input: 'Do you ship to Canada?',
})
console.log(response.output_text)

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.moss.dev/v1",
    api_key=f"{os.environ['MOSS_PROJECT_ID']}:{os.environ['MOSS_PROJECT_KEY']}",
)
response = client.responses.create(
    model="moss/fastlane-1",
    instructions="You are the support agent for Acme.",
    input="Do you ship to Canada?",
)
print(response.output_text)

The model is moss/fastlane-1 unless you name another. The reply is an OpenAI Response object, and the SDKs expose its text as output_text. On a new project nothing is grounded yet, so the answer is generic. Step 5 fixes that.

Stream it

Add stream: true and read response.output_text.delta events as they arrive.

curl -N https://inference.moss.dev/v1/responses \
  -H "Authorization: Bearer $MOSS_PROJECT_ID:$MOSS_PROJECT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "moss/fastlane-1",
    "stream": true,
    "instructions": "You are the support agent for Acme.",
    "input": "Do you ship to Canada?"
  }'

import asyncio
import os

from moss import MossInference

client = MossInference(os.environ["MOSS_PROJECT_ID"], os.environ["MOSS_PROJECT_KEY"])


async def main():
    stream = await client.responses.create(
        instructions="You are the support agent for Acme.",
        input="Do you ship to Canada?",
        stream=True,
    )
    async with stream:
        async for event in stream:
            if event.type == "response.output_text.delta":
                print(event.delta, end="", flush=True)


asyncio.run(main())

import { MossInference } from '@moss-js/moss'

const client = new MossInference(process.env.MOSS_PROJECT_ID!, process.env.MOSS_PROJECT_KEY!)
const stream = await client.responses.create({
  instructions: 'You are the support agent for Acme.',
  input: 'Do you ship to Canada?',
  stream: true,
})
for await (const event of stream) {
  if (event.type === 'response.output_text.delta') process.stdout.write(event.delta ?? '')
}

Streaming lists every event and shows the Chat Completions form.

Ground it in your data

Create an index named faq with four documents, then name it in the call. Pick another name if faq is taken.

import asyncio
import os

from moss import DocumentInfo, MossClient, MossInference

client = MossClient(os.environ["MOSS_PROJECT_ID"], os.environ["MOSS_PROJECT_KEY"])
inference = MossInference(os.environ["MOSS_PROJECT_ID"], os.environ["MOSS_PROJECT_KEY"])

documents = [
    DocumentInfo(id="doc1", text="How do I track my order? You can track your order by logging into your account.", metadata={"category": "shipping"}),
    DocumentInfo(id="doc2", text="What is your return policy? We offer a 30-day return policy for most items.", metadata={"category": "returns"}),
    DocumentInfo(id="doc3", text="How can I change my shipping address? Contact our customer service team.", metadata={"category": "support"}),
    DocumentInfo(id="doc4", text="Do you ship to Canada? Yes, we ship to Canada and the US. Delivery takes 5 to 7 business days.", metadata={"category": "shipping"}),
]


async def main():
    await client.create_index("faq", documents, "moss-minilm")
    response = await inference.responses.create(
        instructions="You are the support agent for Acme.",
        input="Do you ship to Canada?",
        indexes=["faq"],
    )
    print(response.output_text)


asyncio.run(main())

import { MossClient, MossInference, type DocumentInfo } from '@moss-js/moss'

const client = new MossClient(process.env.MOSS_PROJECT_ID!, process.env.MOSS_PROJECT_KEY!)
const inference = new MossInference(process.env.MOSS_PROJECT_ID!, process.env.MOSS_PROJECT_KEY!)

const documents: DocumentInfo[] = [
  { id: 'doc1', text: 'How do I track my order? You can track your order by logging into your account.', metadata: { category: 'shipping' } },
  { id: 'doc2', text: 'What is your return policy? We offer a 30-day return policy for most items.', metadata: { category: 'returns' } },
  { id: 'doc3', text: 'How can I change my shipping address? Contact our customer service team.', metadata: { category: 'support' } },
  { id: 'doc4', text: 'Do you ship to Canada? Yes, we ship to Canada and the US. Delivery takes 5 to 7 business days.', metadata: { category: 'shipping' } },
]

await client.createIndex('faq', documents, { modelId: 'moss-minilm' })
const response = await inference.responses.create({
  instructions: 'You are the support agent for Acme.',
  input: 'Do you ship to Canada?',
  indexes: ['faq'],
})
console.log(response.output_text)

The reply should now mention delivery in 5 to 7 business days. With curl or an OpenAI client, send the header X-Moss-Indexes: faq instead of indexes. Without the header, Moss does not search. Ground answers in your indexes covers the options and the cost.

Pick a model

Name any model from the Models page in model. The search of your indexes works the same on all of them.

curl https://inference.moss.dev/v1/responses \
  -H "Authorization: Bearer $MOSS_PROJECT_ID:$MOSS_PROJECT_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Moss-Indexes: faq" \
  -d '{
    "model": "anthropic/claude-haiku-5.5",
    "instructions": "You are the support agent for Acme.",
    "input": "Do you ship to Canada?"
  }'

import asyncio
import os

from moss import MossInference

inference = MossInference(os.environ["MOSS_PROJECT_ID"], os.environ["MOSS_PROJECT_KEY"])


async def main():
    response = await inference.responses.create(
        model="anthropic/claude-haiku-5.5",
        instructions="You are the support agent for Acme.",
        input="Do you ship to Canada?",
        indexes=["faq"],
    )
    print(response.output_text)


asyncio.run(main())

import { MossInference } from '@moss-js/moss'

const inference = new MossInference(process.env.MOSS_PROJECT_ID!, process.env.MOSS_PROJECT_KEY!)
const response = await inference.responses.create({
  model: 'anthropic/claude-haiku-5.5',
  instructions: 'You are the support agent for Acme.',
  input: 'Do you ship to Canada?',
  indexes: ['faq'],
})
console.log(response.output_text)

Next steps

Models

Prompt limits, prices and notes for every model.

Streaming

Every event, stopping early, errors mid-stream.

Tool calling

Function tools, the loop, structured output.

SDKs

Python, JavaScript, Swift, Android, Elixir, browser and C.

Pricing

Credits, tokens, caching and retrievals.

Errors and limits

Every status and code, limits, retries.

Ground answers in your indexes

Choose indexes, turn retrieval off, what it costs.

Get started

Guides

SDKs

Reference

Quickstart

Next steps

Models

Streaming

Tool calling

SDKs

Pricing

Errors and limits

Ground answers in your indexes

​Next steps

Models

Streaming

Tool calling

SDKs

Pricing

Errors and limits

Ground answers in your indexes

Next steps