Asynchronous inference

The OpenAI-compatible Asynchronous Inference API accepts individual inference requests that complete within 24 hours at reduced cost. Use /v1/slow instead of /v1 in your API calls.

Background inference is cheaper, and easier to build with when your application isn't serving real-time inference.

#How it works

The Asynchronous Inference API is a simple way to make cost-effective inference requests when you don't need an immediate response. With the /v1/slow prefix instead of /v1, you:

1. Get a request ID immediately: your API call returns at once with a unique ID.
2. Save on cost: pricing is significantly lower than synchronous requests.
3. Get flexible completion: requests complete within 24 hours.
4. Use the same API: the request format is the same as the standard endpoints.

Good fits include:

• Large-scale content generation
• Batch document processing
• Non-urgent data analysis
• Cost-sensitive workloads
• Background processing tasks

#Get started

Change your base URL from /v1/ to /v1/slow/. The API stays fully compatible with the OpenAI SDK, and the same API key works for both.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inference.net/v1/slow", // Note the /v1/slow prefix
  apiKey: process.env.INFERENCE_API_KEY,
});

#Make an asynchronous request

#Submit a request

Make requests exactly as you would with the standard API. The response includes a request ID instead of the completion result:

const response = await client.chat.completions.create({
  model: "kimi-k3-fast",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is the capital of France?" },
  ],
  max_tokens: 1000,
});

console.log(response.id); // Returns immediately with request ID

The initial response includes a unique request ID:

{
  "id": "N2mZQjrvh-k_m8nMMN7Jn",
  "choices": [],
  "created": 1749061362809,
  "model": "kimi-k3-fast",
  "object": "chat.completion"
}

#Retrieve the result

Once the request is processed, retrieve the result from the generation endpoint:

const response = await fetch(
  `https://api.inference.net/v1/generation/${generationId}`,
  {
    headers: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY}`,
    },
  },
);
const result = await response.json();

The completed response includes both the original request and the generation result:

{
  "request": {
    "messages": [
      { "content": "You are a helpful assistant.", "role": "system" },
      { "content": "What is the meaning of life?", "role": "user" }
    ],
    "model": "kimi-k3-fast",
    "stream": false,
    "max_tokens": 8,
    "metadata": { "webhook_id": "mPufxRcrw" }
  },
  "response": {
    "id": "N2mZQjrvh-k_m8nMMN7Jn",
    "object": "chat.completion",
    "created": 1749061362,
    "model": "kimi-k3-fast",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "The meaning of life is a complex and",
          "reasoning_content": null,
          "tool_calls": null
        },
        "logprobs": null,
        "finish_reason": "length",
        "matched_stop": null
      }
    ],
    "usage": {
      "prompt_tokens": 48,
      "total_tokens": 56,
      "completion_tokens": 8,
      "prompt_tokens_details": null
    },
    "system_fingerprint": ""
  },
  "state": "Success",
  "stateMessage": "Generation successful",
  "finishedAt": "2025-06-04T18:22:42.912Z"
}

#Request states

#Best practices

1. Store request IDs: always save the returned request ID for later retrieval.
2. Use webhooks: instead of polling, set up webhooks to be notified when requests complete. See Webhooks.
3. Handle failures: have a fallback plan for requests that fail during processing.
4. Batch when possible: for many requests, use the Batch API or the Group API.

#Supported endpoints

• /v1/slow/chat/completions
• /v1/slow/completions
• /v1/slow/embeddings

Replace /v1/ with /v1/slow/ in your existing code to use asynchronous processing.

#Pricing and limits

• Pricing: significantly reduced compared to synchronous requests (contact sales for specific rates)
• Completion time: 24 hours
• Rate limits: more generous than synchronous endpoints. See Rate limits.
• Request expiration: requests expire after 24 hours if not completed

For specific pricing and higher rate limits, contact support@inference.net.