Group API

The Group API submits multiple inference requests together, for related tasks that you want to track as a unit. It supports chat completions and text completions, with up to 50 requests per group.

#When to use it

The Group API submits multiple asynchronous inference requests as a single unit. Unlike the Batch API, which requires JSONL file uploads, the Group API accepts requests directly in the request body. It's a good fit for:

• Small to medium batches: process up to 50 requests at once
• Related tasks: group related inference requests together
• Webhook notifications: get notified when all requests in a group complete
• Simpler integration: no file uploads or JSONL formatting
• Faster implementation: direct JSON API calls without file management

#Group API vs Batch API

#Get started

#Submit a group

Send multiple requests together as an array in the request body:

const response = await fetch(
  "https://api.inference.net/v1/slow/group/chat/completions",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      requests: [
        {
          model: "kimi-k3-fast",
          messages: [
            { role: "system", content: "You are a helpful assistant." },
            { role: "user", content: "What is the capital of France?" },
          ],
          max_tokens: 100,
        },
        {
          model: "kimi-k3-fast",
          messages: [
            { role: "system", content: "You are a helpful assistant." },
            { role: "user", content: "What is the capital of Germany?" },
          ],
          max_tokens: 100,
        },
      ],
      webhook_id: "my-webhook-123", // Optional: attach a webhook for notifications
    }),
  },
);

const result = await response.json();
console.log(result); // { groupId: "group_abc123", groupSize: 2 }

The response includes a group ID and the number of requests:

{
  "groupId": "group_xY3kL9mN2pQ",
  "groupSize": 2
}

#Retrieve group results

Once the group is processed, retrieve all generation results with the group ID:

const response = await fetch(
  `https://api.inference.net/v1/slow/group/${groupId}/generations`,
  {
    headers: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY}`,
    },
  },
);

const result = await response.json();
console.log(result.generations); // Array of all completed generations

The response includes every generation in the group:

{
  "generations": [
    {
      "id": "gen_abc123",
      "state": "Success",
      "request": {
        "model": "kimi-k3-fast",
        "messages": [
          { "role": "system", "content": "You are a helpful assistant." },
          { "role": "user", "content": "What is the capital of France?" }
        ],
        "max_tokens": 100
      },
      "response": {
        "id": "gen_abc123",
        "object": "chat.completion",
        "choices": [
          {
            "message": {
              "role": "assistant",
              "content": "The capital of France is Paris."
            },
            "finish_reason": "stop"
          }
        ],
        "usage": {
          "prompt_tokens": 25,
          "completion_tokens": 8,
          "total_tokens": 33
        }
      }
    },
    {
      "id": "gen_def456",
      "state": "Success",
      "request": {
        "model": "kimi-k3-fast",
        "messages": [
          { "role": "system", "content": "You are a helpful assistant." },
          { "role": "user", "content": "What is the capital of Germany?" }
        ],
        "max_tokens": 100
      },
      "response": {
        "id": "gen_def456",
        "object": "chat.completion",
        "choices": [
          {
            "message": {
              "role": "assistant",
              "content": "The capital of Germany is Berlin."
            },
            "finish_reason": "stop"
          }
        ],
        "usage": {
          "prompt_tokens": 25,
          "completion_tokens": 8,
          "total_tokens": 33
        }
      }
    }
  ]
}

#Use webhooks

Attach a webhook to be notified when the group finishes processing. Include the webhook_id when you submit the group:

const response = await fetch(
  "https://api.inference.net/v1/slow/group/chat/completions",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      requests: [
        /* your requests */
      ],
      webhook_id: "my-webhook-123",
    }),
  },
);

When the group finishes, your webhook receives a slow-group.completed notification. Its data.status is completed when all requests in the group finished, or failed when the group could not finish: processing failed, or the group passed its 72-hour completion window. In a failed group, check each entry's state in generations to see which requests finished.

The notification includes:

• Group ID
• Completion status
• Summary of successful and failed requests
• Custom IDs for each request (if provided)

See Webhooks for setup and the webhooks reference for the payload.

#Text completions

The Group API also supports text completions:

const response = await fetch(
  "https://api.inference.net/v1/slow/group/completions",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.INFERENCE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      requests: [
        {
          model: "kimi-k3-fast",
          prompt: "The capital of France is",
          max_tokens: 10,
        },
        {
          model: "kimi-k3-fast",
          prompt: "The capital of Germany is",
          max_tokens: 10,
        },
      ],
    }),
  },
);

#Limits

• Maximum requests per group: 50
• Request format: direct JSON (no JSONL files)
• Supported endpoints:
  • /v1/slow/group/chat/completions
  • /v1/slow/group/completions
  • /v1/slow/group/embeddings
  • /v1/slow/group/responses
• Completion time: 72 hours
• Request expiration: groups expire after 72 hours if not completed

#Best practices

1. Group related requests: use groups for requests that logically belong together (for example, analyzing multiple documents from the same source).
2. Use webhooks for notifications: instead of polling, configure a webhook to be notified when your group completes.
3. Handle individual failures: some requests in a group may fail while others succeed. Check each generation's state.
4. Stay under the limit: keep groups to 50 requests or fewer. For larger workloads, use the Batch API.
5. Include metadata: add custom IDs or metadata to your requests for easier tracking:

{
  "model": "kimi-k3-fast",
  "messages": [{ "role": "user", "content": "..." }],
  "metadata": {
    "custom_id": "doc_123",
    "type": "summary"
  }
}

#Error handling

The API validates your request structure immediately. A common error:

{
  "error": {
    "message": "Invalid request body.",
    "type": "BadRequestError",
    "fields": {
      "_errors": ["Unrecognized key(s) in object: 'webhook_url'"]
    }
  }
}

Use the correct field name:

• webhook_id (correct)
• webhook_url (incorrect — this is for the Batch API)

#When to choose the Group API

Choose the Group API when you need:

• Quick implementation without file management
• To process 50 or fewer related requests
• Webhook notifications for a set of requests
• Simple JSON-based integration

For workloads of more than 50 requests, use the Batch API instead.