EN ▾
Get API key

LLM API in Python: Chat, Streaming and Tool Calls

This llm api python guide uses the official OpenAI SDK pointed at an OpenAI-compatible endpoint. You get a first call, streaming, a tool call and retry logic in code you can paste and run.

Updated

Ribbon of code blocks flowing from a laptop into chat bubbles, illustrating a Python LLM API call
01

Key points

  • Install openai, set the base URL and key, and use the model ID uncensored.
  • Stream with stream=True and close the stream early to free the concurrency slot.
  • Tool calls return arguments you must parse and validate before running anything.
  • Handle 429 with Retry-After and cap parallel requests at 8 on a full key.

Install the SDK and make the first call

If you want an llm api python setup that works in five minutes, the OpenAI SDK is all you need. The API is OpenAI-compatible, so you keep the official client and change two values: the base URL and the key. The model ID is uncensored.

First get a key. Sign up on the API key page, claim the one-click trial credit, and copy the key. Then install the client and keep the key out of your source code.

pip install openai
export LLM_API_KEY="your-key-here"

Now the smallest useful program:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.llmmodelsapi.com/v1",
    api_key=os.environ["LLM_API_KEY"],
)

resp = client.chat.completions.create(
    model="uncensored",
    messages=[
        {"role": "system", "content": "You are a concise technical assistant."},
        {"role": "user", "content": "Explain what a context window is in two sentences."},
    ],
    temperature=0.7,
)

print(resp.choices[0].message.content)
print(resp.usage)

That is the whole integration. resp.usage gives prompt and completion token counts, which you can multiply by the rates on the pricing page ($0.15 per 1M input tokens, $0.50 per 1M output tokens) to see what each call cost. The same endpoint also accepts POST /v1/responses if your code already uses that format, and GET /v1/models lists the model ID.

What the request and response look like

The OpenAI client hides the HTTP details, but it helps to know them when something goes wrong. The SDK sends a POST to /v1/chat/completions with a bearer token. You can reproduce the call with curl and compare.

curl https://api.llmmodelsapi.com/v1/chat/completions \
  -H "Authorization: Bearer $LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "uncensored", "messages": [{"role": "user", "content": "Say hello"}]}'

Useful parameters that are passed through to the model: temperature, top_p, stop, seed and the frequency and presence penalties. max_tokens is optional. If you leave it out, the completion can use whatever remains of the 100,000-token window (prompt and completion together).

Two response headers are worth logging in every call: X-Request-Id for support questions, and X-Balance-USD for your remaining credit. With the Python SDK you reach them through the raw response:

raw = client.chat.completions.with_raw_response.create(
    model="uncensored",
    messages=[{"role": "user", "content": "Ping"}],
    max_tokens=20,
)
print(raw.headers.get("X-Request-Id"))
print(raw.headers.get("X-Balance-USD"))
completion = raw.parse()
print(completion.choices[0].message.content)

Streaming tokens as they arrive

For chat interfaces and anything a person is waiting on, stream the answer. Set stream=True and iterate over the chunks. The API sends them over server-sent events, and the last chunk carries usage when you ask for it.

stream = client.chat.completions.create(
    model="uncensored",
    messages=[{"role": "user", "content": "Write a short product description for a mechanical keyboard."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)

Two details trip people up. First, some chunks have an empty choices list (the usage chunk, for example), so guard before indexing. Second, a streamed request holds its concurrency slot until the stream ends. If you open a stream and abandon it without closing, that slot stays busy and later calls can fail with a concurrency error. Use the stream in a with block or call stream.close() when you stop reading early.

If a stream cuts off with an unclear message, the article on how to use an LLM model on this site covers the basics of reading errors and retrying.

A tool call against the model

Tool calling lets the model ask your code to run a function and then continue with the result. The flow has three steps: send the tool definitions, run whatever the model requests, and send the output back with role: "tool".

import json

def get_order_status(order_id: str) -> str:
    # Replace with a real lookup
    return json.dumps({"order_id": order_id, "status": "shipped"})

tools = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Look up the status of an order by its ID",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string"}},
            "required": ["order_id"],
        },
    },
}]

messages = [{"role": "user", "content": "Where is order A-1042?"}]
first = client.chat.completions.create(
    model="uncensored", messages=messages, tools=tools, tool_choice="auto"
)

msg = first.choices[0].message
if msg.tool_calls:
    messages.append(msg)
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": get_order_status(**args),
        })
    final = client.chat.completions.create(model="uncensored", messages=messages, tools=tools)
    print(final.choices[0].message.content)
else:
    print(msg.content)

Always parse call.function.arguments as JSON inside a try block in real code, because models occasionally produce arguments that do not match your schema. Validate before you execute anything with side effects. For longer agent loops, see the guide to using this API for agents.

Tool calls also stream: the arguments arrive in pieces in delta.tool_calls, and you concatenate them by index until the stream finishes.

JSON mode for structured output

When you need machine-readable output rather than prose, set response_format to a JSON object. Describe the shape you want in the prompt, since JSON mode guarantees valid JSON syntax but not your particular schema.

resp = client.chat.completions.create(
    model="uncensored",
    messages=[
        {"role": "system", "content": "Reply only with JSON: {\"sentiment\": \"positive|negative|neutral\", \"confidence\": 0-1}"},
        {"role": "user", "content": "The setup took ten minutes and everything just worked."},
    ],
    response_format={"type": "json_object"},
    temperature=0,
)
data = json.loads(resp.choices[0].message.content)
print(data["sentiment"])

Use temperature=0 for extraction and classification so repeated runs give the same labels. Add seed if you want to pin down sampling further, and still validate the parsed result against your own schema.

Limits, retries and error handling

A full key allows 8 concurrent requests and 300 requests per minute. A trial key allows 2 concurrent and 60 per minute. Going over either returns HTTP 429. The error code tells you which limit you hit: rate_limited for requests per minute and concurrency for too many at once. On temporary overload you also get a 429 with a Retry-After: 5 header.

The OpenAI SDK already retries some failures, but you should set the policy yourself:

import os
import time
from openai import OpenAI, RateLimitError, APIConnectionError

client = OpenAI(base_url="https://api.llmmodelsapi.com/v1", api_key=os.environ["LLM_API_KEY"], max_retries=0, timeout=120)

def ask(messages, tries=5):
    delay = 2
    for attempt in range(tries):
        try:
            return client.chat.completions.create(model="uncensored", messages=messages)
        except RateLimitError as e:
            wait = float(e.response.headers.get("Retry-After", delay))
            time.sleep(wait)
            delay = min(delay * 2, 30)
        except APIConnectionError:
            time.sleep(delay)
            delay = min(delay * 2, 30)
    raise RuntimeError("gave up after retries")

If you fan out work with threads or asyncio, cap the pool at 8 (or 2 on a trial key) with a semaphore. That keeps you under the concurrency limit without relying on retries. Failed or refused requests are not charged, so retrying does not cost extra for the failed attempts.

Check your balance from code with GET /v1/balance, or read X-Balance-USD from any response. The full parameter list lives in the API docs.

What it costs to run these examples

The examples above are tiny. A call with 300 prompt tokens and 200 completion tokens costs 300 x $0.15 / 1M plus 200 x $0.50 / 1M, which is $0.000045 plus $0.0001, or $0.000145. Roughly 6,900 such calls fit into one dollar. The $0.50 trial credit covers around 3,400 of them, and it is valid for 7 days with no payment details.

After the trial, top up any whole amount from $20 to $500, with a bonus of 5% from $50, 10% from $100 and 15% from $200. Credit does not expire and there is no subscription. Prompts are not used for training.

02

Questions and answers

Can I use the official OpenAI Python library?

Yes. Set base_url to the API address and api_key to your key, then call chat.completions.create with model uncensored.

Does the API support async Python?

Yes, use AsyncOpenAI with the same base URL and key. Keep concurrent tasks at or below 8 on a full key, or 2 on a trial key.

How do I see how many tokens a call used?

Read resp.usage on a normal call, or pass stream_options with include_usage on a streamed call. Multiply by $0.15 per 1M input and $0.50 per 1M output tokens.

How long can my prompt be?

The context window is 100,000 tokens, prompt and completion combined. The request body is limited to 8 MB.

Your key is one form away

Create an account, copy the key, change the base URL. That is the whole setup.

Get API key