EN ▾
Get API key

LLM Models API for Agents: Tool Calling Guide

An agent is a loop: the model picks a tool, your code runs it, the result goes back, and the model decides what to do next. Here is a working version of that loop, plus the limits and costs you should plan around.

Updated

Isometric loop of an orb passing through tool blocks around a central terminal, illustrating an agent workflow
01

Key points

  • The endpoint supports tools, tool_choice, streamed tool_calls and role tool results.
  • Cap your loop, validate tool arguments and retry 429 responses with a short wait.
  • Keep parallel workers at or under 8 on a full key; streams hold their slot until they end.
  • A six-step agent run with a growing context costs about half a cent.

What an agent needs from the API

If you are choosing an LLM models API for agents, the checklist is shorter than the marketing suggests. You need reliable tool calling, a context window big enough to hold the running transcript, predictable rate limits, and a price that does not punish a loop that makes many calls.

This API covers those points through the standard OpenAI interface. It supports tools and tool_choice, returns tool_calls in both normal and streamed responses, and accepts tool results as messages with role: "tool". JSON mode is available through response_format. The context window is 100,000 tokens for prompt plus completion, and the model ID is uncensored.

Since the framework side is just the OpenAI client, most agent libraries that let you set a base URL will work. You point them at https://api.llmmodelsapi.com/v1, supply your key and name the model. Setup details are in the documentation, and you can get a key from the key page.

Two limits shape the design. First, the model is text only, so any image, audio or file handling has to happen in your tools and come back as text. Second, there are no embeddings on this endpoint, so if your agent does retrieval, the vector search needs another source.

A system prompt that keeps the loop sane

Most agent failures come from vague instructions, not from the API. A good system prompt tells the model who it is, which tools exist, when to stop and what format the final answer takes. Here is one for a research helper.

You are a research assistant. You can call the tools search_notes and read_note. Call a tool only when you need information you do not already have. Use at most five tool calls. When you have enough, reply with a short answer and list the note IDs you used. If the notes do not contain the answer, say so plainly.

Notice what it does. It names the tools, sets a call budget, defines the stopping condition and gives the model permission to say it does not know. That last part matters, because an agent that is forced to answer will invent something.

Set a low temperature for tool use. Values around 0.2 to 0.4 keep tool names and argument formats stable. If your agent also writes creative text at the end, you can raise the temperature for that final call only. The seed parameter is passed through, which helps when you want to reproduce a run while debugging.

For adult fiction and roleplay agents, the same pattern works: put the persona, the scene rules and the tools in the system message. The model is tuned to answer without content refusals for lawful adult use. Sexual content involving minors remains off limits.

A minimal agent loop in Python

The loop below has one tool, a lookup in a dictionary standing in for your real data. It stops when the model replies without a tool call, or after a fixed number of steps.

import json
from openai import OpenAI

client = OpenAI(base_url="https://api.llmmodelsapi.com/v1", api_key="YOUR_KEY")

NOTES = {
    "n1": "Invoice 114 was paid on March 3.",
    "n2": "Invoice 115 is overdue by 12 days.",
}

def search_notes(query):
    return [{"id": k, "text": v} for k, v in NOTES.items() if query.lower() in v.lower()]

TOOLS = [{
    "type": "function",
    "function": {
        "name": "search_notes",
        "description": "Search internal notes by keyword",
        "parameters": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"],
        },
    },
}]

SYSTEM = "You are a research assistant. Use search_notes when needed, at most five calls. Then answer briefly."

def run(question, max_steps=6):
    messages = [
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": question},
    ]
    for step in range(max_steps):
        resp = client.chat.completions.create(
            model="uncensored",
            messages=messages,
            tools=TOOLS,
            temperature=0.3,
            max_tokens=500,
        )
        msg = resp.choices[0].message
        messages.append(msg)
        if not msg.tool_calls:
            return msg.content
        for call in msg.tool_calls:
            try:
                args = json.loads(call.function.arguments)
                result = search_notes(args["query"])
            except Exception as exc:
                result = {"error": str(exc)}
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": json.dumps(result),
            })
    return "Stopped: step limit reached."

print(run("Which invoice is overdue?"))

Three details deserve attention. The step limit stops a confused model from looping until your budget is gone. The try block turns a malformed argument string into an error message that the model can read and correct. And every tool result is sent back with the matching tool_call_id, which is how the model knows which call each answer belongs to.

Handling 429 responses and concurrency

Agents are bursty. A planner may fan out five sub-tasks at once, and each of them runs its own loop. That is where rate limits show up.

A full key allows 8 concurrent requests and 300 requests per minute. A trial key allows 2 and 60. Past the limit the API answers with HTTP 429. The error code tells you which limit you hit: rate_limited for requests per minute, concurrency for too many at once. On temporary overload you also get a 429, with a Retry-After: 5 header.

Treat all of these as retryable. Wait the number of seconds in the header, or a couple of seconds if it is absent, and send the request again. Failed requests are not charged, so retries cost nothing. A small wrapper does the job.

import time
from openai import OpenAI, RateLimitError

client = OpenAI(base_url="https://api.llmmodelsapi.com/v1", api_key="YOUR_KEY", max_retries=0)

def chat_with_retry(**kwargs):
    for attempt in range(5):
        try:
            return client.chat.completions.create(model="uncensored", **kwargs)
        except RateLimitError as err:
            wait = 5
            if err.response is not None:
                wait = int(err.response.headers.get("retry-after", 5))
            time.sleep(wait)
    raise RuntimeError("Still rate limited after 5 attempts")

To avoid the 429 in the first place, cap your worker pool. A semaphore of 8 matches the full key. If you stream responses to a user interface, remember that a stream holds its concurrency slot until the last token arrives, so a slow client reading a long answer occupies a slot the whole time. Reading the X-RateLimit-Limit-Requests and X-RateLimit-Limit-Concurrency headers lets your code adapt to whichever key it is using.

Context, memory and what a run costs

Every step of the loop resends the full transcript. That is how chat APIs work, and it means input tokens grow with each tool call. Take a six-step run where the prompt starts at 2,000 tokens and grows by 1,000 per step, with 300 output tokens per step.

  • Input tokens: 2,000 + 3,000 + 4,000 + 5,000 + 6,000 + 7,000 = 27,000
  • Output tokens: 6 x 300 = 1,800
  • Input cost: 27,000 x 0.15 / 1,000,000 = $0.00405
  • Output cost: 1,800 x 0.50 / 1,000,000 = $0.0009
  • Total: $0.00495 per run

A thousand such runs cost $4.95. That is cheap enough to let an agent think in several steps, but the growth is worth watching. With 100,000 tokens of room, a long-running agent will eventually fill the window, and a prompt that large costs 100,000 x 0.15 / 1,000,000 = $0.015 for input on every single step.

Three habits keep it under control. Truncate large tool results before they go into the transcript, for example the first 2,000 characters of a web page. Summarise old steps into one short message once the history passes a threshold. And store durable facts in your own database instead of relying on the conversation as memory.

The X-Balance-USD header on each response shows the remaining credit, so an agent supervisor can stop a run when the balance drops below a floor. Credit is prepaid from $20, and it never expires. See the pricing page for top-up bonuses.

Honest limits to plan around

Agents built on any model need guardrails, and this one is no exception. Be ready for the following.

  • Malformed arguments. The model can produce tool arguments that are not valid JSON or miss a required field. Parse defensively and return the error as the tool result.
  • Wrong tool choice. Write short, distinct tool descriptions. If two tools sound alike, the model will mix them up. Use tool_choice to force a specific call when the flow requires one.
  • Text only. No image input or output, no audio, no embeddings, no fine-tuning. Do those parts elsewhere and pass the result as text.
  • Single model. There is one open-weight model behind the endpoint, not a router across vendors. You cannot pick a smaller model for easy steps.
  • Destructive tools. Do not give an agent a delete or payment tool without a confirmation step in your own code.

Planning agents also benefit from a second pass. After the loop returns an answer, you can make one more cheap call that asks the model to check the answer against the tool results it saw, using JSON mode to return a verdict and a list of unsupported claims. That adds a few hundred tokens and catches the most common failure, which is a confident answer that ignores what the tools returned.

Finally, log every step. Store the messages, the tool arguments and the X-Request-Id header from each response. When a run goes wrong, the request ID lets you point to the exact call, and the logged transcript shows which tool result misled the model. Agents are easiest to improve when you can replay a bad run and change one thing at a time.

Test an agent with the trial before you commit. New users can claim $0.50 of credit, which at the six-step run above covers about 101 full runs (0.50 / 0.00495). Keep the trial's 2 concurrent requests in mind and run one agent at a time. For a broader look at the service, read the pay-per-token overview. Prompts are not used for training.

02

Questions and answers

Does the API support tool calling for agents?

Yes. It accepts tools and tool_choice, returns tool_calls (also when streaming), and takes results as messages with role tool.

How many agent calls can run in parallel?

A full key allows 8 concurrent requests and 300 per minute. A trial key allows 2 concurrent and 60 per minute.

What should an agent do when it gets a 429?

Wait for the Retry-After value, or a few seconds if there is none, and retry. Rejected requests are not charged.

Can the agent handle images or audio?

No. The model works with text only, so any other media must be processed by your own tools and passed in as text.

Your key is one form away

Create an account, copy the key, change the base URL. That is the whole setup.

Get API key