Tool Calling (Function Calling)

Tool Calling (also known as Function Calling) allows models to go beyond text generation by returning a structured “function call request” when needed. Your program executes the actual logic (check weather, query database, place order), returns the result to the model, and the model generates a final response based on it. RouteAPI supports tool calling on both /v1/chat/completions (OpenAI format) and /v1/messages (Claude format).

1. Tool Calling Overview

What is Tool Calling

Tool calling is a multi-turn exchange:

1. You declare a set of “tools” in the request (each tool has a name, description, and parameter schema).
2. The model determines whether a tool is needed; if so, it returns a call request with parameters instead of a direct answer.
3. Your code parses the parameters, executes the actual logic, and obtains the result.
4. You pass the result back to the model, which generates a natural language response for the user.

The model itself does not execute any code; it only decides “which tool to call” and “what parameters to pass”. Actual execution always happens on your side, meaning security boundaries (permission checks, parameter filtering) are under your control.

Use Cases

Scenario	Description
Real-time data queries	Weather, exchange rates, stock prices, inventory—information not in the model’s training data
Internal system integration	Query databases, call internal APIs, read order status
Execute actions	Place orders, send emails, create tickets—operations with side effects
Structured extraction	Force the model to output according to a fixed schema, equivalent to structured output
Agent orchestration	Agent frameworks use tool calling to drive multi-step tasks

Supported Models

Tool calling capability depends on the selected model. Most mainstream models (OpenAI GPT series, Claude series, Gemini series, etc.) support it, but details like maximum tool count and parallel calling support vary. The model list endpoint does not return a tool-calling capability field — consult the model provider’s documentation, or send one real request with tools to the target model to verify, see Models.

2. Tool Definition Format (OpenAI)

In /v1/chat/completions, tools are declared via a top-level tools array, where each tool is a nested object with type: "function".

tools Array Structure

{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Query current weather for a specified city. Use full city name in Chinese.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": { "type": "string", "description": "City name, e.g., Beijing" },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"],
              "description": "Temperature unit, default celsius"
            }
          },
          "required": ["city"]
        }
      }
    }
  ]
}

function schema

Field	Type	Required	Description
type	string	Yes	Fixed as "function"
function.name	string	Yes	Tool name, can only contain letters, numbers, underscores, and hyphens
function.description	string	Recommended	Tool purpose description, model uses this to decide whether to call
function.parameters	object	No	Parameter definition, standard JSON Schema

The quality of description directly determines whether the model will correctly choose the tool. Clearly describe the purpose, meaning of each parameter, value range, and boundary conditions—more effective than adjusting any sampling parameters.

parameters (JSON Schema)

parameters uses standard JSON Schema to describe parameter structure:

• type: Usually "object".
• properties: Type, description, enum values for each parameter.
• required: List of required parameter names.
• enum: Restrict value range, significantly reduces probability of model passing wrong values.

Tools with no parameters should still provide an empty schema: "parameters": { "type": "object", "properties": {} }.

3. Tool Definition Format (Claude)

In /v1/messages, tool definitions are flat structures without outer type and function wrappers, and the parameter field is named input_schema.

tools Array Differences

{
  "tools": [
    {
      "name": "get_weather",
      "description": "Query current weather for a specified city. Use full city name in Chinese.",
      "input_schema": {
        "type": "object",
        "properties": {
          "city": { "type": "string", "description": "City name, e.g., Beijing" },
          "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
        },
        "required": ["city"]
      }
    }
  ]
}

input_schema

The internal structure of input_schema is identical to OpenAI’s parameters, both standard JSON Schema. The difference is only in the outer wrapper.

Comparison of Two Formats

Item	OpenAI (/v1/chat/completions)	Claude (/v1/messages)
Outer wrapper	{ "type": "function", "function": {...} }	Direct flat structure, no wrapper
Tool name field	function.name	name
Description field	function.description	description
Parameter field	function.parameters	input_schema
Parameter schema	Standard JSON Schema	Standard JSON Schema (identical)
Model return	message.tool_calls array	tool_use block in content
Result passing role	Independent role: "tool" message	tool_result block in user message
Result association field	tool_call_id	tool_use_id

For complete Claude protocol details (including is_error, streaming input_json_delta, etc.), see Claude Messages Protocol. Subsequent examples on this page default to OpenAI format.

4. tool_choice Options

tool_choice controls how the model selects tools.

Value	Behavior
"auto"	Model decides whether to call a tool and which one. Default when tools is present
"none"	Prohibit calling any tool, model only outputs text
"required"	Must call at least one tool, but model chooses which
{ "type": "function", "function": { "name": "get_weather" } }	Force call specified tool

auto (default)

{ "tool_choice": "auto" }

Most common. Suitable for scenarios where “user questions sometimes need tools, sometimes direct answers”.

none

{ "tool_choice": "none" }

Temporarily disable tools while keeping definitions. Often used in the closing stage of “let model summarize, don’t call tools again”.

required

{ "tool_choice": "required" }

Force model to take tool path. Suitable for scenarios like structured extraction that “must produce structured results”.

Specify Specific Tool

{
  "tool_choice": {
    "type": "function",
    "function": { "name": "get_weather" }
  }
}

Claude format equivalent is { "type": "tool", "name": "get_weather" }, required corresponds to { "type": "any" }, none corresponds to { "type": "none" }.

5. Tool Calling Flow

A complete tool call involves at least two rounds of requests.

First Round: Model Returns tool_calls

Send request with tools:

curl https://api.routeapi.ai/v1/chat/completions \
  -H "Authorization: Bearer $ROUTEAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5",
    "messages": [
      { "role": "user", "content": "How is the weather in Beijing now?" }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Query current weather for a specified city.",
          "parameters": {
            "type": "object",
            "properties": { "city": { "type": "string" } },
            "required": ["city"]
          }
        }
      }
    ]
  }'

When the model determines a tool is needed, finish_reason is tool_calls, message.content is null, and tool_calls contains the call request:

{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"city\": \"Beijing\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}

Note arguments is a JSON string, not an object; you need to JSON.parse / json.loads it yourself. The model may occasionally generate invalid JSON, so parsing should be in a try/catch.

Execute Tool

Dispatch by function.name to your actual logic, execute with parsed parameters:

import json

args = json.loads(tool_call["function"]["arguments"])
result = get_weather(**args)  # "Beijing, sunny, 23 celsius"

Second Round: Pass Tool Result

Add the first round’s assistant message (with tool_calls) back to messages as-is, then append a role: "tool" message carrying the result. tool_call_id must exactly match the id from the first round:

{
  "model": "gpt-5.5",
  "messages": [
    { "role": "user", "content": "How is the weather in Beijing now?" },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": { "name": "get_weather", "arguments": "{\"city\": \"Beijing\"}" }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_abc123",
      "content": "Beijing, sunny, temperature 23 celsius, humidity 45%."
    }
  ],
  "tools": []
}

Model Generates Final Response

The second round request returns a natural language result, finish_reason returns to stop:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Beijing is currently sunny with a temperature of 23 celsius and humidity at 45%, quite comfortable."
      },
      "finish_reason": "stop"
    }
  ]
}

6. Multi-Turn Tool Calling

Sequential Multiple Tool Calls

The model may need multiple rounds of tool calls to complete a task: first check order number, then use order number to check logistics. Each round follows the “model returns tool_calls → execute → pass back result” loop until finish_reason changes back to stop. Production code should be written as a loop with a maximum turn limit to prevent infinite loops:

MAX_TURNS = 5
for _ in range(MAX_TURNS):
    resp = call_model(messages)
    msg = resp["choices"][0]["message"]
    messages.append(msg)
    if not msg.get("tool_calls"):
        break  # Model gives final response
    for tc in msg["tool_calls"]:
        result = dispatch(tc)  # Execute and return string
        messages.append({
            "role": "tool",
            "tool_call_id": tc["id"],
            "content": result,
        })

Parallel Tool Calls

In one round, the model may simultaneously request multiple independent tools (e.g., checking weather for both Beijing and Shanghai), resulting in a multi-element tool_calls array. You need to append a corresponding role: "tool" message for each tool_call, with tool_call_id aligned one-to-one; missing one will cause an error in the next round.

To disable parallelism and force the model to call only one tool at a time, add "parallel_tool_calls": false in OpenAI format, or add "disable_parallel_tool_use": true in tool_choice for Claude format.

7. Streaming Tool Calls

When stream: true is set, tool call parameters are returned incrementally in chunks.

tool_calls in Streaming Output

Each SSE chunk’s delta.tool_calls contains index (identifies which tool call), and function.arguments is a fragment of the parameter JSON:

data: {"choices":[{"delta":{"tool_calls":[{"index":0,"id":"call_abc123","type":"function","function":{"name":"get_weather","arguments":""}}]}}]}

data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"city\":"}}]}}]}

data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"Beijing\"}"}}]}}]}

data: {"choices":[{"finish_reason":"tool_calls"}]}

data: [DONE]

How to Handle Incremental delta

Accumulate by index groups: id and name usually only appear in the first fragment, arguments needs all fragments concatenated before parsing. Do not attempt to parse intermediate states during concatenation (that’s incomplete JSON).

buffers = {}  # index -> {"id", "name", "args"}
for chunk in stream:
    for tc in chunk["choices"][0]["delta"].get("tool_calls", []):
        i = tc["index"]
        buf = buffers.setdefault(i, {"id": "", "name": "", "args": ""})
        if tc.get("id"):
            buf["id"] = tc["id"]
        fn = tc.get("function", {})
        if fn.get("name"):
            buf["name"] = fn["name"]
        if fn.get("arguments"):
            buf["args"] += fn["arguments"]

# Parse after stream ends
for buf in buffers.values():
    args = json.loads(buf["args"])

Claude format streaming tool parameters are returned incrementally via input_json_delta event’s partial_json, accumulated by index then parsed; details in Claude Messages Protocol.

8. Model Support Status

Models Supporting Tool Calling

Most mainstream models aggregated by RouteAPI support tool calling, including OpenAI GPT series, Claude series, Gemini series, etc. First use the model list endpoint to confirm the model is available in your account:

curl https://api.routeapi.ai/v1/models \
  -H "Authorization: Bearer $ROUTEAPI_KEY"

This endpoint only returns whether a model is available and which endpoints it can be used with — it does not return tool-calling capability. To determine tool-calling support, rely on the provider’s documentation or one actual request with tools.

Model Limitations

Different models vary in the following dimensions; verify in a test environment before going live:

Dimension	Description
Maximum tool count	Upper limit of tools that can be declared in a single request varies by model
Parallel calling	Some models do not support returning multiple tool_calls in one round
tool_choice support	Not all models support required / specifying specific tools
Parameter complexity	Deeply nested or very large JSON Schema may be truncated or ignored by some models
Streaming increment granularity	arguments fragmentation varies across models; must accumulate by index

9. Complete Examples

Weather Query Tool Example (End-to-End)

The following three code snippets are functionally equivalent: define tool → first round gets tool_calls → execute → second round passes back result → gets final response.

curl (manually complete two rounds):

# First round: send request with tools
curl https://api.routeapi.ai/v1/chat/completions \
  -H "Authorization: Bearer $ROUTEAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5",
    "messages": [{ "role": "user", "content": "How is the weather in Beijing now?" }],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Query current weather for a specified city.",
        "parameters": {
          "type": "object",
          "properties": { "city": { "type": "string" } },
          "required": ["city"]
        }
      }
    }]
  }'

# Second round: pass back tool result (tool_call_id uses value from first round)
curl https://api.routeapi.ai/v1/chat/completions \
  -H "Authorization: Bearer $ROUTEAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5",
    "messages": [
      { "role": "user", "content": "How is the weather in Beijing now?" },
      {
        "role": "assistant",
        "content": null,
        "tool_calls": [{
          "id": "call_abc123",
          "type": "function",
          "function": { "name": "get_weather", "arguments": "{\"city\": \"Beijing\"}" }
        }]
      },
      {
        "role": "tool",
        "tool_call_id": "call_abc123",
        "content": "Beijing, sunny, temperature 23 celsius, humidity 45%."
      }
    ]
  }'

Python (OpenAI SDK, automatically completes two rounds):

import json
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ROUTEAPI_KEY"],
    base_url="https://api.routeapi.ai/v1",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Query current weather for a specified city. Use full city name in Chinese.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name, e.g., Beijing"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["city"],
            },
        },
    }
]


def get_weather(city: str, unit: str = "celsius") -> str:
    # Replace with actual weather service call
    return f"{city}, sunny, temperature 23 celsius, humidity 45%."


messages = [{"role": "user", "content": "How is the weather in Beijing now?"}]

# First round
resp = client.chat.completions.create(
    model="gpt-5.5", messages=messages, tools=tools
)
msg = resp.choices[0].message

if msg.tool_calls:
    # assistant message must be added back as-is, otherwise tool_call_id cannot align
    messages.append(msg)
    for tc in msg.tool_calls:
        args = json.loads(tc.function.arguments)
        result = get_weather(**args)
        messages.append(
            {"role": "tool", "tool_call_id": tc.id, "content": result}
        )

    # Second round
    resp = client.chat.completions.create(
        model="gpt-5.5", messages=messages, tools=tools
    )

print(resp.choices[0].message.content)

Node.js (openai package, automatically completes two rounds):

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.ROUTEAPI_KEY,
  baseURL: 'https://api.routeapi.ai/v1',
});

const tools = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Query current weather for a specified city. Use full city name in Chinese.',
      parameters: {
        type: 'object',
        properties: {
          city: { type: 'string', description: 'City name, e.g., Beijing' },
          unit: { type: 'string', enum: ['celsius', 'fahrenheit'] },
        },
        required: ['city'],
      },
    },
  },
];

function getWeather(city, unit = 'celsius') {
  // Replace with actual weather service call
  return `${city}, sunny, temperature 23 celsius, humidity 45%.`;
}

const messages = [{ role: 'user', content: 'How is the weather in Beijing now?' }];

// First round
let resp = await client.chat.completions.create({
  model: 'gpt-5.5',
  messages,
  tools,
});
const msg = resp.choices[0].message;

if (msg.tool_calls) {
  messages.push(msg); // Add assistant message back as-is
  for (const tc of msg.tool_calls) {
    const args = JSON.parse(tc.function.arguments);
    const result = getWeather(args.city, args.unit);
    messages.push({ role: 'tool', tool_call_id: tc.id, content: result });
  }

  // Second round
  resp = await client.chat.completions.create({
    model: 'gpt-5.5',
    messages,
    tools,
  });
}

console.log(resp.choices[0].message.content);

Database Query Tool Example

Use tools as secure wrappers for internal systems. Key: SQL should not be generated directly by the model; instead, the model selects parameters, and your code constructs parameterized queries to avoid injection.

tools = [
    {
        "type": "function",
        "function": {
            "name": "query_order",
            "description": "Query order status and amount by order number.",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {
                        "type": "string",
                        "description": "Order number, like ORD-20260917-001",
                    }
                },
                "required": ["order_id"],
            },
        },
    }
]


def query_order(order_id: str) -> str:
    # Use parameterized query, never splice model output directly into SQL
    row = db.execute(
        "SELECT status, amount FROM orders WHERE order_id = %s",
        (order_id,),
    ).fetchone()
    if row is None:
        return json.dumps({"found": False})
    return json.dumps({"found": True, "status": row[0], "amount": row[1]})

Tool results are recommended to be passed back as JSON strings; models can more reliably parse structured fields.

Multi-Tool Orchestration Example

Declare multiple tools simultaneously; the model selects as needed or even calls in parallel. Here using weather + exchange rate two tools:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Query current weather for a specified city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "get_exchange_rate",
            "description": "Query exchange rate between two currencies.",
            "parameters": {
                "type": "object",
                "properties": {
                    "from_currency": {"type": "string", "description": "Like USD"},
                    "to_currency": {"type": "string", "description": "Like CNY"},
                },
                "required": ["from_currency", "to_currency"],
            },
        },
    },
]

dispatch = {
    "get_weather": lambda city: f"{city}, sunny, 23 celsius.",
    "get_exchange_rate": lambda from_currency, to_currency: f"1 {from_currency} = 7.2 {to_currency}",
}

messages = [{"role": "user", "content": "How is the weather in Beijing? Also tell me the USD to CNY exchange rate."}]

# Loop to handle multi-turn / parallel tool calls, set limit to prevent infinite loops
for _ in range(5):
    resp = client.chat.completions.create(
        model="gpt-5.5", messages=messages, tools=tools
    )
    msg = resp.choices[0].message
    messages.append(msg)
    if not msg.tool_calls:
        break
    # During parallel calls, tool_calls is a multi-element array; each must be passed back
    for tc in msg.tool_calls:
        args = json.loads(tc.function.arguments)
        result = dispatch[tc.function.name](**args)
        messages.append(
            {"role": "tool", "tool_call_id": tc.id, "content": result}
        )

print(messages[-1]["content"])

10. Error Handling

Tool calling introduces three potential error points: model side, your code side, and upstream service side.

Tool Parameter Validation Failure

The model may return invalid JSON, or miss required parameters, or pass values outside enum range. Make sure to:

• Wrap JSON.parse / json.loads in try/catch.
• After parsing, validate required fields and value ranges according to schema.
• When validation fails, pass the error message back as tool result to let the model self-correct, rather than directly throwing an exception to terminate the conversation:

try:
    args = json.loads(tc.function.arguments)
    city = args["city"]  # Validate required field
except (json.JSONDecodeError, KeyError) as exc:
    result = f"Parameter parsing failed: {exc}. Please provide valid city parameter again."
else:
    result = get_weather(city)

messages.append({"role": "tool", "tool_call_id": tc.id, "content": result})

Tool Execution Timeout

Tools are backed by real services that may timeout or be unavailable. Set a timeout limit for each tool call; if it times out, pass back the result as a clear error description to let the model switch to another strategy:

try:
    result = call_service(args, timeout=5)
except TimeoutError:
    result = "Service timeout, data not retrieved, please retry later or use another method."

Do not automatically retry tools with side effects (place orders, send emails), as they may execute repeatedly. Idempotent design or check-before-write is safer.

Tool Returns Error

Business-level errors (order not found, no permission) should also be passed back to the model, not silently return empty values. Pass back structured error information; the model can provide reasonable explanations to users:

messages.append({
    "role": "tool",
    "tool_call_id": tc.id,
    "content": json.dumps({"error": "order_not_found", "order_id": order_id}),
})

Claude format correspondingly sets "is_error": true on the tool_result block, see Claude Messages Protocol.

Compatibility Reminders

• Tool calling capability, maximum tool count, parallel calling support depend on the selected model; verify in test environment before going live.
• arguments is always a JSON string, not an object; must explicitly parse.
• In streaming scenarios, must accumulate arguments fragments by index, parse after complete concatenation.
• Optional parameters explicitly passed as 0 or false are considered user-set, not treated as defaults and discarded.
• Log request ID, model ID, status code, and token usage for each request for easier troubleshooting. Error structure details in Errors and Debugging.