Structured Outputs

Structured outputs transform model responses from free-form text into strictly formatted JSON. RouteAPI supports two structured output modes: JSON mode (requires valid JSON) and JSON Schema (guarantees conformance to a specific schema).

1. Structured Outputs Overview

What are Structured Outputs

Structured outputs are a mechanism for controlling model response formats. Unlike ordinary conversations where models freely generate text, structured outputs force the model to generate JSON data according to your defined format. This is crucial for scenarios requiring programmatic processing of model outputs (data extraction, form generation, API response parsing).

JSON mode vs JSON Schema

RouteAPI provides two structured output modes:

Comparison	JSON mode	JSON Schema
Guarantee	Output is valid JSON	Output conforms to specified schema
Parameter	`response_format: { type: "json_object" }`	`response_format: { type: "json_schema", json_schema: {...} }`
Schema definition	Not required, but format must be described in prompt	Complete JSON Schema required
Strictness	Only guarantees parseability, not structure	Guarantees fields, types, and required items match exactly
Use cases	Simple formats, model can understand structure from prompt	Complex nested structures, strict type validation needed

Simple explanation: JSON mode only guarantees “can be successfully parsed by JSON.parse” but doesn’t care about fields; JSON Schema guarantees not only validity but also structure, field names, types, and required items all conform to your definition.

Use Cases

Scenario	Description	Recommended mode
Data extraction	Extract structured information from unstructured text (name, address, date)	JSON Schema
Form generation	Have model generate form initial values or configuration objects	JSON Schema
API response parsing	Model output needs to interface with downstream system APIs	JSON Schema
Simple key-value pairs	Only need a few fields, structure is simple and clear	JSON mode
Classification tasks	Output fixed enum values (e.g., sentiment: positive/negative/neutral)	JSON Schema + enum

2. JSON Mode

JSON mode is the simplest structured output method: you set response_format.type to "json_object" in your request, and the model will output valid JSON instead of plain text.

Request Format

{
  "model": "gpt-5.5",
  "messages": [
    {
      "role": "system",
      "content": "You are a data extraction assistant. Extract name, age, and city fields from user input and return in JSON format."
    },
    {
      "role": "user",
      "content": "My name is Li Ming, I'm 28 years old, and I live in Shanghai."
    }
  ],
  "response_format": { "type": "json_object" }
}

Key Points

Must describe JSON format in prompt: The model doesn’t know what fields you want; you must explicitly tell it through system message or user message what fields to output and their types. In the example above, "Extract name, age, and city fields from user input and return in JSON format" is the format description.
Does not guarantee schema conformance: The model might output {"name": "Li Ming", "age": 28, "city": "Shanghai"}, or {"姓名": "Li Ming", "年龄": 28}, or even {"person": {"name": "Li Ming"}}. As long as it’s valid JSON, it’s acceptable.
How to use: Suitable for scenarios with simple formats, few fields, and where the model can understand structure from natural language. If strict validation of field names or types is needed, use JSON Schema.

Complete Example

Request (curl):

curl https://api.routeapi.ai/v1/chat/completions \
  -H "Authorization: Bearer $ROUTEAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5",
    "messages": [
      {
        "role": "system",
        "content": "You are a data extraction assistant. Extract name, age, and city fields from user input and return in JSON format."
      },
      {
        "role": "user",
        "content": "My name is Li Ming, I am 28 years old, and I live in Shanghai."
      }
    ],
    "response_format": { "type": "json_object" }
  }'

Response:

{
  "id": "chatcmpl_xxx",
  "object": "chat.completion",
  "created": 1730000000,
  "model": "gpt-5.5",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\"name\": \"Li Ming\", \"age\": 28, \"city\": \"Shanghai\"}"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 58,
    "completion_tokens": 18,
    "total_tokens": 76
  }
}

Note that message.content is a JSON string, you need to JSON.parse / json.loads it yourself.

3. JSON Schema (Structured Outputs)

JSON Schema mode lets you precisely define the output structure, and the model guarantees the generated JSON fully conforms to your schema definition.

Request Format

{
  "model": "gpt-5.5",
  "messages": [
    {
      "role": "user",
      "content": "My name is Li Ming, I'm 28 years old, and I live in Shanghai."
    }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "person_extraction",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "name": { "type": "string", "description": "Name" },
          "age": { "type": "integer", "description": "Age" },
          "city": { "type": "string", "description": "City" }
        },
        "required": ["name", "age", "city"],
        "additionalProperties": false
      }
    }
  }
}

response_format Structure

Field	Type	Required	Description
`type`	string	Yes	Fixed as `"json_schema"`
`json_schema.name`	string	Yes	Schema name for identification, letters, numbers, underscores, hyphens
`json_schema.strict`	boolean	No	Whether to enable strict mode, default `false`
`json_schema.schema`	object	Yes	Standard JSON Schema definition

Strict Mode vs Non-strict Mode

Mode	`strict`	Behavior
Strict mode	`true`	Model must generate exactly according to schema, field names, types, required items, `additionalProperties` all strictly obeyed
Non-strict mode	`false`	Model tries to conform to schema, but doesn’t guarantee complete consistency, may miss fields or add extra ones

Recommendation: Use strict: true in production; this is the core value of JSON Schema mode. Non-strict mode behavior is similar to JSON mode + prompt description, meaning is limited.

Schema Definition Specification

The schema field follows standard JSON Schema specification (Draft 2020-12), common fields:

Field	Description
`type`	Data type: `"object"`, `"array"`, `"string"`, `"number"`, `"integer"`, `"boolean"`, `"null"`
`properties`	Field definitions for objects (used when type is `"object"`)
`required`	Array of required field names
`additionalProperties`	Whether to allow undefined extra fields (recommend `false` in strict mode)
`items`	Schema for array elements (used when type is `"array"`)
`enum`	List of enum values, restricts value range
`description`	Field description, helps model understand semantics

4. Schema Definition Guide

Basic Types

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "age": { "type": "integer" },
    "score": { "type": "number" },
    "is_active": { "type": "boolean" },
    "notes": { "type": ["string", "null"] }
  }
}

Type descriptions:

"string": String
"integer": Integer
"number": Number (including integers and decimals)
"boolean": Boolean
"null": Null value
["string", "null"]: Allows string or null (optional field)

Objects and Arrays

{
  "type": "object",
  "properties": {
    "user": {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "email": { "type": "string" }
      },
      "required": ["name"]
    },
    "tags": {
      "type": "array",
      "items": { "type": "string" }
    },
    "scores": {
      "type": "array",
      "items": { "type": "number" }
    }
  }
}

Required Fields

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "age": { "type": "integer" },
    "city": { "type": "string" }
  },
  "required": ["name", "age"]
}

The required array lists fields that must exist. In the example above, name and age are required, city is optional.

Enum Values

{
  "type": "object",
  "properties": {
    "sentiment": {
      "type": "string",
      "enum": ["positive", "negative", "neutral"],
      "description": "Sentiment classification result"
    },
    "priority": {
      "type": "integer",
      "enum": [1, 2, 3],
      "description": "Priority: 1-low, 2-medium, 3-high"
    }
  },
  "required": ["sentiment"]
}

enum restricts the field to only values in the list; the model won’t generate other values.

Nested Structures

{
  "type": "object",
  "properties": {
    "user": {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "address": {
          "type": "object",
          "properties": {
            "city": { "type": "string" },
            "street": { "type": "string" }
          },
          "required": ["city"]
        }
      },
      "required": ["name", "address"]
    }
  },
  "required": ["user"]
}

Objects can be nested infinitely, but excessive nesting may affect model generation quality and performance.

Description Fields

{
  "type": "object",
  "properties": {
    "date": {
      "type": "string",
      "description": "Date, format YYYY-MM-DD"
    },
    "amount": {
      "type": "number",
      "description": "Amount in CNY"
    }
  }
}

description is not required, but strongly recommended. It helps the model understand field semantics, value ranges, and format conventions, significantly improving generation accuracy.

5. Schema Examples

User Information Extraction

Extract user information from unstructured text:

{
  "name": "user_info_extraction",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "name": {
        "type": "string",
        "description": "User name"
      },
      "age": {
        "type": "integer",
        "description": "Age"
      },
      "email": {
        "type": ["string", "null"],
        "description": "Email address, null if not in text"
      },
      "phone": {
        "type": ["string", "null"],
        "description": "Phone number, null if not in text"
      },
      "city": {
        "type": "string",
        "description": "City"
      }
    },
    "required": ["name", "age", "city"],
    "additionalProperties": false
  }
}

Product List Generation

Have model generate a product array:

{
  "name": "product_list",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "products": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "name": {
              "type": "string",
              "description": "Product name"
            },
            "price": {
              "type": "number",
              "description": "Price in CNY"
            },
            "category": {
              "type": "string",
              "enum": ["electronics", "clothing", "food", "other"],
              "description": "Product category"
            },
            "in_stock": {
              "type": "boolean",
              "description": "Whether in stock"
            }
          },
          "required": ["name", "price", "category", "in_stock"],
          "additionalProperties": false
        }
      },
      "total_count": {
        "type": "integer",
        "description": "Total product count"
      }
    },
    "required": ["products", "total_count"],
    "additionalProperties": false
  }
}

Complex Nested Object

Order information extraction with multi-level nesting:

{
  "name": "order_extraction",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "Order ID"
      },
      "customer": {
        "type": "object",
        "properties": {
          "name": { "type": "string" },
          "phone": { "type": "string" },
          "address": {
            "type": "object",
            "properties": {
              "province": { "type": "string" },
              "city": { "type": "string" },
              "street": { "type": "string" }
            },
            "required": ["province", "city", "street"],
            "additionalProperties": false
          }
        },
        "required": ["name", "phone", "address"],
        "additionalProperties": false
      },
      "items": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "product_name": { "type": "string" },
            "quantity": { "type": "integer" },
            "unit_price": { "type": "number" }
          },
          "required": ["product_name", "quantity", "unit_price"],
          "additionalProperties": false
        }
      },
      "total_amount": {
        "type": "number",
        "description": "Order total amount"
      }
    },
    "required": ["order_id", "customer", "items", "total_amount"],
    "additionalProperties": false
  }
}

6. Model Support

Models Supporting JSON mode

Most mainstream models aggregated by RouteAPI support JSON mode (response_format: { type: "json_object" }), including:

OpenAI GPT series (gpt-4o, gpt-4-turbo, gpt-3.5-turbo, etc.)
Claude series (claude-3.5-sonnet, claude-3-opus, claude-3-haiku, etc.)
Gemini series (gemini-2.0-flash, gemini-1.5-pro, etc.)
Other models supporting OpenAI format

Models Supporting JSON Schema

JSON Schema (response_format: { type: "json_schema" }) requires higher model capabilities; currently supported models:

OpenAI: gpt-4o series, gpt-4-turbo series (version 2024-08-06 and later)
Claude: claude-3.5-sonnet series, claude-3-opus series
Gemini: gemini-2.0-flash-exp, gemini-1.5-pro series
Others: some new generation models

How to confirm: Query model capabilities through Models API before calling, check the supports_response_format field.

Model Capability Comparison

Capability	JSON mode	JSON Schema	Notes
Model support range	Wide (almost all mainstream models)	Limited (new generation models)	JSON mode has higher support
Schema complexity	N/A	Recommend no more than 3 nesting levels	Excessive depth affects generation quality
Max field count	N/A	Recommend no more than 50 top-level fields	Too many fields affect performance
Strict guarantee	Only guarantees valid JSON	Guarantees schema conformance	100% conformance in strict mode
Performance	Fast	Relatively slower	Schema validation has extra overhead
Token consumption	Low	Slightly higher	Schema definition occupies prompt tokens

Limitations and Notes

Schema size limit: Single schema definition recommended not to exceed 10KB; excessively large schemas may be truncated or rejected.
Nesting depth limit: Recommend nesting levels not exceed 3-4 layers; excessive nesting reduces model generation quality and speed.
Performance impact: JSON Schema mode response time is typically 10%-30% slower than regular requests because the model needs to validate structure in real-time during generation.
Unsupported schema features: Some advanced JSON Schema features (like $ref, allOf, anyOf, oneOf, regex) may not be supported by all models.
Streaming output: JSON Schema mode supports streaming output (stream: true), but content is returned in fragments; complete JSON needs to be concatenated before parsing.

7. Complete Application Examples

Example 1: Extract Structured Information from Unstructured Text

Scenario: Extract customer information and issue classification from customer service conversation.

curl:

curl https://api.routeapi.ai/v1/chat/completions \
  -H "Authorization: Bearer $ROUTEAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "Customer Li Ming (phone 13800138000) reported that order ORD-2024-001 in Chaoyang District, Beijing has not been shipped and is quite urgent."
      }
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "customer_inquiry",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "customer_name": { "type": "string", "description": "Customer name" },
            "phone": { "type": ["string", "null"], "description": "Customer phone" },
            "location": { "type": ["string", "null"], "description": "Customer location" },
            "order_id": { "type": ["string", "null"], "description": "Order ID" },
            "issue_category": {
              "type": "string",
              "enum": ["delivery", "quality", "refund", "other"],
              "description": "Issue category: delivery-logistics, quality-quality, refund-refund, other-other"
            },
            "urgency": {
              "type": "string",
              "enum": ["low", "medium", "high"],
              "description": "Urgency level"
            }
          },
          "required": ["customer_name", "issue_category", "urgency"],
          "additionalProperties": false
        }
      }
    }
  }'

Response:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "{\"customer_name\":\"Li Ming\",\"phone\":\"13800138000\",\"location\":\"Chaoyang District, Beijing\",\"order_id\":\"ORD-2024-001\",\"issue_category\":\"delivery\",\"urgency\":\"high\"}"
      },
      "finish_reason": "stop"
    }
  ]
}

Python:

import os
import json
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ROUTEAPI_KEY"],
    base_url="https://api.routeapi.ai/v1",
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {
            "role": "user",
            "content": "Customer Li Ming (phone 13800138000) reported that order ORD-2024-001 in Chaoyang District, Beijing has not been shipped and is quite urgent.",
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "customer_inquiry",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "customer_name": {"type": "string", "description": "Customer name"},
                    "phone": {"type": ["string", "null"], "description": "Customer phone"},
                    "location": {"type": ["string", "null"], "description": "Customer location"},
                    "order_id": {"type": ["string", "null"], "description": "Order ID"},
                    "issue_category": {
                        "type": "string",
                        "enum": ["delivery", "quality", "refund", "other"],
                        "description": "Issue category",
                    },
                    "urgency": {
                        "type": "string",
                        "enum": ["low", "medium", "high"],
                        "description": "Urgency level",
                    },
                },
                "required": ["customer_name", "issue_category", "urgency"],
                "additionalProperties": False,
            },
        },
    },
)

data = json.loads(response.choices[0].message.content)
print(f"Customer: {data['customer_name']}")
print(f"Issue type: {data['issue_category']}")
print(f"Urgency: {data['urgency']}")

Node.js:

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.ROUTEAPI_KEY,
  baseURL: 'https://api.routeapi.ai/v1',
});

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    {
      role: 'user',
      content: 'Customer Li Ming (phone 13800138000) reported that order ORD-2024-001 in Chaoyang District, Beijing has not been shipped and is quite urgent.',
    },
  ],
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'customer_inquiry',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          customer_name: { type: 'string', description: 'Customer name' },
          phone: { type: ['string', 'null'], description: 'Customer phone' },
          location: { type: ['string', 'null'], description: 'Customer location' },
          order_id: { type: ['string', 'null'], description: 'Order ID' },
          issue_category: {
            type: 'string',
            enum: ['delivery', 'quality', 'refund', 'other'],
            description: 'Issue category',
          },
          urgency: {
            type: 'string',
            enum: ['low', 'medium', 'high'],
            description: 'Urgency level',
          },
        },
        required: ['customer_name', 'issue_category', 'urgency'],
        additionalProperties: false,
      },
    },
  },
});

const data = JSON.parse(response.choices[0].message.content);
console.log(`Customer: ${data.customer_name}`);
console.log(`Issue type: ${data.issue_category}`);
console.log(`Urgency: ${data.urgency}`);

Example 2: Generate Form Data

Scenario: Have model generate form initial values based on natural language description.

Python:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {
            "role": "system",
            "content": "You are a form filling assistant. Generate form data based on user description.",
        },
        {
            "role": "user",
            "content": "Create a new employee onboarding form: Zhang Wei, male, born in 1995, bachelor's degree, software engineer position, monthly salary 15000.",
        },
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "employee_form",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "gender": {"type": "string", "enum": ["male", "female"]},
                    "birth_year": {"type": "integer"},
                    "education": {
                        "type": "string",
                        "enum": ["high_school", "bachelor", "master", "phd"],
                    },
                    "position": {"type": "string"},
                    "salary": {"type": "number", "description": "Monthly salary in CNY"},
                },
                "required": ["name", "gender", "birth_year", "education", "position", "salary"],
                "additionalProperties": False,
            },
        },
    },
)

form_data = json.loads(response.choices[0].message.content)
print(json.dumps(form_data, ensure_ascii=False, indent=2))

Output:

{
  "name": "Zhang Wei",
  "gender": "male",
  "birth_year": 1995,
  "education": "bachelor",
  "position": "Software Engineer",
  "salary": 15000
}

Example 3: API Response Parsing

Scenario: Have model extract key information from unstructured third-party API response.

Node.js:

const apiResponse = `
Order status: Shipped
Logistics company: SF Express
Tracking number: SF1234567890
Estimated delivery: January 20, 2024
Current location: Beijing Distribution Center
`;

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    {
      role: 'user',
      content: `Extract structured data from the following logistics information:\n${apiResponse}`,
    },
  ],
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'logistics_info',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          status: {
            type: 'string',
            enum: ['pending', 'shipped', 'in_transit', 'delivered'],
          },
          carrier: { type: 'string', description: 'Logistics company' },
          tracking_number: { type: 'string', description: 'Tracking number' },
          estimated_delivery: {
            type: ['string', 'null'],
            description: 'Estimated delivery date, format YYYY-MM-DD',
          },
          current_location: { type: ['string', 'null'], description: 'Current location' },
        },
        required: ['status', 'carrier', 'tracking_number'],
        additionalProperties: false,
      },
    },
  },
});

const data = JSON.parse(response.choices[0].message.content);
console.log(data);
// Output: { status: 'shipped', carrier: 'SF Express', tracking_number: 'SF1234567890', estimated_delivery: '2024-01-20', current_location: 'Beijing Distribution Center' }

8. Error Handling

Schema Validation Failure

If your schema definition itself has problems (syntax errors, unsupported features), the request will directly return a 400 error:

{
  "error": {
    "message": "Invalid JSON schema: ...",
    "type": "invalid_request_error",
    "param": "response_format.json_schema.schema",
    "code": "invalid_json_schema"
  }
}

Solution:

Check if schema syntax conforms to JSON Schema specification
Remove unsupported advanced features (like $ref, allOf)
Simplify excessively nested structures

Model Cannot Generate Output Conforming to Schema

In rare cases, the model may be unable to generate content conforming to the schema (e.g., user input completely conflicts with schema requirements); in this case, an error will be returned or it will downgrade to plain text output. Error example:

{
  "error": {
    "message": "Failed to generate valid output matching the provided schema after maximum retries.",
    "type": "model_error",
    "code": "schema_generation_failed"
  }
}

Solution:

Check if prompt is consistent with schema requirements
Simplify schema, remove overly strict constraints
Explicitly state output requirements in prompt
For optional fields, use ["string", "null"] type instead of just "string"

Retry Strategy

For occasional generation failures, you can implement automatic retry:

import time

def call_with_retry(client, **kwargs):
    max_retries = 3
    for attempt in range(max_retries):
        try:
            response = client.chat.completions.create(**kwargs)
            content = response.choices[0].message.content
            data = json.loads(content)  # Verify it can be parsed
            return data
        except (json.JSONDecodeError, Exception) as e:
            if attempt == max_retries - 1:
                raise
            time.sleep(2 ** attempt)  # Exponential backoff

9. Best Practices

Schema Design Principles

Start simple: First verify feasibility with JSON mode, then upgrade to JSON Schema after confirming strict validation is needed.
Clarify required and optional: Put required fields in the required array, use ["type", "null"] for optional fields or simply don’t put them in required.
Use enum to restrict enums: For fields with limited values (status, classification, priority), explicitly list them with enum; this can greatly reduce error rates.
Add description: Add description to each field, explaining meaning, format, value range; the model generates more accurately based on this.
Set additionalProperties: false: In strict mode, adding this prevents the model from outputting undefined extra fields.

Avoid Overly Complex Schemas

Problem	Description	Suggestion
Too deeply nested	More than 4 nesting levels reduces generation quality	Split into multiple flat objects, or extract in steps with multi-turn conversations
Too many fields	Single object exceeds 50 fields	Group by business logic, split into multiple sub-objects
Over-constrained	All fields are required, no flexibility	Only mark core fields as required, allow null for other fields
No description	Model doesn’t understand field semantics	Add description to each field, explain clearly

Performance Optimization

Reduce schema size: Schema definition occupies prompt tokens; excessively large schemas increase latency and cost.
Cache schema definitions: When the same schema is used repeatedly, define it as a constant in code to avoid reconstructing it for each request.
Choose appropriate model: Not all tasks need the strongest model; simple structured extraction can use gpt-3.5-turbo + JSON mode.
Batch processing: If there are multiple similar tasks, design an array schema to have the model process multiple data items at once.

Cost Considerations

Factor	Impact	Optimization suggestion
Schema size	Schema definition occupies prompt tokens	Streamline description, remove redundant fields
Model choice	JSON Schema typically requires stronger models	Use JSON mode + weaker model for simple tasks
Response length	JSON output is typically longer than text (key names, quotes, brackets)	Shorten field names, use enums instead of long strings
Retry count	Generation failure retries double billing	Optimize schema and prompt to reduce failure rate

Practical tip: For high-frequency call scenarios, first test with JSON mode; after confirming the model can stably output correct format, upgrade to JSON Schema. JSON mode typically has 10%-20% lower token consumption and cost than JSON Schema.

Compatibility Notes

JSON mode and JSON Schema support depends on the selected model; please query the supports_response_format field through Models API to confirm.
response_format is mutually exclusive with tools (tool calling): the same request cannot use both structured outputs and tool calling.
In streaming output (stream: true), content is returned in fragments and needs to be fully concatenated before JSON.parse.
Explicitly passed 0 or false optional parameters are treated as user-set explicitly and won’t be treated as default omissions.
Log each request’s request ID, model ID, status code, and token usage for troubleshooting. Error structure details see Errors and Debugging.