Streaming Debugging
Streaming requests involve three layers: the client connection, RouteAPI forwarding, and the model service response. Troubleshooting should distinguish where the issue originates.
Common issues
Section titled “Common issues”| Issue | Possible cause |
|---|---|
| No output | The client did not enable stream, or the current model does not support it |
| Output interrupted | Model service timeout, network disconnect, or client connection closed |
| No usage received | The current model does not support stream_options.include_usage |
| JSON parsing failed | The client is not parsing by SSE line |
Client handling recommendations
Section titled “Client handling recommendations”- Read
data:line by line. - Ignore empty lines.
- End after receiving
[DONE]. - Parse each JSON chunk independently.
- Do not parse the entire response as one JSON document.