Timeouts, retries, and token budgets
On This Lesson 4 topics
This lesson is planned. The outline below lists the topics it will cover. The full lesson has not been published yet.
In this lesson
keep model calls from hurting your service or your bill.
Planned topics
- Model calls are slow and can fail. Set a connect and read timeout.
- Retry only transient failures (timeouts,
429,5xx), a small number of times, with a growing delay. - Set a max output tokens limit per request and limit input size too; this is your token budget.
- Cap user-provided text length before it reaches the prompt.
- Log token usage so cost can be watched; do not log secrets or private data.
Draft outline details
Demonstration
# property names: verify for the chosen provider
app.ai.timeout: 10s
app.ai.max-tokens: 300Check your understanding
Which failures deserve a retry?
Show answer
Transient ones: timeouts, 429, 5xx.
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.