Limiting cost and abuse
On This Lesson 4 topics
This lesson is planned. The outline below lists the topics it will cover. The full lesson has not been published yet.
In this lesson
stop one user from burning the budget.
Planned topics
- Unbounded consumption: huge inputs, long outputs, and many calls cost money and can cause denial of service.
- Set per-user rate limits and token limits, plus maximum input and output sizes.
- Add timeouts and a monthly cost alarm.
- Return 429 when a limit is hit.
- Count usage per authenticated user, not only per IP.
Draft outline details
Demonstration
a rule: 20 calls per user per hour, 1,000 output tokens per call, input capped at 2,000 characters.
Check your understanding
Which HTTP status for too many requests?
Show answer
429.
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.