Skip to content
Ameen Abozaid
Englishالعربية
JNR-056

Calling LLMs from Spring

  • 7 Lessons
  • 3 h practice (estimate)
Start module
On this page 8 topics

Overview

Topics

  • Spring AI ChatClient
  • system prompts
  • structured output to records
  • streaming
  • timeouts and retries
  • token cost
  • API key handling
  • testing with a fake model

Learn first

3 h of practice (estimate)

Learning outcomes

  1. Explain how an LLM API call works: messages, system prompt, model parameters, tokens, and cost.
  2. Call a model from Spring with Spring AI's ChatClient, keeping the provider behind configuration.
  3. Convert model output into Java records with structured output, and validate it before use.
  4. Stream responses to clients and set timeouts, retries, and a token budget per request.
  5. Hide the model behind an application interface and test the feature with a fake implementation.

Lessons

  1. 01What happens in an LLM API call
  2. 02Spring AI concepts: ChatClient, prompts, advisors
  3. 03Structured output to records, and validating it
  4. 04Streaming responses
  5. 05Timeouts, retries, and token budgets
  6. 06Keys, configuration, and testing with a fake model
  7. 07Demo: an AI product-description helper for Shop API

Lab and exercises

Lab

JNR-056-LAB1 · AI helper for Shop

Outcomes: 1–5 · Time: ~3 h Add a ProductDescriptionAssistant (or a natural-language product search) to Shop API behind an interface. The Spring AI implementation returns a validated record; a fake implementation is used in tests.

Acceptance:

  • provider and model are set only in configuration
  • the API key comes from the environment
  • timeout and max-token limit are configured
  • invalid model output is handled with a clear error or a fallback
  • controller and service tests pass without network access

Exercises (assessment: extend the helper with a new field and keep all tests deterministic)

IDOutcomeTask
JNR-056-EX011Given a request with 400 input tokens and 150 output tokens, explain what is billed and how a lower max_tokens changes cost.
JNR-056-EX022Call a model with ChatClient using a system message and a templated user message; show where the provider is configured.
JNR-056-EX033Return a Description record with .entity(...) and reject results that fail Bean Validation.
JNR-056-EX044Add a streaming endpoint that returns pieces as server-sent events.
JNR-056-EX054Configure a timeout, a retry policy for transient errors, and a max-token limit; list which errors you retry.
JNR-056-EX065Extract an interface, write a fake, and test both the valid and the invalid-output path with no network.
Show solutions
  • EX01 You pay for 400 input plus up to 150 output tokens; lowering the limit caps the output share of the cost (and may cut the reply short).
  • EX02 chatClient.prompt().system("...").user(u -> u.text("... {name}").param("name", n)).call().content(); provider and model in application.yml (property names per the Spring AI reference) and the starter dependency.
  • EX03 .entity(Description.class) then validator.validate(d); if the set is not empty, throw an exception mapped to a clear error response.
  • EX04 produces = TEXT_EVENT_STREAM_VALUE returning chatClient.prompt()...stream().content().
  • EX05 Retry timeouts, 429, 5xx; do not retry 400 or authentication errors.
  • EX06 ProductDescriptionAssistant interface; a fake returns fixed values or an invalid record; @WebMvcTest registers the fake as a test bean (verify the current mock/bean annotation for your Spring Boot version).

Outcome 1 is also checked by quiz Q01 to Q03. Add a new field (for example keywords) to the record, validation, fake, and tests for the assessment.

Quiz

Outcomes: O1 how a call works · O2 ChatClient · O3 structured output · O4 streaming, timeouts, budgets · O5 interface and fake.

A self-check for you: nothing is graded, saved or sent anywhere. Choose an answer, then check it. You can try again as often as you like.

  1. 1A system message is used to:O1
  2. 2Cost of a call mainly depends on:O1
  3. 3Same input twice to a model can give:O1
  4. 4Spring AI's ChatClient is used to:O2
  5. 5Where should the provider and model be chosen?O2
  6. 6Why use structured output?O3
  7. 7After mapping to a record, you should:O3
  8. 8Streaming helps because:O4
  9. 9Which failure is reasonable to retry?O4
  10. 10Tests for the AI feature should:O5

Practical part

Six exercises JNR-056-EX01 to EX06 plus the AI-helper lab. Pass: at least five exercises and a lab meeting every acceptance item; the assessment extension (a new field) must keep all tests offline and deterministic.

Common mistakes

Parsing free text instead of requesting structured output; no timeout on model calls; API keys in application.yml; tests that call the real model.

Assessment

Extend the helper with a new field and keep all tests deterministic.

Career relevance

LLM features are increasingly part of backend work; JNR-057 secures them, and MID-067 to MID-070 add retrieval, tools, and evaluation.

Type to search levels, phases, modules and lessons.