Skip to content
Ameen Abozaid
Englishالعربية
MID-068

Retrieval-augmented generation

  • 8 Lessons
  • 3.5 h practice (estimate)
Start module
On this page 8 topics

Overview

Topics

  • Chunking
  • retrieval
  • context assembly
  • citations
  • keeping the index fresh with events
  • retrieval quality

Learn first

3.5 h of practice (estimate)

Learning outcomes

  1. Explain why retrieval-augmented generation (RAG) grounds model answers in your own data and where it still fails.
  2. Chunk documents with sensible size, overlap, and metadata, and retrieve with filters (e.g., language, product, customer tier).
  3. Assemble a prompt from retrieved chunks with source citations, and make the assistant say "I don't know" when retrieval finds nothing relevant.
  4. Keep the index fresh by re-embedding on change events instead of nightly full rebuilds.
  5. Measure retrieval quality (recall of the right chunk) separately from answer quality.

Lessons

  1. 01Why RAG, and its limits
  2. 02Chunking strategies
  3. 03Retrieval, filters, and re-ranking
  4. 04Context assembly and citations
  5. 05Freshness: re-indexing on events
  6. 06Evaluating retrieval separately from generation
  7. 07Demo: the order-support assistant answers with citations
  8. 08Demo: a stale policy answer fixed by event-driven re-indexing

Lab and exercises

Lab

MID-068-LAB1 · Order-support assistant v1

Outcomes: 1–5 · Time: ~3.5 h Answer customer questions about shipping, returns, and payment policies from the MID-067 index, citing the source articles. When an article changes, a HelpArticleUpdated event re-embeds only that article. Measure retrieval recall on the question set separately from answer quality. Acceptance: every answer cites at least one source or explicitly declines; a changed policy is reflected within one minute; retrieval recall measured on the question set.

Exercises (assessment: diagnose three bad answers)

IDOutcomeTask
MID-068-EX011Explain in five lines why RAG helps with Shop's changing policies and list three ways it can still fail.
MID-068-EX022Chunk a 1 500-word returns article: choose size, overlap, and metadata, and justify. Add the filter for an Arabic-speaking customer.
MID-068-EX033Write the prompt for the assistant with numbered sources, citation format, and the decline rule. Say what happens when no chunk passes the threshold.
MID-068-EX044Sketch the consumer for HelpArticleUpdated: steps, idempotency, deletions, and why not a nightly rebuild.
MID-068-EX055Compute recall@4 for 20 questions where the right chunk was in the top 4 for 16; name two fixes for the misses.
MID-068-EX061–5Diagnose three bad answers from logs and fix each at the right layer: (a) right article not in retrieved ids; (b) retrieved chunk dated before the policy change; (c) right chunk retrieved but answer contradicts it.
Show solutions
  • EX01 Model lacks private, current data; RAG injects it and enables citations. Failures: retrieval miss, stale chunk, model ignores context, irrelevant context, wrong filter.
  • EX02 Split by headings, then to a few hundred tokens with small overlap (tune by recall); metadata: article id, section, language, product, updated time. Filter lang = 'ar' and customer audience.
  • EX03 "Answer only from the sources; cite [n]; if not answered, say you could not find it." With no chunk above the threshold: skip the model call and return a fixed decline.
  • EX04 Receive event, load article, chunk, compare hashes, embed only changed chunks, delete old and insert new in one transaction. Idempotent by article version or hash. HelpArticleDeleted removes chunks. Nightly rebuild is slow, costly, and stale for hours.
  • EX05 16/20 = 0.80. Fixes: add hybrid search or synonyms; adjust chunk size; add re-ranking; fix metadata filters.
  • EX06 (a) Retrieval miss: fix chunking, hybrid search, filters, or threshold. (b) Stale chunk: re-index on event, check the hash comparison. (c) Ignored context: tighten the prompt, reduce irrelevant context, check the model; evaluate in MID-070.

Outcomes 3 and 5 are also covered by the quiz and the lab (citations, decline, recall measurement).

Quiz

Outcomes: O1 RAG and its limits · O2 chunking and filters · O3 context and citations · O4 freshness · O5 retrieval evaluation.

A self-check for you: nothing is graded, saved or sent anywhere. Choose an answer, then check it. You can try again as often as you like.

  1. 1RAG mainly helps a model answer from:O1
  2. 2RAG answers can still be wrong when:O1
  3. 3Chunks that are too large:O2
  4. 4Why store the language as chunk metadata?O2
  5. 5When retrieval finds nothing relevant, the assistant should:O3
  6. 6Citations are useful because:O3
  7. 7The best way to reflect an edited article:O4
  8. 8A re-index handler must be idempotent because:O4
  9. 9Recall@4 measures:O5
  10. 10Low retrieval recall means:O5

Practical part

Diagnosis exercise (MID-068-EX06) plus lab MID-068-LAB1. Pass: quiz at least 8/10, three bad answers correctly diagnosed and fixed at the right layer, and a lab where every answer cites a source or declines, a changed policy is reflected within one minute, and retrieval recall is measured.

Common mistakes

Chunks so large they drown the answer; no citations, so wrong answers cannot be traced; answering from model memory when retrieval is empty; judging RAG only by how fluent the answers sound.

Assessment

Diagnose three bad answers (retrieval miss, stale chunk, ignored context) from logs and fix each at the right layer.

Career relevance

The most common pattern in enterprise AI features; evaluated in MID-070, secured in MID-071, operated in SRC-022.

Type to search levels, phases, modules and lessons.