Why assistants sound right and are sometimes wrong
On This Lesson 11 topics
Lesson Overview
In the last lesson you learned that an assistant predicts one likely token after another and does not run code or check facts. This lesson shows the practical result: hallucinations. A hallucination is a confident answer that is invented or false. In code, that means methods that do not exist, library calls with the wrong arguments, outdated advice, and explanations that sound reasonable but are wrong.
You will see four real failures, with the exact messages Java produced, and learn a short checklist to catch them.
Why this matters: wrong code that looks professional is more dangerous than code that obviously fails. A backend developer who ships an unchecked suggestion owns the bug.
This lesson builds on "What an AI coding assistant actually is", and on reading compiler errors and stack traces from your debugging module (BEG-018).
Learning Objectives
By the end of this lesson you will be able to:
- Define a hallucination and give three kinds seen in code answers.
- Explain why the tone of an answer says nothing about its truth.
- Explain what a training-data cutoff is and how it causes outdated answers.
- Recognize an invented method from a
cannot find symbolcompiler error. - Use the compiler, a run, and the official documentation to settle whether a claim is true.
- Recognize a wrong explanation that compiles and runs.
Written Explanation
Why hallucinations happen
The model chooses likely tokens. Sometimes the most likely continuation is not a true fact. Imagine asking how to reverse a string. In the model's training text, reverse appears next to strings, lists, and StringBuilder many times. A token sequence like s.reverse() therefore looks natural, even though Java's String class has no such method.
OWASP, which catalogues risks of LLM applications, describes hallucination as a major cause of misinformation: the model invents plausible content and fills gaps in what it learned with statistical guesses. The same page notes that models can suggest libraries that do not exist, which is a real security risk if someone installs them without checking.
Four typical failures
| Kind | What it looks like | What catches it |
|---|---|---|
| Invented method | s.reverse() on a String | The compiler: cannot find symbol |
| Wrong arguments | list.addAll(5) | The compiler: incompatible types |
| Wrong behaviour | "List.of(...) gives you a list you can add to" | Running the code; the Javadoc |
| Outdated advice | new Integer(5) | Compiler warnings; the Javadoc |
The first two are the friendliest: Java's static typing refuses to compile them. The last two compile (or nearly so) and fail later, or work in a way that is already discouraged. That is why checking only by "it compiled" is not enough.
The training cutoff
A model's training data ends at some date. Java releases a new version every six months, libraries change, and tools change their menus and commands. Anything newer than the cutoff is invisible to the model, and anything that changed after it may be described the old way. The product's own documentation should tell you its cutoff; do not assume the model knows what is current.
When an answer mentions a version, a deprecated API, or "the latest way", check the official documentation for your Java version.
Same tone, right or wrong
The model does not have a "confident" and an "unsure" voice that tracks truth. The phrase "Here is the corrected code" appears with correct and wrong code alike. Some assistants add hedges ("this may work"), but a hedge is also just text. Never use tone to decide how much to check.
A verification checklist
For any answer that contains a claim you will rely on:
- Compile it. Invented methods and wrong arguments fail here.
- Run it on a small input, including an edge case from BEG-016 (empty, one element, invalid).
- Read the official documentation for each new method or class. Look for the exact name, the parameters, and the described behaviour.
- Read compiler warnings. Deprecation warnings are free hints.
- Test the explanation, not just the code. If the assistant says why something happens, find an experiment that would show it was wrong.
What to do when you catch one
Do not argue with the tone. Give the assistant the evidence and ask again: paste the exact compiler message and say what the documentation says. Or skip the assistant and use the documentation. Either way you now know the first answer was wrong, which is itself useful information about how much to trust the rest.
Examples
Example 1: an invented method (simple)
An assistant suggests reversing a string with s.reverse(), and adds list.addAll(5) to add one number to a list. You compile:
import java.util.*;
public class Hallu2 {
public static void main(String[] args) {
List<Integer> list = new ArrayList<>();
list.addAll(5);
String s = "hello";
System.out.println(s.reverse());
}
}Compiler output (JDK 21):
Hallu2.java:6: error: incompatible types: int cannot be converted to Collection<? extends Integer>
list.addAll(5);
^
Hallu2.java:8: error: cannot find symbol
System.out.println(s.reverse());
^
symbol: method reverse()
location: variable s of type String
Note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output
2 errorsThe documentation settles the first error: List.addAll takes a Collection<? extends E>, so adding one element means list.add(5). For the second, String has no reverse(); the usual tool is new StringBuilder(s).reverse().toString(), because StringBuilder does have a reverse() method (running it on hello printed olleh).
Example 2: wrong behaviour that compiles (practical)
An assistant says: "Create the list with List.of(1, 2, 3), then call nums.add(4)." Nothing is wrong at compile time:
import java.util.*;
public class Hallu {
public static void main(String[] args) {
List<Integer> nums = List.of(1, 2, 3);
nums.add(4);
System.out.println(nums);
}
}Exception in thread "main" java.lang.UnsupportedOperationException
at java.base/java.util.ImmutableCollections.uoe(ImmutableCollections.java:142)
at java.base/java.util.ImmutableCollections$AbstractImmutableCollection.add(ImmutableCollections.java:147)
at Hallu.main(Hallu.java:6)The compiler could not know. Running it did. The Javadoc for List explains: lists created by List.of are unmodifiable, and calling any mutator method throws UnsupportedOperationException. A correct version starts from new ArrayList<>(List.of(1, 2, 3)).
Example 3: outdated advice (realistic)
An assistant writes Integer n = new Integer(5);. Older tutorials, which the model has seen a lot of, use this form. Compile with all lint warnings on:
Old.java:3: warning: [removal] Integer(int) in Integer has been deprecated and marked for removal
Integer n = new Integer(5);
^
1 warningIt compiles, so a quick "it works" check would pass. The Javadoc says the constructor has been deprecated since Java 9 and marked for removal, and recommends Integer.valueOf(int). A future Java version may remove it and break the code. The habit that catches this: read warnings and check new classes and methods in the documentation for your Java version.
Backend Insight
In a backend project, wrong behaviour often hides until production: an immutable list in a service that only fails when a rare branch runs, a deprecated call that breaks on a Java upgrade, a library name that is wrong. This is why real teams rely on code review, automated tests, and compiler warnings. Pull requests do not ask who wrote the code.
There is also a supply-chain angle. When Maven (BEG-054) lets you add a dependency by name, a name an assistant invented may not exist, or may belong to someone else's package. Always look up a dependency on the official Maven Central site and check who publishes it before you add it.
Common Mistakes
Mistake 1: "It compiled, so it is right"
Compiling only proves the names and types fit. The List.of(...).add(4) code compiled. Run it, test edge cases, and read the documentation for behaviour.
Mistake 2: Asking the same assistant "are you sure?"
A second answer from the same model is more predicted text, and it may flip or may repeat the error. Use independent evidence: the compiler, a test, the documentation.
Mistake 3: Ignoring warnings
Deprecation warnings are often the only sign that a suggestion is old. Compile with warnings visible, and look up each one.
Mistake 4: Installing a dependency because an assistant named it
Verify that the library and the exact coordinates exist in the official repository before adding anything to pom.xml.
Exercises
Try each exercise yourself before opening the solutions.
Level A: Foundation
A1. Define hallucination in one sentence.
Show solution A1
A1. A hallucination is a confident answer that is invented or false.
A2. Name the four failure kinds in the table.
Show solution A2
A2. Invented method, wrong arguments, wrong behaviour, outdated advice.
A3. What is a training cutoff, and why does it matter for Java answers?
Show solution A3
A3. The cutoff is the date where the model's training data ends. Java and its libraries keep changing, so newer features, versions, and deprecations may be unknown or described the old way.
Level B: Application
B1. An assistant claims Math.max(1, 2, 3) returns the largest of three numbers, and that "abc".contains('b') checks for a character. Say how you would check both claims, and what you expect to find.
Show solution B1
B1. Compile a small program with both calls and read the Javadoc for Math.max and String.contains. Expected result with JDK 21: both lines fail to compile. Math.max has two-argument versions only (int, long, float, double), so the three-argument call gives no suitable method found for max(int,int,int). contains takes a CharSequence, so the char literal gives char cannot be converted to CharSequence. Fix: Math.max(1, Math.max(2, 3)) and "abc".contains("b").
B2. An assistant says: "Arrays.asList(1, 2) gives a list you can freely add to." Write a three-line experiment that tests this claim and say what outcome refutes the assistant.
Show solution B2
B2. For example:
List<Integer> a = Arrays.asList(1, 2);
a.set(0, 9); // works
a.add(3); // the claim says this should workRunning this with JDK 21 printed [9, 2] after the set, and then add threw UnsupportedOperationException. If add throws, the claim is refuted: the list has a fixed size, but its elements can be replaced.
B3. An assistant cites a method with a plausible name that you cannot find in the Javadoc. List the steps you take before deciding it is a hallucination.
Show solution B3
B3. Check the spelling and the class (it may be real but on another class), search the Javadoc for your Java version, try to compile a one-line call, read the exact compiler message, and check whether the method exists in another language or library (models mix them up). If the compiler says cannot find symbol and the Javadoc has nothing, treat it as a hallucination.
Level C: Challenge
C1. An assistant says: "A NullPointerException happens when you call a method on a variable set to null; to fix it, wrap everything in try/catch (Exception e)." One part is right and one part is poor advice. Separate them and explain.
Show solution C1
C1. The cause is right: a NullPointerException occurs when you use a null reference as if it were an object. The fix is poor: catching Exception around everything hides the failure instead of fixing where the null came from, and it can swallow unrelated errors. Find out which variable is null (stack trace, debugger) and fix the source.
C2. Write a short rule (three or four lines) for your own use about when "it compiles and runs" is enough evidence and when it is not.
Show solution C2
C2. For example: "Compiling shows the names and types fit. Running one example shows it works once. I also need an edge case, the documentation for new methods, and no unexplained warnings. For anything that touches data or money, I want a test as well."
Mini Challenge
Pick three methods you have used in Java (for example from String, List, or Math). For each, write one wrong but plausible claim an assistant might make (a wrong method name, a wrong argument, or wrong behaviour). Then compile or run each claim and record the exact compiler message or runtime outcome, plus the documentation sentence that settles it.
Check your work: each entry has the claim, the evidence, and a one-line correction. At least one claim should compile but behave differently from what was claimed.
Summary & Next Lesson
Key takeaways
- A hallucination is a confident answer that is invented or false: invented methods, wrong arguments, wrong behaviour, outdated advice.
- Tone is the same whether the answer is right or wrong.
- Training data has a cutoff, so recent changes may be missing.
- Verify with the compiler, a run with edge cases, the official documentation, and warnings.
- When you catch an error, give the assistant the evidence, or go to the documentation.
Next lesson
Next, "Asking good questions: goal, code, error, attempts", teaches you how to write a question that gives an assistant what it needs, which also makes the answers easier for you to check.
Self-check
- Which kind of hallucination does the compiler catch most easily?
- Why did
List.of(1, 2, 3).add(4)get past the compiler? - Does a confident tone mean an answer is correct?
- Why is a deprecation warning useful when checking an assistant's code?
- Why is asking the same assistant "are you sure?" weak evidence?
Show answers
- Invented methods and wrong arguments, through
cannot find symbolorincompatible types. addexists on theListinterface, so the types fit; only running the code showed that the list fromList.ofis unmodifiable.- No. The same process produces both tones; verify every answer.
- It often reveals advice that was common in older code but is discouraged or scheduled for removal.
- The answer is more predicted text from the same model; independent evidence (compiler, test, documentation) is stronger.
References
- Oracle, Java SE 21
ListJavadoc. Supports that lists fromList.ofare unmodifiable and throwUnsupportedOperationExceptionon mutators, and theaddAll(Collection<? extends E>)signature. - Oracle, Java SE 21
IntegerJavadoc. Supports that theInteger(int)constructor is deprecated since Java 9 and marked for removal, withInteger.valueOfrecommended. - OWASP GenAI Security Project, LLM09:2025 Misinformation. Supports hallucination as a cause of false but credible output, and the risk of models suggesting non-existent libraries.
- I did not fetch the
StringorStringBuilderJavadoc pages; theStringBuilder.reverse()claim, theMath.maxoverloads, and all compiler messages shown were confirmed by compiling and running on JDK 21.
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.