اختبار الكود المولَّد وفق المتطلب
في هذا الدرس 11 موضوعات
هذا الدرس متاح حاليًا باللغة الإنجليزية فقط، ولم تُنشر نسخته العربية بعد.
Lesson Overview
Tracing by hand (last lesson) finds bugs, but it is slow and you can forget a case. Automated tests make the check repeatable: you write them once, and every later change to the code gets checked again for free.
The key idea of this lesson is whose behaviour the tests describe. If you write tests by looking at the generated code, they describe the code, bugs included. If you write them from the requirement, before you look at the draft, they describe what the code is supposed to do, and they can fail.
Why this matters: a test suite is the only fast, honest answer to "does this draft do what I asked?". In a team, tests written from the ticket are also how a reviewer knows that a generated change is safe.
You need JUnit 5 from BEG-045 (@Test, assertEquals, assertThrows, and reading failures) and the boundary and equivalence-class method from the same module. The next lesson, "Licensing, integrity, and disclosure", covers the rules for using generated code honestly.
Learning Objectives
By the end of this lesson you will be able to:
- Explain why tests must be written from the requirement, before reading the generated code.
- Derive a test list with normal cases, boundaries, and invalid input for a short requirement.
- Recognize a test that merely copies the code's current behaviour.
- Add a size test to catch a solution that is correct but too slow, and know its limits.
- Decide, when a test fails, whether the code or the test is wrong.
Written Explanation
Tests first, from the requirement alone
The order is:
- Write the requirement (lesson 1).
- Write the tests, without looking at the generated code. Better still, before you ask for the code.
- Get the draft, run the tests, and read the failures.
Why before? Once you have read a draft, it is hard to unsee it. You will unconsciously choose inputs that the draft handles, and you will write expected values that match what it returns. A test written that way cannot catch the draft's mistakes, because the draft's mistakes are in the test too.
The trap: expected values copied from the code's output
A common shortcut is: run the generated method, see what it prints, and paste that value into assertEquals. This only proves that the code returns what the code returns. A teammate who does this with a buggy method gets a green test suite and a bug that is now protected by tests. The expected value must always come from the requirement, worked out by hand or from a trusted source.
What to cover: the checklist
For each requirement, list inputs from these groups (BEG-045 L05):
| Group | Examples |
|---|---|
| Normal | Typical inputs, in a different order, with negative values if allowed |
| Boundaries | Empty, one item, exactly at a limit, one below, one above, first item, last item |
| Invalid | null, a negative where it is not allowed; the requirement says what must happen (usually an exception, tested with assertThrows) |
| Size | One large input, to catch hidden O(n²) work |
| Duplicates and ties | Repeated values, equal neighbours, all-equal input |
Give each test a name that states the behaviour, such as largestAppearsTwice, and test one thing per test method. Tests must not depend on each other (BEG-045).
A size test, with its limits
Some drafts are correct and still too slow. To catch that, run the method on a large input and demand it finish within a generous limit. JUnit has assertTimeout(Duration, Executable) for this. (This one assertion was not taught in BEG-045; treat it as a small preview.) Timing tests have a weakness: they depend on the machine, so use a limit that is generous (seconds, not milliseconds) and a size big enough that a bad algorithm is clearly out of range. They are a safety net, not a benchmark.
When a test fails: code or test?
A failing test means "the code and the test disagree". Do not assume the code is wrong, and do not edit the test until it passes. Reread the requirement and decide:
- The test follows the requirement and the code does not: fix the code.
- The code follows the requirement and the test does not: fix the test (you made a mistake).
- The requirement does not say: the requirement is incomplete. Decide what is right, write it down, and then fix whichever side disagrees.
Examples
Example 1: a test that copies the bug (simple)
Requirement: sum(int[] a) returns the sum of the items. The draft (from lesson 1) has a loop that stops one item early. A careless tester runs sum(new int[]{3, 4}), sees 3, and writes:
@Test
void sumOfTwo() {
assertEquals(3, sum(new int[]{3, 4})); // copied from the output
}The test passes, and the bug is now guarded. The same test written from the requirement is assertEquals(7, sum(new int[]{3, 4})), because 3 + 4 is 7. That version fails on the draft, which is the whole point.
Example 2: tests from a requirement find two bugs (practical)
Requirement: secondLargest(int[] a) returns the second largest distinct value. If the array is null or has fewer than two distinct values, it throws IllegalArgumentException.
Tests written first, from the requirement only:
class StatsTest {
@Test void normalCase() { assertEquals(7, Stats.secondLargest(new int[]{3, 9, 7, 1})); }
@Test void negativeValues() { assertEquals(-5, Stats.secondLargest(new int[]{-8, -3, -5})); }
@Test void exactlyTwoValues() { assertEquals(2, Stats.secondLargest(new int[]{9, 2})); }
@Test void largestAppearsTwice() { assertEquals(7, Stats.secondLargest(new int[]{9, 9, 7})); }
@Test void allValuesEqualIsInvalid(){ assertThrows(IllegalArgumentException.class,
() -> Stats.secondLargest(new int[]{4, 4, 4})); }
@Test void oneValueIsInvalid() { assertThrows(IllegalArgumentException.class,
() -> Stats.secondLargest(new int[]{4})); }
@Test void nullIsInvalid() { assertThrows(IllegalArgumentException.class,
() -> Stats.secondLargest(null)); }
}Then the draft arrives. It tracks first and second, starting both at Integer.MIN_VALUE, and updates them in a loop with if (x > first) {...} else if (x > second) {...}. It checks null and length. Running the suite (Maven with JUnit 5.12.2):
Tests run: 7, Failures: 2
StatsTest.allValuesEqualIsInvalid:9 Expected java.lang.IllegalArgumentException to be thrown, but nothing was thrown.
StatsTest.largestAppearsTwice:8 expected: <7> but was: <9>Five of seven tests pass, so the draft looks fine on ordinary input. Two edge tests fail, and both come from the word "distinct" in the requirement. For [9, 9, 7] the second 9 is greater than second, so the draft treats it as the second largest. For [4, 4, 4] nothing signals "no second value", because the sentinel starting value is returned instead.
A fix that only counts values strictly smaller than the largest, and remembers whether a second value was found:
int first = a[0];
boolean hasSecond = false;
int second = 0;
for (int i = 1; i < a.length; i++) {
int x = a[i];
if (x > first) { second = first; hasSecond = true; first = x; }
else if (x < first && (!hasSecond || x > second)) { second = x; hasSecond = true; }
}
if (!hasSecond) throw new IllegalArgumentException("need two different values");
return second;(preceded by the null and length < 2 check). Rerun: Tests run: 7, Failures: 0.
Example 3: a size test catches a slow draft (realistic)
Requirement: hasDuplicate(List<Integer> xs) returns true when a value appears twice. We write the usual tests (a duplicate, no duplicate, empty list) plus a size test:
@Test
void largeDistinctInputFinishesQuickly() {
List<Integer> big = new ArrayList<>();
for (int i = 0; i < 100_000; i++) big.add(i);
assertTimeout(Duration.ofSeconds(1), () -> Dup.hasDuplicate(big));
}The ArrayList plus contains draft from lesson 2 passes the three small tests and fails the size test:
DupTest.largeDistinctInputFinishesQuickly:13 execution exceeded timeout of 1000 ms by 6806 msAfter switching to the HashSet version, all four tests pass. The failure message shows the size of the problem, not just that something is wrong. Because the exact time depends on the machine, the failure above is from one run and yours will differ; the limit is chosen so that the gap is obvious.
Backend Insight
Every real backend project keeps its tests in the repository and runs them on every change (later levels connect this to Git and automated builds). When a pull request contains generated code, the tests are the reviewer's evidence, and tests derived from the ticket carry the business rules: the order limit, the stock check, the price rounding.
One practical habit that comes from this lesson: put the requirement in the test, for example in the test name or a one-line comment that quotes the rule ("free shipping from exactly 50"). When a future change breaks the test, the failure message tells the next developer which rule was broken, not just that a number differs.
Common Mistakes
Mistake 1: Writing tests after reading the draft
- Incorrect: reading the code, then writing tests for what you saw.
- Correct: writing the tests from the requirement first, then running them against the draft.
Mistake 2: Expected values copied from the output
If the expected value in assertEquals came from running the code, the test cannot fail for that input. Work the answer out from the requirement.
Mistake 3: Only happy-path tests
Five normal tests would have passed the secondLargest draft in Example 2. The two failing tests were a repeated maximum and an all-equal array. Always add boundaries, duplicates, and invalid input.
Mistake 4: "Fixing" a failing test until it passes
When a test fails, reread the requirement. Changing the expected value without a requirement-based reason hides the defect.
Exercises
Level A: Foundation
A1. Why should the tests be written before you read the generated code?
أظهر الحل A1
A1. Tests written after reading the code tend to copy the code's assumptions and results, so the code's bugs end up in the tests. Tests written from the requirement describe correct behaviour, so they can fail.
A2. Requirement: isValidPin(String s) returns true when s is exactly four digits, and false otherwise (including null). List five test inputs with their expected results.
أظهر الحل A2
A2. For example: "1234" gives true; "123" gives false (too short); "12345" gives false (too long); "12a4" gives false (non-digit); null gives false. An empty string "" giving false is another good one.
A3. A teammate writes assertEquals(1, countPositives(new int[]{5, -1, 2})) after running the lesson 2 draft and seeing it print 1. What two things are wrong with this?
أظهر الحل A3
A3. (1) The expected value was copied from the code's output, so the test only confirms what the code does. (2) Worked out from the requirement, the answer is 2 (the items 5 and 2), so the test expects a wrong value and now protects the bug.
Level B: Application
B1. Requirement: shippingFee(double total) returns 0 when the total is 50 or more, 4.99 when it is below 50, and throws IllegalArgumentException for a negative total. Write five JUnit tests (the bodies only) that cover normal, boundary, and invalid input. The draft uses total > 50. Which test fails, and what is the message?
أظهر الحل B1
B1.
@Test void smallOrderPays() { assertEquals(4.99, Shipping.shippingFee(20.0), 0.001); }
@Test void justBelowThresholdPays() { assertEquals(4.99, Shipping.shippingFee(49.99), 0.001); }
@Test void exactlyFiftyIsFree() { assertEquals(0.0, Shipping.shippingFee(50.0), 0.001); }
@Test void largeOrderIsFree() { assertEquals(0.0, Shipping.shippingFee(120.0), 0.001); }
@Test void negativeTotalIsInvalid() { assertThrows(IllegalArgumentException.class,
() -> Shipping.shippingFee(-1)); }With total > 50, exactlyFiftyIsFree fails. We ran it: expected: <0.0> but was: <4.99>. The fix is total >= 50. (The third argument of assertEquals is the allowed difference for double values, as in BEG-045.)
B2. For each failing test decide whether the code or the test is wrong, using this requirement: "A person aged 18 or over is an adult." (a) assertFalse(isAdult(18)) fails because isAdult(18) returns true. (b) assertTrue(isAdult(17)) fails because isAdult(17) returns false.
أظهر الحل B2
B2. (a) The test is wrong: 18 is an adult by the requirement, so the code returning true is correct. Fix the test to assertTrue(isAdult(18)). (b) The test is wrong too: 17 is not an adult, so false is correct, and the test should use assertFalse. (If the requirement were silent on 18, you would first settle the requirement.)
B3. Label each test as normal, boundary, invalid, or size: (a) largeInput: 200,000 distinct numbers. (b) emptyList. (c) nullThrows. (d) threeBooksAtLimitThree. (e) typicalOrder.
أظهر الحل B3
B3. (a) size. (b) boundary (empty). (c) invalid. (d) boundary (exactly at the limit). (e) normal.
Level C: Challenge
C1. Requirement: "median(int[] sorted) returns the middle value of a sorted array." List what is unclear, rewrite the requirement so that tests can be written, and list four tests with expected values.
أظهر الحل C1
C1. Unclear: what to return for an even number of items (two middle values: the lower one? their average?), whether the result is an int or a double, and what happens for an empty array. A rewrite: "Returns the median as a double. For an odd length it is the middle item; for an even length it is the average of the two middle items. An empty or null array throws IllegalArgumentException." Tests: [1, 3, 5] gives 3.0; [1, 3] gives 2.0; [7] gives 7.0; [] throws.
C2. The draft findLast from lesson 2 (exercise B2) uses break at the first match. Write two tests, from this requirement only, that catch it: "findLast(int[] a, int t) returns the index of the last occurrence of t, or -1 if absent."
أظهر الحل C2
C2. For example: findLast(new int[]{4, 7, 4}, 4) must be 2, and findLast(new int[]{9, 9}, 9) must be 1. Add findLast(new int[]{1, 2}, 5) equals -1 for the absent case. The draft returns 0 for the first (we ran it), so the test fails and exposes the break.
Mini Challenge
Take this requirement: "lastWord(String s) returns the last word of a sentence; words are separated by single spaces; an empty string or null throws IllegalArgumentException."
- Write at least six tests (the bodies only), covering normal input, a one-word string, a boundary, and invalid input, before writing or generating any code. Decide what to do about a trailing space and write the decision into the requirement.
- Optionally ask an assistant for the method, run your tests, and note which fail.
- For each failure, state whether the code or the test was wrong, with the sentence of the requirement that decides it.
Check your work: every expected value comes from the requirement, at least two tests are invalid-input tests, and the trailing-space decision is written down.
Summary & Next Lesson
Key takeaways
- Write tests from the requirement, before you read the generated code, so they can fail.
- Never copy expected values from the code's output; that only confirms its bugs.
- Cover normal cases, boundaries, invalid input, duplicates, and one large input for speed.
- A size test with a generous
assertTimeoutlimit catches slow solutions, but depends on the machine. - When a test fails, reread the requirement to decide whether the code, the test, or the requirement itself is wrong.
Next lesson
The next lesson, "Licensing, integrity, and disclosure", covers the non-technical half of using generated code: what to check about licences, what never to paste into an assistant, and how to disclose AI use honestly.
Self-check
- Why do tests written after reading the code tend to miss its bugs?
- Where must the expected value in an assertion come from?
- Name three groups of inputs a good test list covers.
- In Example 2, which word of the requirement led to the two failing tests?
- A test fails. What is the first thing to do?
أظهر الإجابات
- They copy the code's assumptions and results, so the bugs are in both.
- From the requirement, worked out by hand or from a trusted source.
- Any three of: normal, boundaries, invalid input, size, duplicates and ties.
- "Distinct": it makes a repeated maximum and an all-equal array special cases.
- Reread the requirement and decide whether the code, the test, or the requirement is wrong.
References
- JUnit Team, Assertions API (JUnit 5). Its index lists
assertTimeout(Duration, Executable)andassertThrows(Class, Executable). The page text we could read did not describe howassertTimeoutbehaves; that comes from our own run, where it failed a slow call with an "exceeded timeout" message. - GitHub Docs, Responsible use of GitHub Copilot code completion. It advises reviewing, testing, and validating generated suggestions before accepting them.
- All test outputs in this lesson come from compiling and running the examples with JDK 21 and JUnit 5.12.2 in a scratch Maven project.
التطبيق
طبّق هذا الدرس بمختبر الوحدة وتمارينها واختبارها.
تعليم الدرس كمكتمل اختياري ويمكنك التراجع عنه في أي وقت. يبقى تقدمك محفوظًا في هذا المتصفح.