Running tests with Maven
On This Lesson 11 topics
Lesson Overview
In module BEG-045 you ran JUnit tests by clicking a green arrow in the IDE. This lesson moves that to the terminal with mvn test. The tests are the same; what changes is who runs them and how: Maven finds the test classes by itself, runs them all, prints a summary, and fails the build if any test fails.
In the recipe analogy, the tasting step is now part of the card: the assistant tastes the dish at the right moment, every time, and refuses to serve it if it is bad.
Why this matters: a build server cannot click a green arrow. It runs the same mvn test you do, and decides from the result whether your change is acceptable. This lesson builds on lesson 4 (the lifecycle) and on BEG-045 (writing and reading JUnit tests).
Learning Objectives
By the end of this lesson you will be able to:
- Run all tests with
mvn testand read theTests run:summary. - Explain which test classes Maven (Surefire) picks up, including its naming convention.
- Run a single test class or a single test method from the command line.
- Tell a test failure from a test error in Maven's output.
- Explain why a failing test fails the build.
- Notice the quiet case in which no tests run at all.
Written Explanation
The command and its summary
Run, from the folder that contains pom.xml:
mvn testMaven compiles the main code and the tests, then runs the tests with a plugin called Surefire (lesson 4 showed it as surefire:test). The part of the output you care about, from the example project (real output):
[INFO] T E S T S
[INFO] Running com.example.GreeterTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.095 s -- in com.example.GreeterTest
[INFO] Results:
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESSEach test class gets one line; the final Tests run: line is the total. The line to look for is BUILD SUCCESS (all green) or BUILD FAILURE. Reports are also written to target/surefire-reports/, one text file and one XML file per test class.
Which classes are tests
You never list your test classes. Surefire scans the compiled test classes and runs those whose names follow a convention. By default these patterns are Test*, *Test, *Tests, and *TestCase (for example GreeterTest, TestGreeter). A test class with some other name is silently ignored. I checked this: a class named Checks containing a failing @Test method was not run, and mvn test ended with BUILD SUCCESS and Tests run: 2. After renaming it to ChecksTest, it ran:
[INFO] Running com.example.ChecksTest
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 ... <<< FAILURE! -- in com.example.ChecksTest
[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0
[INFO] BUILD FAILURERule of thumb: end every test class name with Test.
Also, tests must be in src/test/java. Maven compiles and scans only that tree for tests. Surefire also needs the JUnit libraries: you gave them to Maven with the test-scoped junit-jupiter dependency from lesson 3. (The Surefire documentation says it selects the JUnit Jupiter engine when the JUnit Platform libraries are on the class path.)
Failure versus error
Maven reports two kinds of red:
- A failure means an assertion was false: the test ran, and the result was not the expected one.
- An error means something unexpected happened before the assertion could decide: an exception such as
ArithmeticExceptionescaped the test.
Real output of each, with the details trimmed:
[ERROR] ChecksTest.wouldFail:9 expected: <1> but was: <2>
[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0[ERROR] ChecksTest.crashes:9 Arithmetic / by zero
[ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0In both cases the line after the dot names the test method and the number after the colon is the line in the test file. You learned to read this style in lesson BEG-045-L06; it is the same message with Maven's prefix.
The test reports
Besides the console, Surefire saves its results in target/surefire-reports/. For the example project that folder holds com.example.GreeterTest.txt and TEST-com.example.GreeterTest.xml. The text file is a small summary, for example (real content):
Test set: com.example.GreeterTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.095 s -- in com.example.GreeterTestThe XML file holds the same results in a format that tools understand, such as build servers that draw a test dashboard. For you, the text file is useful when the console output scrolled away: open it instead of running the build again. These files are part of target/, so they are deleted by mvn clean and never committed. A report from an earlier run can also linger if you do not clean; in my experiment, reports from a previous run were still in the folder after I moved the tests away, another reason to read the console total instead of trusting old files.
A failing test fails the build
Both a failure and an error end with BUILD FAILURE. Because tests run before packaging (lesson 4), no JAR is built after a red run. A teammate or a server that runs mvn test therefore learns the same thing you do, without anyone having to inspect the report.
Running only some tests
While fixing one test you do not want to wait for all of them. The -Dtest option selects tests (real output):
mvn -Dtest=GreeterTest test # one class
mvn -Dtest=GreeterTest#greetsEmptyName test # one method[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.114 s -- in com.example.GreeterTestThis is for your own speed while working. Before committing, always run the whole suite with plain mvn test.
The silent case: no tests ran
In an experiment I moved the test file out of src/test/java into a folder named src/test/mytests. mvn test finished with BUILD SUCCESS, and printed no Tests run: line at all. Nothing was wrong from Maven's point of view: there simply were no tests to run. That is the dangerous kind of success. Make it a habit to read the Tests run: total, and check that it matches the number of tests you expect.
Where the analogy stops
A cook knows when a dish tastes wrong. Surefire only knows what your tests check: a green build means "the checks you wrote passed", not "the program is correct". That is why BEG-045 taught you to choose tests carefully, with boundaries and exceptional cases.
Examples
Example 1: a green run (simple)
$ mvn test
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESSTwo tests, both passed, build succeeded.
Example 2: a red assertion (practical)
After changing an expected value in GreeterTest to the wrong text:
[ERROR] GreeterTest.greetsByName:9 expected: <Hi, Ana!> but was: <Hello, Ana!>
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
[INFO] BUILD FAILURERead left to right: the test class, the method, the line, expected and actual. Fix either the code or the expectation, whichever is wrong, and run again.
Example 3: a class that never runs (realistic)
A teammate writes class OrderChecks with ten tests and commits it. CI is green and everyone is happy. Months later someone notices the total never included those ten tests, because the class name does not match Surefire's patterns. The fix is to rename the class to OrderChecksTest. The lesson: a green build is only as good as the tests that actually ran.
Backend Insight
Real backend projects often have hundreds or thousands of tests. The server runs them on every push and blocks changes that break any. This is the practical payoff of BEG-045: the test suite becomes a safety net that runs without anyone remembering to run it. Later you will also meet slower integration tests that run in the verify phase, which is one reason that phase exists, and a naming scheme (not just *Test) that separates them.
A useful habit follows from this: before pushing a change, run mvn clean test (or mvn clean package) yourself. If it is green on your machine from a clean target/, the server will almost always agree, and your teammates will not meet a red build caused by your change.
Common Mistakes
Mistake 1: Naming a test class without Test
- Incorrect:
class StockCheckswith@Testmethods. - Correct:
class StockTest.
Mistake 2: Putting tests in the wrong folder
Tests outside src/test/java are not compiled as tests, and Maven will not complain. Check the Tests run: total.
Mistake 3: Reading only the last line
BUILD SUCCESS with Tests run: 0 (or no such line) means nothing was verified.
Mistake 4: Running only one test and committing
-Dtest=... is a convenience. A change that breaks a different test will only be caught by the full run.
Mistake 5: Using JUnit 4 imports in a JUnit 5 project
If you copy an older example with org.junit.Test, the method is not recognized by the Jupiter engine, and its test may not run (module BEG-045 warned about this). Use org.junit.jupiter.api.Test.
Exercises
Try each exercise yourself before opening the solutions.
Level A: Foundation
A1. What command runs all tests, and from which folder?
Show solution A1
A1. mvn test, from the folder containing pom.xml.
A2. Name three class names that Maven's default pattern treats as test classes.
Show solution A2
A2. For example AccountTest, TestAccount, AccountTests, AccountTestCase (any three that match Test*, *Test, *Tests, *TestCase).
A3. What is the difference between a failure and an error?
Show solution A3
A3. A failure means an assertion was false. An error means an unexpected exception stopped the test.
Level B: Application
B1. A project's output ends with Tests run: 0 and BUILD SUCCESS. List three possible reasons.
Show solution B1
B1. The tests are in the wrong folder (not src/test/java); the classes' names do not match the Surefire patterns; there are no test methods or they have the wrong annotation import; or tests were skipped on purpose.
B2. Write the command that runs only the method withdrawRejectsNegative of class AccountTest.
Show solution B2
B2. mvn -Dtest=AccountTest#withdrawRejectsNegative test.
B3. Explain why a failing test fails the whole build instead of being just a warning.
Show solution B3
B3. The test phase is part of the lifecycle before package; if it fails, Maven stops. This ensures that a build which passes has also passed its checks, and that a broken program is never packaged unnoticed.
Level C: Challenge
C1. In a scratch project, write one passing test and one test that throws an exception on purpose. Run mvn test and write down the totals and which one is the failure and which the error.
Show solution C1
C1. With one passing test and one test that throws, expect Tests run: 2, Failures: 0, Errors: 1 and BUILD FAILURE. The throwing test is the error. (A test with a wrong assertion would be counted as a failure instead.)
C2. A teammate's CI is green but a test class is never run. How would you discover that, and what would you change?
Show solution C2
C2. Compare the number of test classes in src/test/java with the Running ... lines in the build output (or the files in target/surefire-reports). Rename the missing class so the name ends in Test.
Mini Challenge
In the project of lesson 4, add a second test class EdgeCasesTest with two tests, one that passes and one that fails on purpose. Predict the final Tests run: line before running mvn test, then check yourself. Next fix the failing test, and finish by running a single method with -Dtest=EdgeCasesTest#.... Write down every Tests run: line you saw.
Check your work: your prediction matched the output (or you can say why not), and the full run ends with BUILD SUCCESS and your expected total.
Summary & Next Lesson
Key takeaways
mvn testfinds your test classes by naming convention (such as*Test), runs them with Surefire, and prints a summary.- A failure is a false assertion; an error is an unexpected exception.
- Any red test makes the build fail, so nothing is packaged after it.
-Dtest=Class#methodruns a subset for speed; always run the full suite before committing.- Read the
Tests run:total: a success with no tests proves nothing.
Next lesson
The next lesson, "Demo: converting a project to Maven", puts the whole module to work: you will convert an existing project step by step and make mvn test pass.
Self-check
- Which command runs the tests?
- Does a class named
PriceChecksrun by default? - What does an error (not a failure) mean?
- How do you run one test method?
- Why is "BUILD SUCCESS" alone not enough?
Show answers
mvn test.- No: the name does not match the default patterns such as
*Test. - An unexpected exception escaped the test before an assertion could decide.
mvn -Dtest=ClassName#methodName test.- If no tests ran, nothing was verified; check the
Tests run:total.
References
- Apache Maven Surefire, Using JUnit 5 Platform. Supports the default test class name patterns (
Test*,*Test,*Tests,*TestCase) and that Surefire runs JUnit 5 tests via the JUnit Platform when the Jupiter engine is on the class path. - Apache Maven, Introduction to the Build Lifecycle. Supports
surefire:testbeing bound to thetestphase, beforepackage. - All output shown was produced by Maven 3.9.11 with Surefire 3.2.5 and JUnit 5.12.2 on JDK 21. I did not find a fetched page that states a failing test fails the build by default; that is stated from my own runs (
BUILD FAILURE). Newer Surefire versions may print slightly different text.
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.