Streams, readers, and buffering
On This Lesson 11 topics
Lesson Overview
The one-line methods from the last lesson load a whole file at once. That is perfect for small files and wrong for big ones. This lesson shows the tool for the general case: open the file, take one line at a time, and close it again.
You will also learn the two families of Java I/O types. A byte stream moves raw bytes and is the right tool for images, PDFs, and archives. A character stream moves text. In the filing-cabinet picture, a byte stream hands you pages without caring what is written on them, while a character stream reads the writing. The classes BufferedReader and BufferedWriter add a buffer, a small holding area that makes reading and writing efficient.
Why this matters for a backend: log files and imports are often far larger than memory is comfortable with, and every server that forgets to close a file eventually runs into trouble. Reading line by line inside try-with-resources is the habit that avoids both problems.
The previous lesson, "Reading and writing text", covered readAllLines and writeString. The next lesson, "Character encoding (UTF-8)", explains how characters become bytes.
Learning Objectives
By the end of this lesson you will be able to:
- Explain the difference between byte streams (
InputStream,OutputStream) and character streams (Reader,Writer), and choose between them. - Explain what buffering is and why it helps.
- Read a text file line by line with
BufferedReaderandreadLine(). - Write text with
BufferedWriter, usingwriteandnewLine(). - Open readers and writers with
Files.newBufferedReaderandFiles.newBufferedWriter. - Close every resource with
try-with-resourcesand reportIOExceptionclearly.
Written Explanation
Bytes and characters
Everything in a file is stored as bytes, numbers from 0 to 255. Java offers two families of types on top of that:
| Family | Base types | Moves | Use for |
|---|---|---|---|
| Byte streams | InputStream, OutputStream | raw bytes | images, PDFs, archives, anything binary |
| Character streams | Reader, Writer | characters (text) | text files such as .txt, .csv, .log |
A character stream is a byte stream plus a translation step: bytes are decoded into characters when reading and encoded back when writing. The translation uses a character encoding, which the next lesson covers. If you copy a photo through a character stream, the translation damages it, because a photo's bytes are not text. If you read text through a byte stream, you get numbers and have to translate them yourself. So: bytes for binary, characters for text.
Why buffering
Asking the disk for one character at a time would be slow, because each request has a cost. A buffer is a block of memory: the buffered class fetches a large chunk once and then serves your small requests from memory. Writing works the same way in reverse: small writes collect in the buffer and go to the disk in bigger pieces. The documentation of BufferedReader makes exactly this point and recommends wrapping costly readers in it. The default buffer size is described as large enough for most purposes, so you rarely need to choose it.
Reading line by line
try (BufferedReader reader = Files.newBufferedReader(file)) {
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
}readLine() returns the next line without its terminator, and returns null when the end of the file is reached. That null is the signal to stop. The odd-looking loop condition assigns the line and compares it in one step; it is a standard idiom, so learn to read it: "read a line, store it in line, and continue while it is not null".
A blank line comes back as an empty string "", which is not null, so blank lines do not end the loop.
Files.newBufferedReader(path) opens the file for you and decodes it as UTF-8. Only one line is in your hands at a time, so memory use stays small however big the file is.
Writing with BufferedWriter
try (BufferedWriter out = Files.newBufferedWriter(file)) {
out.write("first line");
out.newLine();
}write(String) writes text and adds nothing. newLine() writes the platform's line separator, so you do not type \n yourself. newBufferedWriter follows the same defaults as the earlier methods: it creates the file and truncates an existing one. Pass StandardOpenOption.APPEND and CREATE to add instead.
Closing: always
A reader or writer holds an operating-system resource: an open file. If you never close it, the resource leaks, and on some systems an open file cannot be deleted or renamed by another program. A buffered writer also has a second reason: data sitting in the buffer reaches the disk when the writer is flushed or closed, so a writer you forget to close can leave a file with missing lines. The documentation for newBufferedWriter adds that errors may only show up during write, flush, or close, because of the buffering.
The tool is the one from the exceptions module: try-with-resources. Resources declared in the parentheses are closed automatically when the block ends, whether normally or by an exception. With two resources, they are closed in the reverse order of declaration. Unlike readAllLines, newBufferedReader and newBufferedWriter do not close themselves, so this is your job.
Put the catch (IOException e) after the try to report problems. A good message includes the file name and the exception's message. Note that for a missing file the message is often just the path, so add your own words around it.
Examples
Example 1: read a file line by line and number the lines (simple)
import java.io.*;
import java.nio.file.*;
import java.util.List;
public class Ex1 {
public static void main(String[] args) throws IOException {
Path file = Path.of("data", "words.txt");
Files.createDirectories(file.getParent());
Files.write(file, List.of("alpha", "", "beta", "gamma"));
int count = 0;
try (BufferedReader reader = Files.newBufferedReader(file)) {
String line;
while ((line = reader.readLine()) != null) {
count++;
System.out.println(count + ": [" + line + "]");
}
}
System.out.println("lines: " + count);
}
}Output:
1: [alpha]
2: []
3: [beta]
4: [gamma]
lines: 4The blank second line is delivered as an empty string and the loop continues. The loop ends only when readLine returns null after gamma.
Example 2: write with BufferedWriter and report errors (practical)
import java.io.*;
import java.nio.file.*;
public class Ex2 {
public static void main(String[] args) {
Path file = Path.of("data", "squares.txt");
try {
Files.createDirectories(file.getParent());
try (BufferedWriter out = Files.newBufferedWriter(file)) {
for (int i = 1; i <= 5; i++) {
out.write(i + " squared is " + (i * i));
out.newLine();
}
}
try (BufferedReader in = Files.newBufferedReader(file)) {
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
}
} catch (IOException e) {
System.out.println("I/O problem with " + file + ": " + e.getMessage());
}
}
}Output:
1 squared is 1
2 squared is 4
3 squared is 9
4 squared is 16
5 squared is 25The writer's try block ends before the reading begins, which closes the writer and flushes its buffer. That is why the reader sees all five lines. One outer catch handles a problem in any of the steps and names the file.
Example 3: filter-copy a text file and copy a binary file (realistic)
import java.io.*;
import java.nio.file.*;
import java.util.List;
public class Ex3 {
static void copyNonBlank(Path src, Path dst) throws IOException {
try (BufferedReader in = Files.newBufferedReader(src);
BufferedWriter out = Files.newBufferedWriter(dst)) {
String line;
while ((line = in.readLine()) != null) {
if (!line.isBlank()) {
out.write(line.strip());
out.newLine();
}
}
}
}
static long copyBytes(Path src, Path dst) throws IOException {
long total = 0;
try (InputStream in = Files.newInputStream(src);
OutputStream out = Files.newOutputStream(dst)) {
byte[] buf = new byte[8192];
int n;
while ((n = in.read(buf)) != -1) {
out.write(buf, 0, n);
total += n;
}
}
return total;
}
public static void main(String[] args) throws IOException {
Files.createDirectories(Path.of("data"));
Path src = Path.of("data", "messy.txt");
Files.write(src, List.of(" one ", "", " ", "two"));
Path dst = Path.of("data", "clean.txt");
copyNonBlank(src, dst);
System.out.println(Files.readAllLines(dst));
Path bin = Path.of("data", "blob.bin");
Files.write(bin, new byte[] {(byte) 0x89, 0x50, 0x4E, 0x47, 0, (byte) 0xFF});
long n = copyBytes(bin, Path.of("data", "blob-copy.bin"));
System.out.println(n + " bytes, identical: "
+ java.util.Arrays.equals(Files.readAllBytes(bin), Files.readAllBytes(Path.of("data", "blob-copy.bin"))));
try {
copyNonBlank(Path.of("data", "nothing.txt"), dst);
} catch (IOException e) {
System.out.println("Copy failed: " + e.getMessage());
}
}
}Output:
[one, two]
6 bytes, identical: true
Copy failed: data\nothing.txtThe first method is a character stream job: it works with lines, trims them, and skips blank ones. The second is a byte stream job: read(buf) fills the array and returns how many bytes it got, or -1 at the end, and the bytes are written back untouched. The file blob.bin is just six arbitrary bytes standing in for a binary file. Last, the copy of a missing file fails with an IOException whose message is only the path, so a good handler adds its own wording.
Backend Insight
Servers handle streams all the time: an uploaded image arrives as a stream of bytes, a CSV import is read line by line, and a log file is written continuously. The reason to learn streams, even when readAllLines exists, is scale and failure. A stream-based loop uses roughly the same memory for a ten-line file and a ten-million-line file, and it lets you stop early or report the line where the problem occurred.
Unclosed resources are a classic production incident. Each leaked file handle stays open until the program ends; the operating system allows only a limited number per process, so a long-running service that leaks one per request eventually fails in surprising ways. Teams therefore treat a missing try-with-resources as a code review comment, every time.
Common Mistakes
Mistake 1: Not closing the file
- Incorrect:
BufferedReader r = Files.newBufferedReader(f); ...with no close. - Correct: declare it in
try (...). If the loop throws halfway through, the file is still closed.
Mistake 2: Copying binary files with a Reader or Writer
A photo copied through characters is corrupted, because its bytes are not text. Use InputStream and OutputStream (or Files.copy) for anything binary.
Mistake 3: Stopping the loop on an empty line
- Incorrect:
while (!(line = r.readLine()).isEmpty()), which also crashes withNullPointerExceptionat the end of the file. - Correct: compare with
null. An empty line is a valid line.
Mistake 4: Writing lines without newLine()
out.write("a"); out.write("b"); produces ab on one line. Call newLine() after each line.
Mistake 5: Using a writer after its try block
The writer is closed at the end of the block. Writing to it afterwards throws an IOException. Keep all writes inside the block.
Exercises
Try each exercise yourself before opening the solutions.
Level A: Foundation
A1. For each task say byte stream or character stream: copying a .png image, reading a .csv file, saving a downloaded .zip archive, writing a log line.
Show solution A1
A1. Copying a .png: byte stream. Reading a .csv: character stream. Saving a .zip: byte stream. Writing a log line: character stream.
A2. What does readLine() return when the file has no more lines?
Show solution A2
A2. null.
A3. Give one reason to use a BufferedReader instead of reading one character at a time.
Show solution A3
A3. Each read from the disk has a cost. The buffer fetches a large chunk once and serves your small requests from memory, so reading is faster. It also gives you readLine().
Level B: Application
B1. Write static int countMatching(Path f, String word) throws IOException that counts the lines containing word, reading line by line.
Show solution B1
B1.
static int countMatching(Path f, String word) throws IOException {
int n = 0;
try (BufferedReader r = Files.newBufferedReader(f)) {
String line;
while ((line = r.readLine()) != null) {
if (line.contains(word)) n++;
}
}
return n;
}On the lines the cat, a dog, the bird sings, counting the gave 2 (run and verified).
B2. Write static void numbered(Path src, Path dst) throws IOException that copies each line of src to dst with a prefix 1: , 2: , and so on.
Show solution B2
B2.
static void numbered(Path src, Path dst) throws IOException {
try (BufferedReader r = Files.newBufferedReader(src);
BufferedWriter w = Files.newBufferedWriter(dst)) {
String line;
int i = 1;
while ((line = r.readLine()) != null) {
w.write(i++ + ": " + line);
w.newLine();
}
}
}The result for the three lines above was [1: the cat, 2: a dog, 3: the bird sings].
B3. This code has two problems. Find and fix them.
BufferedWriter w = Files.newBufferedWriter(Path.of("out.txt"));
w.write("one");
w.write("two");Show solution B3
B3. Problem one: the writer is never closed, so the file handle leaks and buffered text may never reach the disk. Problem two: no newLine(), so the file would hold onetwo on one line. Fix:
try (BufferedWriter w = Files.newBufferedWriter(Path.of("out.txt"))) {
w.write("one");
w.newLine();
w.write("two");
w.newLine();
}The method that contains this code must handle or declare IOException.
Level C: Challenge
C1. Write static String longest(Path f) throws IOException that returns the longest line of a file, or null for an empty file, without loading the whole file into a list.
Show solution C1
C1.
static String longest(Path f) throws IOException {
String best = null;
try (BufferedReader r = Files.newBufferedReader(f)) {
String line;
while ((line = r.readLine()) != null) {
if (best == null || line.length() > best.length()) best = line;
}
}
return best;
}It returned the bird sings for the file above and null after the file was emptied.
C2. Write static void stripComments(Path src, Path dst) that copies every line except those starting with #. It must close both files in every case and print Could not copy <src> to <dst>: <message> when an IOException occurs, instead of crashing.
Show solution C2
C2.
static void stripComments(Path src, Path dst) {
try (BufferedReader in = Files.newBufferedReader(src);
BufferedWriter out = Files.newBufferedWriter(dst)) {
String line;
while ((line = in.readLine()) != null) {
if (!line.startsWith("#")) {
out.write(line);
out.newLine();
}
}
} catch (IOException e) {
System.out.println("Could not copy " + src + " to " + dst + ": " + e.getMessage());
}
}Run on # comment, port=80, #x, host=h it produced [port=80, host=h]; for a missing source it printed the message and continued.
Mini Challenge
Write a small program LogFilter with a method static int filterLog(Path in, Path out, String level) that reads a log file line by line and writes to out only the lines that contain the given level text (for example ERROR), then returns how many lines it kept. In main, create a sample log with at least six lines using Files.write, call the method, and print the count and the result file.
Check your work: both files are opened in one try-with-resources, an IOException is reported with the file names and does not crash main, and the code never loads the whole input into a list.
Summary & Next Lesson
Key takeaways
- Byte streams (
InputStream,OutputStream) move raw bytes for binary data; character streams (Reader,Writer) move text. - Buffering collects data in memory so the disk is used in larger steps.
BufferedReader.readLine()returns one line at a time andnullat the end. Blank lines are empty strings, notnull.BufferedWriterneedsnewLine()for line breaks. It is only reliably flushed when closed.Files.newBufferedReaderandnewBufferedWriterdo not close themselves: usetry-with-resourcesand reportIOExceptionwith the file name.
Next lesson
You have seen that character streams translate bytes into characters. The next lesson, "Character encoding (UTF-8)", explains that translation, and what goes wrong when two programs disagree about it.
Self-check
- Which kind of stream should you use to copy an image?
- What does
readLine()return at the end of the file? - Why is
try-with-resourcesthe safest way to close a reader? - What does
newLine()do thatwrite("\n")does not promise? - Why can a forgotten
close()on a writer lose data?
Show answers
- A byte stream (
InputStreamandOutputStream). null.- The resource is closed automatically when the block ends, whether it finishes normally or throws.
- It writes the platform's own line separator, so the same code produces the right line break on each operating system.
- Data may still be waiting in the buffer. It reaches the file on flush or close, so an unclosed writer can leave the file incomplete.
References
- Oracle, BufferedReader (Java SE 21 API). It supports the purpose of buffering,
readLine()returningnullat the end, the line terminators, the default buffer size, and use with try-with-resources. - Oracle, Reader (Java SE 21 API). It supports
Readeras the base class for character streams (the page itself does not describe byte streams; that contrast, and the role ofInputStreamReaderas a bridge, are general Java knowledge). - Oracle, Files (Java SE 21 API). It supports the UTF-8 default of
newBufferedReaderandnewBufferedWriter, the default write options, that the caller must close those streams, and that write errors may only appear later because of buffering. - Not verified against a source: the remark about per-process limits on open files is general operating-system knowledge. The example outputs come from running the code on JDK 21.
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.