Structured vs unstructured data
On This Lesson 11 topics
Lesson Overview
Not all data looks the same. A list of customers with a name, an email, and a phone number is very different from a voice message or a holiday photo. The shape of your data decides where it fits best, and a database is much better at some shapes than others.
In this lesson you will learn three shapes: structured, semi-structured, and unstructured data. You will classify real examples, see the same contact written as a CSV line and as JSON, and learn the usual backend habit for big files: keep the file in file storage and keep a small structured record about it in the database.
You need the previous lesson: a file has no idea what its bytes mean, and a database adds rules and fast search. You also need to recognise CSV (from the file-handling module) and JSON (from the earlier modules that used it). No new Java is needed.
Why this matters for a backend: when you design what to store, the first question is not "which database?" but "what shape is this data?" The answer guides every later choice.
Learning Objectives
By the end of this lesson you will be able to:
- Define structured, semi-structured, and unstructured data.
- Classify everyday examples into the three shapes, and explain the reasoning.
- Explain why a CSV row is structured and why a JSON object is semi-structured.
- Explain why fixed fields make data easy to search and to validate.
- Describe how a backend stores a photo: the file in file storage, and a structured record in the database.
- Explain why "unstructured" does not mean "useless" or "cannot be stored".
Written Explanation
Structured data: fixed fields
Structured data has a fixed, known set of fields, and every record has the same ones. A contacts table has id, name, and email; every contact has all three, in the same order, with the same kind of value in each. A spreadsheet with column headings is structured. So is a CSV file where every line has the same columns.
Why it is useful: because the fields are known in advance, a program can ask precise questions ("all contacts whose email ends in @example.com"), check values ("the id must be a whole number"), and sort or total columns. The structure is a promise, and software can rely on that promise.
Semi-structured data: fields that can vary
Semi-structured data has labelled fields, but the set of fields can differ from record to record, and values can be nested inside other values. The most common example for a backend is JSON, the text format that web services use to send data. JSON describes objects as named values: each value can be text, a number, true or false, null, a list, or another object. The labels travel with the data, so each record describes itself.
Here are two contacts. The first has a name and an email. The second has no email, but a list of phone numbers:
{"id": 7, "name": "Sara Ali", "email": "sara@example.com"}
{"id": 8, "name": "Omar", "phones": ["0100", "0111"]}Both are valid JSON, and there is no rule that says they must have the same fields. That is the flexibility. The price is that the reader cannot assume a field exists: email might simply be missing. Other examples of semi-structured data are XML documents and the configuration files used by many tools.
Unstructured data: no fixed shape
Unstructured data has no predefined fields that a program can rely on. Photos, videos, audio recordings, scanned documents, and free text such as an email body or a customer review are the usual examples. The content has plenty of meaning for a person, but a program cannot ask "what is the value of field X?" without first doing work to extract meaning, for example speech recognition for audio or image analysis for photos.
Careful: unstructured does not mean "messy" or "bad". It means the data does not come in labelled fields. A high-quality photo is unstructured. It is stored, copied, and shown every day without trouble.
A spectrum, not three boxes
The three names are a convenient spectrum, not strict categories. A text file of free sentences is unstructured, but if each line starts with a date, a program can split it. An email is partly structured (sender, subject, date) and partly unstructured (the body). When you classify, say which part you mean. A useful test is: "Could a program reliably find a given field by name or position?"
- Always, in every record: structured.
- By name, but not every record has it: semi-structured.
- No: unstructured.
What this means for storing data
Databases work best with structured data, because fixed fields allow the rules and fast search from the previous lesson. Many databases also accept semi-structured values (for example a JSON column or a document store, which the next lesson covers). Large unstructured items such as photos and videos usually do not live inside the database. They go into file storage: a plain file system or a cloud service that stores files and gives each one an address. The database keeps a small structured record about the file: its id, who uploaded it, its file name, its size, and where it is stored. This is a pattern you will see in almost every backend that handles uploads.
Examples
Example 1: one contact, three shapes (simple)
This program prints the same contact the way a CSV file and a JSON document would hold it, and then shows what each format does with a missing value.
import java.util.*;
public class L02a {
public static void main(String[] args) {
String csvLine = "7,Sara Ali,sara@example.com";
String json = "{\"id\": 7, \"name\": \"Sara Ali\", \"email\": \"sara@example.com\"}";
System.out.println(csvLine);
System.out.println(json);
String json2 = "{\"id\": 8, \"name\": \"Omar\", \"phones\": [\"0100\", \"0111\"]}";
System.out.println(json2);
String[] f = csvLine.split(",");
System.out.println("name = " + f[1] + ", fields = " + f.length);
String[] g = "9,Lina,,0100".split(",");
System.out.println("fields = " + g.length + ", email = '" + g[2] + "'");
}
}Output with JDK 21:
7,Sara Ali,sara@example.com
{"id": 7, "name": "Sara Ali", "email": "sara@example.com"}
{"id": 8, "name": "Omar", "phones": ["0100", "0111"]}
name = Sara Ali, fields = 3
fields = 4, email = ''In the CSV, the meaning of each value comes from its position. The third value is the email only because the format says so. The row with the empty email still needs its empty slot (two commas in a row), or every later value shifts to the wrong column. In the JSON version, each value carries its own label, so a missing email can simply be absent. Note also that Lina's row has four fields while the others have three: nothing in a CSV file stops that.
Example 2: classify an online shop's data (practical)
An online shop stores several kinds of data. Classify each, and say where it would usually live.
| Data | Shape | Where it usually lives |
|---|---|---|
| Customer table: id, name, email | Structured | Database |
| Product catalogue where each product has different attributes (a laptop has RAM, a shirt has size) | Semi-structured (or structured with many optional columns) | Database, often with a flexible or JSON-valued field |
| Product photos | Unstructured | File storage, with a database record pointing to each |
| Customer reviews (free text) | Unstructured text with a structured wrapper: who wrote it, for which product, when, the star rating | Database for the wrapper and, in many designs, the text too |
| Server log lines | Semi-structured (a date and a message) | Files or a log system |
The review row shows why the spectrum matters. The star rating is a structured number you can average. The sentence "arrived late but works well" is unstructured: counting how many reviews mention "late" requires extra work.
Example 3: storing a photo upload (realistic)
A user uploads a profile photo of 2 MB. A backend does not put the 2 MB inside a contacts table. It does this:
- Receives the upload and saves the bytes as a file in file storage under a generated name such as
photos/7-profile.jpg. - Inserts a structured record in the database:
| id | owner_id | file_name | size_bytes | stored_at |
|---|---|---|---|---|
| 31 | 7 | profile.jpg | 2000000 | photos/7-profile.jpg |
- When the profile page is requested, the backend reads the record, finds
stored_at, and returns the photo's address to the browser.
You can see the first part with Java you already know. Files.size(path) returns the size of any file in bytes, whatever is in it, and that number is the structured fact you would record:
long size = Files.size(Path.of("photos", "7-profile.jpg"));This line needs a real file at that path (it throws an IOException otherwise), so it was not run here. The database holds facts about the photo. File storage holds the photo. Each is used for what it does best: the database searches ("all photos uploaded by user 7") and file storage serves big bytes.
Backend Insight
Real backends receive semi-structured data constantly. A client sends an order as JSON, and the backend must treat it as untrusted: a field may be missing, have the wrong type, or contain unexpected text. The flexibility of the format does not mean the program may be relaxed. A good backend checks the fields it needs and turns the JSON into a strict structure (for example a Java object with typed fields) before storing it.
A second point: choosing flexible storage for data that is actually regular usually costs you later. If every order has an id, a customer, and a total, say so with a fixed structure, and let the database enforce it.
Common Mistakes
Mistake 1: "Unstructured means no structure at all"
- Incorrect: "A photo has no structure, so we cannot store information about it."
- Correct: "A photo's content has no fields we can query, but we can store structured facts about it: owner, size, date, location of the file."
Mistake 2: Treating JSON as "no rules"
JSON labels the fields, but it does not require them. If your program expects email, it must still check that email is present.
Mistake 3: Putting big files straight into a database table by default
It is possible in some systems, but the usual, simpler design keeps big files in file storage and records their address. Storing big files in the database is a decision to justify, not a default.
Mistake 4: Classifying a whole thing as one shape
An email has structured headers and an unstructured body. Say which part you are classifying.
Exercises
Level A: Foundation
A1. Define structured, semi-structured, and unstructured data in one sentence each.
Show solution A1
A1. Structured: fixed, known fields in every record. Semi-structured: labelled fields that can differ between records or be nested. Unstructured: no predefined fields a program can rely on.
A2. Classify as structured, semi-structured, or unstructured: (a) a table of exam scores, (b) a JSON response from a weather service, (c) a podcast recording, (d) a text message you typed.
Show solution A2
A2. (a) Structured. (b) Semi-structured. (c) Unstructured. (d) Unstructured (free text).
A3. Why can a program check "the id must be a whole number" easily in structured data?
Show solution A3
A3. The field and its type are known in advance, so the program can test each value against the rule.
Level B: Application
B1. A library stores books. Some books have an isbn, old ones do not. Some have many authors. Write two books as JSON objects that show these differences.
Show solution B1
B1. For example {"title": "Java Basics", "isbn": "978-0-00-000000-0", "authors": ["Ana", "Sam"]} and {"title": "Old Poems", "authors": ["Hala"]}. The second has no isbn, and the first has two authors. (The ISBN is a made-up placeholder.)
B2. Describe how a school system should store 300 uploaded homework PDFs. Say what goes in file storage, and design one structured record (column names) for the database.
Show solution B2
B2. The PDF files go in file storage. A record might have: id, student_id, file_name, size_bytes, uploaded_at, stored_at.
B3. Look at this CSV line: 12,Lee,,0100. List the four values and say which column each belongs to if the columns are id,name,email,phone. What goes wrong if the empty email is left out so the line reads 12,Lee,0100?
Show solution B3
B3. 12 is the id, Lee the name, the empty value the email, and 0100 the phone. Without the empty slot the line has three values, so 0100 would be read as the email and the phone would be missing.
Level C: Challenge
C1. A support team receives customer emails. Divide one email into its structured parts and its unstructured part. Suggest one thing you could store as a structured field after reading the unstructured part, and say how you would get it.
Show solution C1
C1. Structured parts: sender, recipient, subject, date. Unstructured: the body. Example: after a person (or a program) reads the body, it could record a category such as "refund request" in a structured category field.
C2. A teammate says: "JSON is flexible, so we will store every kind of data as JSON blobs and skip designing tables." Give two risks.
Show solution C2
C2. Risks: fields may be missing or misspelled and nothing warns you; you lose rules such as required values and uniqueness; queries across records become harder to write. Any two reasonable risks are fine.
Mini Challenge
Choose an app such as a music player. List at least six pieces of data it handles. Classify each as structured, semi-structured, or unstructured, and state where it would live: database, file storage, or both. For at least one item, write the structured record you would keep in the database about a file.
Check your work: every item has a shape and a place, and at least one item uses the "file plus database record" pattern.
Summary & Next Lesson
Key takeaways
- Structured data has fixed fields in every record; semi-structured data has labelled fields that can vary; unstructured data has no fixed fields.
- The three shapes are a spectrum, and one item can have parts of different shapes.
- Fixed fields allow precise queries, validation, and fast search.
- Big unstructured files usually live in file storage, with a structured record in the database.
- JSON is flexible, but a program must still check what it receives.
Next lesson
The next lesson, "Relational vs nonrelational", compares the two big families of databases at a high level and shows how each handles these data shapes.
Self-check
- Is a photo structured, semi-structured, or unstructured?
- Why is a JSON object called semi-structured?
- In a CSV row, what tells you which value is the email?
- Where does a backend usually keep a 2 MB uploaded photo?
- Name one fact about that photo you would store in the database.
Show answers
- Unstructured.
- Its values are labelled, but the set of labels can differ between records and values can be nested.
- Its position in the row: the format says which column is which.
- In file storage, with a record in the database pointing to it.
- For example its owner, file name, size, or storage location.
References
- Internet Engineering Task Force, RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format. Supports the description of JSON as a text format of objects (name/value pairs), arrays, strings, numbers, booleans, and null, where values can be nested.
- Internet Engineering Task Force, RFC 4180: Common Format and MIME Type for CSV Files. Supports the description of CSV as comma-separated values; it is an informational document that records common practice rather than an Internet standard.
- The definitions of structured, semi-structured, and unstructured data are the usual teaching vocabulary; this lesson did not verify them against a single primary source. The Java code was compiled and run with JDK 21 (the
Files.sizeline was not run).
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.