Building a Water Filter
Concept:
The pipe | is the most powerful concept in bash. It takes the output of one command and feeds it as input to the next. Unlike ; (which just runs commands in sequence), | connects them β data flows through.
head -n N β shows the first N lines.
tail -n N β shows the last N lines.
sort β sorts lines alphabetically or numerically (-n).
uniq β removes adjacent duplicate lines (always sort first).
wc -l β counts lines.
Terminal:
You've learned to find things. Now learn to transform them. In the wilderness, raw water needs filtering β each stage removes something. Bash pipelines work the same way.
You:
Pipelines? Is that like the semicolon from before?
Terminal:
Similar but different. Semicolon ';' just runs commands in sequence β they don't talk to each other. The pipe '|' is smarter: it takes the output of one command and feeds it as input to the next. Like water flowing through a series of filters. 'cat data.txt | sort | uniq' β read the data, sort it, remove duplicates. Three stages, one clean result.
You:
So each command does one thing, and the pipe chains them together?
Terminal:
That's the Unix philosophy β small tools that do one thing well, connected by pipes. 'head -n 5' takes the first 5 lines. 'tail -n 3' takes the last 3. 'sort' orders everything. 'uniq' removes duplicates. 'wc -l' counts lines.
You:
Like building a survival tool from smaller parts.
Terminal:
Exactly. There's an access log from the old outpost β it has repeated entries. Filter it: read it, sort it, remove duplicates, count what's left. How many unique signals were received?
Example Code:
# Pipe: command1 | command2 | command3
cat names.txt | sort
# Alice
# Bob
# Charlie
cat names.txt | sort | uniq
# removes duplicates
cat server.log | grep "error" | wc -l
# 3 (counts error lines)Your Assignment
Use a pipeline to read 'access.log', sort the lines, remove duplicates, and count how many unique lines remain. Your output should be just a number.
Bash Console
bash>