Foundations Updated 2026-09 View as Markdown

Redirection and Here Documents

Where output goes, where input comes from, and how to embed a block of text in a script without fighting quotes.

Every process starts with three open file descriptors:

0  stdin    input
1  stdout   normal output
2  stderr   diagnostics

Redirection changes where they point.

cmd > out.txt         # stdout to a file, TRUNCATING it
cmd >> out.txt        # stdout appended
cmd 2> err.txt        # stderr to a file
cmd < in.txt          # stdin from a file
cmd > /dev/null       # discard stdout

Combining streams#

cmd > all.txt 2>&1        # stdout to the file, then stderr to wherever stdout goes
cmd &> all.txt            # bash shorthand for the same
cmd >> all.txt 2>&1       # appending version
cmd 2>&1 | tee log.txt    # both streams into a pipe AND a file

Sending diagnostics to stderr#

echo "processing $file"            # to stdout — part of the output
echo "warning: skipping" >&2       # to stderr — a diagnostic

Getting this right is what lets a caller do ./script.sh > results.txt and still see the warnings. Anything that is not the script's actual output belongs on stderr — usage text, progress, errors.

die() { printf '%s\n' "$*" >&2; exit 1; }

Here documents#

A block of text fed to a command's stdin.

cat <<EOF
Deploying $APP_NAME
Target: $TARGET
User:   $(whoami)
EOF

Variables and command substitutions are expanded. Quote the delimiter to switch that off:

cat <<'EOF'
This $VARIABLE is printed literally.
So is $(this) and `this`.
EOF

The quoted form is what you want whenever the text contains shell syntax you do not want evaluated — a config file, a generated script, an awk program.

generate_service_file() {
  cat > /etc/systemd/system/myapp.service <<EOF
[Unit]
Description=$APP_NAME

[Service]
ExecStart=$INSTALL_DIR/bin/myapp
Restart=always
EOF
}

Note the mixed use: the delimiter is unquoted here because we want $APP_NAME and $INSTALL_DIR expanded.

Indented here documents#

<<- strips leading tabs (not spaces), so the block can be indented with the surrounding code:

if [[ -n "$verbose" ]]; then
	cat <<-'EOF'
	This text is indented in the source
	but printed flush left.
	EOF
fi

The indentation must be actual tab characters. That requirement catches people out often enough that many scripts just leave here-docs unindented.

Here strings#

For a single line, <<< is shorter:

grep -q "error" <<< "$log_line"
jq -r '.name' <<< "$json"
read -r first second <<< "$line"

<<< adds a trailing newline, which matters when hashing or comparing exact bytes — use printf '%s' piped instead if that is a problem.

Process substitution#

<(cmd) makes a command's output look like a file:

diff <(sort a.txt) <(sort b.txt)          # compare without temp files
comm -13 <(sort old.txt) <(sort new.txt)  # lines only in new

And the crucial use — feeding a while read loop without a subshell:

count=0
while read -r line; do
  count=$((count + 1))
done < <(find . -name '*.log')
echo "$count"           # correct

count=0
find . -name '*.log' | while read -r line; do
  count=$((count + 1))  # runs in a SUBSHELL
done
echo "$count"           # 0 — the variable never escaped

That second form is one of the most common shell bugs, and process substitution is the fix. Note the space in < <( — <<( is a syntax error.

>(cmd) works the other way, treating a command's input as a file:

tar czf >(ssh backup-host 'cat > backup.tgz') /data

Reading files safely#

while IFS= read -r line; do
  printf '%s\n' "$line"
done < input.txt

Three deliberate parts: IFS= preserves leading and trailing whitespace, -r stops backslash interpretation, and redirecting the file avoids the subshell. Generated scripts routinely omit all three.

# handle a final line with no trailing newline
while IFS= read -r line || [[ -n "$line" ]]; do
  printf '%s\n' "$line"
done < input.txt

Exercise#

#!/bin/bash
set -euo pipefail

# 1. Write a `usage` function printing a multi-line here-doc to STDERR
#    with the delimiter quoted so nothing is expanded.
# 2. Write `render_config` that produces a config block WITH $APP and $PORT
#    expanded, redirected to a file.
# 3. Count the lines of `ls` output in a while loop such that the count is
#    still correct after the loop.
APP=myapp PORT=8080
# write your code here

Common questions#

Why did cmd 2>&1 > file not capture my errors?#

Because redirections apply left to right. 2>&1 binds stderr to whatever stdout points at at that moment — still the terminal. Move stdout first: cmd > file 2>&1.

Quoted or unquoted here-doc delimiter?#

Quote it (<<'EOF') unless you specifically want expansion. Unquoted is right for templating in variables; quoted is right for embedding config, scripts, or anything containing $ you want preserved literally.

Why does my variable lose its value after a while loop?#

The right-hand side of a pipe runs in a subshell, so assignments inside it do not survive. Use process substitution — done < <(command) — and the loop runs in the current shell.