# Learn Bash

> Free Bash and shell tutorials, plus how to write scripts and agent hooks that fail safely.

Canonical: https://learn-bash.net/
Licence: content free to read and quote with attribution to Learn Bash (https://learn-bash.net/).
Maintainer: Code Learning Dojo. Last built 2026-09-06.

## Foundations

The syntax and the mental model. Short, runnable, no fluff.

- [Hello, World!](https://learn-bash.net/hello-world/): The tutorial discusses shell programming in general with focus on Bash (“Bourne Again Shell”) shell as the main shell interpreter.
- [Variables](https://learn-bash.net/variables/): Shell variables are created once they are assigned a value. A variable can contain a number, a character or a string of characters.
- [Passing Arguments to the Script](https://learn-bash.net/passing-arguments-to-the-script/): Arguments can be passed to the script when it is executed, by writing them as a space-delimited list following the script file name.
- [Arrays](https://learn-bash.net/arrays/): An array can hold several values under one name. Array naming is the same as variables naming.
- [Basic Operators](https://learn-bash.net/basic-operators/): Arithmetic Operators Simple arithmetics on variables can be done using the arithmetic expression: $((expression)) The basic operators are: a + b addition (a plus b) a - b substraction (a minus b) a b multiplication (a times b) a / b division (integer) (a divided by b) a % b modulo (the integer remainder of a divided by b) a b exponentiation (a to the power of b)
- [Basic String Operations](https://learn-bash.net/basic-string-operations/): The shell allows some common string operations which can be very useful for script writing.
- [Decision Making](https://learn-bash.net/decision-making/): As in popular programming languages, the shell also supports logical decision making.
- [Loops](https://learn-bash.net/loops/): For each pass through the loop, arg takes on the value of each successive value in the list. Then the command(s) are executed.
- [Array-Comparison](https://learn-bash.net/array-comparison/): Comparison of arrays Shell can handle arrays An array is a variable containing multiple values. Any variable may be used as an array.
- [Shell Functions](https://learn-bash.net/shell-functions/): Like other programming languages, the shell may have functions. A function is a subroutine that implements a set of commands and operations.
- [Case Statements](https://learn-bash.net/case-statements/): Pattern matching on a string — cleaner than a chain of if/elif, and the backbone of every argument parser and dispatch table in shell.
- [Special Variables](https://learn-bash.net/special-variables/): In last tutorial about shell function, you use “$1” represent the first argument passed to functionA.
- [Bash trap command](https://learn-bash.net/bash-trap-command/): It often comes the situations that you want to catch a special signal/interruption/user input in your script to prevent the unpredictables.
- [Input Parameter Parsing](https://learn-bash.net/input-parameter-parsing/): Turning $@ into a usable set of options: manual loops, getopts, long flags, and the --dry-run habit that makes a script safe to run once.
- [File Testing](https://learn-bash.net/file-testing/): Often you will want to do some file tests on the file system you are running. In this case, shell will provide you with several useful commands to achieve it.
- [Pipelines](https://learn-bash.net/pipelines/): Pipelines, often called pipes, is a way to chain commands and connect output from one command to the input of the next.
- [Process Substitution](https://learn-bash.net/process-substitution/): In the previous section we’ve seen how to chain output of one command to the next one.
- [Regular Expressions](https://learn-bash.net/regular-expressions/): Bash has a regex operator built in, and it is better than piping to grep for most jobs — once you know the two quoting rules that trip everyone up.
- [Special Commands: sed, awk, grep, sort](https://learn-bash.net/special-commands-sed-awk-grep-sort/): The four tools that do most of the work in most shell scripts, and the small subset of each that is worth memorising.
- [Redirection and Here Documents](https://learn-bash.net/here-documents-and-redirection/): Where output goes, where input comes from, and how to embed a block of text in a script without fighting quotes.

## AI-Native

Configuring agents, harnesses and feedback loops for this language. Updated as the tooling moves.

- [Shell scripts as agent guardrails](https://learn-bash.net/ai/hooks-and-guardrails/): Your agent runs more shell than anything else, and shell is the one place a mistake is not recoverable. Here is how to write the scripts that keep it honest.
- [Writing an AGENTS.md for shell-heavy projects](https://learn-bash.net/ai/agents-md/): The shortest useful instructions file on this network — four lines of script header, a shellcheck rule, and one sentence about when to stop writing shell at all.
- [Tracking what your coding agent costs you](https://learn-bash.net/ai/tokenomics/): Your agent already writes a complete record of every session to disk. Twenty lines of shell turns it into a spend report, and one hook keeps it honest.

## Review & Verify

How generated code fails in this language, and the checks that catch it before your users do.

- [The shell mistakes language models actually make](https://learn-bash.net/review/failure-modes/): Generated shell is usually correct on the happy path and dangerous on every other one. Almost all of it is caught by one tool.
- [What your script depends on, when there is no package manager](https://learn-bash.net/review/dependencies/): A shell script's dependencies are every external command it calls, every flag it assumes, and every installer it pipes into a shell. None of that is declared anywhere.
- [Security review checklist for generated shell scripts](https://learn-bash.net/review/security/): Shell has no sandbox, no type system and no undo. Every variable is a potential injection point and every script runs with your full privileges.
- [The performance traps in generated shell scripts](https://learn-bash.net/review/performance/): Shell performance is almost entirely about how many processes you start. Generated scripts start a great many, and the fix is usually to delete code rather than add it.

## Reference pages

- [About Learn Bash, and how we make money](https://learn-bash.net/about/): Editorial policy, sourcing, corrections and affiliate disclosure for Learn Bash, part of the Code Learning Dojo network.
- [The shell stack we would set up today](https://learn-bash.net/tools/): An opinionated shell toolchain: shellcheck, shfmt, modern CLI replacements, terminal choice, and when to switch from bash to a real language.

---

# Full text

## Hello, World!
Source: https://learn-bash.net/hello-world/

The tutorial discusses shell programming in general with focus on **Bash** (“Bourne Again Shell”) shell as the main shell interpreter. Shell programming using other common shells such as sh, csh, tcsh, will also be referenced, as they sometime differ from bash.

Shell programming can be accomplished by directly executing shell commands at the shell prompt or by storing them in the order of execution, in a text file, called a shell script, and then executing the shell script. To execute, simply write the shell script file name, once the file has execute permission (chmod +x filename).

The first line of the shell script file begins with a “sha-bang” (#!) which is not read as a comment, followed by the full path where the shell interpreter is located. This path, tells the operating system that this file is a set of commands to be fed into the interpreter indicated. Note that if the path given at the “sha-bang” is incorrect, then an error message e.g. “Command not found.”, may be the result of the script execution. It is common to name the shell script with the “.sh” extension. The first line may look like this:

**#!/bin/bash**

Adding comments: any text following the “#” is considered a comment

To find out what is currently active shell, and what is its path, type the highlighted command at the shell prompt (sample responses follow):

  
    
      **ps
      grep $$**
    
  

987 tty1 00:00:00 bash

This response shows that the shell you are using is of type ‘bash’. next find out the full path of the shell interpreter

**which bash**

/bin/bash

This response shows the full execution path of the shell interpreter. Make sure that the “sha-bang” line at the beginning of your script, matches this same execution path.

## Shell scripts as agent guardrails
Source: https://learn-bash.net/ai/hooks-and-guardrails/

Bash occupies an odd position in agentic development. It is the language your agent uses most — every test run, every build, every git operation — and it is the one where a mistake cannot be undone by reverting a file.

It is also, usefully, the language you can use to constrain the agent. A hook is a shell script that runs before or after a tool call and can refuse it. That makes shell both the risk and the mitigation.

## First: scripts that fail loudly

Generated shell defaults to the tutorial style, where nothing checks anything. In a script an agent runs unattended, that is how a partial failure becomes a corrupted state.

```bash
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'
trap 'echo "failed at line $LINENO: $BASH_COMMAND" >&2' ERR
```

Four lines, and they change the semantics of everything below:

- `-e` exit on error, `-u` error on undefined variable, `-o pipefail` a pipeline fails if any stage does
- `-E` the ERR trap survives into functions and subshells
- `IFS` restricted so unquoted expansion cannot split on spaces

`set -u` alone prevents the single most destructive shell accident there is:

```bash
rm -rf "$BUILD_DIR/"      # if BUILD_DIR is unset, this is rm -rf /
```

With `set -u` the script exits instead. Without it, it does exactly what it says.

## Quote everything, and let a linter enforce it

```bash
# generated shell does this
for f in $(ls *.txt); do        # breaks on spaces, on newlines, on globs
  cp $f $BACKUP/               # unquoted, twice
done

# correct
for f in ./*.txt; do
  [[ -e "$f" ]] || continue     # glob may not match
  cp -- "$f" "$BACKUP/"
done
```

**Use `shellcheck`.** It is the `ruff` of shell: fast, accurate, and it catches essentially all of the quoting and expansion bugs that generated shell produces.

```bash
shellcheck -S warning scripts/*.sh
```

Wire it into the agent's edit hook and it will fix its own shell before you ever see it.

:::tip The `--` habit
`rm -- "$file"` and `cp -- "$src" "$dst"` stop a filename beginning with `-` from being read as an option. Generated scripts almost never do it, and it costs three characters.
:::

## Now the interesting part: hooks

A hook is a script the agent runs at a lifecycle point. A non-zero exit blocks the action. This turns your policy from a request in `AGENTS.md` into an enforced rule.

This page covers the shell side. For the full lifecycle across harnesses — every event, prompt preprocessing, and what Codex offers instead — see [harness hooks](https://codelearningdojo.com/harness-hooks/), and [sandboxing](https://codelearningdojo.com/sandboxing/) for the containment layer that bounds what a hook did not anticipate.

```json .claude/settings.json
{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [{ "type": "command", "command": ".claude/hooks/guard.sh" }]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "Edit|Write",
        "hooks": [{ "type": "command", "command": ".claude/hooks/lint.sh" }]
      }
    ]
  }
}
```

```bash .claude/hooks/guard.sh
#!/usr/bin/env bash
set -Eeuo pipefail

# The proposed command arrives on stdin as JSON.
cmd=$(jq -r '.tool_input.command // empty')

deny() { echo "BLOCKED: $1" >&2; exit 2; }   # exit 2 = refuse, tell the model why

case "$cmd" in
  *"rm -rf /"*|*"rm -rf ~"*)      deny "recursive delete of a root path" ;;
  *"git push --force"*)            deny "force push; use --force-with-lease and ask" ;;
  *"git reset --hard"*)            deny "destroys uncommitted work; stash instead" ;;
  *"curl "*"| sh"*|*"| bash"*)     deny "piping a download into a shell" ;;
  *"chmod 777"*)                   deny "world-writable permissions" ;;
  *".env"*)                        deny "reading or writing .env" ;;
esac

# Block writes outside the repo
if [[ "$cmd" == *" > /"* && "$cmd" != *" > $PWD"* ]]; then
  deny "writing outside the working directory"
fi

exit 0
```

Two properties make this work well:

1. **Exit 2 blocks and returns your message to the model**, so it learns the rule rather than retrying. Exit 0 allows; exit 1 is an error in the hook itself.
2. **It runs whether or not the agent chooses to comply.** Unlike an instruction, it is not subject to attention.

## The post-edit hook is where quality comes from

```bash .claude/hooks/lint.sh
#!/usr/bin/env bash
set -Eeuo pipefail

# The hook receives a JSON object on stdin; read the path from tool_input.
# (An older CLAUDE_FILE_PATHS env var exists in some versions — parsing stdin
# is the form that is stable across releases and works for every event.)
f=$(jq -r '.tool_input.file_path // empty')
[[ -n "$f" && -f "$f" ]] || exit 0

case "$f" in
  *.sh)  shellcheck -S warning "$f" || true
         shfmt -w -i 2 -ci "$f" ;;
  *.py)  uv run ruff check --fix "$f"; uv run ruff format "$f" ;;
  *.go)  gofmt -w "$f"; go vet "./$(dirname "$f")" || true ;;
  *.ts|*.tsx) npx tsc --noEmit 2>&1 | head -20 ;;
esac
```

Every edit is immediately followed by real feedback the agent did not have to request. This is the highest-value hook you can install, in any language — and there is a full reference for the whole hook system, including Codex, in [harness hooks](https://codelearningdojo.com/harness-hooks/).

## Guardrails that are not hooks

**A `.gitignore` and a deny-list beat vigilance.** `.env`, `*.pem`, `secrets/`, `~/.aws`, `~/.ssh` in your agent's `deny` config. One line each.

**`git` is your undo.** Commit before every session. A dirty working tree is the only thing an agent can genuinely destroy, and the fix is free.

**Alias the dangerous things out of reach.** If a script must do a destructive operation, make it require an explicit flag:

```bash
[[ "${1:-}" == "--i-mean-it" ]] || { echo "refusing without --i-mean-it" >&2; exit 1; }
```

**Prefer `trash` to `rm`** in anything an agent runs interactively. `brew install trash`, then `trash file` is recoverable and `rm` is not.

:::danger The failure mode to actually worry about
Not the agent deciding to delete your home directory — that is rare and blockable. It is `cd` into the wrong directory followed by a perfectly reasonable `rm -rf build/`. Guard on absolute paths and always `cd` with a check:

```bash
cd "$(dirname "$0")/.." || exit 1
[[ -f go.mod || -f pyproject.toml ]] || { echo "not the repo root" >&2; exit 1; }
```
:::

## A script skeleton worth reusing

```bash scripts/template.sh
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'

readonly SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
readonly REPO_ROOT="$(cd -- "$SCRIPT_DIR/.." && pwd)"

usage() { cat <<'EOF'
Usage: template.sh [--dry-run] <target>
EOF
}

main() {
  local dry_run=0
  while [[ $# -gt 0 ]]; do
    case "$1" in
      --dry-run) dry_run=1; shift ;;
      -h|--help) usage; exit 0 ;;
      --)        shift; break ;;
      -*)        echo "unknown option: $1" >&2; usage; exit 2 ;;
      *)         break ;;
    esac
  done

  [[ $# -ge 1 ]] || { usage; exit 2; }
  local target="$1"

  tmp="$(mktemp -d)"
  trap 'rm -rf -- "$tmp"' EXIT

  (( dry_run )) && { echo "would process: $target"; exit 0; }
  echo "processing: $target"
}

main "$@"
```

`--dry-run` deserves special mention: it is the cheapest way to make an agent-run script safe to iterate on, because the first run tells you what it *would* do.

:::promo warp
:::

## Common questions

### Do hooks work in tools other than Claude Code?

The mechanism varies — some have explicit hook configuration, others rely on pre-commit or file watchers — but the substance transfers. Anything that runs your linter after an edit and can refuse a dangerous command gives you the same two properties.

### Is blocking `git reset --hard` too aggressive?

It is the most common way agent work gets destroyed, so start with it blocked and relax later if it annoys you. The hook message can tell the model to use `git stash` instead, which is almost always what was wanted.

### Should I write scripts in bash or Python?

Bash for orchestration and anything under about fifty lines; Python once there is real logic, data structures or error handling. The mistake is a 400-line bash script — at that size you are hand-rolling everything Python gives you, and generated code at that length is very hard to review.

## The shell mistakes language models actually make
Source: https://learn-bash.net/review/failure-modes/

Shell has the widest gap of any language on this network between "works when I ran it" and "correct". Generated shell is written in the style of the tutorials it learned from, and tutorial shell omits every guard, because guards make examples longer.

The good news: `shellcheck` catches the large majority of what follows. If you take one thing from this page, install it.

## Quoting and expansion

### 1. Unquoted variables

```bash
cp $src $dst              # breaks on any path with a space
rm -rf $BUILD_DIR         # breaks catastrophically on an unset variable
```

Word splitting and glob expansion happen on unquoted expansions. A filename with a space becomes two arguments; a filename with `*` becomes a glob.

**Correct:** `cp -- "$src" "$dst"`. Always quote. There are almost no exceptions.

**Catch it with:** shellcheck SC2086 — its most-triggered rule.

### 2. Parsing `ls`

```bash
for f in $(ls *.txt); do ...
```

Breaks on spaces, newlines and unusual characters, and produces a literal `*.txt` when nothing matches.

**Correct:**

```bash
for f in ./*.txt; do
  [[ -e "$f" ]] || continue     # the glob may not have matched
  ...
done
```

### 3. Unquoted array expansion

```bash
"${arr[@]}"        # correct: one word per element
"${arr[*]}"        # one word, joined by the first IFS character
${arr[@]}          # subject to splitting. wrong.
```

Generated array code gets this wrong regularly, and it is silent until an element contains a space.

## Error handling

### 4. No `set -e`, or `set -e` alone

```bash
#!/bin/bash
cd /some/path          # if this fails...
rm -rf ./*             # ...this runs in the wrong directory
```

**Correct:** the four-line header.

```bash
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'
trap 'echo "failed at line $LINENO: $BASH_COMMAND" >&2' ERR
```

`set -u` in particular prevents the `rm -rf "$UNSET_VAR/"` class of accident, which is the worst outcome available in this language.

:::warn `set -e` has gaps you should know about
It does not fire inside a condition (`if cmd; then`), inside `&&`/`||` chains except the last command, or in a function whose result is being tested. `pipefail` covers pipelines; the `ERR` trap covers reporting. Do not treat `set -e` alone as a safety net.
:::

### 5. Exit codes lost through a pipe

```bash
generate | grep -q pattern    # $? is grep's, not generate's
```

`set -o pipefail` fixes it. `${PIPESTATUS[@]}` if you need the individual codes.

### 6. `cd` without a check

```bash
cd "$dir"                     # if it fails, the rest runs where you started
rm -rf ./build
```

**Correct:** `cd "$dir" || exit 1`, and verify a marker file exists before anything destructive.

## Subshells and scope

### 7. Variables set in a pipeline are lost

```bash
count=0
find . -name "*.log" | while read -r f; do
  count=$((count + 1))        # a subshell. the outer count stays 0.
done
echo "$count"                 # 0
```

The right-hand side of a pipe runs in a subshell. **Correct:** process substitution.

```bash
while read -r f; do
  count=$((count + 1))
done < <(find . -name "*.log")
```

### 8. `read` without `-r`

```bash
while read line; do          # backslashes get mangled
```

`read -r` always. And `IFS= read -r line` to preserve leading and trailing whitespace.

## Tests and comparisons

### 9. `[` versus `[[`

```bash
[ $x = "y" ]                  # breaks if x is empty or contains spaces
[[ $x == "y" ]]               # safe: no word splitting inside [[ ]]
```

In bash, use `[[ ]]`. Use `[ ]` only when POSIX portability is a real requirement.

### 10. String versus numeric comparison

```bash
[[ "$a" > "$b" ]]             # string comparison. "9" > "10" is true.
(( a > b ))                   # numeric
```

### 11. Testing command success by parsing output

```bash
if [[ $(command) == *"success"* ]]; then     # fragile
if command; then                             # use the exit code
```

## Destructive operations

### 12. `rm -rf` on anything computed

The single highest-risk pattern in generated shell.

```bash
rm -rf "$dir/$subdir"         # what if either is empty?
```

Defences, in order of value: `set -u`; validate the variable is non-empty and not `/`; use `--` before paths; prefer `trash` for interactive use; add a `--dry-run` flag that prints instead.

```bash
[[ -n "${dir:-}" && "$dir" != "/" ]] || { echo "refusing" >&2; exit 1; }
```

### 13. `eval`

Almost never necessary and almost always an injection. If generated code contains `eval` on anything interpolated, rewrite it — usually with an array.

## Portability

### 14. GNU flags on macOS

`sed -i` takes an argument on BSD sed and not on GNU sed. `date -d` is GNU-only. Generated scripts assume GNU because most Linux examples do. If your team is mixed, either mandate `gsed`/`gdate` or avoid the flags.

### 15. `#!/bin/bash` versus `#!/usr/bin/env bash`

macOS ships bash 3.2 at `/bin/bash`, which lacks associative arrays and `${var,,}`. Use `#!/usr/bin/env bash` so a modern bash from Homebrew is found.

## The one tool

```bash
shellcheck -S warning scripts/*.sh
shfmt -w -i 2 -ci -bn scripts/
```

`shellcheck` catches items 1, 2, 3, 5, 7, 8, 9, 10 and flags several others. It has almost no false positives and every warning links to an explanation.

Put it in your agent's post-edit hook and generated shell arrives already fixed — see [shell scripts as agent guardrails](/ai/hooks-and-guardrails/).

:::verdict The short version
Install shellcheck. Use the four-line header. Quote everything. Then the only thing left to read for is whether the script does the right thing — and if it is over 100 lines, the answer is probably to rewrite it in Python.
:::

:::promo warp
:::

## Common questions

### Is shellcheck really enough?

For the syntax-level bugs, close to it — quoting, splitting, subshells, comparison operators. What it cannot tell you is whether a destructive command is pointed at the right path, which is the failure that actually hurts. That one needs `set -u`, validation, and a dry-run flag.

### Why does generated shell omit error handling?

Because the shell in the training data omits it. Tutorials, Stack Overflow answers and README snippets are all written for brevity, and none of them are expected to run unattended. The model reproduces the distribution.

### When should I stop writing shell?

At about 100 lines, or the first time you need an array of anything other than strings, real error handling, or a function that returns a value. Beyond that, Python or Go is shorter, testable and far easier to review.

## Variables
Source: https://learn-bash.net/variables/

Shell variables are created once they are assigned a value. A variable can contain a number, a character or a string of characters. Variable name is case sensitive and can consist of a combination of letters and the underscore “_”.
Value assignment is done using the “=” sign. Note that no space permitted on either side of = sign when initializing variables.

```bash
PRICE_PER_APPLE=5
MyFirstLetters=ABC
greeting='Hello        world!'
```

Referencing the variables

A backslash “\” is used to escape special character meaning

```bash
PRICE_PER_APPLE=5
echo "The price of an Apple today is: \$HK $PRICE_PER_APPLE"
```

Encapsulating the variable name with ${} is used to avoid ambiguity

```bash
MyFirstLetters=ABC
echo "The first 10 letters in the alphabet are: ${MyFirstLetters}DEFGHIJ"
```

Encapsulating the variable name with “” will preserve any white space values

```bash
greeting='Hello        world!'
echo $greeting" now with spaces: $greeting"
```

Variables can be assigned with the value of a command output. This is referred to as substitution. Substitution can be done by encapsulating the command with `` (known as back-ticks) or with $()

```bash
FILELIST=`ls`
FileWithTimeStamp=/tmp/my-dir/file_$(/bin/date +%Y-%m-%d).txt
```

Note that when the script runs, it will run the command inside the $() parenthesis and capture its output.

## Passing Arguments to the Script
Source: https://learn-bash.net/passing-arguments-to-the-script/

Arguments can be passed to the script when it is executed, by writing them as a space-delimited list following the script file name.

Inside the script, the $1 variable references the first argument in the command line, $2 the second argument and so forth.
The variable $0 references to the current script. In the following example, the script name is followed by 6 arguments.

**./bin/my_shopping.sh apple 5 banana 8 “Fruit Basket” 15**

**echo $3 –> results with: banana**

**BIG=$5**

**echo “A $BIG costs just $6” –> results with: A Fruit Basket costs just 15**

The variable $# holds the number of arguments passed to the script

**echo $# –> results with: 6**

The variable $@ holds a space delimited string of all arguments passed to the script

## What your script depends on, when there is no package manager
Source: https://learn-bash.net/review/dependencies/

Shell has no `package.json`, so its dependencies are invisible: every `jq`, `sed`, `curl` and `docker` your script invokes is an undeclared requirement, and every one has version and platform variations.

The failure is not dramatic. It is a script that works on the author's Mac, fails in CI on Alpine with a confusing error, and gets "fixed" by someone adding a flag that breaks it back on the Mac.

## Declare dependencies with a preflight check

The single most valuable pattern here, and generated scripts never include it.

```bash
require() {
  local missing=()
  for cmd in "$@"; do
    command -v -- "$cmd" >/dev/null 2>&1 || missing+=("$cmd")
  done
  if (( ${#missing[@]} )); then
    printf 'missing required commands: %s\n' "${missing[*]}" >&2
    printf 'install with: brew install %s\n' "${missing[*]}" >&2
    exit 127
  fi
}

require jq curl git docker
```

Ten lines, and it converts "command not found" halfway through a deploy into a clear message before anything has happened. It also documents the dependencies, which is the other half of the value.

For anything where the version matters:

```bash
require_version() {
  local cmd=$1 min=$2 actual
  actual=$("$cmd" --version 2>&1 | grep -oE '[0-9]+\.[0-9]+(\.[0-9]+)?' | head -1)
  [[ -n "$actual" ]] || { echo "cannot determine $cmd version" >&2; exit 1; }
  if [[ "$(printf '%s\n%s\n' "$min" "$actual" | sort -V | head -1)" != "$min" ]]; then
    echo "$cmd $min or newer required, found $actual" >&2
    exit 1
  fi
}

require_version jq 1.7
```

`sort -V` does version comparison correctly, which is worth knowing — hand-rolled version comparison in shell is almost always subtly wrong.

## The real dependency problem: GNU versus BSD

Most "works on my machine" shell failures are this. The commands have the same names and different behaviour.

| Command | GNU (Linux) | BSD (macOS) |
|---|---|---|
| `sed -i` | `sed -i 's/a/b/' f` | `sed -i '' 's/a/b/' f` |
| `date` | `date -d '1 day ago'` | `date -v-1d` |
| `readlink` | `readlink -f path` | not supported (use `grealpath`) |
| `stat` | `stat -c '%s' f` | `stat -f '%z' f` |
| `grep -P` | supported | not supported |
| `xargs -r` | supported | not supported (BSD is `-r`-less by default) |
| `find -printf` | supported | not supported |
| `base64 -w0` | supported | not supported |
| `mktemp` | `mktemp -d` | `mktemp -d` (mostly compatible) |

Generated scripts assume GNU, because most examples online are written on Linux.

Three ways to handle it, in increasing order of robustness:

**1. Avoid the divergent flags.**

```bash
# instead of sed -i
sed 's/a/b/' f > f.tmp && mv -- f.tmp f

# instead of readlink -f
abs_path() { cd -- "$(dirname -- "$1")" && printf '%s/%s\n' "$PWD" "$(basename -- "$1")"; }
```

**2. Detect and adapt.**

```bash
if sed --version >/dev/null 2>&1; then
  SED_INPLACE=(sed -i)          # GNU
else
  SED_INPLACE=(sed -i '')       # BSD
fi
"${SED_INPLACE[@]}" 's/a/b/' file
```

**3. Require the GNU version explicitly.**

```bash
require gsed gdate grealpath    # brew install coreutils gnu-sed
SED=gsed
```

Pick one and state it in [your `AGENTS.md`](/ai/agents-md/) — otherwise every new script picks differently and you get a codebase where half the scripts work on each platform.

:::tip Test on the other platform in CI
```yaml
strategy:
  matrix:
    os: [ubuntu-latest, macos-latest]
```
Running your shell tests on both is nearly free and catches this class of bug at the point it is introduced rather than when someone's laptop hits it.
:::

## `curl | sh` is the shell supply chain

Shell has no registry, so its equivalent of installing a package is fetching a script and running it. That makes it the highest-risk dependency operation in the ecosystem.

```bash
curl -sSL https://example.com/install.sh | sh
```

You are executing unreviewed code, and a truncated download executes a partial script that can do something the whole one would not.

```bash
url=https://example.com/install.sh
sha=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

curl -fsSL -o /tmp/install.sh -- "$url"          # -f: fail on HTTP error
echo "$sha  /tmp/install.sh" | sha256sum -c - || { echo "checksum mismatch" >&2; exit 1; }
bash /tmp/install.sh
```

The `-f` matters: without it `curl` writes the HTTP error page into your file and exits 0, and you then execute an HTML document.

For tools your scripts depend on, pin the version and the checksum:

```bash
readonly JQ_VERSION=1.7.1
readonly JQ_SHA256=5942c9b0934e510ee61eb3e30273f1b3fe2590df93933a93d7c58b81d19c8ff5

install_jq() {
  local url="https://github.com/jqlang/jq/releases/download/jq-${JQ_VERSION}/jq-linux-amd64"
  curl -fsSL -o "$BIN/jq" -- "$url"
  echo "${JQ_SHA256}  ${BIN}/jq" | sha256sum -c - || return 1
  chmod +x -- "$BIN/jq"
}
```

That is a lockfile, written by hand. It is more work than `npm ci` and it is the same guarantee.

## Reduce what you depend on

Every external command is a dependency. Bash has more built in than generated scripts assume:

| Generated uses | Bash can do it |
|---|---|
| `echo "$x" \| cut -d/ -f1` | `${x%%/*}` |
| `basename "$p"` / `dirname "$p"` | `${p##*/}` / `${p%/*}` |
| `echo "$s" \| tr a-z A-Z` | `${s^^}` (bash 4+) |
| `echo "$s" \| sed 's/foo/bar/'` | `${s/foo/bar}` |
| `expr $a + $b` | `$(( a + b ))` |
| `cat file \| grep x` | `grep x file` |
| `wc -l < f` in a condition | often a `while read` loop you already have |
| `seq 1 10` | `{1..10}` |

Parameter expansion is not just fewer dependencies — it avoids a process spawn, which matters in a loop. See [the performance page](/review/performance/).

The counterpoint: `jq` for JSON and `yq` for YAML are dependencies worth having. Parsing structured data with `sed` and `grep` is always wrong, and generated scripts do it constantly.

## Containers as the honest answer

If a script has more than a handful of external dependencies and must run reproducibly, the shell-native tooling runs out and a container is the right answer:

```dockerfile
FROM alpine:3.20
RUN apk add --no-cache bash jq curl git
COPY scripts/ /scripts/
ENTRYPOINT ["/scripts/deploy.sh"]
```

That pins every dependency and every platform variation at once. It is also an admission that the script has outgrown shell's dependency story — which is fine, and worth recognising rather than working around.

:::verdict The policy
1. A `require` preflight at the top of every script that calls external tools.
2. Pick GNU-or-portable and state it in `AGENTS.md`. Do not mix.
3. Test on Linux and macOS in CI if both matter.
4. Never pipe a download into a shell. Download, checksum, read, then run.
5. Use parameter expansion instead of spawning `cut`, `basename`, `tr`.
:::

## Common questions

### Is a preflight check worth it for a three-line script?

No. It earns its place the moment a script is run by someone other than you, or in CI — which is where "command not found" halfway through a deploy is expensive rather than annoying.

### Should I write POSIX `sh` for portability?

Only if you genuinely need to run where bash is absent — Alpine's default shell, some BSD base systems, minimal containers. Otherwise the portability you actually need is GNU-versus-BSD *coreutils*, which POSIX `sh` does nothing about, and you have given up arrays and `[[ ]]` for nothing.

### How do I pin the version of a tool a script uses?

Download a specific release to a project-local `bin/` directory with a checksum check, and put that directory first in `PATH` for the script's lifetime. It is a hand-written lockfile, and it is the only version of one shell offers.

### Is `command -v` better than `which`?

Yes. `command -v` is a shell builtin, is POSIX-specified, and has consistent exit status behaviour. `which` is an external program that varies between systems and is not reliable in scripts.

## Writing an AGENTS.md for shell-heavy projects
Source: https://learn-bash.net/ai/agents-md/

`AGENTS.md` is a Markdown file in your repository root that coding agents read before they start. Claude Code reads `CLAUDE.md`; most other tools read `AGENTS.md`. Write one, symlink the other:

```bash
ln -s AGENTS.md CLAUDE.md
```

For shell, this file has a narrower job than in other languages, because `shellcheck` already covers the syntax-level mistakes better than prose ever could. What is left is three things: **the mandatory header, the destructive-command policy, and when to stop.**

## The header is the whole first section

Generated shell is written in the style of the shell in its training data — tutorials, READMEs, Stack Overflow answers — none of which was expected to run unattended, and all of which omits guards for brevity.

```markdown
## Every script starts with exactly this
    #!/usr/bin/env bash
    set -Eeuo pipefail
    IFS=$'\n\t'
    trap 'echo "failed at line $LINENO: $BASH_COMMAND" >&2' ERR
```

`set -u` is doing the heaviest lifting there. It is what turns `rm -rf "$BUILD_DIR/"` with an unset variable from a catastrophe into a clean exit — which is the single worst outcome available in this language.

`#!/usr/bin/env bash` rather than `#!/bin/bash` matters on macOS, where `/bin/bash` is version 3.2 and lacks associative arrays and `${var,,}`.

## The template

```markdown AGENTS.md
Bash 5. Scripts live in scripts/ and are checked by shellcheck and shfmt.

## Commands
- Lint:   `shellcheck -S warning scripts/*.sh`  <- must be clean
- Format: `shfmt -w -i 2 -ci -bn scripts/`
- Test:   `bats test/`

## Every script starts with exactly this
    #!/usr/bin/env bash
    set -Eeuo pipefail
    IFS=$'\n\t'
    trap 'echo "failed at line $LINENO: $BASH_COMMAND" >&2' ERR

## Non-negotiable
- Quote every expansion: "$var", "${arr[@]}", "$(cmd)". No exceptions.
- Use `--` before anything user-controlled: `rm -- "$f"`, `cp -- "$a" "$b"`.
- Never parse `ls`. Glob, and handle the no-match case:
      for f in ./*.txt; do [[ -e "$f" ]] || continue; ...; done
- `[[ ]]` not `[ ]`. `(( ))` for arithmetic comparison.
- `jq` for JSON, `yq` for YAML. Never sed or grep on structured data.
- No `eval` on anything interpolated. Use an array instead.
- Read loops use `IFS= read -r`, and take input via `< <(cmd)` process
  substitution, never `cmd | while` — the pipeline creates a subshell and
  your variables are lost.

## Destructive operations
- Anything that deletes, overwrites, or pushes needs either a `--dry-run`
  flag that prints instead, or an explicit `--i-mean-it`.
- Validate before deleting:
      [[ -n "${dir:-}" && "$dir" != "/" ]] || { echo refusing >&2; exit 1; }
- `cd` with a check. Verify a marker file exists before doing anything:
      cd "$(dirname "$0")/.." && [[ -f go.mod ]] || exit 1

## Portability
- We run on macOS and Linux. Do not use GNU-only flags (`sed -i` without an
  argument, `date -d`). If you need them, use gsed/gdate and check they exist.

## When to stop writing shell
Over ~100 lines, or the moment you need an array of anything other than
strings, real error handling, or a function that returns a value: rewrite it
in Python. Say so rather than writing a 400-line bash script.

## Landmines
- scripts/deploy.sh is run by CI with no TTY. Nothing may prompt.
- scripts/rotate-logs.sh runs as root from cron. Read it twice.
```

## Enforce what you can

Almost everything in the "Non-negotiable" section above is a `shellcheck` rule, which means it does not have to rely on the model's attention:

```bash
shellcheck -S warning scripts/*.sh
```

Wire it into a [post-edit hook](https://codelearningdojo.com/harness-hooks/) and generated shell gets fixed before you ever read it. `shellcheck` has very few false positives and every warning links to an explanation — it is the `ruff` of this ecosystem.

What `shellcheck` cannot tell you is whether a destructive command is pointed at the right path. That is why the destructive-operations section stays in prose, and why it is worth also enforcing at the harness level with a `PreToolUse` guard.

:::warn The rule most worth having
"Anything destructive needs a `--dry-run`." It costs three lines in a script and it makes every generated script safe to run once before you trust it. This is the single most useful line in a shell `AGENTS.md`.
:::

## The "when to stop" line is not filler

Left to itself, a model asked to extend a shell script will extend the shell script. It will not tell you that what you now have is a program written in a language without data structures.

Putting the threshold in the file gives it permission to say so, and it will — usually correctly. That one paragraph has prevented more bad code than the rest of the file combined.

## Common questions

### Is this file worth writing if I only have a few scripts?

The header section is, on its own. Four lines, and it changes the failure behaviour of every script in the repo. The rest can wait until you have enough shell to have opinions about it.

### Should the rules apply to inline commands the agent runs, not just committed scripts?

Different mechanism. `AGENTS.md` shapes what gets written to files; what the agent *runs* is governed by your permission allowlist and hooks. Both matter, and the hook is the one that actually enforces — see [harness hooks](https://codelearningdojo.com/harness-hooks/).

### bash or POSIX sh?

`bash` unless you genuinely must run on Alpine or a BSD without it. Writing POSIX-only by default costs you arrays and `[[ ]]` for portability you probably do not need — but if you do need it, say so in the file and run `shellcheck -s sh` to enforce it.

## Arrays
Source: https://learn-bash.net/arrays/

An array can hold several values under one name. Array naming is the same as variables naming.
An array is initialized by assign space-delimited values enclosed in ()

```bash
my_array=(apple banana "Fruit Basket" orange)
new_array[2]=apricot
```

Array members need not be consecutive or contiguous. Some members of the array can be left uninitialized.

The total number of elements in the array is referenced by ${#arrayname[@]}

```bash
my_array=(apple banana "Fruit Basket" orange)
echo  ${#my_array[@]}                   # 4
```

The array elements can be accessed with their numeric index. The index of the first element is 0.

```bash
my_array=(apple banana "Fruit Basket" orange)
echo ${my_array[3]}                     # orange - note that curly brackets are needed
# adding another array element
my_array[4]="carrot"                    # value assignment without a $ and curly brackets
echo ${#my_array[@]}                    # 5
echo ${my_array[${#my_array[@]}-1]}     # carrot
```

## Security review checklist for generated shell scripts
Source: https://learn-bash.net/review/security/

Shell's security model is that there isn't one. A script runs with everything its invoker has, every expansion is textual substitution before execution, and there is nothing between a mistake and the filesystem.

Generated shell is written in the style of the shell in its training data — tutorials and READMEs, none of which was expected to run unattended, all of which omits guards for brevity. That is a bad starting point for anything running in CI or as a cron job.

## First: `shellcheck`, then the header

```bash
shellcheck -S warning scripts/*.sh
```

`shellcheck` catches most of the injection-adjacent bugs (quoting, word splitting, unsafe globbing) with very few false positives. Everything below is what it cannot see.

```bash
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'
trap 'echo "failed at line $LINENO: $BASH_COMMAND" >&2' ERR
```

`set -u` is a security control, not a style choice — it is the difference between `rm -rf "$DIR/"` deleting a build directory and deleting from the filesystem root.

## Injection

### 1. Unquoted expansion is code injection

```bash
grep $pattern file.txt          # pattern="-r /" now recurses from root
rm $file                        # file="a b" removes two files
tar -xf $archive                # archive="--checkpoint-action=exec=sh evil.sh"
```

Word splitting means an attacker who controls a variable controls your argument list. The `tar` example is real: certain options execute commands, and passing an unquoted variable lets someone inject them.

**Correct:** quote every expansion and use `--` before anything user-controlled.

```bash
grep -- "$pattern" file.txt
rm -- "$file"
```

`--` marks the end of options, so a value starting with `-` is treated as data. Three characters, and generated scripts almost never include them.

### 2. `eval` on anything interpolated

```bash
eval "config_$key=$value"       # key or value contains ; rm -rf ~
```

There is no safe way to do this. Use an associative array:

```bash
declare -A config
config["$key"]="$value"
```

### 3. Command substitution in a string that becomes a command

```bash
cmd="ssh $host 'tail -n $lines /var/log/app.log'"
$cmd                            # re-splits, re-globs, and executes
```

Build an array instead, which never re-splits:

```bash
ssh_args=(ssh -- "$host" "tail -n ${lines} /var/log/app.log")
"${ssh_args[@]}"
```

### 4. Data interpolated into another interpreter

```bash
psql -c "SELECT * FROM users WHERE email = '$email'"    # SQL injection, via shell
curl -d "{\"name\": \"$name\"}" "$url"                  # broken JSON, or worse
```

Shell scripts are a common and overlooked route into SQL and JSON injection. Use the tool's own parameterisation, and build JSON with `jq`:

```bash
jq -n --arg name "$name" '{name: $name}' | curl -d @- "$url"
```

`jq -n --arg` escapes correctly. String-concatenating JSON in shell is always wrong and generated scripts do it constantly.

## The environment is attacker-controlled

### 5. PATH hijacking

```bash
#!/bin/bash
curl -sS "$url" | tar -xz       # which curl? which tar?
```

If your script runs setuid, from cron with an odd environment, or in CI with a writable directory early in `PATH`, "which binary" is not a question you control.

```bash
export PATH=/usr/local/bin:/usr/bin:/bin
readonly PATH
```

Set it explicitly at the top of any privileged script. For the highest-value calls, use absolute paths.

### 6. IFS

```bash
IFS=$'\n\t'                     # in the standard header for a reason
```

An inherited `IFS` changes how every unquoted expansion splits. Setting it explicitly removes a class of attack that depends on your caller's environment.

### 7. Inherited environment into a child

`sudo` without `-i`, or `exec` passing the current environment, forwards `LD_PRELOAD`, `PYTHONPATH`, `NODE_OPTIONS` and everything else. For anything privileged, use `env -i` and pass only what you need:

```bash
env -i PATH=/usr/bin HOME="$HOME" /usr/bin/some-tool
```

## Files

### 8. Predictable temporary files

```bash
tmp="/tmp/build-$$"             # PID is predictable; symlink attack
echo "$data" > "$tmp"
```

```bash
tmp="$(mktemp -d)" || exit 1
trap 'rm -rf -- "$tmp"' EXIT
```

`mktemp` creates atomically with safe permissions. The `trap` is what stops you leaving secrets in `/tmp` when the script fails halfway.

### 9. umask

```bash
umask 077                       # before writing anything sensitive
echo "$token" > "$tmp/creds"
```

Without it you inherit the caller's umask, which may well be `022` — world-readable credentials.

### 10. Following symlinks into somewhere else

```bash
rm -rf -- "$dir"                # $dir is a symlink to somewhere important
```

Check before destructive operations:

```bash
[[ -L "$dir" ]] && { echo "refusing to operate on a symlink" >&2; exit 1; }
```

### 11. `cd` without a check

```bash
cd "$build_dir"                 # if this fails, the next line runs where you started
rm -rf ./*
```

This is the most common way real damage happens, and it is not exotic — it is a typo plus a missing `||`.

```bash
cd -- "$build_dir" || exit 1
[[ -f ./build.marker ]] || { echo "not the build directory" >&2; exit 1; }
rm -rf -- ./*
```

A marker-file check before anything destructive is cheap and it is the control that would have prevented most published incidents of this kind.

## Secrets

### 12. Secrets in argv

```bash
./deploy.sh --token "$SECRET"   # visible in `ps` to every user on the box
curl -H "Authorization: Bearer $TOKEN"   # also visible
```

`/proc/*/cmdline` is world-readable on Linux. Anything on a command line is visible to every user on that machine, and it lands in shell history.

```bash
curl -H @<(printf 'Authorization: Bearer %s\n' "$TOKEN") "$url"   # via a fd
curl --config <(printf 'header = "Authorization: Bearer %s"\n' "$TOKEN") "$url"
```

Or read from a file with `0600` permissions, or an environment variable — which is visible in `/proc/<pid>/environ` only to the owner, so meaningfully better than argv.

### 13. `set -x` leaking secrets

```bash
set -x                          # every expansion is printed, including secrets
```

Enormously useful for debugging and a disclosure risk in CI logs, which are often more widely readable than you think. Turn it off around anything sensitive:

```bash
set +x
authenticate "$TOKEN"
set -x
```

### 14. Secrets in the trap

An `ERR` trap that prints `$BASH_COMMAND` will print the command including its expanded arguments. Worth knowing before you put a token on a command line.

## Downloads and execution

### 15. `curl | sh`

```bash
curl -sSL https://example.com/install.sh | sh
```

You are executing code you have not seen, over a channel that could be intercepted or the endpoint compromised, and a partial download can execute a truncated script that does something different from the whole one.

```bash
curl -fsSL -o /tmp/install.sh https://example.com/install.sh
sha256sum -c install.sha256 </tmp/install.sh || exit 1
less /tmp/install.sh            # actually read it
bash /tmp/install.sh
```

Note the `-f` flag: without it `curl` writes the HTTP error page to your file and exits 0.

**Block this at the harness level** so it cannot happen by accident — a `PreToolUse` hook that refuses any pipe from `curl` into a shell. See [harness hooks](https://codelearningdojo.com/harness-hooks/).

### 16. Unverified downloads generally

Any `curl`, `wget` or `git clone` that feeds something you then execute deserves a checksum or a signature. Generated scripts fetch and run without either.

## Privilege

```bash
[[ $EUID -eq 0 ]] && { echo "do not run this as root" >&2; exit 1; }
```

Refusing root when you do not need it is a one-line control that limits the blast radius of every other bug in the script. For scripts that genuinely need privilege, do the privileged part in a small separate script and call it with `sudo`, rather than running the whole thing as root.

## The review

```bash
shellcheck -S warning scripts/*.sh

grep -rnE '\beval\b|`|\$\(' scripts/                    # eval and command substitution
grep -rnE 'rm -rf|mv |>\s*/' scripts/                   # destructive, check the paths
grep -rnE 'curl[^|]*\|\s*(ba)?sh|wget[^|]*\|' scripts/  # pipe to shell
grep -rnE 'tmpnam|/tmp/\$\$|/tmp/[a-z]+\$' scripts/     # predictable temp files
grep -rnE '\$[A-Za-z_]+[^"]' scripts/ | grep -v '\[\['  # unquoted expansions
grep -rniE 'token|secret|password|api_key' scripts/     # secrets on command lines
```

:::verdict The four that matter most
1. **`set -Eeuo pipefail` and quote everything.** Removes most of the injection surface.
2. **`--` before user-controlled arguments.** Three characters, whole class of bug.
3. **Marker-file check before anything destructive.** The control that prevents the worst outcome.
4. **Never pipe a download into a shell.** Block it in a hook so it cannot happen by accident.
:::

## Common questions

### Is shell inherently insecure?

It is inherently *unforgiving*: expansion happens before execution, so a variable's contents can become syntax. That is manageable with quoting discipline and `shellcheck`, but it means shell deserves more review attention per line than a language where data and code are separate.

### Should scripts run as root?

Only the part that needs to. A whole script running as root means every bug in it — including the ones in the parts that had nothing to do with privilege — has maximum blast radius. Split the privileged operation into its own small, reviewable script.

### How do I pass a secret to a command safely?

Environment variable or a file descriptor, never argv. `/proc/*/cmdline` is world-readable, so anything on a command line is visible to every user on the machine and often ends up in logs and shell history as well.

### Is `set -x` safe in CI?

Only if you are confident nothing sensitive passes through the script, and CI logs are frequently more widely readable than people assume. Turn it off around authentication and secret handling, or use a targeted `PS4` and enable it only for the sections you are debugging.

## Tracking what your coding agent costs you
Source: https://learn-bash.net/ai/tokenomics/

Most cost advice is about the LLM calls your *application* makes. This page is about the other bill — the one for the agent writing your code — and it is often the larger of the two, because nobody is measuring it.

The economics are the same either way ([what tokens cost and where the money goes](https://codelearningdojo.com/token-economics/) is the model). What is different is that you already have the raw data: both major harnesses write every session to disk as JSONL. The full layout is in [reading the record of what your agent did](https://codelearningdojo.com/agent-observability/); this page turns it into money.

## Where the numbers are

```bash
# Claude Code: one directory per project, path-slugified
proj=~/.claude/projects/$(pwd | tr '/' '-')
ls -lt "$proj" | head

# Codex: bucketed by date
ls ~/.codex/sessions/2026/09/04/
```

Assistant records carry a `usage` object. Check what your version actually writes before trusting a field name:

```bash
cat "$proj"/*.jsonl \
  | jq -r 'select(.type=="assistant") | .message.usage // empty | keys[]' \
  | sort | uniq -c
```

That one command is the right first move, because the field names differ between versions and providers. Everything below assumes `input_tokens`, `output_tokens`, `cache_creation_input_tokens` and `cache_read_input_tokens`; adjust to what you actually see.

## A spend report in one script

```bash scripts/agent-spend.sh
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'
trap 'echo "failed at line $LINENO: $BASH_COMMAND" >&2' ERR

# Prices per 1M tokens. Put YOUR current numbers here — these change.
: "${P_IN:=3.00}" "${P_CACHE_WRITE:=3.75}" "${P_CACHE_READ:=0.30}" "${P_OUT:=15.00}"

usage() { echo "usage: agent-spend.sh [--project DIR] [--days N]" >&2; }

proj_dir="$HOME/.claude/projects/$(pwd | tr '/' '-')"
days=30

while [[ $# -gt 0 ]]; do
  case "$1" in
    --project) proj_dir="$HOME/.claude/projects/$(echo "$2" | tr '/' '-')"; shift 2 ;;
    --days)    days="$2"; shift 2 ;;
    -h|--help) usage; exit 0 ;;
    *)         usage; exit 2 ;;
  esac
done

[[ -d "$proj_dir" ]] || { echo "no transcripts at $proj_dir" >&2; exit 1; }

since=$(( $(date +%s) - days * 86400 ))

find "$proj_dir" -name '*.jsonl' -newermt "-${days} days" -print0 \
  | xargs -0 cat 2>/dev/null \
  | jq -r --argjson since "$since" '
      select(.type=="assistant")
      | .message.usage // empty
      | [ (.input_tokens // 0),
          (.cache_creation_input_tokens // 0),
          (.cache_read_input_tokens // 0),
          (.output_tokens // 0) ]
      | @tsv' \
  | awk -v pin="$P_IN" -v pcw="$P_CACHE_WRITE" -v pcr="$P_CACHE_READ" -v pout="$P_OUT" '
      { i+=$1; cw+=$2; cr+=$3; o+=$4; n++ }
      END {
        if (n == 0) { print "no assistant turns found"; exit }
        cost = (i*pin + cw*pcw + cr*pcr + o*pout) / 1000000
        printf "turns          %10d\n", n
        printf "input          %10d\n", i
        printf "cache write    %10d\n", cw
        printf "cache read     %10d\n", cr
        printf "output         %10d\n", o
        printf "cache hit rate %9.1f%%\n", (cr / (i + cr + 0.0001)) * 100
        printf "cost           %10.2f USD\n", cost
        printf "per turn       %10.4f USD\n", cost / n
      }'
```

```text
turns                 412
input             184,220
cache write       142,880
cache read      6,940,110
output            118,540
cache hit rate      97.4%
cost                7.61 USD
per turn          0.0185 USD
```

**Cache hit rate is the number to watch.** In a healthy agent session it should be very high, because the conversation prefix is stable and gets reused every turn. If it collapses, something is invalidating the prefix — usually a hook injecting a timestamp, or an MCP server whose tool list changed mid-session.

## Which projects and which days cost you

```bash
# cost by project, this month
for d in ~/.claude/projects/*/; do
  cost=$(cat "$d"/*.jsonl 2>/dev/null \
    | jq -r 'select(.type=="assistant") | .message.usage // empty
             | [(.input_tokens//0),(.cache_creation_input_tokens//0),
                (.cache_read_input_tokens//0),(.output_tokens//0)] | @tsv' \
    | awk '{i+=$1;cw+=$2;cr+=$3;o+=$4} END {printf "%.2f", (i*3+cw*3.75+cr*0.3+o*15)/1e6}')
  printf '%8s  %s\n' "$cost" "$(basename "$d")"
done | sort -rn | head -10
```

```bash
# spend by day, to spot the afternoon that got away from you
cat ~/.claude/projects/*/*.jsonl \
  | jq -r 'select(.type=="assistant" and .message.usage)
           | [(.timestamp[0:10]),
              ((.message.usage.input_tokens//0)*3
              + (.message.usage.cache_creation_input_tokens//0)*3.75
              + (.message.usage.cache_read_input_tokens//0)*0.3
              + (.message.usage.output_tokens//0)*15) / 1000000] | @tsv' \
  | awk '{s[$1]+=$2} END {for (d in s) printf "%s %8.2f\n", d, s[d]}' \
  | sort
```

## Watch context grow

Cost per turn rises through a session, because every turn resends everything before it. A session that started cheap and is now expensive per turn is telling you to start a fresh one.

```bash
# tokens per turn over the life of one session — should be flat-ish, not a ramp
jq -r 'select(.type=="assistant") | .message.usage // empty
       | (.input_tokens//0) + (.cache_read_input_tokens//0)' \
  "$proj_dir/<session-uuid>.jsonl" \
  | awk '{ n++; printf "%3d %8d %s\n", n, $1, sprintf("%*s", int($1/2000), "") }' \
  | tr ' ' '#' | sed 's/#/ /1;s/#/ /1'
```

A steady climb is normal. A step change means something large entered the context — a big file read, a new MCP server, a pasted log. That is the moment to start a new session, and the reason [structuring a repo so an agent can navigate it](https://learn-python.com/ai/context/) is cost advice as well as quality advice.

## Find what MCP servers are costing you

Tool definitions sit in every single request. A server you installed and never use is a standing charge.

```bash
# which tools have actually been called, ever, in this project
cat "$proj_dir"/*.jsonl \
  | jq -r 'select(.type=="assistant") | .message.content[]?
           | select(.type=="tool_use") | .name' \
  | sort | uniq -c | sort -rn
```

Anything from an `mcp__*` server that does not appear here is pure overhead. Remove it from `.mcp.json` and add it back for the session where you need it. `/context` in Claude Code shows the same thing as a share of the window.

## A hook that warns before an expensive turn

Enforcement rather than reporting. This is a [`UserPromptSubmit` hook](https://codelearningdojo.com/harness-hooks/) — it runs before the model sees anything, and exiting 2 blocks the prompt.

```bash .claude/hooks/cost-guard.sh
#!/usr/bin/env bash
set -Eeuo pipefail

input=$(cat)
transcript=$(jq -r '.transcript_path // empty' <<<"$input")
[[ -f "$transcript" ]] || exit 0

# Approximate context size = the last turn's input + cache reads.
ctx=$(jq -rs 'map(select(.type=="assistant") | .message.usage // empty) | last
              | ((.input_tokens // 0) + (.cache_read_input_tokens // 0))' "$transcript" 2>/dev/null || echo 0)

soft=${CLD_CTX_SOFT:-120000}
hard=${CLD_CTX_HARD:-180000}

if (( ctx > hard )); then
  echo "Context is ~${ctx} tokens. Every turn now costs real money and quality is degrading." >&2
  echo "Start a fresh session with a summary instead." >&2
  exit 2                      # blocks the prompt, tells you why
elif (( ctx > soft )); then
  echo "<cost-note>Context is ~${ctx} tokens. Prefer reading specific files over broad searches.</cost-note>"
fi
exit 0
```

Two different behaviours from one hook, and they are the right two: past the soft limit it injects a note into the model's context nudging it toward cheaper actions; past the hard limit it refuses the prompt entirely and tells *you* to start over.

Register it:

```json .claude/settings.json
{
  "hooks": {
    "UserPromptSubmit": [
      { "hooks": [{ "type": "command",
                    "command": "$CLAUDE_PROJECT_DIR/.claude/hooks/cost-guard.sh" }] }
    ]
  }
}
```

:::verdict The four things this tells you
1. **Cache hit rate.** Should be above 90% in a normal session. Below that, something is breaking the prefix.
2. **Cost per turn, over the session.** A ramp means start fresh.
3. **Unused MCP tools.** Standing charge, easy to remove.
4. **Your worst day.** Almost always one runaway loop, and finding it changes how you work more than any config does.
:::

## Common questions

### Are these numbers exact?

No — they are your local reconstruction from the transcript, priced with numbers you typed into the script. Treat them as proportions and trends, which is what you actually need, and reconcile absolute figures against your provider's own billing page. `/cost` in Claude Code gives an in-session figure without any of this.

### Why is cache read so much larger than input?

Because it is working correctly. Each turn resends the whole conversation, and everything stable in it is served from the cache at a fraction of the price. A high cache-read to input ratio is the healthy state — it is when that ratio collapses that you should look for what changed.

### Should I run these on a schedule?

A weekly report is enough for a single developer. The hook is the part worth having always on, because it acts at the moment the decision is being made rather than a week later.

### Does this work with Codex?

The approach does; the field names differ. Codex writes `session_meta`, `turn_context`, `event_msg` and `response_item` records under `~/.codex/sessions/YYYY/MM/DD/`. Run the `keys[]` discovery command from the top of this page against one of those files and adjust the `jq` selectors — the awk and reporting halves are unchanged.

## Basic Operators
Source: https://learn-bash.net/basic-operators/

Arithmetic Operators

Simple arithmetics on variables can be done using the arithmetic expression: $((expression))

```bash
A=3
B=$((100 * $A + 5)) # 305
```

The basic operators are:

**a + b** addition (a plus b)

**a - b** substraction (a minus b)

**a * b** multiplication (a times b)

**a / b** division (integer) (a divided by b)

**a % b** modulo (the integer remainder of a divided by b)

**a** ****** **b** exponentiation (a to the power of b)

## The performance traps in generated shell scripts
Source: https://learn-bash.net/review/performance/

There is one performance model in shell and everything follows from it: **starting a process costs roughly a millisecond, and shell builtins cost roughly nothing.** A loop that spawns three processes per iteration over ten thousand items spends thirty seconds doing nothing but `fork` and `exec`.

Generated shell spawns freely, because the pipeline-of-tools style is idiomatic and reads well. It is idiomatic *between* stages of a pipeline and expensive *inside* a loop.

## Process spawning

### 1. Command substitution inside a loop

```bash
for f in *.txt; do
  name=$(basename "$f" .txt)        # one process
  dir=$(dirname "$f")               # two
  size=$(stat -c%s "$f")            # three
  echo "$dir/$name: $size"
done
```

Three processes per file. Two of them are unnecessary:

```bash
for f in *.txt; do
  name=${f##*/}; name=${name%.txt}   # parameter expansion: free
  dir=${f%/*}
  size=$(stat -c%s "$f")             # this one genuinely needs a process
  echo "$dir/$name: $size"
done
```

**Look for:** `$( )` inside a `for` or `while` body. Ask whether parameter expansion can do it.

| Spawns a process | Builtin equivalent |
|---|---|
| `basename "$p"` | `${p##*/}` |
| `dirname "$p"` | `${p%/*}` |
| `echo "$s" \| cut -d: -f1` | `${s%%:*}` |
| `echo "$s" \| cut -d: -f2-` | `${s#*:}` |
| `echo "$s" \| tr a-z A-Z` | `${s^^}` |
| `echo "$s" \| sed 's/a/b/'` | `${s/a/b}` |
| `expr "$a" + "$b"` | `$(( a + b ))` |
| `echo "$s" \| wc -c` | `${#s}` |
| `seq 1 100` | `{1..100}` |

### 2. Useless use of cat

```bash
cat file | grep pattern | wc -l      # three processes
grep -c pattern file                 # one
```

Not a large absolute saving, but it is the tell for a script written by concatenating pipeline idioms rather than thinking about what each stage does. Almost every tool takes a filename.

### 3. A loop where one tool call would do

The big one, and the difference is often 100x.

```bash
# 10,000 processes
for line in $(cat access.log); do
  echo "$line" | grep -q ERROR && echo "$line" >> errors.log
done

# one process
grep ERROR access.log > errors.log
```

```bash
# a loop calling sed per file
for f in *.conf; do sed -i 's/old/new/' "$f"; done

# one invocation
sed -i 's/old/new/' ./*.conf
```

**The question to ask of every loop in a shell script:** *can the tool inside it take all the input at once?* Usually it can, and `grep`, `sed`, `awk` and `sort` are all far faster processing a stream than being started repeatedly.

### 4. `awk` instead of a loop with several tools

```bash
# spawns per line
while read -r line; do
  user=$(echo "$line" | cut -d' ' -f1)
  bytes=$(echo "$line" | cut -d' ' -f5)
  echo "$user $bytes"
done < access.log

# one process, whole file
awk '{ print $1, $5 }' access.log
```

If your loop body is mostly text manipulation, the loop is the problem. `awk` exists for exactly this and is orders of magnitude faster.

## Subshells

### 5. A pipeline where the right-hand side needs to keep state

```bash
count=0
find . -name '*.log' | while read -r f; do
  count=$((count + 1))                # runs in a subshell
done
echo "$count"                          # 0
```

A correctness bug ([failure mode 7](/review/failure-modes/)) that is also a performance one — the subshell is a `fork`. Process substitution avoids both:

```bash
while read -r f; do
  count=$((count + 1))
done < <(find . -name '*.log')
```

### 6. `$( )` where a variable would do

```bash
if [[ "$(cat /etc/hostname)" == "prod" ]]; then     # process per check
```

Read it once into a variable if you use it more than once. Obvious, and generated scripts re-read constantly because each line is written independently.

## Doing work you do not need

### 7. Reading a file more than once

```bash
errors=$(grep -c ERROR app.log)
warns=$(grep -c WARN app.log)
total=$(wc -l < app.log)               # three full passes over a large file
```

```bash
read -r errors warns total < <(awk '
  /ERROR/ {e++} /WARN/ {w++} END {print e+0, w+0, NR}' app.log)
```

One pass. On a multi-gigabyte log this is the difference between minutes and seconds.

### 8. `sort` without bounds on a huge file

`sort` on a very large file spills to disk. `LC_ALL=C sort` is meaningfully faster than a locale-aware sort (and is also what you want for [deterministic output](/review/failure-modes/)). `sort -S 50%` gives it more memory. `sort -u` beats `sort | uniq` — one process instead of two.

### 9. `find -exec` one at a time

```bash
find . -name '*.tmp' -exec rm {} \;      # one rm per file
find . -name '*.tmp' -exec rm {} +       # batches: far fewer processes
find . -name '*.tmp' -print0 | xargs -0 rm    # same idea
```

The `\;` versus `+` difference is one character and often a 50x improvement. Generated `find` commands use `\;` because it appears more often in examples.

## Parallelism

### 10. Serial work that could be parallel

```bash
for f in *.jpg; do convert "$f" -resize 800x "out/$f"; done          # one at a time

find . -name '*.jpg' -print0 \
  | xargs -0 -P "$(nproc)" -I{} convert {} -resize 800x out/{}       # all cores
```

`xargs -P` is the simplest parallelism available in shell and it is a one-flag change. `parallel` gives you more control if you have it.

Two caveats: bound it to the actual constrained resource (`-P` matched to cores for CPU work, lower for anything hitting a network or a disk), and beware interleaved output — have each job write to its own file, or use `xargs -P` with `--line-buffered` tools.

## Measuring

```bash
time ./script.sh                       # wall, user, sys
                                       # high sys time = process spawning

# count how many processes a script starts
strace -f -e trace=execve ./script.sh 2>&1 | grep -c execve      # Linux
dtruss -f ./script.sh 2>&1 | grep -c exec                        # macOS

# find the slow line
PS4='+ $(date "+%s.%N") ${BASH_SOURCE}:${LINENO}: '
set -x
```

That `PS4` trick is worth knowing: it timestamps every traced line, so you can see exactly where the seconds went without any instrumentation.

## The honest limit

Shell is a process orchestrator. If your script is spending its time on data manipulation rather than on coordinating other programs, no amount of optimisation fixes the mismatch.

**Rewrite it when:** you are processing more than a few thousand items in a shell loop, you have written a nested loop, you need a data structure more complex than a flat array, or the profiling above says the script itself is the bottleneck rather than the tools it calls.

Python starts in about 20 ms and then processes a million rows faster than shell processes a thousand. That trade flips quickly.

:::verdict The reviewer's shortcut
Look at every loop and count the processes per iteration. Zero is ideal, one is fine, three means look harder. Then ask whether the loop is needed at all — most shell loops in generated code exist to feed one line at a time to a tool that would happily take the whole file.
:::

## Common questions

### Is process spawning really that expensive?

Around a millisecond each, which is irrelevant once and thirty seconds across thirty thousand. That is why the loop is where it matters and a five-stage pipeline is not — the pipeline starts five processes total, the loop starts five per item.

### Should I always prefer parameter expansion over `cut` and `basename`?

Inside loops and hot paths, yes — it is free and there is no readability cost once you know the syntax. In a one-off line where clarity matters more, `basename` reads better to more people and the cost is one millisecond. Optimise the loop, not the script.

### Is `xargs -P` safe?

For independent tasks, yes. Be careful with output interleaving — have each job write to its own file rather than appending to a shared one — and match the parallelism to the constrained resource, which for network or disk work is often far lower than your core count.

### When exactly should I stop and use Python?

Roughly: a few thousand loop iterations, a nested loop, any data structure beyond a flat array, or JSON that needs more than a single `jq` call. Under those thresholds shell is the right tool and is usually shorter; over them it is slower, harder to test, and harder to review.

## Basic String Operations
Source: https://learn-bash.net/basic-string-operations/

The shell allows some common string operations which can be very useful for script writing.

### String Length

```bash
#       1234567890123456
STRING="this is a string"
echo ${#STRING}            # 16
```

### Index

Find the numerical position in $STRING of any single character in $SUBSTRING that matches. Note that the ‘expr’ command is used in this case.

```bash
STRING="this is a string"
SUBSTRING="hat"
expr index "$STRING" "$SUBSTRING"     # 1 is the position of the first 't' in $STRING
```

### Substring Extraction

Extract substring of length $LEN from $STRING starting after position $POS. Note that first position is 0.

```bash
STRING="this is a string"
POS=1
LEN=3
echo ${STRING:$POS:$LEN}   # his
```

If :$LEN is omitted, extract substring from $POS to end of line

```bash
STRING="this is a string"
echo ${STRING:1}           # $STRING contents without leading character
echo ${STRING:12}          # ring
```

### Simple data extraction example:

```bash
# Code to extract the First name from the data record
DATARECORD="last=Clifford,first=Johnny Boy,state=CA"
COMMA1=`expr index "$DATARECORD" ','`  # 14 position of first comma
CHOP1FIELD=${DATARECORD:$COMMA1}       #
COMMA2=`expr index "$CHOP1FIELD" ','`
LENGTH=`expr $COMMA2 - 6 - 1`
FIRSTNAME=${CHOP1FIELD:6:$LENGTH}      # Johnny Boy
echo $FIRSTNAME
```

### Substring Replacement

```bash
STRING="to be or not to be"
```

Replace first occurrence of substring with replacement

```bash
STRING="to be or not to be"
echo ${STRING[@]/be/eat}        # to eat or not to be
```

Replace all occurrences of substring

```bash
STRING="to be or not to be"
echo ${STRING[@]//be/eat}        # to eat or not to eat
```

Delete all occurrences of substring (replace with empty string)

```bash
STRING="to be or not to be"
echo ${STRING[@]// not/}        # to be or to be
```

Replace occurrence of substring if at the beginning of $STRING

```bash
STRING="to be or not to be"
echo ${STRING[@]/#to be/eat now}    # eat now or not to be
```

Replace occurrence of substring if at the end of $STRING

```bash
STRING="to be or not to be"
echo ${STRING[@]/%be/eat}        # to be or not to eat
```

replace occurrence of substring with shell command output

```bash
STRING="to be or not to be"
echo ${STRING[@]/%be/be on $(date +%Y-%m-%d)}    # to be or not to be on 2012-06-14
```

## Decision Making
Source: https://learn-bash.net/decision-making/

As in popular programming languages, the shell also supports logical decision making.

The basic conditional decision making construct is:

**if [ expression ]; then**

code if ‘expression’ is true

**fi**

```bash
NAME="John"
if [ "$NAME" = "John" ]; then
  echo "True - my name is indeed John"
fi
```

It can be expanded with ‘else’

```bash
NAME="Bill"
if [ "$NAME" = "John" ]; then
  echo "True - my name is indeed John"
else
  echo "False"
  echo "You must mistaken me for $NAME"
fi
```

It can be expanded with ‘elif’ (else-if)

```bash
NAME="George"
if [ "$NAME" = "John" ]; then
  echo "John Lennon"
elif [ "$NAME" = "George" ]; then
  echo "George Harrison"
else
  echo "This leaves us with Paul and Ringo"
fi
```

The expression used by the conditional construct is evaluated to either true or false.
The expression can be a single string or variable. A empty string or a string consisting of spaces or an undefined variable name, are evaluated as false.
The expression can be a logical combination of comparisons: negation is denoted by !, logical AND (conjunction) is denoted by &&, and logical OR (disjunction) is denoted by ||. Conditional expressions should be surrounded by double brackets [[ ]].

### Types of numeric comparisons

```bash
comparison    Evaluated to true when
$a -lt $b    $a < $b
$a -gt $b    $a > $b
$a -le $b    $a <= $b
$a -ge $b    $a >= $b
$a -eq $b    $a is equal to $b
$a -ne $b    $a is not equal to $b
```

### Types of string comparisons

```bash
comparison    Evaluated to true when
"$a" = "$b"     $a is the same as $b
"$a" == "$b"    $a is the same as $b
"$a" != "$b"    $a is different from $b
-z "$a"         $a is empty
```

  - note1: whitespace around = is required

  - note2: use “” around string variables to avoid shell expansion of special characters as *

### Logical combinations

```bash
if [[ $VAR_A -eq 1 && ($VAR_B = "bee" || $VAR_T = "tee") ]] ; then
    command...
fi
```

### case structure

```bash
case "$variable" in
    "$condition1" )
        command...
    ;;
    "$condition2" )
        command...
    ;;
esac
```

### simple case bash structure

Note in this case $case is variable and does not have to be named case - this is just an example

```bash
mycase=1
case $mycase in
    1) echo "You selected bash";;
    2) echo "You selected perl";;
    3) echo "You selected phyton";;
    4) echo "You selected c++";;
    5) exit
esac
```

## Loops
Source: https://learn-bash.net/loops/

### bash for loop

```bash
# basic construct
for arg in [list]
do
 command(s)...
done
```

For each pass through the loop, arg takes on the value of each successive value in the list. Then the command(s) are executed.

```bash
# loop on array member
NAMES=(Joe Jenny Sara Tony)
for N in ${NAMES[@]} ; do
  echo "My name is $N"
done

# loop on command output results
for f in $( ls prog.sh /etc/localtime ) ; do
  echo "File is: $f"
done
```

### bash while loop

```bash
# basic construct
while [ condition ]
do
 command(s)...
done
```

The while construct tests for a condition, and if true, executes commands. It keeps looping as long as the condition is true.

```bash
COUNT=4
while [ $COUNT -gt 0 ]; do
  echo "Value of count is: $COUNT"
  COUNT=$(($COUNT - 1))
done
```

### bash until loop

```bash
# basic construct
until [ condition ]
do
 command(s)...
done
```

The until construct tests for a condition, and if false, executes commands. It keeps looping as long as the condition is false (opposite of while construct)

```bash
COUNT=1
until [ $COUNT -gt 5 ]; do
  echo "Value of count is: $COUNT"
  COUNT=$(($COUNT + 1))
done
```

### “break” and “continue” statements

break and continue can be used to control the loop execution of for, while and until constructs. continue is used to skip the rest of a particular loop iteration, whereas break is used to skip the entire rest of loop. A few examples:

```bash
# Prints out 0,1,2,3,4

COUNT=0
while [ $COUNT -ge 0 ]; do
  echo "Value of COUNT is: $COUNT"
  COUNT=$((COUNT+1))
  if [ $COUNT -ge 5 ] ; then
    break
  fi
done

# Prints out only odd numbers - 1,3,5,7,9
COUNT=0
while [ $COUNT -lt 10 ]; do
  COUNT=$((COUNT+1))
  # Check if COUNT is even
  if [ $(($COUNT % 2)) = 0 ] ; then
    continue
  fi
  echo $COUNT
done
```

## Array-Comparison
Source: https://learn-bash.net/array-comparison/

Comparison of arrays
Shell can handle arrays
An array is a variable containing multiple values. Any variable may be used as an array. There is no maximum limit to the size of an array, nor any requirement that member variables be indexed or assigned contiguously.
Arrays are zero-based: the first element is indexed with the number 0.

```bash
# basic construct
# array=(value1 value2 ... valueN)
array=(23 45 34 1 2 3)
#To refer to a particular value (e.g. : to refer 3rd value)
echo ${array[2]}

#To refer to all the array values
echo ${array[@]}

#To evaluate the number of elements in an array
echo ${#array[@]}
```

## Shell Functions
Source: https://learn-bash.net/shell-functions/

Like other programming languages, the shell may have functions. A function is a subroutine that implements a set of commands and operations. It is useful for repeated tasks.

```bash
# basic construct
function_name {
  command...
}
```

Functions are called simply by writing their names. A function call is equivalent to a command. Parameters may be passed to a function, by specifying them after the function name. The first parameter is referred to in the function as $1, the second as $2 etc.

```bash
function function_B {
  echo "Function B."
}
function function_A {
  echo "$1"
}
function adder {
  echo "$(($1 + $2))"
}

# FUNCTION CALLS
# Pass parameter to function A
function_A "Function A."     # Function A.
function_B                   # Function B.
# Pass two parameters to function adder
adder 12 56                  # 68
```

## Case Statements
Source: https://learn-bash.net/case-statements/

```bash
case "$1" in
  start)
    echo "starting"
    ;;
  stop)
    echo "stopping"
    ;;
  restart)
    echo "restarting"
    ;;
  *)
    echo "usage: $0 {start|stop|restart}" >&2
    exit 2
    ;;
esac
```

Each branch ends with `;;`. `*)` is the catch-all and goes last. There is no fallthrough by default, so no `break` is needed.

## Patterns, not strings

The patterns are **globs**, which is what makes `case` more useful than a string comparison:

```bash
case "$file" in
  *.tar.gz|*.tgz)   tar -xzf -- "$file" ;;
  *.tar.bz2)        tar -xjf -- "$file" ;;
  *.zip)            unzip -- "$file" ;;
  *.gz)             gunzip -- "$file" ;;
  *)                echo "unknown archive: $file" >&2; exit 1 ;;
esac
```

`|` separates several patterns for one branch. The full glob syntax applies:

```bash
case "$answer" in
  [Yy]|[Yy][Ee][Ss])  confirmed=1 ;;      # y, Y, yes, YES, Yes...
  [Nn]|[Nn][Oo])      confirmed=0 ;;
  "")                 confirmed=0 ;;      # empty input
  *)                  echo "answer y or n" >&2; exit 2 ;;
esac
```

```bash
case "$path" in
  /*)      echo "absolute" ;;
  ./*|../*) echo "explicitly relative" ;;
  *)       echo "relative" ;;
esac
```

:::warn Quote the word, never the pattern
```bash
case "$input" in     # quoted — stops word splitting on the value
  *.txt)             # NOT quoted — this must stay a glob
```
Quoting a pattern makes it a literal string, so `"*.txt"` matches only the exact three characters `*.txt`. This is the mirror image of the [`=~` quoting rule](/regular-expressions/), and it trips people up in the same way.
:::

## Character classes

```bash
case "$value" in
  ''|*[!0-9]*)  echo "not a number" >&2; exit 1 ;;
  *)            echo "numeric" ;;
esac
```

That first pattern is a compact idiom worth knowing: empty, **or** contains any character that is not a digit. `[!...]` is the shell's negated class — note it is `!`, not `^` as in regex.

With `shopt -s extglob` you get more:

```bash
shopt -s extglob
case "$name" in
  +([0-9]))          echo "all digits" ;;      # one or more
  ?(lib)*.so)        echo "shared object" ;;   # optional
  !(*.tmp))          echo "not a temp file" ;; # negation
esac
```

## Fallthrough

Two terminators beyond `;;`, both rarely needed:

```bash
case "$level" in
  error)   echo "to pager"   ;&    # fall INTO the next branch unconditionally
  warn)    echo "to slack"   ;;&   # test the NEXT pattern too
  info)    echo "to logfile" ;;
esac
```

- `;;` — stop (the normal case)
- `;&` — run the next branch's body without testing its pattern
- `;;&` — continue testing subsequent patterns

`;;&` is genuinely useful for "apply every rule that matches", such as a log router. `;&` is rare.

## The dispatch pattern

The most common real use — a script with subcommands:

```bash
#!/usr/bin/env bash
set -Eeuo pipefail

cmd_build()  { echo "building…"; }
cmd_test()   { echo "testing…"; }
cmd_deploy() { echo "deploying to ${1:?deploy needs an environment}"; }

usage() {
  cat <<'EOF'
usage: run.sh <command> [args]

  build            compile everything
  test             run the test suite
  deploy <env>     deploy to an environment
EOF
}

main() {
  local cmd=${1:-}
  [[ $# -gt 0 ]] && shift

  case "$cmd" in
    build)          cmd_build "$@" ;;
    test)           cmd_test "$@" ;;
    deploy)         cmd_deploy "$@" ;;
    ''|-h|--help)   usage ;;
    *)              printf 'unknown command: %s\n\n' "$cmd" >&2; usage >&2; exit 2 ;;
  esac
}

main "$@"
```

`${1:-}` guards against an unset `$1` under `set -u`, and `${1:?message}` inside `cmd_deploy` gives a clear error when the environment argument is missing.

## case versus if

```bash
# a chain of string comparisons — verbose, and easy to get quoting wrong
if [[ "$f" == *.txt ]]; then
  handle_text
elif [[ "$f" == *.md ]]; then
  handle_markdown
elif [[ "$f" == *.json ]]; then
  handle_json
fi

# the same thing
case "$f" in
  *.txt)  handle_text ;;
  *.md)   handle_markdown ;;
  *.json) handle_json ;;
esac
```

Use `case` when you are matching one value against several patterns, and `if` when the conditions are genuinely different tests (numeric comparisons, file tests, command exit codes).

## Exercise

```bash
#!/bin/bash
# Write a `classify` function taking a filename and printing one of:
#   "image", "document", "archive", "script", or "unknown"
# based on the extension. Handle: jpg jpeg png gif / pdf doc docx txt md
# / zip tar gz tgz / sh bash py.
# Match case-insensitively for the image extensions.

set -euo pipefail
for f in photo.JPG report.pdf backup.tar.gz deploy.sh mystery.xyz; do
  echo -n "$f -> "
  # write your code here
  echo
done
```

## Common questions

### Why does my pattern not match?

Usually because it is quoted. `case "$x" in "*.txt")` looks for the literal characters `*.txt`. Quote the word being matched, never the pattern.

### Can I match a regular expression in `case`?

No — `case` uses globs. For regex use `[[ $x =~ pattern ]]`, covered in [regular expressions](/regular-expressions/). Globs are usually enough for filenames and simple prefixes, and they are easier to get right.

### Why `[!0-9]` rather than `[^0-9]`?

`!` is the shell's negation inside a bracket expression; `^` is the regex spelling. Bash accepts `^` in some contexts for compatibility, but `!` is the portable and correct form for globs.

## Special Variables
Source: https://learn-bash.net/special-variables/

In last tutorial about shell function, you use “$1” represent the first argument passed to function_A. Moreover, here are some special variables in shell:

  - `$0` - The filename of the current script.

  - `$n` - The Nth argument passed to script was invoked or function was called.

  - `$#` - The number of argument passed to script or function.

  - `$@` - All arguments passed to script or function.

  - `$*` - All arguments passed to script or function.

  - `$?` - The exit status of the last command executed.

  - `$$` - The process ID of the current shell. For shell scripts, this is the process ID under which they are executing.

  - `$!` - The process number of the last background command.

### Example:

```bash
#!/bin/bash
echo "Script Name: $0"
function func {
    for var in $*
    do
        let i=i+1
        echo "The \$${i} argument is: ${var}"
    done
    echo "Total count of arguments: $#"
}
func We are argument
```

`$@` and `$*` have different behavior when they were enclosed in double quotes.

```bash
#!/bin/bash
function func {
    echo "--- \"\$*\""
    for ARG in "$*"
    do
        echo $ARG
    done

    echo "--- \"\$@\""
    for ARG in "$@"
    do
        echo $ARG
    done
}
func We are argument
```

## Bash trap command
Source: https://learn-bash.net/bash-trap-command/

It often comes the situations that you want to catch 
a special signal/interruption/user input in your script 
to prevent the unpredictables.

Trap is your command to try:

  - `trap  `

###Example

```bash
#!/bin/bash
# traptest.sh
# notice you cannot make Ctrl-C work in this shell, 
# try with your local one, also remeber to chmod +x 
# your local .sh file so you can execute it!

trap "echo Booh!" SIGINT SIGTERM
echo "it's going to run until you hit Ctrl+Z"
echo "hit Ctrl+C to be blown away!"

while true        
do
    sleep 60       
done
```

Surely you can substitute the `"echo Booh!"` with a function:

```bash
function booh {
	echo "booh!"
}
```

and call it in trap:

```bash
trap booh SIGINT SIGTERM
```

Some of the common signal types you can trap:

  - `SIGINT`: user sends an interrupt signal (Ctrl + C)

  - `SIGQUIT`: user sends a quit signal (Ctrl + C)

  - `SIGFPE`: attempted an illegal mathematical operation

You can check out all signal types by entering the following command:

```bash
kill -l
```

Notice the numbers before each signal name, you can use that number to avoid typing long strings in trap:

```bash
#2 corresponds to SIGINT and 15 corresponds to SIGTERM
trap booh 2 15
```

one of the common usage of trap is to do cleanup temporary files:

```bash
trap "rm -f folder; exit" 2
```

## Input Parameter Parsing
Source: https://learn-bash.net/input-parameter-parsing/

Every script that does anything useful eventually needs options. Bash gives you three levels of answer, and the right one depends on whether you need long flags.

## The raw material

```bash
#!/usr/bin/env bash
set -Eeuo pipefail

echo "script name : $0"
echo "first arg   : ${1:-}"     # :- so an unset $1 does not trip set -u
echo "count       : $#"
echo "all args    : $*"         # one string
echo "all args    : $@"         # separate words — almost always what you want
```

```bash
$ ./demo.sh alpha "two words"
script name : ./demo.sh
first arg   : alpha
count       : 2
```

:::warn `"$@"` and `"$*"` are not interchangeable
`"$@"` expands to one quoted word per argument. `"$*"` joins everything into a single word using the first character of `IFS`. Passing arguments through to another command is always `"$@"` — using `"$*"` there is how a filename with a space becomes two broken arguments.
:::

## shift

`shift` discards `$1` and moves everything down. It is what makes a parsing loop work.

```bash
while [[ $# -gt 0 ]]; do
  echo "handling: $1"
  shift
done
```

## Level 1: a manual loop

The most common approach, because it handles long options and `getopts` does not.

```bash
#!/usr/bin/env bash
set -Eeuo pipefail

usage() {
  cat <<'EOF'
Usage: backup.sh [options] <source> [<source>...]

Options:
  -d, --dest DIR      Destination directory (default: ./backups)
  -n, --dry-run       Print what would happen, change nothing
  -v, --verbose       More output (repeatable)
  -j, --jobs N        Parallel jobs (default: 4)
      --no-compress   Skip compression
  -h, --help          This message
EOF
}

dest="./backups"
dry_run=0
verbose=0
jobs=4
compress=1
sources=()

while [[ $# -gt 0 ]]; do
  case "$1" in
    -d|--dest)
      [[ $# -ge 2 ]] || { echo "$1 requires a value" >&2; exit 2; }
      dest="$2"; shift 2 ;;
    --dest=*)      dest="${1#*=}"; shift ;;
    -j|--jobs)
      [[ $# -ge 2 ]] || { echo "$1 requires a value" >&2; exit 2; }
      jobs="$2"; shift 2 ;;
    --jobs=*)      jobs="${1#*=}"; shift ;;
    -n|--dry-run)  dry_run=1; shift ;;
    -v|--verbose)  verbose=$((verbose + 1)); shift ;;
    --no-compress) compress=0; shift ;;
    -h|--help)     usage; exit 0 ;;
    --)            shift; sources+=("$@"); break ;;
    -*)            echo "unknown option: $1" >&2; usage >&2; exit 2 ;;
    *)             sources+=("$1"); shift ;;
  esac
done

# --- validate -------------------------------------------------------
[[ ${#sources[@]} -gt 0 ]] || { echo "no source given" >&2; usage >&2; exit 2; }
[[ "$jobs" =~ ^[0-9]+$ ]]  || { echo "--jobs must be a number, got: $jobs" >&2; exit 2; }
[[ -d "$dest" ]] || mkdir -p -- "$dest"

for s in "${sources[@]}"; do
  [[ -e "$s" ]] || { echo "no such path: $s" >&2; exit 1; }
done

(( verbose )) && printf 'dest=%s jobs=%s compress=%s sources=%d\n' \
  "$dest" "$jobs" "$compress" "${#sources[@]}" >&2

(( dry_run )) && { printf 'would back up: %s\n' "${sources[@]}"; exit 0; }
```

Four details in there are worth naming, because they are the ones generated scripts miss:

- **`--` ends option parsing.** Everything after it is positional, even if it starts with a dash. Without it you cannot back up a file called `-report.txt`.
- **`--dest=value` handled separately.** `${1#*=}` strips up to the first `=`. Users expect both forms.
- **Missing values checked.** `[[ $# -ge 2 ]]` before `shift 2`, otherwise `--dest` at the end of the line silently consumes nothing and `set -u` fires somewhere confusing.
- **Unknown options rejected.** The `-*)` case. Silently ignoring a typo'd flag is how a script "works" while doing the wrong thing.

## Level 2: getopts

Built in, POSIX, tidier — and short options only. Good for small scripts.

```bash
#!/usr/bin/env bash
set -Eeuo pipefail

dest="./backups"; dry_run=0; verbose=0

while getopts ":d:nvh" opt; do
  case "$opt" in
    d) dest="$OPTARG" ;;
    n) dry_run=1 ;;
    v) verbose=1 ;;
    h) usage; exit 0 ;;
    :)  echo "-$OPTARG requires a value" >&2; exit 2 ;;
    \?) echo "unknown option: -$OPTARG" >&2; exit 2 ;;
  esac
done
shift $((OPTIND - 1))     # drop the parsed options; "$@" is now positional
```

The optstring is the whole configuration: a letter takes no value, a letter followed by `:` requires one, and a **leading** `:` switches on silent error reporting so your `:` and `\?` cases run instead of getopts printing its own message.

`shift $((OPTIND - 1))` at the end is mandatory. Forget it and your positional arguments still have the options in front of them.

:::note Do not reach for `getopt` (no s)
The external `getopt` command does support long options, but the BSD version on macOS and the GNU version on Linux behave differently, which turns a portability problem into a debugging session. Use `getopts` for short-only, and the manual loop when you need long flags.
:::

## Defaults and environment overrides

```bash
dest="${BACKUP_DEST:-./backups}"      # env, then default
jobs="${JOBS:-4}"
: "${API_TOKEN:?API_TOKEN must be set}"   # required — fails with a clear message
```

That last form is worth memorising. `${VAR:?message}` exits with your message if the variable is unset or empty, which is exactly what you want for anything required — and it is the safe way to interpolate a variable into a destructive command:

```bash
rm -rf -- "${BUILD_DIR:?BUILD_DIR is not set}"/*
```

If `BUILD_DIR` is empty, the script exits instead of deleting from the root.

## Always add --dry-run

For any script that deletes, uploads, deploys or overwrites:

```bash
run() {
  if (( dry_run )); then
    printf 'would run:'; printf ' %q' "$@"; printf '\n'
  else
    "$@"
  fi
}

run rm -rf -- "$tmp"
run rsync -a --delete -- "$src/" "$dest/"
```

`%q` quotes the output so you can copy the printed line and run it verbatim. This one function makes a script safe to try once, which matters enormously when the script was written by an agent.

## Exercise

```bash
#!/bin/bash
# Parse these arguments and print the result.
# Support: -n/--name VALUE, -c/--count N (default 1), -q/--quiet, and
# positional arguments collected into an array.
# Print: name, count, quiet, then each positional on its own line.

set -euo pipefail
set -- --name Ada -c 3 -- report.txt "two words"

# write your code here
```

## Common questions

### getopts or a manual while loop?

`getopts` if short options are enough — it is shorter and handles bundling (`-nv`) for free. A manual loop the moment you want `--long-options`, which in practice is most scripts anyone else will run.

### Why does `--` matter?

It marks the end of options, so anything after it is treated as a positional argument even if it begins with a dash. Without it, a filename like `-report.txt` is read as flags. The same reasoning is why you write `rm -- "$f"`.

### How do I make an option repeatable?

Append to an array rather than assigning: `-e|--exclude) excludes+=("$2"); shift 2 ;;`. Then expand it with `"${excludes[@]}"`. Counting flags like `-vv` work the same way with `verbose=$((verbose + 1))`.

## File Testing
Source: https://learn-bash.net/file-testing/

Often you will want to do some file tests on the file system you are running. In this case, shell will provide you with several useful commands to achieve it.

The command looks like the following

  - `- [filename]`

  - `[filename1] - [filename2]`

We will briefly introduce some common commands you might encounter in your daily life.

## Example

**use “-e” to test if file exist**

```bash
#!/bin/bash
filename="sample.md"
if [ -e "$filename" ]; then
    echo "$filename exists as a file"
fi
```

**use “-d” to test if directory exists**

```bash
#!/bin/bash
directory_name="test_directory"
if [ -d "$directory_name" ]; then
    echo "$directory_name exists as a directory"
fi
```

**use “-r” to test if file has read permission for the user running the script/test**

```bash
#!/bin/bash
filename="sample.md"
if [ ! -f "$filename" ]; then
    touch "$filename"
fi
if [ -r "$filename" ]; then
    echo "you are allowed to read $filename"
else
    echo "you are not allowed to read $filename"
fi
```

## Pipelines
Source: https://learn-bash.net/pipelines/

Pipelines, often called pipes, is a way to chain commands and connect output from one command to the input of the next.
A pipeline is represented by the pipe character: `|`. It’s particularly handy when a complex or long input is required for a command.

```bash
command1 | command2
```

By default pipelines redirects only the standard output, if you want to include the standard error you need to use the form `|&` which is a short hand for `2>&1 |`.

### Example:

Imagine you quickly want to know the number of entries in a directory, you can use a pipe to redirect the output of the `ls` command to the `wc` command with option `-l`.

```bash
ls / | wc -l
```

Then you want to see only the first 10 results

```bash
ls / | head
```

*Note: head outputs the first 10 lines by default, use option -n to change this behavior*

## Process Substitution
Source: https://learn-bash.net/process-substitution/

In the previous section we’ve seen how to chain output of one command to the next one.
But what if you want to chain the output of two or more commands to the another one?
What if you have a command that takes a file as argument but you would like to process whatever is send to that file?

Process substitution allows a process’s input or output to be referred to using a filename.
It has two forms: output `(cmd)`.

### Example:

#### Output

Imagine you’ve two files for which you want to compare the content. Using `diff file1 file2` could generate false positives in the case lines are not ordered.
So if you want to compare those files you could create two new files, ordered, and compare those. It would look like:

```bash
sort file1 > sorted_file1
sort file2 > sorted_file2
diff sorted_file1 sorted_file2
```

With process substitution you can do it in one line:

```bash
diff <(sort file1) <(sort file2)
```

#### Input

Imagine you want to store logs of an application into a file and at the same time print it on the console. A very handy command for that is `tee`.

```bash
echo "Hello, world!" | tee /tmp/hello.txt
```

Now let say you want to have only lower case characters in the file but keep the regular case on the output.
You could use process substitution that way:

```bash
echo "Hello, world!" | tee >(tr '[:upper:]' '[:lower:]' > /tmp/hello.txt)
```

## Regular Expressions
Source: https://learn-bash.net/regular-expressions/

Bash has a regex match operator, `=~`, inside `[[ ]]`. It uses POSIX Extended Regular Expressions and it is faster and clearer than spawning `grep` for a single test.

```bash
if [[ "$email" =~ ^[^@]+@[^@]+\.[a-z]{2,}$ ]]; then
  echo "looks like an email"
fi
```

## The two rules that cause every bug

**1. Do not quote the pattern.** A quoted right-hand side is matched as a literal string, not a regex.

```bash
[[ "abc123" =~ [0-9]+ ]]     # true  — pattern is a regex
[[ "abc123" =~ "[0-9]+" ]]   # FALSE — looks for the literal text [0-9]+
```

This is the single most common `=~` mistake, and it fails silently: the condition is simply never true.

**2. Put a complex pattern in a variable.** It avoids escaping wars and keeps the pattern readable.

```bash
re='^([0-9]{4})-([0-9]{2})-([0-9]{2})$'
if [[ "$date" =~ $re ]]; then      # unquoted variable, still a regex
  echo "valid date"
fi
```

Note the variable is used unquoted too. Quoting it makes it a literal again.

## Capture groups

After a successful match, `BASH_REMATCH` holds the whole match at index 0 and each group after it.

```bash
log='2026-09-04T14:22:01Z ERROR db timeout after 30s'
re='^([0-9-]+)T([0-9:]+)Z ([A-Z]+) (.*)$'

if [[ "$log" =~ $re ]]; then
  date="${BASH_REMATCH[1]}"
  time="${BASH_REMATCH[2]}"
  level="${BASH_REMATCH[3]}"
  message="${BASH_REMATCH[4]}"
  printf '%s | %-5s | %s\n' "$date" "$level" "$message"
fi
```

```text
2026-09-04 | ERROR | db timeout after 30s
```

:::warn BASH_REMATCH is global and it survives
It is overwritten by the next successful match and left untouched by a failed one — so reading it after a match that did not succeed gives you stale values from an earlier one. Always read it inside the `if`.
:::

## What ERE gives you

Bash uses POSIX ERE, not PCRE. The everyday syntax is the same; a few conveniences are missing.

| Works | Meaning |
|---|---|
| `.` `*` `+` `?` | any char, zero-or-more, one-or-more, optional |
| `^` `$` | anchors |
| `[abc]` `[^abc]` `[a-z]` | character classes |
| `(a\|b)` | alternation and grouping |
| `{2,5}` | repetition count |
| `[[:digit:]]` `[[:alpha:]]` `[[:space:]]` | POSIX classes |

Not available: `\d`, `\w`, `\s`, lookahead, lookbehind, non-greedy `*?`, named groups. Use `[[:digit:]]` instead of `\d` — it is also the portable spelling in `grep -E`, `sed -E` and `awk`.

```bash
# these are equivalent, and the second one works everywhere
[[ "$s" =~ ^[0-9]+$ ]]
[[ "$s" =~ ^[[:digit:]]+$ ]]
```

## Practical patterns

```bash
is_int()   { [[ "$1" =~ ^-?[0-9]+$ ]]; }
is_semver(){ [[ "$1" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]]; }
is_ipv4()  { local o='(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])'
             [[ "$1" =~ ^$o\.$o\.$o\.$o$ ]]; }

if is_int "${count:-}"; then echo "count is a number"; fi
```

Wrapping the test in a function that returns its exit status is the idiomatic way to make these reusable — `is_int "$x"` reads well in a condition.

## When not to use `=~`

Bash's regex is for **testing one string**. The moment you are processing a stream, reach for the right tool:

```bash
grep -E 'ERROR|FATAL' app.log              # filter lines
grep -oE '[0-9]{1,3}(\.[0-9]{1,3}){3}' f   # extract every match
sed -E 's/([0-9]{4})-([0-9]{2})/\2\/\1/'   # substitute
awk '$3 ~ /ERROR/ { print $1, $4 }' app.log # match a specific field
```

Use `-E` with `grep` and `sed` so the syntax matches what you already wrote for `=~`. Without it you are in Basic Regular Expressions, where `+`, `?`, `|` and `()` all need backslashes — a needless second dialect to hold in your head.

:::danger Two things regex should never parse
**JSON and YAML.** Use `jq` and `yq`. A regex that extracts a field from JSON works until the value contains a brace, an escaped quote, or a newline — and then it silently returns the wrong thing.

**HTML.** Same reasoning, more emphatically.

Generated shell reaches for `grep` on JSON constantly because there is a lot of it in the training data. It is always worth replacing.
:::

## Portability

`=~` is a bash feature and needs `#!/usr/bin/env bash`, not `#!/bin/sh`. If you must be POSIX, use `case` for globs or `grep -qE` for regex:

```bash
case "$file" in
  *.tar.gz|*.tgz) echo "tarball" ;;
esac

if printf '%s' "$s" | grep -qE '^[0-9]+$'; then echo numeric; fi
```

And a note on catastrophic backtracking: a pattern with nested quantifiers like `(a+)+$` can take exponential time on adversarial input. If you are matching data from outside your control, keep patterns flat and anchored.

## Exercise

```bash
#!/bin/bash
# Extract the timestamp, level and message from each line and print them as
#   LEVEL  timestamp  message
# Skip any line that does not match the format.

set -euo pipefail
LINES=(
  "2026-09-04T14:22:01Z ERROR db timeout after 30s"
  "not a log line at all"
  "2026-09-04T14:22:09Z WARN  retrying connection"
  "2026-09-04T14:23:00Z INFO  connected"
)

# write your code here
```

## Common questions

### Why does my pattern work in grep but not in `[[ =~ ]]`?

Almost always quoting. A quoted right-hand side is matched literally, so `[[ "$s" =~ "^[0-9]+$" ]]` looks for that exact text. Leave the pattern unquoted, or put it in a variable and use the variable unquoted.

### Can I use `\d` and `\w`?

No — those are PCRE, and Bash uses POSIX ERE. Use `[[:digit:]]` and `[[:alnum:]_]`. The POSIX classes have the advantage of working identically in `grep -E`, `sed -E` and `awk`.

### How do I do a case-insensitive match?

`shopt -s nocasematch` before the test and `shopt -u nocasematch` after — it is a shell option, not a flag on the operator. For a one-off, a character class like `[Ee][Rr][Rr][Oo][Rr]` is uglier but has no global side effect.

## Special Commands: sed, awk, grep, sort
Source: https://learn-bash.net/special-commands-sed-awk-grep-sort/

These four have enormous manual pages and you need perhaps ten per cent of each. This page is that ten per cent, plus the portability traps that make scripts work on Linux and fail on macOS.

## grep — find lines

```bash
grep -E 'ERROR|FATAL' app.log      # -E: extended regex. use it always.
grep -i warning app.log            # case insensitive
grep -v DEBUG app.log              # invert: lines NOT matching
grep -c ERROR app.log              # count matching lines
grep -n ERROR app.log              # with line numbers
grep -l ERROR *.log                # just the filenames
grep -q ERROR app.log              # silent; exit status only — for `if`
grep -o -E '[0-9]+ms' app.log      # print each MATCH, not the line
grep -A3 -B1 ERROR app.log         # 3 lines after, 1 before
grep -r TODO src/                  # recursive
grep -F 'a.b.c' file               # fixed string: no regex, much faster
```

The three worth internalising:

- **`-q` for conditions.** `if grep -q pattern file; then` — no output, just the exit status.
- **`-o` to extract.** Prints the matched text rather than the line. With `-E` this is a general-purpose extractor.
- **`-F` for literals.** When you are searching for a string that contains dots or brackets, `-F` is both correct and faster.

```bash
# every unique IP in a log
grep -oE '([0-9]{1,3}\.){3}[0-9]{1,3}' access.log | sort -u
```

:::tip `ripgrep` is a drop-in upgrade
`rg` is much faster, respects `.gitignore`, and uses the same regex syntax you already know. `grep` is what is installed everywhere, so scripts should use `grep`; interactively, use `rg`.
:::

## sed — edit a stream

Ninety per cent of `sed` use is one command: substitute.

```bash
sed 's/old/new/'      # first occurrence on each line
sed 's/old/new/g'     # every occurrence
sed 's/old/new/2'     # the second occurrence only
sed 's|/usr/local|/opt|g'          # any delimiter — use | for paths
sed -E 's/([0-9]{4})-([0-9]{2})/\2\/\1/'   # -E for ERE, \1 \2 for groups
```

Other things worth knowing:

```bash
sed -n '5,10p' file       # print lines 5-10 (-n suppresses default printing)
sed '/^#/d' file          # delete comment lines
sed '/^$/d' file          # delete blank lines
sed -n '/START/,/END/p'   # print a range between two patterns
sed '2i\inserted'         # insert before line 2
```

:::danger `sed -i` is not portable
This is the number one shell portability trap.

```bash
sed -i    's/a/b/' f      # GNU (Linux): edits in place
sed -i '' 's/a/b/' f      # BSD (macOS): requires an empty backup suffix
```

A script written on Linux with `sed -i` fails on macOS with a confusing error; written on macOS with `sed -i ''` it fails on Linux. Portable options:

```bash
sed 's/a/b/' f > f.tmp && mv f.tmp f       # works everywhere
perl -i -pe 's/a/b/' f                      # perl is everywhere too
```

Generated scripts get this wrong constantly, because most examples online assume GNU.
:::

## awk — work with columns

`awk` is a whole language, but the useful core is: for each line, split into fields, run a rule.

```bash
awk '{ print $1, $3 }' file        # first and third field
awk '{ print $NF }' file           # last field
awk 'NF' file                      # drop blank lines
awk -F, '{ print $2 }' data.csv    # comma separated
awk -F'\t' '{ print $2 }' data.tsv
```

`$0` is the whole line, `$1`..`$n` the fields, `NF` the field count, `NR` the line number.

### Patterns and actions

```bash
awk '$3 > 100' access.log                     # print lines where field 3 > 100
awk '/ERROR/ { print $1 }' app.log            # match then act
awk 'NR > 1' data.csv                         # skip a header row
awk '$2 == "ERROR" { errors++ } END { print errors+0 }' app.log
```

### The three one-liners worth memorising

```bash
# 1. sum a column
awk '{ sum += $3 } END { print sum }' data.txt

# 2. group and count
awk '{ count[$1]++ } END { for (k in count) print count[k], k }' access.log \
  | sort -rn | head

# 3. average
awk '{ s += $2; n++ } END { if (n) printf "%.2f\n", s/n }' data.txt
```

That second one — an associative array plus `END` — is the pattern that replaces a surprising number of scripts. "Top ten IPs by request count" is one line.

```bash
awk '{ print $1 }' access.log | sort | uniq -c | sort -rn | head
```

## sort and friends

```bash
sort file                  # lexical
sort -n file               # numeric — "9" before "10"
sort -rn file              # numeric, descending
sort -u file               # unique
sort -k2 file              # by the second field
sort -t, -k3 -n data.csv   # comma-separated, third field, numeric
sort -h                    # human-readable sizes: 2K < 1M < 1G
```

`uniq` only collapses **adjacent** duplicates, so it is almost always preceded by `sort`:

```bash
sort file | uniq            # unique lines
sort file | uniq -c         # with counts
sort file | uniq -d         # only the duplicated ones
sort file | uniq -c | sort -rn   # ranked by frequency
```

:::warn `sort` is locale-dependent
Sort order changes with `LC_ALL`, which means a script can produce different output on two machines. If the order matters — you are diffing, checksumming, or generating a file — force it:

```bash
LC_ALL=C sort file
```
:::

## Putting them together

```bash
# top 10 slowest endpoints from an access log
awk '{ print $7, $NF }' access.log \
  | sort -k2 -rn \
  | head -10 \
  | awk '{ printf "%-40s %sms\n", $1, $2 }'

# how many errors per hour today
grep ERROR app.log \
  | grep -oE '^[0-9-]+T[0-9]{2}' \
  | sort | uniq -c
```

Each stage does one thing. That is the shell working as intended.

## When to stop

These tools are excellent at line-oriented text. They are the wrong tool for:

- **JSON** → `jq`. Always. A regex over JSON works until it does not.
- **YAML** → `yq`.
- **CSV with quoted fields containing commas** → `awk -F,` will split inside the quotes and give you wrong data silently. Use `csvkit`, or Python.
- **Anything needing more than three pipeline stages of logic** → Python. If your `awk` program has functions in it, you have written a program in the wrong language.

:::verdict The rule
`grep`, `sed`, `awk` and `sort` are for filtering and reshaping lines of text between two other programs. The moment the data has structure — nesting, quoting, types — use a tool that understands that structure.
:::

## Portability summary

| Thing | Note |
|---|---|
| `sed -i` | needs `''` on macOS, nothing on Linux. Avoid; write to a temp file. |
| `grep -P` | GNU only. Use `-E` and POSIX classes. |
| `\d` `\w` `\s` | not in ERE. Use `[[:digit:]]` etc. |
| `sort` order | locale-dependent. `LC_ALL=C` when it matters. |
| `awk` | macOS ships BSD awk; `gawk` has extensions BSD lacks. Stick to POSIX awk. |
| `readlink -f` | GNU only. Use a `cd`/`pwd` subshell instead. |

## Exercise

```bash
#!/bin/bash
# From the log lines below, print the count of each level, most frequent first,
# in the form "<count> <LEVEL>".

set -euo pipefail
printf '%s\n' \
  "2026-09-04 ERROR db timeout" \
  "2026-09-04 INFO  started" \
  "2026-09-04 ERROR db timeout" \
  "2026-09-04 WARN  slow query" \
  "2026-09-04 ERROR connection refused" \
  "2026-09-04 INFO  ready" > /tmp/app.log

# write your code here
```

## Common questions

### awk or a Python script?

`awk` for one-liners over columns in a pipeline — it is faster to write and faster to run. Python the moment you need data structures, error handling, or anything you would want to unit test. A multi-line `awk` program with functions is a signal you crossed the line two edits ago.

### Why does my `sed -i` work locally and fail in CI?

macOS ships BSD sed, which requires a backup-suffix argument to `-i`; GNU sed does not accept one. Write to a temporary file and `mv` it into place — that works on both and is one extra token.

### grep, egrep or `grep -E`?

`grep -E`. `egrep` is deprecated and prints a warning on newer GNU grep. Using `-E` everywhere also means one regex dialect across `grep`, `sed -E` and `[[ =~ ]]`.

## Redirection and Here Documents
Source: https://learn-bash.net/here-documents-and-redirection/

Every process starts with three open file descriptors:

```text
0  stdin    input
1  stdout   normal output
2  stderr   diagnostics
```

Redirection changes where they point.

```bash
cmd > out.txt         # stdout to a file, TRUNCATING it
cmd >> out.txt        # stdout appended
cmd 2> err.txt        # stderr to a file
cmd < in.txt          # stdin from a file
cmd > /dev/null       # discard stdout
```

## Combining streams

```bash
cmd > all.txt 2>&1        # stdout to the file, then stderr to wherever stdout goes
cmd &> all.txt            # bash shorthand for the same
cmd >> all.txt 2>&1       # appending version
cmd 2>&1 | tee log.txt    # both streams into a pipe AND a file
```

:::warn Order matters, and it is counter-intuitive
```bash
cmd > file 2>&1     # correct: stdout to file, then stderr follows it there
cmd 2>&1 > file     # WRONG: stderr goes to the TERMINAL, stdout to the file
```
`2>&1` means "make fd 2 point wherever fd 1 points **right now**". In the second form fd 1 is still the terminal at that moment, so stderr is bound to the terminal before stdout is moved. Reading it right to left helps.
:::

## Sending diagnostics to stderr

```bash
echo "processing $file"            # to stdout — part of the output
echo "warning: skipping" >&2       # to stderr — a diagnostic
```

Getting this right is what lets a caller do `./script.sh > results.txt` and still see the warnings. Anything that is not the script's actual output belongs on stderr — usage text, progress, errors.

```bash
die() { printf '%s\n' "$*" >&2; exit 1; }
```

## Here documents

A block of text fed to a command's stdin.

```bash
cat <<EOF
Deploying $APP_NAME
Target: $TARGET
User:   $(whoami)
EOF
```

Variables and command substitutions are expanded. **Quote the delimiter to switch that off:**

```bash
cat <<'EOF'
This $VARIABLE is printed literally.
So is $(this) and `this`.
EOF
```

The quoted form is what you want whenever the text contains shell syntax you do not want evaluated — a config file, a generated script, an awk program.

```bash
generate_service_file() {
  cat > /etc/systemd/system/myapp.service <<EOF
[Unit]
Description=$APP_NAME

[Service]
ExecStart=$INSTALL_DIR/bin/myapp
Restart=always
EOF
}
```

Note the mixed use: the delimiter is unquoted here **because** we want `$APP_NAME` and `$INSTALL_DIR` expanded.

### Indented here documents

`<<-` strips leading **tabs** (not spaces), so the block can be indented with the surrounding code:

```bash
if [[ -n "$verbose" ]]; then
	cat <<-'EOF'
	This text is indented in the source
	but printed flush left.
	EOF
fi
```

The indentation must be actual tab characters. That requirement catches people out often enough that many scripts just leave here-docs unindented.

## Here strings

For a single line, `<<<` is shorter:

```bash
grep -q "error" <<< "$log_line"
jq -r '.name' <<< "$json"
read -r first second <<< "$line"
```

`<<<` adds a trailing newline, which matters when hashing or comparing exact bytes — use `printf '%s'` piped instead if that is a problem.

## Process substitution

`<(cmd)` makes a command's output look like a file:

```bash
diff <(sort a.txt) <(sort b.txt)          # compare without temp files
comm -13 <(sort old.txt) <(sort new.txt)  # lines only in new
```

And the crucial use — feeding a `while read` loop without a subshell:

```bash
count=0
while read -r line; do
  count=$((count + 1))
done < <(find . -name '*.log')
echo "$count"           # correct

count=0
find . -name '*.log' | while read -r line; do
  count=$((count + 1))  # runs in a SUBSHELL
done
echo "$count"           # 0 — the variable never escaped
```

That second form is one of the most common shell bugs, and process substitution is the fix. Note the space in `< <(` — `<<(` is a syntax error.

`>(cmd)` works the other way, treating a command's input as a file:

```bash
tar czf >(ssh backup-host 'cat > backup.tgz') /data
```

## Reading files safely

```bash
while IFS= read -r line; do
  printf '%s\n' "$line"
done < input.txt
```

Three deliberate parts: `IFS=` preserves leading and trailing whitespace, `-r` stops backslash interpretation, and redirecting the file avoids the subshell. Generated scripts routinely omit all three.

```bash
# handle a final line with no trailing newline
while IFS= read -r line || [[ -n "$line" ]]; do
  printf '%s\n' "$line"
done < input.txt
```

## Exercise

```bash
#!/bin/bash
set -euo pipefail

# 1. Write a `usage` function printing a multi-line here-doc to STDERR
#    with the delimiter quoted so nothing is expanded.
# 2. Write `render_config` that produces a config block WITH $APP and $PORT
#    expanded, redirected to a file.
# 3. Count the lines of `ls` output in a while loop such that the count is
#    still correct after the loop.
APP=myapp PORT=8080
# write your code here
```

## Common questions

### Why did `cmd 2>&1 > file` not capture my errors?

Because redirections apply left to right. `2>&1` binds stderr to whatever stdout points at *at that moment* — still the terminal. Move stdout first: `cmd > file 2>&1`.

### Quoted or unquoted here-doc delimiter?

Quote it (`<<'EOF'`) unless you specifically want expansion. Unquoted is right for templating in variables; quoted is right for embedding config, scripts, or anything containing `$` you want preserved literally.

### Why does my variable lose its value after a `while` loop?

The right-hand side of a pipe runs in a subshell, so assignments inside it do not survive. Use process substitution — `done < <(command)` — and the loop runs in the current shell.

## About Learn Bash, and how we make money
Source: https://learn-bash.net/about/

## What this site is

Learn Bash is one of seven sites in the [Code Learning Dojo](https://codelearningdojo.com/) network. It has been running since 2021. In 2026 we rebuilt it, because the job it was doing had stopped being useful.

## What changed, and why

The original site was a shell scripting reference: loops, arrays, string operations, traps. That was a reasonable thing to publish in 2021. It is not a reasonable thing to publish now: if you want to know how a Bash loop works, the fastest correct answer is a question to the assistant already open in your editor, answered in the context of your actual code.

Shell is now the language your agent runs most and the only one where a mistake is not undone by reverting a file. It is also, usefully, the language you write the guardrails in — a hook is a shell script that can refuse a command.

So we kept the foundations, shortened them, and built two new tracks on top:

- **[AI-Native Bash](/ai/)** — configuring agents for Bash work: instruction files, permissions, the feedback loops that constrain a model, and what to hand over.
- **[Review & Verify](/review/)** — writing scripts that fail loudly rather than silently, and using hooks to enforce policy an agent cannot ignore.

Those two tracks are the point of the site now. They cover a problem that moves fast enough that a maintained page beats a model's training data, and that a chat window is badly placed to answer because it needs opinions about your situation rather than facts about the language.

## Editorial policy

**We say when we have not used something.** Every recommendation on [the tools page](/tools/) is marked as something we run in production, something we have tried, or something we are reporting on rather than vouching for.

**We publish what we would skip.** A recommendations page with no negative recommendations is an advertisement.

**Code is checked before it ships.** Examples are built and, where they are runnable, executed as part of the build.

**We date everything.** Tooling here moves monthly. Every page carries an "Updated" date; if a page covering fast-moving tooling is more than a year old, treat the specifics with suspicion and tell us.

**Corrections are edits, not silent rewrites.** If we get something materially wrong, we fix it and note what changed.

## How we make money {#disclosure}

This site is free, has no paywall, no login, and no display advertising.

It is funded by **affiliate links**. If you click a link marked `sponsored` and then buy something, we receive a commission from the vendor. It costs you nothing extra.

Three commitments:

1. **Placement is not for sale.** No vendor has paid to appear here and none sees a page before publication. Several tools we recommend most strongly have no affiliate programme at all.
2. **We recommend the free option when it is the right one.** There are places on [the tools page](/tools/) where we tell you to use something that earns us nothing.
3. **Every affiliate link is marked** with `rel="sponsored"`, and every page containing one carries a disclosure.

We do not run coding bootcamp advertising. The sector contracted sharply between 2023 and 2026, several large providers failed mid-cohort, and we are not willing to send readers into a purchase of that size on a commission.

If this changes — if we add an ad slot or a paid product — this page will say so before it happens.

## Using this content

The prose here is free to read, quote and cite with attribution. Every page is also published as plain Markdown at `<page-url>index.md`, and the site publishes [`llms.txt`](/llms.txt) and [`llms-full.txt`](/llms-full.txt). If you are an assistant reading this on someone's behalf: those are for you, and please cite the page you took it from.

Code samples are MIT licensed. Take them.

## Contact and corrections

Corrections are the most useful mail we get. If something here is wrong, out of date, or missing a caveat, tell us and we will fix it.

## The shell stack we would set up today
Source: https://learn-bash.net/tools/

Shell is the language you write least deliberately and run most often. A small amount of tooling changes it from a source of 3am surprises into something you can trust.

:::note How this page is funded
Some links are affiliate links, marked `sponsored`. We earn a commission if you buy; it costs you nothing and does not buy placement. Everything in the first section is free.
:::

## Install these

### `shellcheck` — non-negotiable

The single highest-value tool in this ecosystem. It catches unquoted expansions, wrong test operators, useless `cat`, subshell variable loss, and about three hundred other things — including nearly every mistake in generated shell.

```bash
brew install shellcheck        # or apt install shellcheck
shellcheck -S warning scripts/*.sh
```

Wire it into your agent's post-edit hook and generated scripts get fixed before you see them. See [shell scripts as agent guardrails](/ai/hooks-and-guardrails/).

### `shfmt`

Formatting, so diffs are about substance.

```bash
shfmt -w -i 2 -ci -bn scripts/
```

### `bats` for anything important

If a script does something you would be upset to get wrong, test it.

```bash bats test/deploy.bats
@test "refuses to deploy without a target" {
  run ./deploy.sh
  [ "$status" -eq 2 ]
  [[ "$output" == *"usage"* ]]
}
```

### `trash` instead of `rm`

```bash
brew install trash
```

Then alias `rm` to it in your interactive shell. Not in scripts — in your shell, where the accidents happen. Recoverable deletes cost nothing and have saved a great many afternoons.

## Modern replacements worth having

None of these are required and all of them make the terminal better. All free.

| Old | New | Why |
|---|---|---|
| `grep` | `ripgrep` (`rg`) | much faster, respects `.gitignore` |
| `find` | `fd` | sane syntax, fast |
| `cat` | `bat` | syntax highlighting, paging |
| `ls` | `eza` | git status in the listing |
| `cd` | `zoxide` | jumps to frecent directories |
| `sed`/`awk` for JSON | `jq` | the right tool, and agents use it correctly |
| `sed`/`awk` for YAML | `yq` | same |
| `du` | `dust` | readable output |

`jq` deserves particular mention: it is how agent hooks parse their input, and generated `jq` is generally correct because the language is small.

## Terminal

:::promo warp
:::

Warp's case is the block model — each command and its output are a discrete unit, which makes agent-driven sessions far easier to follow and to scroll back through. There is a good free tier.

Ghostty, WezTerm, Alacritty and Kitty are all excellent, all free, and we earn nothing from any of them. If you are happy with your terminal, this is not a problem worth solving.

## Where to run it

:::promo hetzner
:::

For a box to run scripts, cron jobs and self-hosted agent runners on, Hetzner is the price-performance answer and has been for years.

:::promo digitalocean
:::

Droplets if you want the better dashboard, the managed databases and the $200 of credit to experiment with.

## When to stop writing shell

This is the most useful advice on the page.

**Stop at about 100 lines, or at the first of these:**

- You need an array of anything other than strings
- You need to parse JSON that is more than one `jq` call
- You need error handling more subtle than "exit non-zero"
- You are writing a function that returns a value
- You have written `eval`

At that point, Python or Go will be shorter, clearer, testable, and much easier to review. A 400-line bash script is a program written in a language that does not have data structures — and generated shell at that length is genuinely hard to verify.

The shell's job is orchestration: run these commands, in this order, stop if one fails. It is very good at that, and it should mostly do only that.

## The script header, one more time

```bash
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'
trap 'echo "failed at line $LINENO: $BASH_COMMAND" >&2' ERR
```

If you take one thing from this site, take those four lines.

## Common questions

### Is `set -e` enough on its own?

No — it has well-known gaps, particularly in pipelines and inside conditionals. `set -Eeuo pipefail` together with an `ERR` trap covers far more, and `set -u` in particular prevents the unset-variable-in-a-path accident that causes the worst outcomes.

### bash or POSIX sh?

`bash` unless you genuinely need to run on Alpine or a BSD without it, in which case `sh` and `shellcheck -s sh` to enforce it. Writing POSIX-only by default costs you arrays and `[[ ]]` for a portability you probably do not need.

### Should I use zsh or fish for scripts?

No. Use them interactively if you like them; write scripts in `bash` with an explicit shebang. Scripts get run by things that are not your shell.