CLI Design Principles - Building Command-Line Tools Engineers Love

Reading time: ~28 minutes | Level: Foundation → Engineering

# The difference between a tool you reach for and one you avoid:

# Bad CLI - no help, cryptic errors, wrong exit code
$ process data.csv
Error
$ echo $?
0        # reported success even though it failed

# Good CLI \text{---} self-documenting, composable, honest
$ process --help
Usage: process [OPTIONS] INPUT_FILE

  Process a CSV file and output cleaned data.

Options:
  -o, --output PATH    Write output to file (default: stdout)
  --dry-run            Show what would happen without doing it
  -v, --verbose        Enable verbose logging
  --help               Show this message and exit.

$ process data.csv | sort | uniq -c | sort -rn
# works seamlessly with every other Unix tool

A well-designed CLI is a force multiplier: it works in pipelines, integrates with CI systems, accepts automation without modification, and teaches new users through its own help text. A poorly designed CLI fights every environment it touches.

Python is the dominant language for CLI tools in data engineering, DevOps, and platform tooling. This lesson teaches the principles behind tools that engineers actually enjoy using.

What You Will Learn

• The Unix philosophy and why it produces tools that compose
• Exit codes and why they matter for shell scripts and CI pipelines
• Building professional CLIs with argparse - subcommands, types, error handling
• Click's modern approach: decorators, type coercion, color output, progress bars
• stdin/stdout/stderr conventions and the --output file flag pattern
• The --dry-run flag - indispensable for any destructive operation
• Verbose/quiet flag patterns and what -vvv actually means
• Config file and environment variable integration
• Building a complete, production-quality CLI tool from scratch

Prerequisites

• Comfortable with Python functions, classes, and exception handling
• Basic familiarity with running programs from the terminal
• Understanding of Python's sys module and file I/O

Why CLIs Matter for Python Engineers

Before diving into implementation, understand where CLIs fit in real engineering work:

Automation and scripting. CLI tools are the building blocks of shell scripts, Makefiles, and CI/CD pipelines. A tool with a clean interface can be composed with cron, xargs, parallel, and dozens of other standard tools without any modification.

DevOps and platform tooling. Tools like kubectl, aws, docker, and git are CLIs. When you build internal platform tools, you are building for an audience of engineers who think in terminal commands.

Data pipelines. Processing pipelines are often chains of CLI tools: extract | transform | load. Each tool does one thing and passes data forward through pipes.

Testability. A CLI with proper exit codes and structured output is trivially testable with subprocess.run(). A GUI or interactive tool is not.

The Unix Philosophy for CLIs

The Unix philosophy, formulated by Doug McIlroy in 1978, has three rules that still define good CLI design:

1. Do one thing well. A tool that does one thing can be composed with tools that do other things. A tool that tries to do everything becomes impossible to compose and hard to reason about.

# Bad: one tool that does everything
process --read-csv --clean --sort --deduplicate --output-json data.csv

# Good: each tool does one thing
csv-to-json data.csv | clean | sort | uniq > output.json

2. Work with text streams. Text is the universal interface. A tool that reads from stdin and writes to stdout can be chained with any other Unix tool - including ones that don't exist yet.

3. Output structure, not presentation. When a tool's output is meant to be consumed by another program, output parseable data (JSON, CSV, TSV), not formatted tables with borders and colors. Reserve formatting for human-facing output.

# Check if output is going to a terminal or a pipe
import sys
if sys.stdout.isatty():
    # Human is watching - pretty print with colors
    print_table(results)
else:
    # Output is being piped - machine-readable
    for row in results:
        print(json.dumps(row))

Exit Codes - The Silent API

Exit codes are the most overlooked aspect of CLI design and the one that causes the most pain in CI/CD pipelines.

The Standard Exit Codes

Code	Meaning	When to use it
0	Success	Everything worked correctly
1	General error	Something failed (file not found, processing error)
2	Usage error	Wrong arguments, invalid flags

import sys

# Success
sys.exit(0)

# General error - processing failed
print("Error: input file is empty", file=sys.stderr)
sys.exit(1)

# Usage error - wrong arguments
print("Error: --output requires a file path", file=sys.stderr)
sys.exit(2)

Why Exit Codes Are Non-Negotiable

# Shell scripts check exit codes with && and ||
process input.csv && upload output.json || notify_failure

# CI systems (GitHub Actions, Jenkins) fail the build on non-zero exit
- run: python process.py input.csv
  # If this exits with code 1, the entire workflow fails

# xargs stops on failure
find . -name "*.csv" | xargs -I{} process {}

A tool that always exits with 0 - even on failure - will silently corrupt data pipelines and pass CI checks when it should fail them. This is one of the most common bugs in internally built tooling.

argparse - The Standard Library CLI

argparse is part of Python's standard library. It handles argument parsing, type conversion, help generation, and error messages without any external dependencies.

Basic Structure

import argparse
import sys
from pathlib import Path

def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(
        prog="process",
        description="Process CSV files and output cleaned data.",
        epilog="Examples:\n  process data.csv\n  process data.csv -o output.csv",
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )

    # Positional argument - required, no flag
    parser.add_argument(
        "input",
        type=Path,
        metavar="INPUT_FILE",
        help="Path to the input CSV file",
    )

    # Optional argument - has a flag, has a default
    parser.add_argument(
        "-o", "--output",
        type=Path,
        default=None,
        metavar="OUTPUT_FILE",
        help="Write output to FILE (default: stdout)",
    )

    # Flag - boolean, True when present
    parser.add_argument(
        "--dry-run",
        action="store_true",
        help="Show what would happen without making changes",
    )

    # Choice constraint
    parser.add_argument(
        "--format",
        choices=["csv", "json", "tsv"],
        default="csv",
        help="Output format (default: csv)",
    )

    # Multiple values
    parser.add_argument(
        "--columns",
        nargs="+",
        metavar="COLUMN",
        help="Only include these columns in the output",
    )

    # Count - -v, -vv, -vvv
    parser.add_argument(
        "-v", "--verbose",
        action="count",
        default=0,
        help="Increase verbosity (use multiple times: -v, -vv, -vvv)",
    )

    return parser


def main() -> None:
    parser = build_parser()
    args = parser.parse_args()

    # Type was already converted to Path by argparse
    if not args.input.exists():
        parser.error(f"Input file not found: {args.input}")
        # parser.error() prints the message and exits with code 2

    if args.dry_run:
        print(f"Would process: {args.input}")
        print(f"Output format: {args.format}")
        sys.exit(0)

    try:
        result = process_file(args.input, args.format, args.columns)
        if args.output:
            args.output.write_text(result)
            if args.verbose:
                print(f"Written to {args.output}")
        else:
            print(result)
    except ProcessingError as e:
        print(f"Error: {e}", file=sys.stderr)
        sys.exit(1)


if __name__ == "__main__":
    main()

Subcommands with add_subparsers

Git, docker, and kubectl are all subcommand-style CLIs. argparse supports this with add_subparsers:

import argparse
import sys
from pathlib import Path


def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(
        prog="myapp",
        description="A tool with multiple subcommands",
    )

    # Global options that apply to all subcommands
    parser.add_argument(
        "--config",
        type=Path,
        default=Path.home() / ".myapp.toml",
        help="Path to config file",
    )

    subparsers = parser.add_subparsers(
        dest="command",
        metavar="COMMAND",
        help="Available commands",
    )
    subparsers.required = True  # error if no subcommand given

    # --- 'process' subcommand ---
    process_parser = subparsers.add_parser(
        "process",
        help="Process input files",
        description="Read, clean, and output data files.",
    )
    process_parser.add_argument("input", type=Path, help="Input file")
    process_parser.add_argument("-o", "--output", type=Path, help="Output file")
    process_parser.set_defaults(func=cmd_process)

    # --- 'validate' subcommand ---
    validate_parser = subparsers.add_parser(
        "validate",
        help="Validate file format without processing",
    )
    validate_parser.add_argument("input", type=Path, help="File to validate")
    validate_parser.set_defaults(func=cmd_validate)

    # --- 'report' subcommand ---
    report_parser = subparsers.add_parser(
        "report",
        help="Generate a summary report",
    )
    report_parser.add_argument("input", type=Path, help="File to report on")
    report_parser.add_argument(
        "--format", choices=["text", "json"], default="text"
    )
    report_parser.set_defaults(func=cmd_report)

    return parser


def cmd_process(args: argparse.Namespace) -> None:
    print(f"Processing {args.input}...")
    # actual processing logic here


def cmd_validate(args: argparse.Namespace) -> None:
    print(f"Validating {args.input}...")
    # validation logic here


def cmd_report(args: argparse.Namespace) -> None:
    print(f"Generating report for {args.input} in {args.format} format...")
    # report logic here


def main() -> None:
    parser = build_parser()
    args = parser.parse_args()
    args.func(args)  # dispatch to the right function


if __name__ == "__main__":
    main()

Custom Type Validation

argparse type converters run before your main logic, producing clean error messages at argument parse time:

import argparse
from pathlib import Path


def existing_file(value: str) -> Path:
    """Argparse type that requires the file to exist."""
    path = Path(value)
    if not path.is_file():
        raise argparse.ArgumentTypeError(f"File not found: {value}")
    return path


def port_number(value: str) -> int:
    """Argparse type that requires a valid port number."""
    try:
        port = int(value)
    except ValueError:
        raise argparse.ArgumentTypeError(f"Not a number: {value}")
    if not 1 <= port <= 65535:
        raise argparse.ArgumentTypeError(f"Port must be 1-65535, got {port}")
    return port


parser = argparse.ArgumentParser()
parser.add_argument("input", type=existing_file)
parser.add_argument("--port", type=port_number, default=8080)

Click - The Modern Alternative

Click is a third-party library (pip install click) that uses Python decorators to define CLIs. It produces cleaner code for complex CLIs and has excellent built-in support for types, colors, progress bars, and testing.

Basic Click Command

import click
import sys
from pathlib import Path


@click.command()
@click.argument("input_file", type=click.Path(exists=True, path_type=Path))
@click.option(
    "-o", "--output",
    type=click.Path(path_type=Path),
    default=None,
    help="Write output to FILE (default: stdout)",
)
@click.option(
    "--format",
    type=click.Choice(["csv", "json", "tsv"], case_sensitive=False),
    default="csv",
    show_default=True,
    help="Output format",
)
@click.option("--dry-run", is_flag=True, help="Show what would happen without doing it")
@click.option("-v", "--verbose", count=True, help="Increase verbosity")
def process(
    input_file: Path,
    output: Path | None,
    format: str,
    dry_run: bool,
    verbose: int,
) -> None:
    """Process CSV files and output cleaned data.

    INPUT_FILE is the path to the CSV file to process.

    Examples:

        process data.csv

        process data.csv --output result.csv --format json

        process data.csv --dry-run
    """
    if dry_run:
        click.echo(f"Would process: {input_file}")
        click.echo(f"Output format: {format}")
        return

    if verbose >= 2:
        click.echo(f"Processing {input_file} with format={format}", err=True)

    try:
        result = _do_process(input_file, format)
        if output:
            output.write_text(result)
            if verbose:
                click.echo(f"Written to {output}", err=True)
        else:
            click.echo(result)
    except ProcessingError as e:
        click.echo(f"Error: {e}", err=True)
        sys.exit(1)


if __name__ == "__main__":
    process()

Notice several things:

• The docstring becomes the help text - write it for humans
• click.Path(exists=True) validates file existence at argument parse time, not in your code
• click.echo(..., err=True) writes to stderr, keeping stdout clean for piping
• The count=True option on --verbose supports -v, -vv, -vvv

Click Groups - Subcommands

import click
import sys
from pathlib import Path


@click.group()
@click.option(
    "--config",
    type=click.Path(path_type=Path),
    default=Path.home() / ".myapp.toml",
    envvar="MYAPP_CONFIG",
    help="Path to config file",
)
@click.pass_context
def cli(ctx: click.Context, config: Path) -> None:
    """myapp - A professional file processing tool.

    Use --help with any subcommand for details.
    """
    ctx.ensure_object(dict)
    ctx.obj["config"] = config


@cli.command()
@click.argument("input_file", type=click.Path(exists=True, path_type=Path))
@click.option("-o", "--output", type=click.Path(path_type=Path))
@click.pass_context
def process(ctx: click.Context, input_file: Path, output: Path | None) -> None:
    """Process input files."""
    config = ctx.obj["config"]
    click.echo(f"Using config: {config}")
    click.echo(f"Processing: {input_file}")


@cli.command()
@click.argument("input_file", type=click.Path(exists=True, path_type=Path))
@click.pass_context
def validate(ctx: click.Context, input_file: Path) -> None:
    """Validate file format without processing."""
    click.echo(f"Validating: {input_file}")


@cli.command()
@click.argument("input_file", type=click.Path(exists=True, path_type=Path))
@click.option(
    "--format",
    type=click.Choice(["text", "json"]),
    default="text",
)
def report(input_file: Path, format: str) -> None:
    """Generate a summary report."""
    click.echo(f"Report for {input_file} ({format} format)")


if __name__ == "__main__":
    cli()

Click Types in Depth

import click

# File types - Click opens the file for you
@click.command()
@click.argument("input", type=click.File("r"))    # opens for reading
@click.option("-o", "--output", type=click.File("w"), default="-")  # "-" = stdout
def cat(input, output):
    """Read INPUT and write to OUTPUT."""
    for line in input:
        output.write(line)

# Path types - validates existence, type, permissions
@click.command()
@click.argument(
    "directory",
    type=click.Path(
        exists=True,
        file_okay=False,     # must be a directory
        dir_okay=True,
        readable=True,
        path_type=Path,
    ),
)
def list_dir(directory: Path):
    for item in directory.iterdir():
        click.echo(item.name)

# IntRange - validates numeric ranges
@click.command()
@click.option(
    "--workers",
    type=click.IntRange(1, 32),
    default=4,
    help="Number of worker threads (1-32)",
)
def run(workers: int):
    click.echo(f"Using {workers} workers")

Progress Bars

import click
import time
from pathlib import Path

@click.command()
@click.argument("files", nargs=-1, type=click.Path(exists=True, path_type=Path))
def process_many(files: tuple[Path, ...]) -> None:
    """Process multiple files with a progress bar."""
    with click.progressbar(
        files,
        label="Processing files",
        item_show_func=lambda f: str(f) if f else "",
    ) as bar:
        for file in bar:
            _process_one(file)
            time.sleep(0.1)  # simulated work
    click.echo(f"Done. Processed {len(files)} files.")

Color Output

import click
import sys

def print_status(message: str, level: str = "info") -> None:
    """Print colored status messages to stderr."""
    colors = {
        "info":    ("blue",   "[INFO]"),
        "success": ("green",  "[OK]  "),
        "warning": ("yellow", "[WARN]"),
        "error":   ("red",    "[ERR] "),
    }
    color, prefix = colors.get(level, ("white", "[???]"))
    click.echo(
        click.style(prefix, fg=color, bold=True) + f" {message}",
        err=True,  # status goes to stderr
    )

# Usage
print_status("Starting processing...", "info")
print_status("File validated successfully", "success")
print_status("Missing optional field 'description'", "warning")
print_status("File not found: data.csv", "error")

Environment Variable Overrides

import click

@click.command()
@click.option(
    "--api-key",
    envvar="MYAPP_API_KEY",      # reads from MYAPP_API_KEY env var
    required=True,
    help="API key (or set MYAPP_API_KEY env var)",
)
@click.option(
    "--host",
    envvar="MYAPP_HOST",
    default="localhost",
    show_default=True,
    help="Server host (or set MYAPP_HOST env var)",
)
@click.option(
    "--port",
    envvar="MYAPP_PORT",
    type=int,
    default=8080,
    show_default=True,
)
def connect(api_key: str, host: str, port: int) -> None:
    """Connect to the server."""
    click.echo(f"Connecting to {host}:{port}")

The precedence order is: command-line flags override environment variables, which override defaults. This is the standard pattern for twelve-factor apps.

stdin / stdout / stderr Conventions

These three streams have clear, standard roles:

Stream	Role	Example
stdin	Input data	File contents, piped data
stdout	Output data	Processed results, JSON output
stderr	Diagnostics	Progress messages, warnings, errors

Never mix output data with diagnostic messages. If you print Processing 1000 rows... to stdout, you break any pipeline that tries to parse your output.

Reading from stdin

import sys
import click
from pathlib import Path


@click.command()
@click.argument("input_file", type=click.Path(path_type=Path), default=None, required=False)
def process(input_file: Path | None) -> None:
    """Process data from a file or stdin.

    Read from INPUT_FILE, or from stdin if no file is given.

    Examples:

        process data.csv

        cat data.csv | process

        process < data.csv
    """
    if input_file is not None:
        data = input_file.read_text()
    elif not sys.stdin.isatty():
        # stdin has data (pipe or redirect)
        data = sys.stdin.read()
    else:
        # No file and no pipe - user ran it interactively with no input
        click.echo("Error: provide a file or pipe data to stdin", err=True)
        sys.exit(2)

    result = _transform(data)
    click.echo(result)  # goes to stdout - pipeable

The --output Flag Pattern

import click
import sys
from pathlib import Path


@click.command()
@click.argument("input_file", type=click.Path(exists=True, path_type=Path))
@click.option(
    "-o", "--output",
    type=click.Path(path_type=Path),
    default=None,
    help="Write output to FILE. Defaults to stdout.",
)
def process(input_file: Path, output: Path | None) -> None:
    """Process a file."""
    result = _transform(input_file.read_text())

    if output is None:
        click.echo(result)           # stdout - composable
    else:
        output.write_text(result)
        click.echo(f"Written to {output}", err=True)   # confirmation to stderr

This pattern lets users do either:

process data.csv | grep "important"      # pipe stdout
process data.csv -o result.csv           # write to file

The --dry-run Flag

--dry-run is the professional's testing tool. Any command that modifies files, sends emails, posts to APIs, or changes databases should support it.

import click
from pathlib import Path


@click.command()
@click.argument("source", type=click.Path(exists=True, path_type=Path))
@click.argument("destination", type=click.Path(path_type=Path))
@click.option("--dry-run", is_flag=True, help="Show what would happen without doing it")
@click.option("-v", "--verbose", is_flag=True)
def migrate(source: Path, destination: Path, dry_run: bool, verbose: bool) -> None:
    """Migrate data from SOURCE to DESTINATION."""
    files = list(source.glob("**/*.csv"))

    if dry_run:
        click.echo(click.style("DRY RUN - no changes will be made", fg="yellow", bold=True), err=True)

    for f in files:
        target = destination / f.relative_to(source)

        if dry_run:
            click.echo(f"  would copy: {f} -> {target}")
        else:
            target.parent.mkdir(parents=True, exist_ok=True)
            target.write_bytes(f.read_bytes())
            if verbose:
                click.echo(f"  copied: {f} -> {target}")

    summary = f"{'Would process' if dry_run else 'Processed'} {len(files)} files"
    click.echo(summary)

Verbose / Quiet Flag Patterns

The standard pattern for verbosity levels:

import click
import logging
import sys


def setup_logging(verbosity: int, quiet: bool) -> None:
    """Configure logging based on verbosity flags.

    -q / --quiet:    WARNING and above only
    (default):       INFO
    -v:              DEBUG
    -vv:             DEBUG + show file/line numbers
    """
    if quiet:
        level = logging.WARNING
    elif verbosity == 0:
        level = logging.INFO
    elif verbosity >= 1:
        level = logging.DEBUG

    fmt = "%(message)s"
    if verbosity >= 2:
        fmt = "%(levelname)s %(name)s:%(lineno)d %(message)s"

    logging.basicConfig(
        level=level,
        format=fmt,
        stream=sys.stderr,     # logs go to stderr, not stdout
    )


@click.command()
@click.option("-v", "--verbose", count=True)
@click.option("-q", "--quiet", is_flag=True)
def run(verbose: int, quiet: bool) -> None:
    """Run with configurable verbosity."""
    if verbose and quiet:
        raise click.UsageError("--verbose and --quiet are mutually exclusive")

    setup_logging(verbose, quiet)
    log = logging.getLogger(__name__)

    log.debug("Debug message (only with -v)")
    log.info("Info message (default)")
    log.warning("Warning message (always shown)")

    click.echo("Processing complete")   # to stdout

Config File Integration

Professional tools load defaults from a config file, overridden by environment variables, overridden by command-line flags - in that order. This is the twelve-factor app configuration hierarchy.

import click
import sys
import tomllib
from pathlib import Path


def load_config(config_path: Path) -> dict:
    """Load TOML config file, returning empty dict if not found."""
    if not config_path.exists():
        return {}
    with config_path.open("rb") as f:
        return tomllib.load(f)


class ConfigAwareCommand(click.Command):
    """A Click command that merges config file defaults with CLI args."""

    def invoke(self, ctx: click.Context) -> None:
        config_path = ctx.params.get("config") or Path.home() / ".myapp.toml"
        config = load_config(config_path)

        # Merge config into defaults for params that weren't explicitly set
        for param in self.params:
            if param.name in config and ctx.get_parameter_source(param.name) == click.core.ParameterSource.DEFAULT:
                ctx.params[param.name] = config[param.name]

        return super().invoke(ctx)


@click.command(cls=ConfigAwareCommand)
@click.option("--config", type=click.Path(path_type=Path), default=None, hidden=True)
@click.option("--host", default="localhost", envvar="MYAPP_HOST")
@click.option("--port", type=int, default=8080, envvar="MYAPP_PORT")
@click.option("--api-key", envvar="MYAPP_API_KEY")
def connect(config: Path | None, host: str, port: int, api_key: str | None) -> None:
    """Connect to the server using layered configuration."""
    click.echo(f"Connecting to {host}:{port}")

A typical ~/.myapp.toml:

host = "production.example.com"
port = 443
api_key = "secret-key-here"

A Complete CLI Tool: File Processor

Putting it all together - a production-quality CLI tool with subcommands, type validation, exit codes, stdin support, and dry-run mode:

#!/usr/bin/env python3
"""
fileproc - A professional file processing CLI tool.

Usage:
    fileproc process data.csv
    fileproc validate data.csv
    fileproc report data.csv --format json
    cat data.csv | fileproc process --output result.json --format json
"""
import json
import sys
import csv
import io
from pathlib import Path

import click


# ---------------------------------------------------------------------------
# Shared utilities
# ---------------------------------------------------------------------------

def read_input(input_file: Path | None) -> str:
    """Read from file or stdin."""
    if input_file is not None:
        return input_file.read_text()
    if not sys.stdin.isatty():
        return sys.stdin.read()
    click.echo("Error: provide a file or pipe data via stdin", err=True)
    sys.exit(2)


def write_output(data: str, output: Path | None) -> None:
    """Write to file or stdout."""
    if output is None:
        click.echo(data)
    else:
        output.write_text(data)
        click.echo(f"Written to {output}", err=True)


# ---------------------------------------------------------------------------
# CLI definition
# ---------------------------------------------------------------------------

@click.group()
@click.version_option("1.0.0")
def cli() -> None:
    """fileproc - Process, validate, and report on data files."""


@cli.command()
@click.argument("input_file", type=click.Path(exists=True, path_type=Path), required=False)
@click.option("-o", "--output", type=click.Path(path_type=Path), help="Output file (default: stdout)")
@click.option(
    "--format",
    type=click.Choice(["csv", "json", "tsv"]),
    default="csv",
    show_default=True,
)
@click.option("--columns", multiple=True, metavar="COL", help="Only include these columns")
@click.option("--dry-run", is_flag=True, help="Show what would happen without doing it")
@click.option("-v", "--verbose", count=True)
def process(
    input_file: Path | None,
    output: Path | None,
    format: str,
    columns: tuple[str, ...],
    dry_run: bool,
    verbose: int,
) -> None:
    """Process a CSV file - clean, filter, and reformat.

    Read from INPUT_FILE or stdin if not provided.
    """
    if dry_run:
        click.echo(
            click.style("DRY RUN - no changes will be made", fg="yellow"),
            err=True,
        )
        click.echo(f"  input:   {input_file or 'stdin'}", err=True)
        click.echo(f"  output:  {output or 'stdout'}", err=True)
        click.echo(f"  format:  {format}", err=True)
        if columns:
            click.echo(f"  columns: {', '.join(columns)}", err=True)
        sys.exit(0)

    raw = read_input(input_file)

    try:
        reader = csv.DictReader(io.StringIO(raw))
        rows = list(reader)
    except csv.Error as e:
        click.echo(f"Error: invalid CSV - {e}", err=True)
        sys.exit(1)

    if columns:
        missing = [c for c in columns if c not in (rows[0] if rows else {})]
        if missing:
            click.echo(f"Error: columns not found: {', '.join(missing)}", err=True)
            sys.exit(1)
        rows = [{c: r[c] for c in columns} for r in rows]

    if format == "json":
        result = json.dumps(rows, indent=2)
    elif format == "tsv":
        if not rows:
            result = ""
        else:
            headers = list(rows[0].keys())
            lines = ["\t".join(headers)]
            lines += ["\t".join(str(r[h]) for h in headers) for r in rows]
            result = "\n".join(lines)
    else:  # csv
        if not rows:
            result = ""
        else:
            buf = io.StringIO()
            writer = csv.DictWriter(buf, fieldnames=list(rows[0].keys()))
            writer.writeheader()
            writer.writerows(rows)
            result = buf.getvalue()

    if verbose:
        click.echo(f"Processed {len(rows)} rows", err=True)

    write_output(result, output)


@cli.command()
@click.argument("input_file", type=click.Path(exists=True, path_type=Path), required=False)
def validate(input_file: Path | None) -> None:
    """Validate that a file is well-formed CSV.

    Exits with code 0 if valid, 1 if invalid.
    """
    raw = read_input(input_file)

    try:
        reader = csv.DictReader(io.StringIO(raw))
        rows = list(reader)
        click.echo(
            click.style("VALID", fg="green", bold=True)
            + f" - {len(rows)} rows, {len(reader.fieldnames or [])} columns"
        )
    except csv.Error as e:
        click.echo(click.style("INVALID", fg="red", bold=True) + f" - {e}")
        sys.exit(1)


@cli.command()
@click.argument("input_file", type=click.Path(exists=True, path_type=Path), required=False)
@click.option("--format", type=click.Choice(["text", "json"]), default="text")
def report(input_file: Path | None, format: str) -> None:
    """Generate a summary report of a CSV file."""
    raw = read_input(input_file)

    try:
        reader = csv.DictReader(io.StringIO(raw))
        rows = list(reader)
    except csv.Error as e:
        click.echo(f"Error: {e}", err=True)
        sys.exit(1)

    columns = reader.fieldnames or []
    summary = {
        "row_count": len(rows),
        "column_count": len(columns),
        "columns": columns,
        "empty_cells": sum(
            1 for row in rows for val in row.values() if not val.strip()
        ),
    }

    if format == "json":
        click.echo(json.dumps(summary, indent=2))
    else:
        click.echo(f"Rows:         {summary['row_count']}")
        click.echo(f"Columns:      {summary['column_count']}")
        click.echo(f"Column names: {', '.join(columns)}")
        click.echo(f"Empty cells:  {summary['empty_cells']}")


if __name__ == "__main__":
    cli()

Interview Questions

Q1: What are exit codes and why do they matter for CLI tools?

Answer: Exit codes are integers that a process returns to its parent when it finishes. The universal convention is 0 for success and non-zero for failure - specifically 1 for general errors and 2 for usage errors like wrong arguments. They matter because shell scripts and CI systems depend on exit codes to determine whether a command succeeded and whether to continue running. The && operator in bash only runs the next command if the previous exited with 0. GitHub Actions fails a workflow step if the command exits non-zero. A tool that always exits with 0 - even on failure - silently corrupts pipelines and gives CI systems false confidence. This is one of the most common bugs in internally built tooling, and it often goes unnoticed until a silent data corruption incident.

Q2: What is the Unix philosophy and how should it guide CLI design?

Answer: The Unix philosophy has three core principles: do one thing well, work with text streams, and output structure rather than presentation. For CLI tools, this means each command should have a single clear purpose rather than trying to be a Swiss Army knife. It also means reading from stdin and writing to stdout by default, so tools compose naturally with pipes and redirection. Status messages and errors belong on stderr, keeping stdout clean for machine-readable output. The practical test is whether your tool works in a pipeline like cat data.csv | myapp process | sort | uniq > result.txt. If it does, you've followed the philosophy.

Q3: What is the difference between argparse and Click, and when should you use each?

Answer: Both handle argument parsing, help generation, and type validation, but they take very different approaches. argparse is part of the standard library with no external dependencies - the right choice when you cannot add dependencies or need maximum portability. Its API is more verbose and imperative. Click uses Python decorators, which produce more readable code and less boilerplate for complex CLIs. Click also has better built-in support for colors, progress bars, file handling, and testing. The practical rule: use argparse for simple tools or when you want zero dependencies, and Click for complex CLIs with subcommands, color output, or progress bars. Both are legitimate professional choices; the principles of good CLI design (exit codes, stdin/stdout conventions, help text) apply equally to both.

Q4: What should go to stdout versus stderr in a CLI tool?

Answer: stdout is for output data - the result of what the tool computed. It should be clean, parseable, and suitable for piping into the next tool. stderr is for diagnostics - progress messages, warnings, status updates, and error messages. This separation is critical: if you write "Processing 1000 rows..." to stdout, you break any pipeline that tries to parse your output as CSV or JSON. A good test is to run your tool in a pipeline: mytool input.csv | grep "important". If the grep sees your progress messages mixed with your output, stderr is broken. In Python, print(..., file=sys.stderr) and click.echo(..., err=True) both write to stderr.

Q5: How do you implement a --dry-run flag and why is it important?

Answer: --dry-run is a boolean flag (no argument) that causes the tool to show what it would do without actually doing it. It's essential for any command that modifies files, sends requests, posts to APIs, or makes irreversible changes. The implementation pattern is to check the flag early, print each action that would be taken, and exit with 0 without performing any of them. A good dry-run shows exactly the same information the real run would show about its plan - which files would be modified, which records would be updated - just without executing. This lets users confidently verify the tool's behavior before running it against production data. Senior engineers consider it a prerequisite for any tool used in production automation.

Q6: How do environment variables and config files fit into a CLI tool's configuration hierarchy?

Answer: The professional standard is a three-level hierarchy: config file sets the defaults, environment variables override the config file, and command-line flags override everything. This matches the twelve-factor app methodology. Config files store team or machine defaults that shouldn't be typed every time. Environment variables are used in deployment and CI environments where you cannot pass flags. Command-line flags provide per-invocation overrides. In Click, you implement this with envvar= on options for environment variable support, and a custom command class or startup logic that reads a config file and applies its values to any option that wasn't explicitly set on the command line. The config file location itself should default to ~/.toolname.toml but be overridable with a --config flag.

Practice Challenges

Beginner - Add Exit Codes to a Broken Tool

The following tool always exits with 0, even on error. Add proper exit codes (0 = success, 1 = error, 2 = usage error) and move all status messages to stderr:

import sys
from pathlib import Path

def main():
    if len(sys.argv) != 2:
        print("Usage: tool <filename>")
        return  # should exit 2

    path = Path(sys.argv[1])
    if not path.exists():
        print(f"File not found: {path}")
        return  # should exit 1

    lines = path.read_text().splitlines()
    print(f"File has {len(lines)} lines")  # this output is fine on stdout
    print(f"Processing {path}...")         # this status should go to stderr

main()

Solution

import sys
from pathlib import Path


def main() -> None:
    if len(sys.argv) != 2:
        print("Usage: tool <filename>", file=sys.stderr)
        sys.exit(2)

    path = Path(sys.argv[1])
    if not path.exists():
        print(f"Error: file not found: {path}", file=sys.stderr)
        sys.exit(1)

    print(f"Processing {path}...", file=sys.stderr)   # status to stderr
    lines = path.read_text().splitlines()
    print(f"{len(lines)}")                             # data result to stdout
    sys.exit(0)


if __name__ == "__main__":
    main()

Key changes:

1. return replaced with sys.exit(N) at each failure point
2. Usage error now exits with 2, not 1
3. Status message "Processing {path}..." moved to stderr
4. Output (line count) stays on stdout - it's data
5. Explicit sys.exit(0) at the end for clarity

Intermediate - Build a Click CLI with Subcommands

Build a notes CLI tool using Click with three subcommands:

• notes add "My note text" - appends a timestamped note to ~/.notes.txt
• notes list - prints all notes, newest first
• notes search QUERY - finds notes containing the query text

Each subcommand should have --help text and proper exit codes.

Solution

#!/usr/bin/env python3
"""notes - A simple command-line notes tool."""
import sys
from datetime import datetime
from pathlib import Path

import click

NOTES_FILE = Path.home() / ".notes.txt"


def get_notes() -> list[dict]:
    """Read all notes from the notes file."""
    if not NOTES_FILE.exists():
        return []
    notes = []
    for line in NOTES_FILE.read_text().strip().splitlines():
        if "|" in line:
            timestamp, _, text = line.partition("|")
            notes.append({"timestamp": timestamp.strip(), "text": text.strip()})
    return notes


def save_note(text: str) -> None:
    """Append a new note with current timestamp."""
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M")
    with NOTES_FILE.open("a") as f:
        f.write(f"{timestamp} | {text}\n")


@click.group()
def cli() -> None:
    """notes - A simple command-line note-taking tool.

    Notes are stored in ~/.notes.txt.
    """


@cli.command()
@click.argument("text")
def add(text: str) -> None:
    """Add a new note.

    TEXT is the note content to save.
    """
    save_note(text)
    click.echo(click.style("Added:", fg="green") + f" {text}", err=True)


@cli.command("list")
@click.option("-n", "--count", type=int, default=None, help="Show at most N notes")
def list_notes(count: int | None) -> None:
    """List all notes, newest first."""
    notes = get_notes()
    if not notes:
        click.echo("No notes yet. Use `notes add` to create one.", err=True)
        sys.exit(0)

    notes_to_show = list(reversed(notes))
    if count is not None:
        notes_to_show = notes_to_show[:count]

    for note in notes_to_show:
        ts = click.style(note["timestamp"], fg="cyan")
        click.echo(f"{ts}  {note['text']}")


@cli.command()
@click.argument("query")
@click.option("-i", "--ignore-case", is_flag=True, help="Case-insensitive search")
def search(query: str, ignore_case: bool) -> None:
    """Search notes for QUERY text."""
    notes = get_notes()
    compare = (lambda s: s.lower()) if ignore_case else (lambda s: s)
    matches = [n for n in notes if compare(query) in compare(n["text"])]

    if not matches:
        click.echo(f"No notes matching: {query}", err=True)
        sys.exit(1)

    for note in reversed(matches):
        ts = click.style(note["timestamp"], fg="cyan")
        # highlight the match
        text = note["text"]
        if ignore_case:
            idx = text.lower().find(query.lower())
        else:
            idx = text.find(query)
        if idx != -1:
            highlighted = (
                text[:idx]
                + click.style(text[idx:idx + len(query)], fg="yellow", bold=True)
                + text[idx + len(query):]
            )
        else:
            highlighted = text
        click.echo(f"{ts}  {highlighted}")


if __name__ == "__main__":
    cli()

Advanced - Pipe-Composable Data Transformer

Build a CLI tool that reads CSV data from stdin or a file, applies a chain of transformations specified via flags, and writes the result to stdout or a file. It should support:

• --filter COL=VALUE - keep only rows where COL equals VALUE
• --select COL1,COL2 - keep only specified columns
• --rename OLD:NEW - rename a column
• --sort COL - sort rows by column
• --limit N - keep only the first N rows

The tool should work seamlessly in Unix pipes and exit with proper codes.

Solution

#!/usr/bin/env python3
"""transform - A pipe-composable CSV transformation tool.

Examples:
    transform data.csv --filter status=active --select name,email
    cat data.csv | transform --sort name --limit 10
    transform data.csv --rename old_name:name | sort | uniq
"""
import csv
import io
import sys
from typing import Any

import click


def read_csv(source: str) -> tuple[list[str], list[dict]]:
    reader = csv.DictReader(io.StringIO(source))
    rows = list(reader)
    return list(reader.fieldnames or []), rows


def write_csv(fieldnames: list[str], rows: list[dict]) -> str:
    buf = io.StringIO()
    writer = csv.DictWriter(buf, fieldnames=fieldnames, extrasaction="ignore")
    writer.writeheader()
    writer.writerows(rows)
    return buf.getvalue()


@click.command()
@click.argument("input_file", type=click.Path(exists=True), required=False)
@click.option("-o", "--output", type=click.Path(), default=None)
@click.option("--filter", "filters", multiple=True, metavar="COL=VALUE",
              help="Keep rows where COL equals VALUE")
@click.option("--select", "select_cols", default=None, metavar="COL1,COL2",
              help="Keep only these columns (comma-separated)")
@click.option("--rename", "renames", multiple=True, metavar="OLD:NEW",
              help="Rename column OLD to NEW")
@click.option("--sort", "sort_col", default=None, metavar="COL",
              help="Sort rows by this column")
@click.option("--limit", type=int, default=None, metavar="N",
              help="Keep only the first N rows")
@click.option("--dry-run", is_flag=True, help="Show transformation plan without executing")
def transform(
    input_file: str | None,
    output: str | None,
    filters: tuple[str, ...],
    select_cols: str | None,
    renames: tuple[str, ...],
    sort_col: str | None,
    limit: int | None,
    dry_run: bool,
) -> None:
    """Transform CSV data with a chain of operations.

    Read from INPUT_FILE or stdin if not provided.
    """
    if dry_run:
        click.echo(click.style("Transformation plan:", fg="cyan", bold=True), err=True)
        for f in filters:
            click.echo(f"  filter:  {f}", err=True)
        if select_cols:
            click.echo(f"  select:  {select_cols}", err=True)
        for r in renames:
            click.echo(f"  rename:  {r}", err=True)
        if sort_col:
            click.echo(f"  sort:    {sort_col}", err=True)
        if limit is not None:
            click.echo(f"  limit:   {limit}", err=True)
        sys.exit(0)

    # Read input
    if input_file:
        raw = open(input_file).read()
    elif not sys.stdin.isatty():
        raw = sys.stdin.read()
    else:
        click.echo("Error: provide a file or pipe data via stdin", err=True)
        sys.exit(2)

    try:
        fieldnames, rows = read_csv(raw)
    except csv.Error as e:
        click.echo(f"Error: invalid CSV - {e}", err=True)
        sys.exit(1)

    # Apply filters
    for f in filters:
        if "=" not in f:
            click.echo(f"Error: --filter must be COL=VALUE, got: {f}", err=True)
            sys.exit(2)
        col, _, val = f.partition("=")
        if col not in fieldnames:
            click.echo(f"Error: column not found: {col}", err=True)
            sys.exit(1)
        rows = [r for r in rows if r.get(col) == val]

    # Apply renames
    for r in renames:
        if ":" not in r:
            click.echo(f"Error: --rename must be OLD:NEW, got: {r}", err=True)
            sys.exit(2)
        old, _, new = r.partition(":")
        if old not in fieldnames:
            click.echo(f"Error: column not found: {old}", err=True)
            sys.exit(1)
        fieldnames = [new if c == old else c for c in fieldnames]
        rows = [{(new if k == old else k): v for k, v in row.items()} for row in rows]

    # Apply column selection
    if select_cols:
        cols = [c.strip() for c in select_cols.split(",")]
        missing = [c for c in cols if c not in fieldnames]
        if missing:
            click.echo(f"Error: columns not found: {', '.join(missing)}", err=True)
            sys.exit(1)
        fieldnames = cols
        rows = [{c: r[c] for c in cols} for r in rows]

    # Apply sort
    if sort_col:
        if sort_col not in fieldnames:
            click.echo(f"Error: sort column not found: {sort_col}", err=True)
            sys.exit(1)
        rows = sorted(rows, key=lambda r: r.get(sort_col, ""))

    # Apply limit
    if limit is not None:
        rows = rows[:limit]

    result = write_csv(fieldnames, rows)

    if output:
        open(output, "w").write(result)
        click.echo(
            f"Written {len(rows)} rows to {output}",
            err=True,
        )
    else:
        sys.stdout.write(result)


if __name__ == "__main__":
    transform()

Quick Reference

Concept	Pattern
Exit codes	0 success, 1 error, 2 usage error
argparse errors	parser.error("message") - prints and exits 2
Click errors	raise click.UsageError("message")
Status output	Always sys.stderr or click.echo(..., err=True)
Data output	Always sys.stdout / click.echo(result)
Dry run	Check flag early, print plan, sys.exit(0)
Verbosity	action="count" (argparse) or count=True (Click)
Env var override	envvar="MYAPP_HOST" in Click option
stdin detection	sys.stdin.isatty() returns True if no pipe
File or stdin	Accept optional positional arg, fall back to stdin

Key Takeaways

• Exit codes are the silent API of your CLI. Tools that always exit with 0 silently corrupt shell pipelines and CI systems - this is one of the most consequential bugs you can ship.
• The Unix philosophy - do one thing well, work with text streams - produces tools that compose naturally with every other tool in the ecosystem, including ones that don't exist yet.
• Status messages, progress output, warnings, and errors belong on stderr. Data output belongs on stdout. This separation is what makes a tool pipeable.
• argparse is the zero-dependency standard library choice; Click is the modern choice for complex CLIs. Both support all the same design principles.
• --dry-run is non-negotiable for any command that modifies files, sends requests, or makes irreversible changes. Senior engineers expect it.
• Environment variable overrides (envvar=) make tools work naturally in CI, Docker, and twelve-factor deployments without requiring every invocation to pass every flag.
• The docstring in a Click command IS the help text - write it for the next engineer who reads --help at 2am trying to understand what this tool does.