while Loops and Control Flow - State-Driven Iteration, Retry Logic, and Infinite Loop Mastery

Reading time: ~23 minutes | Level: Foundation → Engineering

Before reading further, predict the output of this code:

import time

attempts = 0
result = None

while attempts < 4:
    attempts += 1
    if attempts == 3:
        result = "success"
        break
else:
    result = "all_failed"

print(result)
print(attempts)

Many developers guess "all_failed" because they associate else with the "other" branch. The actual output is:

success
3

The while/else clause works exactly like for/else: the else block runs only if the loop terminates without a break. Since break fires on attempt 3, the else block never executes. This is a preview of one of Python's most misunderstood features. But the deeper lesson is structural: the pattern above - while attempts < max: ... break ... else: handle_exhaustion - is the precise skeleton of every well-engineered retry loop in production Python systems. This lesson teaches you to build them correctly.

What You Will Learn

• The fundamental difference between while and for: condition-based vs sequence-based iteration
• The while loop execution model at the interpreter level
• while True with internal break: why it is often cleaner than a flag variable
• State-based iteration: using while to advance through state until a condition is met
• Retry loops with exponential backoff: the canonical pattern for resilient network code
• The sentinel value pattern and the walrus operator (:=) for elegant termination
• Infinite loop prevention: identifying the termination guarantee in every while loop
• Input validation loops: the correct pattern for prompting until valid input is received
• Converting recursion to while loops for stack safety
• The else clause on while loops
• ASCII diagrams: the while loop flowchart and a state machine implemented with while
• Pitfalls: off-by-one errors, forgotten increments, using while where for is appropriate

Prerequisites

You should be comfortable with:

• Python functions, variables, and basic types
• if/elif/else branching
• Python exceptions: raise, try/except
• break and continue as loop control statements

while vs for: Choosing the Right Tool

Both for and while repeat code. The distinction is the termination mechanism, and choosing the wrong one makes code harder to read and more prone to bugs.

Use for when iterating over a known sequence or iterable - the loop is driven by the items in a collection:

# for: sequence-driven - the list determines iteration count
for item in shopping_cart:
    apply_discount(item)

Use while when the number of iterations is determined by a condition that changes as the program runs - the loop is driven by state:

# while: condition-driven - state determines when to stop
while queue:
    job = queue.pop(0)
    process(job)

The engineering rule: if you can identify a fixed iterable at the point where the loop starts, use for. If the termination depends on something that changes dynamically during execution - user input, network responses, computed results, queue depth - use while.

note

If you find yourself writing for i in range(some_condition): where some_condition is not a simple integer but something that might change, that is a signal to use while instead. The range() argument is evaluated once at loop entry; while's condition is re-evaluated on every iteration.

The While Loop Execution Model

The while loop evaluates its condition, executes the body if the condition is truthy, then re-evaluates the condition. This cycle repeats until the condition is falsy or until break exits the loop.

Critical observation: the condition is evaluated before the first iteration. A while loop with a false condition at entry executes zero times.

while False:
    print("This never runs")

count = 0
while count > 0:
    print("Nor does this - count starts at 0")

while True with Internal break

The while True: pattern - an explicitly infinite loop broken internally - is idiomatic Python for situations where the exit condition is known only after entering the loop body. It is cleaner than the alternative of tracking a boolean flag:

With a flag variable (less clean):

keep_running = True
while keep_running:
    command = input("Enter command (quit to exit): ")
    if command == "quit":
        keep_running = False
    else:
        handle_command(command)

With while True and break (cleaner):

while True:
    command = input("Enter command (quit to exit): ")
    if command == "quit":
        break
    handle_command(command)

The while True version eliminates the flag variable entirely. The exit condition is expressed once, at the point where it is checked. There is no mutable boolean to track through the loop body. This pattern is used throughout Python's standard library and powers event loops, REPL implementations, and network server handlers.

When the body itself performs an operation that might fail and should retry, while True with an internal break combined with try/except is canonical:

while True:
    try:
        value = int(input("Enter an integer: "))
        break  # success - exit the loop
    except ValueError:
        print("That is not an integer. Please try again.")
# value is guaranteed to be a valid integer here

tip

The while True: pattern is not a code smell - it is the correct tool when the exit condition is naturally determined inside the loop body. The code smell would be a while True: loop without any path to a break statement, which is an actual infinite loop.

State-Based Iteration

The most powerful use of while is advancing through state: repeatedly applying an operation until the system reaches a desired condition. The state drives the loop, not a fixed iterable.

Processing a queue until empty:

job_queue = ["parse_data", "validate_schema", "write_output", "notify_client"]

while job_queue:
    job = job_queue.pop(0)  # Remove and return first item
    print(f"Processing: {job}")
    # ... execute job ...

# After the loop: queue is guaranteed empty
print(f"All {len(job_queue)} remaining jobs: done")

Reading from a socket until EOF:

import socket

def receive_all(sock: socket.socket, buffer_size: int = 4096) -> bytes:
    """Read from socket until connection closed."""
    chunks = []
    while True:
        chunk = sock.recv(buffer_size)
        if not chunk:
            break  # Empty bytes signals connection closed
        chunks.append(chunk)
    return b"".join(chunks)

Binary search as state iteration:

def binary_search(sorted_list, target):
    left, right = 0, len(sorted_list) - 1

    while left <= right:
        mid = (left + right) // 2
        if sorted_list[mid] == target:
            return mid
        elif sorted_list[mid] < target:
            left = mid + 1      # State advances: narrow search right
        else:
            right = mid - 1     # State advances: narrow search left

    return -1  # Not found

Each iteration of the binary search changes the state (left and right) in a way that guarantees the search space shrinks by at least half. The while condition left <= right directly expresses when the state makes further iteration meaningless.

State Machine Using while

Complex systems that cycle through distinct states are naturally expressed as while loops with an explicit state variable:

def manage_connection(host, port, max_retries=3):
    state = "DISCONNECTED"
    retries = 0
    conn = None

    while state != "DONE":
        if state == "DISCONNECTED":
            print("Attempting to connect...")
            state = "CONNECTING"

        elif state == "CONNECTING":
            try:
                conn = connect(host, port)
                state = "CONNECTED"
                retries = 0  # Reset on success
            except ConnectionError:
                retries += 1
                if retries >= max_retries:
                    state = "FAILED"
                else:
                    state = "DISCONNECTED"

        elif state == "CONNECTED":
            try:
                data = conn.read()
                process(data)
            except EOFError:
                conn.close()
                state = "DONE"
            except ConnectionError:
                conn = None
                state = "DISCONNECTED"

        elif state == "FAILED":
            raise RuntimeError(f"Failed to connect after {max_retries} attempts")

The while loop runs the state machine indefinitely. Each iteration handles the current state and transitions to the next. The exit condition is reaching "DONE" or raising an exception. This pattern is used in protocol parsers, game engines, workflow engines, and networking code.

Retry Loops with Exponential Backoff

When making network requests, database calls, or any operation that can fail transiently, a naive retry loop hammers the server:

# WRONG: retry immediately, flood the server
for attempt in range(max_retries):
    try:
        return make_request(url)
    except RequestError:
        continue  # Immediately retries - makes the problem worse

The professional pattern is exponential backoff with optional jitter. The wait time doubles with each attempt, preventing thundering herd problems where many clients retry simultaneously after a server recovers:

import time
import random

def fetch_with_backoff(url, max_attempts=5, base_delay=1.0):
    """
    Fetch URL with exponential backoff retry logic.

    Wait times: 1s, 2s, 4s, 8s before each retry.
    Final attempt raises if it also fails.
    """
    last_exception = None

    for attempt in range(max_attempts):
        try:
            response = make_request(url)
            return response  # Success - return immediately

        except TransientError as exc:
            last_exception = exc
            if attempt == max_attempts - 1:
                # Last attempt failed - do not sleep, just raise
                break

            # Exponential backoff: 2^attempt * base_delay seconds
            delay = (2 ** attempt) * base_delay

            # Add jitter: randomize by ±25% to prevent synchronized retries
            jitter = delay * random.uniform(-0.25, 0.25)
            sleep_time = max(0, delay + jitter)

            print(
                f"Attempt {attempt + 1} failed: {exc}. "
                f"Retrying in {sleep_time:.1f}s..."
            )
            time.sleep(sleep_time)

        except FatalError as exc:
            # Non-transient - do not retry
            raise

    raise last_exception

The delay schedule: attempt=0 → 1s, attempt=1 → 2s, attempt=2 → 4s, attempt=3 → 8s. With jitter, these become randomized intervals around those values, spreading out retries from multiple clients.

warning

Always distinguish transient errors (network timeout, temporary 503) from fatal errors (401 Unauthorized, 404 Not Found, malformed request). Retrying a 401 five times does not fix authentication. Only retry errors that could succeed on a subsequent attempt without changing anything else.

The Sentinel Value Pattern and the Walrus Operator

A sentinel is a special value that signals "stop processing." The classic sentinel pattern reads until a termination signal:

results = []
while True:
    line = input("Enter data (empty line to finish): ")
    if line == "":
        break
    results.append(line)

Python 3.8 introduced the walrus operator (:=), which assigns and evaluates in one expression. This enables a cleaner sentinel pattern - the assignment and the condition are unified:

results = []
while line := input("Enter data (empty line to finish): "):
    # Assigns input() result to 'line' and checks its truthiness
    # Loop exits when line is empty string (falsy)
    results.append(line)

The walrus operator is particularly powerful when reading from streams or files:

# Reading from a file chunk by chunk
import io

def process_stream(stream: io.RawIOBase, chunk_size: int = 4096):
    chunks_processed = 0
    while chunk := stream.read(chunk_size):
        # 'chunk' is assigned stream.read() - loop exits when read() returns b""
        process_chunk(chunk)
        chunks_processed += 1
    return chunks_processed

# Reading from a socket
def receive_messages(sock):
    messages = []
    while data := sock.recv(1024):
        messages.append(data.decode())
    return messages

The walrus operator also works with match operations:

import re

log_line = "ERROR 2024-01-15 Database connection failed"

if match := re.search(r"ERROR (.+)", log_line):
    # match is both assigned and evaluated in one step
    print(f"Found error: {match.group(1)}")

note

The walrus operator (:=) was added in Python 3.8. If you need to support Python 3.7 or earlier, use the traditional two-step pattern (assign then check). In Python 3.8+, the walrus operator in while conditions is idiomatic for stream-reading loops.

Infinite Loop Prevention

Every while loop must have a clear answer to the question: "What makes this condition eventually become False?" If you cannot answer that question, the loop is likely infinite.

There are three common patterns that guarantee termination:

1. A counter with a known bound:

attempts = 0
max_attempts = 10
while attempts < max_attempts:
    # ... do work ...
    attempts += 1  # ← This must always execute, every iteration

2. Consuming a finite resource:

while queue:       # queue shrinks each iteration
    queue.pop()

while data:        # data shrinks each iteration
    process(data.pop())

3. Narrowing a search space:

while left <= right:    # space shrinks every iteration
    mid = (left + right) // 2
    if target < sorted_list[mid]:
        right = mid - 1     # ← shrinks
    else:
        left = mid + 1      # ← shrinks

Common infinite loop bugs:

Forgetting to update the loop variable:

# BUG: i never changes
i = 0
while i < 5:
    print(i)
    # i += 1  ← missing

Update inside a conditional branch that does not always execute:

# BUG: if condition is False, i never increments
i = 0
while i < 5:
    if some_condition:
        i += 1
    do_work()
# If some_condition is always False, infinite loop

Floating-point equality comparison:

# BUG: floating-point arithmetic may never produce exactly 1.0
x = 0.0
while x != 1.0:
    x += 0.1    # 0.1 + 0.1 + ... ≠ 1.0 in floating point

# FIX: use inequality with a tolerance
while x < 1.0 - 1e-9:
    x += 0.1

Adding a safety bound is a practical defense when the termination logic is complex:

MAX_ITERATIONS = 100_000  # Safety net against logic bugs

iterations = 0
while complex_condition() and iterations < MAX_ITERATIONS:
    do_work()
    iterations += 1

if iterations >= MAX_ITERATIONS:
    raise RuntimeError("Loop did not terminate - likely a logic bug")

Input Validation Loops

The canonical pattern for prompting until the user provides valid input:

def get_valid_age():
    """Prompt repeatedly until the user enters a valid age."""
    while True:
        raw = input("Enter your age (1-120): ").strip()
        if not raw.isdigit():
            print("Please enter a whole number.")
            continue
        age = int(raw)
        if not (1 <= age <= 120):
            print(f"Age must be between 1 and 120, got {age}.")
            continue
        return age  # Valid - exit the loop and return

age = get_valid_age()
print(f"Age recorded: {age}")

The while True: combined with continue for invalid input and return for valid input is the clearest structure for this pattern. Each validation failure uses continue to restart the prompt. The return at the end acts as the implicit break - when the function returns, the loop is over.

For more complex validation, use a helper function that raises on failure (the guard clause pattern from the previous lesson):

def validate_username(name: str) -> str:
    if not name:
        raise ValueError("Username cannot be empty.")
    if len(name) < 3:
        raise ValueError("Username must be at least 3 characters.")
    if not name.isalnum():
        raise ValueError("Username may only contain letters and digits.")
    return name.lower()

def prompt_username():
    while True:
        raw = input("Choose a username: ").strip()
        try:
            return validate_username(raw)
        except ValueError as e:
            print(f"Invalid: {e}")

Converting Recursion to while Loops

Python has a default recursion limit of 1000 frames (sys.getrecursionlimit()). Deep recursive algorithms can exceed this limit and raise RecursionError. Converting recursion to an explicit while loop eliminates the stack depth limitation entirely.

The conversion requires making the implicit call stack explicit, usually with a Python list used as a stack:

Recursive factorial (hits limit at ~1000):

def factorial_recursive(n):
    if n <= 1:
        return 1
    return n * factorial_recursive(n - 1)

While loop factorial (no limit):

def factorial_iterative(n):
    if n < 0:
        raise ValueError("n must be non-negative")
    result = 1
    while n > 1:
        result *= n
        n -= 1
    return result

Recursive tree traversal converted to while with explicit stack:

# Recursive depth-first search - O(depth) stack frames
def dfs_recursive(node, target):
    if node is None:
        return False
    if node.value == target:
        return True
    return dfs_recursive(node.left, target) or dfs_recursive(node.right, target)

# Iterative DFS with explicit stack - no recursion limit
def dfs_iterative(root, target):
    if root is None:
        return False
    stack = [root]
    while stack:
        node = stack.pop()
        if node.value == target:
            return True
        if node.right:
            stack.append(node.right)
        if node.left:
            stack.append(node.left)
    return False

The pattern: each recursive call becomes a stack.append(). The while stack: loop replaces the recursive calls. The stack explicitly maintains what recursion maintains implicitly in the call stack.

tip

sys.setrecursionlimit(n) can increase the limit, but this is not a real solution - each call frame consumes real stack memory at the OS level. Deep recursion eventually causes a C-level stack overflow that Python cannot catch, crashing the interpreter entirely. For algorithms that can recurse thousands of levels deep, the while loop with explicit stack is always the safer design.

The else Clause on while Loops

Like for loops, while loops have an else clause that executes when the loop terminates naturally - when the condition becomes False - but does not execute if the loop is exited via break:

max_attempts = 3
attempts = 0

while attempts < max_attempts:
    password = input("Enter password: ")
    if password == "correct_password":
        print("Access granted")
        break
    attempts += 1
    print(f"Wrong password. {max_attempts - attempts} attempts remaining.")
else:
    # Runs only if the while condition became False (all attempts exhausted)
    # Does NOT run if break fired (correct password entered)
    print("Account locked after too many failed attempts.")

This is the cleanest expression of the retry-with-failure pattern: attempt the operation in the while body, break on success, and handle the exhaustion case in the else. No sentinel variable needed. The control flow directly expresses the intent.

Compare to the equivalent code without while/else:

# Without while/else - requires a flag variable
success = False
attempts = 0

while attempts < max_attempts:
    password = input("Enter password: ")
    if password == "correct_password":
        success = True
        break
    attempts += 1

if not success:
    print("Account locked after too many failed attempts.")

The while/else version eliminates the success flag. It is more concise and the structure directly communicates: "run the loop; if it ran to completion without breaking, do this."

Producing Sequences with while

Sometimes while is clearer than a generator when the sequence generation logic requires complex state:

Fibonacci sequence with while:

def fibonacci_up_to(limit):
    """Yield Fibonacci numbers up to limit."""
    a, b = 0, 1
    while a <= limit:
        yield a
        a, b = b, a + b

for n in fibonacci_up_to(100):
    print(n)
# 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89

Collatz sequence:

def collatz(n):
    """Generate the Collatz sequence starting from n."""
    while n != 1:
        yield n
        if n % 2 == 0:
            n //= 2
        else:
            n = 3 * n + 1
    yield 1  # Final value

print(list(collatz(6)))  # [6, 3, 10, 5, 16, 8, 4, 2, 1]

The Collatz sequence has no known closed-form termination condition - it always reaches 1 for all tested values, but this is unproven mathematically. The while n != 1 loop expresses the algorithm exactly as stated: keep going until you reach 1.

Pitfalls

Pitfall 1: Off-by-one in the condition.

# Intended: process items at indices 0, 1, 2, 3, 4
i = 0
while i < 5:    # ← correct: < 5 gives 0,1,2,3,4
    process(items[i])
    i += 1

# Bug: <= 5 processes index 5, causing IndexError on a 5-element list
while i <= 5:   # ← off-by-one: also processes index 5
    process(items[i])
    i += 1

Pitfall 2: Forgetting to update state in all code paths.

while total < target:
    item = get_next_item()
    if item.value > 0:
        total += item.value
        # If item.value <= 0, total never changes - potential infinite loop

Fix: ensure every iteration moves the state toward termination, or add an explicit fallback:

while total < target:
    item = get_next_item()
    if item is None:
        break  # Explicit exit if source is exhausted
    total += max(0, item.value)  # Ensure total always moves forward

Pitfall 3: Using while where for is cleaner.

# WRONG: while used to iterate a known sequence
i = 0
while i < len(items):
    print(items[i])
    i += 1

# CORRECT: use for
for item in items:
    print(item)

If you know the iterable at loop entry, for is always clearer. while with an explicit index is harder to read and easier to get wrong.

Pitfall 4: Modifying the termination condition variable inside the loop incorrectly.

# Intended: count down from 10 to 1
count = 10
while count > 0:
    print(count)
    count -= 1
    if some_condition:
        count = 10  # Reset - but this can cause very long running loops

Be explicit about when resets can happen and ensure the overall loop still terminates.

Interview Questions and Detailed Answers

Q1: When should you use while instead of for?

Use while when the number of iterations is not known in advance and depends on a condition that changes during execution. Examples include: reading from a network socket until the connection closes, retrying an operation until it succeeds, processing a queue until it is empty, advancing through algorithm state (binary search, Collatz sequence), and event loops that run until a shutdown signal. Use for when iterating over a known iterable - a list, range, dict, file - where the collection determines the iteration count. The practical test: if you can write for x in something: without awkward range(len(...)) workarounds, use for. If the termination is conditioned on computed state, use while.

Q2: What is the while True: pattern and when is it appropriate?

while True: creates an intentionally infinite loop that can only exit via break, return, or an unhandled exception. It is appropriate when the exit condition is naturally determined inside the loop body rather than at the loop header - for example, when you must execute the body at least once before knowing whether to continue, or when the exit condition requires processing that happens mid-body. It is the canonical pattern for event loops (while True: event = get_event(); handle(event)), input validation loops, and retry loops. It is NOT appropriate when a finite bound is known and could be expressed as a while condition or for loop - leaving the bound implicit inside the body rather than explicit in the condition makes infinite-loop bugs harder to detect.

Q3: What is exponential backoff and why does adding jitter matter?

Exponential backoff is a retry strategy where the wait time between attempts doubles with each failure: 1s, 2s, 4s, 8s, etc. This prevents the client from continuously hammering a struggling server. However, if many clients all start retrying at the same time (for example, after a server restart), they all experience the same backoff schedule and attempt to reconnect simultaneously on each step - this is the thundering herd problem. Jitter adds a random offset to each wait time, distributing the retry attempts across a time window. A typical implementation is delay = (2 ** attempt) * base + random.uniform(-base * 0.25, base * 0.25). The jitter ensures that even if thousands of clients all start retrying at the same moment, their actual retry times are spread across several seconds rather than synchronized.

Q4: Explain the walrus operator in a while loop. What problem does it solve?

The walrus operator (:=) assigns a value and evaluates it as an expression in one step. In while loops, it solves the pattern where you need to assign a value and immediately test it as the loop condition. The traditional approach requires either repeating the assignment or using while True: with an internal check: data = stream.read(); while data: process(data); data = stream.read(). The walrus operator collapses this to while data := stream.read(): process(data), which assigns the read result to data, evaluates it for truthiness, and uses it in the loop body - all in one expression. This eliminates the duplicated read call and makes the sentinel condition explicit in the while header rather than hidden in an internal if.

Q5: How do you prevent infinite loops in production code?

Several techniques work together. First, always identify the termination guarantee: what variable or condition changes each iteration, and why does it eventually satisfy the exit condition? If you cannot answer this, redesign the loop. Second, add a maximum iteration count as a safety bound for complex loops where the termination logic is non-trivial: while condition and iterations < MAX; iterations += 1. Third, distinguish transient from fatal conditions in retry loops - never retry errors that cannot recover. Fourth, use timeouts for loops that depend on external systems: while not done and time.time() < deadline:. Fifth, in testing, be suspicious of any while loop that does not finish within an expected time bound - add logging to track iteration counts.

Q6: How do you convert a recursive function to a while loop and why would you?

The motivation is Python's recursion limit (default 1000 frames). For algorithms that can recurse thousands of levels deep - DFS on deep graphs, large Fibonacci sequences, deeply recursive parsers - the while loop version avoids RecursionError. The conversion makes the implicit call stack explicit: create a Python list as a stack, initialize it with the starting state, and use while stack: instead of recursive calls. When a recursive function would call itself, the while version does stack.append(new_state). When a recursive function would return, the while version does stack.pop(). For accumulation (like factorial), track the accumulator in a local variable. The key insight: all information that recursion maintains implicitly in call frames must be stored explicitly in the stack list.

Graded Practice Challenges

Level 1 - Predict the Output

x = 16
steps = 0

while x != 1:
    if x % 2 == 0:
        x //= 2
    else:
        x = 3 * x + 1
    steps += 1

print(x, steps)

# Second loop
n = 0
while n < 3:
    n += 1
else:
    print(f"Finished at n={n}")

# Third loop
n = 5
while n < 3:
    n += 1
else:
    print(f"Third loop else, n={n}")

What is the complete output?

Show Answer

1 4
Finished at n=3
Third loop else, n=5

First loop (Collatz from 16):

• x=16 (even): x = 8, steps=1
• x=8 (even): x = 4, steps=2
• x=4 (even): x = 2, steps=3
• x=2 (even): x = 1, steps=4
• x=1: condition x != 1 is False, exit
• Prints: 1 4

Second loop:

• n starts at 0, increments to 3 over 3 iterations. Condition n < 3 becomes False. else executes because no break. Prints: Finished at n=3

Third loop:

• n=5. Condition n < 3 is immediately False - the body never executes. The else still executes because the loop terminated without break (it ran zero times, which counts as natural termination). Prints: Third loop else, n=5

Key insight: the else clause runs whenever the loop exits via condition becoming False - even if the condition was False from the very start (zero iterations).

Level 2 - Debug the Retry Loop

This retry function has three bugs. Find and fix all of them.

import time

def fetch_data(url, max_retries=3):
    attempt = 1
    while attempt <= max_retries:
        try:
            response = make_request(url)
            return response
        except Exception:
            delay = 2 ** attempt
            print(f"Attempt {attempt} failed. Retrying in {delay}s")
            time.sleep(delay)
    raise RuntimeError(f"Failed after {max_retries} retries")

Show Answer

Bug 1: attempt is never incremented. The while loop condition is attempt <= max_retries, but attempt is never increased inside the loop body. This creates an infinite loop on the first failure - it will retry forever.

Bug 2: All exceptions are caught, including fatal ones. except Exception catches every exception including those that should not be retried (e.g., ValueError for a malformed URL, TypeError from a programming error). Only transient, retryable errors should trigger a retry. In a real implementation, you would catch specific exception types like ConnectionError or TimeoutError.

Bug 3: The raise after the while loop is unreachable. Because attempt is never incremented, either the request succeeds (return) or it loops forever. Even if we fix Bug 1, the raise after the loop would work correctly - but as written, the loop never exits naturally, so the raise is dead code. Once Bug 1 is fixed, the raise becomes reachable and correct.

Fixed function:

import time

def fetch_data(url, max_retries=3, base_delay=1.0):
    last_error = None

    for attempt in range(max_retries):  # Using for - iteration count is known
        try:
            response = make_request(url)
            return response

        except (ConnectionError, TimeoutError) as exc:
            # Only retry transient network errors
            last_error = exc
            if attempt == max_retries - 1:
                break  # Last attempt - don't sleep, just fall through to raise

            delay = (2 ** attempt) * base_delay
            print(f"Attempt {attempt + 1} failed: {exc}. Retrying in {delay:.1f}s")
            time.sleep(delay)

        except ValueError:
            # Non-transient - malformed URL, don't retry
            raise

    raise RuntimeError(
        f"Failed after {max_retries} attempts. Last error: {last_error}"
    )

Note: a fixed iteration count makes for more appropriate here than while, since the number of attempts is known upfront.

Level 3 - Design a Connection Retry System with Exponential Backoff and Jitter

Design a ConnectionManager class that manages a connection to an external service with robust retry logic. Requirements:

1. connect(host, port): attempts to connect. Uses exponential backoff with jitter. The base delay starts at 0.5 seconds and caps at 30 seconds. Gives up after max_attempts (default 5) and raises ConnectionError.
2. send(data): sends data on the active connection. If the connection dropped, automatically attempts one reconnect before raising.
3. The class tracks connection state (DISCONNECTED, CONNECTED, FAILED) using a while loop for the connection attempt cycle.
4. All retry attempts are logged with the attempt number, error, and calculated delay.

Implement with a simulated _make_connection method that fails the first two times and succeeds on the third.

Show Answer

import time
import random
import logging

logging.basicConfig(level=logging.INFO, format="%(message)s")
logger = logging.getLogger(__name__)


class SimulatedSocket:
    """A mock socket that works after being instantiated."""
    def send(self, data):
        return len(data)
    def close(self):
        pass


class ConnectionManager:
    """
    Manages a connection with automatic exponential backoff retry.

    States: DISCONNECTED → (CONNECTING attempts) → CONNECTED or FAILED
    """

    MAX_DELAY = 30.0  # Cap backoff at 30 seconds

    def __init__(self, max_attempts: int = 5, base_delay: float = 0.5):
        self.max_attempts = max_attempts
        self.base_delay = base_delay
        self._state = "DISCONNECTED"
        self._conn = None
        self._host = None
        self._port = None
        self._simulated_call_count = 0  # For simulation only

    def _make_connection(self, host: str, port: int):
        """Simulated connection: fails first 2 times, succeeds on 3rd."""
        self._simulated_call_count += 1
        if self._simulated_call_count <= 2:
            raise ConnectionRefusedError(
                f"Connection refused (simulated failure #{self._simulated_call_count})"
            )
        return SimulatedSocket()

    def _calculate_delay(self, attempt: int) -> float:
        """Exponential backoff with ±25% jitter, capped at MAX_DELAY."""
        raw_delay = (2 ** attempt) * self.base_delay
        capped_delay = min(raw_delay, self.MAX_DELAY)
        jitter = capped_delay * random.uniform(-0.25, 0.25)
        return max(0.0, capped_delay + jitter)

    def connect(self, host: str, port: int) -> None:
        """
        Establish a connection with exponential backoff retry.
        Raises ConnectionError if all attempts are exhausted.
        """
        self._host = host
        self._port = port
        self._state = "DISCONNECTED"
        last_error = None
        attempt = 0

        while attempt < self.max_attempts:
            self._state = "CONNECTING"
            logger.info(f"Connection attempt {attempt + 1}/{self.max_attempts}...")

            try:
                self._conn = self._make_connection(host, port)
                self._state = "CONNECTED"
                logger.info(f"Connected to {host}:{port} on attempt {attempt + 1}")
                return  # Success - exit the retry loop

            except (ConnectionRefusedError, TimeoutError, OSError) as exc:
                last_error = exc
                attempt += 1

                if attempt >= self.max_attempts:
                    # Exhausted all attempts
                    break

                delay = self._calculate_delay(attempt - 1)
                logger.warning(
                    f"Attempt {attempt} failed: {exc}. "
                    f"Retrying in {delay:.2f}s..."
                )
                time.sleep(delay)

        # All attempts exhausted
        self._state = "FAILED"
        raise ConnectionError(
            f"Could not connect to {host}:{port} after {self.max_attempts} "
            f"attempts. Last error: {last_error}"
        )

    def send(self, data: bytes) -> int:
        """
        Send data. Automatically reconnects once if disconnected.
        """
        if self._state != "CONNECTED":
            if self._host is None:
                raise RuntimeError("No connection has been established. Call connect() first.")
            logger.warning("Connection lost. Attempting reconnect before send...")
            # Reset simulation counter for reconnect demonstration
            self._simulated_call_count = 2  # Will succeed on next attempt
            self.connect(self._host, self._port)

        try:
            sent = self._conn.send(data)
            logger.info(f"Sent {sent} bytes")
            return sent
        except OSError as exc:
            self._state = "DISCONNECTED"
            self._conn = None
            raise ConnectionError(f"Send failed: {exc}") from exc

    def disconnect(self) -> None:
        """Close the connection gracefully."""
        if self._conn is not None:
            self._conn.close()
            self._conn = None
        self._state = "DISCONNECTED"
        logger.info("Disconnected.")

    @property
    def state(self) -> str:
        return self._state


# Demonstration
if __name__ == "__main__":
    manager = ConnectionManager(max_attempts=5, base_delay=0.1)

    # Connect - will fail twice, succeed on third attempt
    manager.connect("api.example.com", 443)
    print(f"State after connect: {manager.state}")  # CONNECTED

    # Send data
    bytes_sent = manager.send(b"GET /api/data HTTP/1.1\r\n\r\n")
    print(f"Bytes sent: {bytes_sent}")

    # Simulate disconnect and auto-reconnect
    manager._state = "DISCONNECTED"
    manager._conn = None
    bytes_sent = manager.send(b"GET /api/more HTTP/1.1\r\n\r\n")
    print(f"Bytes sent after auto-reconnect: {bytes_sent}")

    manager.disconnect()
    print(f"Final state: {manager.state}")  # DISCONNECTED

Design highlights:

• State tracking: explicit _state variable makes the current connection status always queryable.
• Attempt counter as while condition: while attempt < self.max_attempts gives a clear, provable termination guarantee - the loop runs at most max_attempts times.
• Delay calculation separated: _calculate_delay is a pure function - easy to test, easy to modify the backoff formula.
• Jitter: prevents thundering herd from multiple ConnectionManager instances retrying simultaneously.
• Cap: delays never exceed MAX_DELAY regardless of attempt count.
• Fatal vs transient distinction: only ConnectionRefusedError, TimeoutError, and OSError trigger retries. A ValueError (bad host format) would propagate immediately.
• Auto-reconnect on send: the send method handles the common case where a connection drops between sends, attempting one reconnect before failing.

Quick Reference Cheatsheet

Pattern	Use Case	Code Shape
Basic while	Condition-driven iteration	while condition: body; update
while True + break	Exit condition known only inside body	while True: ... if done: break
State machine	Complex state transitions	while state != "DONE": if state == X: ...
Retry with backoff	Resilient network/IO operations	while attempts < max: try; except; sleep(2**n)
Walrus operator sentinel	Stream reading	while data := stream.read(): process(data)
Input validation	Prompt until valid	while True: try: validate; break; except: print
Queue drain	Process until empty	while queue: item = queue.pop(); process(item)
Recursion → while	Stack-safe deep iteration	stack=[start]; while stack: node=stack.pop()
while/else	Detect loop exhaustion	while cond: ... break; else: handle_no_break
Safety bound	Guard against infinite loop bug	while cond and iters < MAX: ...; iters += 1

Key Takeaways

• while is for condition-driven iteration where the number of repetitions depends on runtime state; for is for sequence-driven iteration over a known iterable.
• The while loop evaluates its condition before every iteration, including the first - a false condition at entry means zero executions.
• while True: with an internal break is idiomatic and correct when the exit condition is naturally determined inside the loop body; it eliminates the need for a flag variable.
• Every while loop must have an identifiable termination guarantee - a variable that changes, a resource that shrinks, or a search space that narrows each iteration.
• Exponential backoff with jitter is the production standard for retry loops: double the delay each attempt, add randomness to spread simultaneous retries, and cap the maximum delay.
• The walrus operator (:=) in while conditions unifies assignment and testing in one expression, eliminating duplicated calls in stream-reading sentinel patterns.
• The while/else clause runs when the loop condition becomes False but not when break fires - this is the cleanest way to detect whether a retry loop exhausted all attempts.
• Converting deep recursion to an iterative while loop with an explicit stack eliminates Python's recursion limit and prevents C-level stack overflows.
• The three most common while loop bugs are: forgetting to update the state variable, updating state only in conditional branches that may not execute, and using floating-point equality in the condition.
• If you find yourself writing while i < len(items): ... i += 1, refactor to a for loop - the while version is harder to read and easier to get wrong.