try / except / else / finally - Python's Exception Handling Syntax
Reading time: ~17 minutes | Level: Foundation → Engineering
Here is behavior that surprises most Python developers:
def what_returns():
try:
return "from try"
finally:
return "from finally"
print(what_returns())
Output:
from finally
The return "from try" was overridden. finally always runs - even when there is a return inside try. If finally itself returns a value, that value wins.
And there is another clause most developers forget exists:
def read_number(text):
try:
value = int(text)
except ValueError:
print("Not a valid integer")
return None
else:
# Only runs if NO exception was raised in try
print(f"Successfully parsed: {value}")
return value
read_number("42") # Successfully parsed: 42
read_number("abc") # Not a valid integer
The else clause runs only when the try block completed without raising. It is not the same as putting code after the try/except block - and understanding the difference is a mark of production-level Python.
What You Will Learn
- The full syntax: all four clauses (
try,except,else,finally) and their execution semantics - The
elseclause: what it does that putting code aftertry/exceptdoes not - The
finallyguarantee: why it runs even withreturn,break, orcontinue - Multiple
exceptclauses and why order matters - Catching multiple exceptions in a single clause
- The
asbinding and Python 3's variable scoping quirk inexceptblocks - EAFP vs LBYL: two philosophies for handling errors
- Performance characteristics: no overhead when no exception is raised
- Context managers as the cleaner alternative to
try/finallyfor resource cleanup - Real-world patterns: retry logic, circuit breakers, resource cleanup
Prerequisites
- Understanding of Python exceptions as objects (see: Exceptions Explained)
- Python functions, return values, and control flow
- Basic understanding of Python classes (for the context manager section)
The Full Syntax
| Clause | When It Runs |
|---|---|
try: | Always - contains code that might raise |
except SomeException as e: | If SomeException (or a subclass) was raised in try |
except (TypeError, ValueError) as e: | If either TypeError or ValueError was raised |
except Exception as e: | Catch-all for all remaining Exception subclasses |
else: | Only if try completed with no exception |
finally: | Always - with or without exception, with or without return/break/continue |
Not all clauses are required. The minimum is try plus at least one of except or finally.
Part 1 - The try and except Clauses
Basic Usage
def safe_divide(a, b):
try:
result = a / b
except ZeroDivisionError:
print("Cannot divide by zero")
return None
return result
print(safe_divide(10, 2)) # 5.0
print(safe_divide(10, 0)) # Cannot divide by zero → None
Multiple except Clauses
You can have multiple except clauses to handle different exceptions differently:
def parse_and_lookup(data, key):
try:
# Could raise TypeError if data is not subscriptable
# Could raise KeyError if key is not in data
# Could raise ValueError if the value cannot be converted
value = int(data[key])
return value
except TypeError:
print(f"data must be a dict-like object, got {type(data).__name__}")
return None
except KeyError:
print(f"Key {key!r} not found in data")
return None
except ValueError as e:
print(f"Value at {key!r} is not an integer: {e}")
return None
parse_and_lookup({"count": "42"}, "count") # 42
parse_and_lookup({"count": "abc"}, "count") # Value at 'count' is not an integer
parse_and_lookup({"count": "42"}, "total") # Key 'total' not found in data
parse_and_lookup(None, "count") # data must be dict-like, got NoneType
Order Matters: Most Specific First
Python checks except clauses in order and executes the first match. Because exceptions use class inheritance, a parent class handler will catch all subclasses - so always put more specific exceptions before more general ones:
# Wrong order - the broad handler catches everything first
try:
open("/nonexistent")
except OSError: # matches FileNotFoundError (it's a subclass)
print("Some OS error") # this always runs
except FileNotFoundError: # never reached!
print("File not found")
# Correct order - specific before general
try:
open("/nonexistent")
except FileNotFoundError: # checked first, matches
print("File not found")
except OSError: # catches other OS errors
print("Some other OS error")
:::warning Order Your Except Clauses Most-Specific First
Python does not warn you if a more specific except clause is unreachable because a broader clause above it catches everything. The flake8 linter with the flake8-bugbear plugin can catch this, but you need to know the exception hierarchy to write correct ordering.
:::
Catching Multiple Exceptions in One Clause
When two exceptions require the same handling, group them in a tuple:
def fetch_value(source, key):
try:
return source[key]
except (KeyError, IndexError) as e:
# Both mean "that key/index does not exist"
print(f"Key/index not found: {e}")
return None
fetch_value({"a": 1}, "b") # Key/index not found: 'b'
fetch_value([1, 2, 3], 10) # Key/index not found: list index out of range
Part 2 - Binding with as
The as e syntax binds the exception object to the name e for the duration of the except block:
try:
result = 1 / 0
except ZeroDivisionError as e:
print(type(e)) # <class 'ZeroDivisionError'>
print(e.args) # ('division by zero',)
print(str(e)) # division by zero
print(repr(e)) # ZeroDivisionError('division by zero')
# Log it, chain it, inspect it - e is a full object
import logging
logging.error("Division failed: %s", e)
Python 3's Scoping Quirk: e Is Deleted After the Block
This surprises developers coming from other languages:
try:
raise ValueError("test")
except ValueError as e:
message = str(e) # Save it before e disappears
print(f"Inside except: e = {e}") # Works
# After the except block, e is deleted from the local namespace
try:
print(e) # NameError: name 'e' is not defined
except NameError:
print("e was deleted after the except block") # This runs
print(message) # Works - we saved the string before e was deleted
Why does Python delete it? Because exception objects hold a reference to the traceback, and the traceback holds references to frame locals. If e persisted, you would have a reference cycle that delays garbage collection.
:::tip Always Save What You Need from an Exception
If you need the exception message or attributes after the except block, save them to a differently-named variable inside the block: message = str(e), code = e.status_code, etc.
:::
Part 3 - The else Clause
The else clause is the most overlooked part of Python's exception syntax.
def load_config(path):
try:
f = open(path)
except FileNotFoundError:
print(f"Config file not found: {path}")
return {}
else:
# Only runs if open() succeeded - no exception from try
config = f.read()
f.close()
return config
else vs Putting Code After try/except
These two look similar but behave differently:
# Version A: code in else
try:
value = risky_parse(data)
except ParseError as e:
handle_error(e)
else:
process(value) # Only runs if risky_parse succeeded
# Version B: code after the block
try:
value = risky_parse(data)
except ParseError as e:
handle_error(e)
process(value) # Runs even if handle_error raised, or if value is undefined!
In Version B, process(value) runs even if handle_error(e) raised a new exception (it would propagate). It also runs after the except block executes, but value might not be defined if risky_parse raised before assigning. In Version A, else only executes if the try block completed normally - it gives you a clean guarantee.
The else clause also makes code clearer: it separates "the operation that might fail" (in try) from "what to do when it succeeds" (in else).
# Best practice: use else to separate failure handling from success path
def connect_and_query(host, query):
try:
conn = db.connect(host)
except ConnectionError as e:
logger.error("Database connection failed: %s", e)
raise
else:
# We only reach here if connect succeeded
try:
return conn.execute(query)
finally:
conn.close()
Part 4 - The finally Clause
finally runs no matter what: whether the try block succeeded, whether an exception was raised and caught, whether an exception was raised and not caught, and even if return, break, or continue is used inside try or except.
finally and Normal Execution
def open_file(path):
try:
f = open(path)
return f.read()
except FileNotFoundError:
return ""
finally:
print("finally always runs")
# Note: f might not be defined if open() failed
open_file("/tmp/test.txt")
# Output:
# finally always runs
finally Overrides return
def tricky():
try:
print("in try")
return 1
finally:
print("in finally")
return 2 # This overrides the return 1
print(tricky())
# Output:
# in try
# in finally
# 2 ← finally's return wins
:::danger Don't Return from finally
A return in finally silently discards any exception that was in flight. This can suppress errors without any warning. Only use finally for cleanup, never for returning values or suppressing exceptions.
:::
finally with an Unhandled Exception
def process():
try:
raise RuntimeError("fatal error")
finally:
print("cleanup runs even without a handler")
# Resources are released here before the exception propagates
try:
process()
except RuntimeError as e:
print(f"Caught after cleanup: {e}")
Output:
cleanup runs even without a handler
Caught after cleanup: fatal error
The finally block runs, then the exception continues propagating to the outer handler.
The Resource Cleanup Pattern
import sqlite3
def run_query(db_path, query):
conn = None
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
cursor.execute(query)
return cursor.fetchall()
except sqlite3.OperationalError as e:
print(f"Query failed: {e}")
return []
finally:
if conn is not None:
conn.close() # Guaranteed to run - connection is always closed
The if conn is not None check handles the case where sqlite3.connect() itself failed (before conn was assigned), which would leave conn = None.
Part 5 - Execution Order: The Full Picture
Understanding exactly when each clause runs is essential for writing correct code:
| Scenario | Execution Order | Notes |
|---|---|---|
| No exception raised | try → else → finally | The happy path |
| Exception raised, handler matches | try (partial) → except → finally | else does NOT run |
| Exception raised, no handler matches | try (partial) → finally → exception propagates | else does NOT run |
Exception raised in except block | try → except (partial) → finally → new exception propagates | |
Exception raised in else block | try → else (partial) → finally → exception propagates | NOT caught by the except clauses above |
Scenario 5 is a common gotcha: exceptions in the else block are not caught by the except clauses of the same try statement:
try:
x = 1
except ValueError:
print("caught ValueError")
else:
raise ValueError("from else") # NOT caught by the except above!
# This ValueError propagates upward
This is by design - the except clauses guard the try block, not the else block.
Part 6 - EAFP vs LBYL
Python culture has two distinct approaches to handling potential errors. Understanding both, and knowing when to use each, is a hallmark of experienced Python developers.
LBYL: Look Before You Leap
Check conditions before performing an operation:
# LBYL style
import os
import json
def read_config(path):
if not os.path.exists(path):
return {}
if not os.access(path, os.R_OK):
raise PermissionError(f"Cannot read {path}")
with open(path) as f:
return json.load(f)
Pros: Explicit about preconditions; can give more specific error messages.
Cons: Race conditions (the file might be deleted between the check and the open); verbose; sometimes the check itself can fail.
EAFP: Easier to Ask Forgiveness than Permission
Try the operation and handle the exception if it fails:
# EAFP style
import json
def read_config(path):
try:
with open(path) as f:
return json.load(f)
except FileNotFoundError:
return {}
except PermissionError as e:
raise PermissionError(f"Cannot read config: {e}") from e
Pros: No race conditions; cleaner code for the happy path; matches Python's idioms.
Cons: Can be less readable if many different exceptions need different handling.
When to Use Each
| Style | Use When |
|---|---|
| EAFP | The failure case is rare (e.g., file usually exists) |
| EAFP | Atomic operations where checks and actions cannot be split safely |
| EAFP | Working with external resources (files, network, databases) |
| EAFP | Idiomatic Python - the community expects EAFP in many contexts |
| LBYL | The check is cheap and the operation is expensive |
| LBYL | Validation logic is complex and clearer as explicit conditions |
| LBYL | You need to validate before allocating expensive resources |
| LBYL | Working with function arguments (validate inputs at entry point) |
# Idiomatic Python EAFP: dict lookup
user = {"name": "Alice", "role": "admin"}
# LBYL - unnecessarily verbose
if "email" in user:
email = user["email"]
else:
email = "unknown"
# EAFP - Pythonic
try:
email = user["email"]
except KeyError:
email = "unknown"
# Even more Pythonic: use dict.get() for simple defaults
email = user.get("email", "unknown")
Part 7 - Performance Characteristics
A common misconception is that try/except adds significant overhead. The reality is nuanced:
| Condition | Performance Impact | Recommendation |
|---|---|---|
| No exception raised | ~0 overhead (a few bytecode instructions; except never executes) | Use try/except freely when the happy path is common |
| Exception IS raised | Stack unwinding + exception object creation (memory + traceback build) - more expensive than a branch check | Avoid using exceptions for normal flow control |
| Exceptions are rare (network, file, DB) | EAFP is fast | Prefer EAFP style |
| Failure case is common (user input) | Exception overhead adds up | Prefer LBYL style |
# Bad: using exceptions for control flow when failure is common
def find_item_bad(items, target):
"""Returns index of target, -1 if not found."""
try:
return items.index(target)
except ValueError:
return -1 # This path is common - exception overhead adds up
# Better: use the LBYL approach when failure is equally likely
def find_item_good(items, target):
if target in items:
return items.index(target)
return -1
Part 8 - Context Managers as Cleaner try/finally
The with statement is Python's built-in mechanism for guaranteed cleanup. It is equivalent to try/finally but more readable:
# Verbose try/finally approach
conn = None
try:
conn = database.connect()
result = conn.query("SELECT * FROM users")
return result
finally:
if conn is not None:
conn.close()
# Clean with statement - identical behavior
with database.connect() as conn:
result = conn.query("SELECT * FROM users")
return result
# conn.close() is called automatically, even if an exception occurs
The with statement calls __enter__ on entry and __exit__ on exit (normal or exceptional). If __exit__ returns a truthy value, exceptions are suppressed - which is how contextlib.suppress works:
from contextlib import suppress
import os
# Suppress FileNotFoundError silently - only do this when you mean it
with suppress(FileNotFoundError):
os.remove("/tmp/old_file.txt")
Writing Your Own Context Manager
from contextlib import contextmanager
import time
@contextmanager
def timed_operation(name):
"""Context manager that times a block of code."""
start = time.perf_counter()
try:
yield # Code inside the with block runs here
except Exception as e:
elapsed = time.perf_counter() - start
print(f"{name} FAILED after {elapsed:.3f}s: {e}")
raise # Re-raise - don't suppress the exception
else:
elapsed = time.perf_counter() - start
print(f"{name} completed in {elapsed:.3f}s")
with timed_operation("database query"):
time.sleep(0.1) # Simulate work
# Output: database query completed in 0.100s
Part 9 - Real-World Patterns
Pattern 1: Retry Logic with Exponential Backoff
import time
import logging
logger = logging.getLogger(__name__)
def retry(max_attempts=3, delay=1.0, backoff=2.0, exceptions=(Exception,)):
"""Decorator that retries a function on specified exceptions."""
def decorator(func):
def wrapper(*args, **kwargs):
current_delay = delay
last_exc = None
for attempt in range(1, max_attempts + 1):
try:
return func(*args, **kwargs)
except exceptions as e:
last_exc = e
if attempt == max_attempts:
logger.error(
"%s failed after %d attempts: %s",
func.__name__, max_attempts, e
)
raise
logger.warning(
"%s attempt %d/%d failed: %s. Retrying in %.1fs",
func.__name__, attempt, max_attempts, e, current_delay
)
time.sleep(current_delay)
current_delay *= backoff
return wrapper
return decorator
@retry(max_attempts=3, delay=0.5, exceptions=(ConnectionError, TimeoutError))
def fetch_user(user_id):
# Simulate an unstable network call
import random
if random.random() < 0.7:
raise ConnectionError("temporary network failure")
return {"id": user_id, "name": "Alice"}
# Usage
try:
user = fetch_user(42)
print(user)
except (ConnectionError, TimeoutError):
print("All retries exhausted")
Pattern 2: Resource Cleanup with Multiple Resources
# Verbose but explicit try/finally for multiple resources
def process_files(input_path, output_path, log_path):
"""Process input, write output, log results. Guarantee all files are closed."""
input_file = None
output_file = None
log_file = None
try:
input_file = open(input_path, "r")
output_file = open(output_path, "w")
log_file = open(log_path, "a")
for line in input_file:
processed = line.strip().upper()
output_file.write(processed + "\n")
log_file.write(f"Processed: {line!r} -> {processed!r}\n")
except FileNotFoundError as e:
print(f"File not found: {e}")
raise
except PermissionError as e:
print(f"Permission denied: {e}")
raise
finally:
# Guaranteed cleanup - each close is independent
for f in [input_file, output_file, log_file]:
if f is not None:
try:
f.close()
except Exception:
pass # Best effort on cleanup
# Cleaner with contextlib.ExitStack:
from contextlib import ExitStack
def process_files_clean(input_path, output_path, log_path):
with ExitStack() as stack:
input_file = stack.enter_context(open(input_path, "r"))
output_file = stack.enter_context(open(output_path, "w"))
log_file = stack.enter_context(open(log_path, "a"))
for line in input_file:
processed = line.strip().upper()
output_file.write(processed + "\n")
log_file.write(f"Processed: {line!r} -> {processed!r}\n")
Pattern 3: Circuit Breaker
import time
from enum import Enum
class CircuitState(Enum):
CLOSED = "closed" # Normal - requests pass through
OPEN = "open" # Failing - requests are rejected immediately
HALF_OPEN = "half_open" # Testing - one request allowed through
class CircuitBreaker:
"""Stops sending requests to a failing service after too many failures."""
def __init__(self, failure_threshold=5, recovery_timeout=30.0):
self.failure_threshold = failure_threshold
self.recovery_timeout = recovery_timeout
self.failure_count = 0
self.last_failure_time = None
self.state = CircuitState.CLOSED
def call(self, func, *args, **kwargs):
if self.state == CircuitState.OPEN:
if time.time() - self.last_failure_time > self.recovery_timeout:
self.state = CircuitState.HALF_OPEN
else:
raise RuntimeError("Circuit breaker OPEN - service unavailable")
try:
result = func(*args, **kwargs)
self._on_success()
return result
except Exception:
self._on_failure()
raise
def _on_success(self):
self.failure_count = 0
self.state = CircuitState.CLOSED
def _on_failure(self):
self.failure_count += 1
self.last_failure_time = time.time()
if self.failure_count >= self.failure_threshold:
self.state = CircuitState.OPEN
print(f"Circuit breaker OPENED after {self.failure_count} failures")
# Usage
cb = CircuitBreaker(failure_threshold=3)
def unreliable_api_call():
raise ConnectionError("service down")
for i in range(5):
try:
cb.call(unreliable_api_call)
except (ConnectionError, RuntimeError) as e:
print(f"Request {i+1}: {type(e).__name__}: {e}")
Output:
Request 1: ConnectionError: service down
Request 2: ConnectionError: service down
Request 3: ConnectionError: service down
Circuit breaker OPENED after 3 failures
Request 4: RuntimeError: Circuit breaker OPEN - service unavailable
Request 5: RuntimeError: Circuit breaker OPEN - service unavailable
Interview Questions
Q1: What is the else clause in a try/except block, and when does it run?
Answer: The else clause runs only when the try block completes without raising any exception. It does not run if an exception was raised and caught by an except clause, and it does not run if an exception was raised and not caught. Putting code in else rather than at the end of try has two benefits: first, exceptions raised in else are not caught by the except clauses of the same try statement (making intent clear); second, it makes the code structure express "this is what I do when the operation succeeds" vs "this is what I do when it fails."
Q2: What happens when a return statement is inside a finally block?
Answer: The return in finally overrides any return in the try or except block. More dangerously, it also suppresses any exception that was in flight. If an exception was raised in try, normally it would propagate after finally runs - but if finally executes a return, the exception is silently discarded and the caller receives the finally return value as if nothing went wrong. This is almost always a bug. The rule is: never use return, break, or continue in a finally block.
Q3: Why is the as e variable deleted after the except block in Python 3?
Answer: Python 3 explicitly deletes the exception binding (e) at the end of the except block to break a reference cycle. Exception objects hold a reference to their traceback, and tracebacks hold references to the frame's local variables. If e persisted after the except block, you would have a cycle: e references the exception, which references the traceback, which references the frame, which contains e. This cycle would prevent the garbage collector from reclaiming the frame promptly. Python 3 breaks this by deleting the name from the local namespace when the except block exits. Save what you need before the block ends: message = str(e).
Q4: What is the difference between EAFP and LBYL?
Answer: LBYL (Look Before You Leap) checks preconditions before performing an operation: if os.path.exists(path): open(path). EAFP (Easier to Ask Forgiveness than Permission) attempts the operation and handles the exception if it fails: try: open(path) except FileNotFoundError: .... Python culture prefers EAFP for external resources (files, network, databases) because it avoids race conditions (a file can be deleted between the exists check and the open), produces cleaner code for the happy path, and maps naturally to Python's exception model. LBYL is better when validating function arguments (where you want to give the caller a specific error message) and when the check is much cheaper than the operation.
Q5: Does try/except add overhead when no exception is raised?
Answer: No - the overhead of a try/except block when no exception is raised is essentially zero. CPython compiles try blocks to bytecode that simply marks the boundaries of the guarded region. The exception table is a lookup structure that is only consulted when an exception actually occurs. The overhead when an exception is raised is more significant: Python must build the traceback object, walk the exception table to find a handler, and unwind the stack. This is why using exceptions for normal control flow (like iterating with StopIteration manually) is slower than a simple loop - but using try/except around code that rarely fails adds negligible cost.
Q6: When would you use contextlib.ExitStack instead of nested with statements?
Answer: ExitStack is useful when you need to open a dynamic number of resources, or when you want to conditionally open resources. With nested with statements, the number of resources must be known at write time. ExitStack acts like a stack of context managers: you enter_context() for each resource, and when the ExitStack exits (normally or with an exception), it calls __exit__ on each in reverse order, guaranteeing cleanup even if some cleanup steps fail. It is also ideal when opening files from a list: for path in paths: stack.enter_context(open(path)).
Practice Challenges
Beginner - Execution Order Tracer
Without running the code, write out what each line prints, in order. Then run it to verify.
def trace():
print("1: start of try")
try:
print("2: inside try")
result = 10 / 2
print("3: after division")
except ZeroDivisionError:
print("4: inside except")
else:
print("5: inside else")
finally:
print("6: inside finally")
print("7: after try block")
trace()
Now predict the output for this version where division is by zero:
def trace_error():
try:
print("1: before error")
result = 10 / 0
print("2: after error (never reached)")
except ZeroDivisionError:
print("3: inside except")
else:
print("4: inside else (skipped)")
finally:
print("5: inside finally")
print("6: after try block")
trace_error()
Solution
First function (no exception):
1: start of try
2: inside try
3: after division
5: inside else
6: inside finally
7: after try block
else runs because no exception was raised. finally runs. Execution continues normally to line 7.
Second function (ZeroDivisionError):
1: before error
3: inside except
5: inside finally
6: after try block
Line "2: after error" is never printed because the exception was raised before it. The else clause is skipped because an exception occurred. finally runs. Execution continues to line 6 because the exception was handled.
Intermediate - Robust File Parser
Write a function parse_csv_safe(path, column) that:
- Opens a CSV file and reads values from the named column
- Handles
FileNotFoundErrorby returning an empty list with a warning - Handles
KeyError(column not found) by returning an empty list with a warning - Handles
ValueErroron individual rows (non-numeric values) by skipping that row - Uses
elseandfinallycorrectly - the file is always closed - Returns a list of floats
Solution
import csv
import logging
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger(__name__)
def parse_csv_safe(path: str, column: str) -> list:
"""Parse numeric values from a named column in a CSV file.
Returns an empty list if the file is missing or the column does not exist.
Skips rows with non-numeric values, logging a warning for each.
"""
file_handle = None
values = []
try:
file_handle = open(path, newline="", encoding="utf-8")
except FileNotFoundError:
logger.warning("CSV file not found: %s", path)
return []
except PermissionError:
logger.warning("Permission denied reading: %s", path)
return []
else:
# File opened successfully - now parse it
reader = csv.DictReader(file_handle)
first_row = next(reader, None)
if first_row is None:
logger.warning("CSV file is empty: %s", path)
return []
if column not in first_row:
available = list(first_row.keys())
logger.warning(
"Column %r not found in %s. Available: %s",
column, path, available
)
return []
# Process the first row and all subsequent rows
for row_num, row in enumerate([first_row] + list(reader), start=1):
raw = row.get(column, "").strip()
try:
values.append(float(raw))
except ValueError:
logger.warning(
"Row %d: cannot convert %r to float - skipping",
row_num, raw
)
finally:
if file_handle is not None:
file_handle.close()
logger.info("File closed: %s", path)
return values
# Test setup
import tempfile
import os
csv_content = """name,score,grade
Alice,92.5,A
Bob,not_a_number,B
Carol,87.3,B+
Dave,,A-
Eve,95.0,A+
"""
with tempfile.NamedTemporaryFile(
mode="w", suffix=".csv", delete=False, encoding="utf-8"
) as f:
f.write(csv_content)
tmp_path = f.name
try:
print("=== Valid column ===")
scores = parse_csv_safe(tmp_path, "score")
print(f"Parsed scores: {scores}")
print(f"Average: {sum(scores)/len(scores):.2f}")
print("\n=== Missing column ===")
result = parse_csv_safe(tmp_path, "points")
print(f"Result: {result}")
print("\n=== Missing file ===")
result = parse_csv_safe("/nonexistent/file.csv", "score")
print(f"Result: {result}")
finally:
os.unlink(tmp_path)
Output:
INFO: File closed: /tmp/tmpXXXXXX.csv
=== Valid column ===
WARNING: Row 2: cannot convert 'not_a_number' to float - skipping
WARNING: Row 4: cannot convert '' to float - skipping
Parsed scores: [92.5, 87.3, 95.0]
Average: 91.60
=== Missing column ===
WARNING: Column 'points' not found in /tmp/tmpXXXXXX.csv. Available: ['name', 'score', 'grade']
Result: []
=== Missing file ===
WARNING: CSV file not found: /nonexistent/file.csv
Result: []
Key design decisions:
try/exceptaround only theopen()call - isolate what might failelsefor parsing - only runs when the file opened successfullyfinallyguarantees the file handle is closed even if parsing raisesif file_handle is not Noneguards against failure before assignment
Advanced - Generic Retry Decorator with Per-Exception Configuration
Build a @retry decorator that:
- Accepts a mapping of exception types to retry counts:
{ConnectionError: 3, TimeoutError: 5} - Uses exponential backoff with jitter (random ±20% variation on the delay)
- Calls an optional
on_retrycallback with the exception, attempt number, and next delay - Calls an optional
on_failurecallback when all retries are exhausted - Preserves the original function's docstring and name using
functools.wraps - Works correctly with both regular and
asyncfunctions
Solution
import asyncio
import functools
import inspect
import logging
import random
import time
from typing import Callable, Optional, Type
logger = logging.getLogger(__name__)
def retry(
exceptions: dict,
base_delay: float = 1.0,
backoff: float = 2.0,
jitter: float = 0.2,
on_retry: Optional[Callable] = None,
on_failure: Optional[Callable] = None,
):
"""Retry decorator with per-exception retry counts and exponential backoff.
Args:
exceptions: Mapping of exception type to max retry count.
e.g. {ConnectionError: 3, TimeoutError: 5}
base_delay: Initial delay in seconds between retries.
backoff: Multiplier applied to delay after each retry.
jitter: Fraction of delay to randomize (+/- jitter).
on_retry: Optional callback(exc, attempt, next_delay) before each retry.
on_failure: Optional callback(exc, attempts) when all retries exhausted.
"""
exc_tuple = tuple(exceptions.keys())
def decorator(func):
is_async = inspect.iscoroutinefunction(func)
@functools.wraps(func)
def sync_wrapper(*args, **kwargs):
attempt_counts = {exc_type: 0 for exc_type in exceptions}
while True:
try:
return func(*args, **kwargs)
except exc_tuple as exc:
matched_type = next(
t for t in exceptions if isinstance(exc, t)
)
attempt_counts[matched_type] += 1
attempt = attempt_counts[matched_type]
max_attempts = exceptions[matched_type]
if attempt >= max_attempts:
if on_failure:
on_failure(exc, attempt)
raise
delay = base_delay * (backoff ** (attempt - 1))
jitter_amt = delay * jitter
actual_delay = max(
0.0, delay + random.uniform(-jitter_amt, jitter_amt)
)
if on_retry:
on_retry(exc, attempt, actual_delay)
else:
logger.warning(
"%s: %s (attempt %d/%d), retrying in %.2fs",
func.__name__, exc, attempt, max_attempts, actual_delay
)
time.sleep(actual_delay)
@functools.wraps(func)
async def async_wrapper(*args, **kwargs):
attempt_counts = {exc_type: 0 for exc_type in exceptions}
while True:
try:
return await func(*args, **kwargs)
except exc_tuple as exc:
matched_type = next(
t for t in exceptions if isinstance(exc, t)
)
attempt_counts[matched_type] += 1
attempt = attempt_counts[matched_type]
max_attempts = exceptions[matched_type]
if attempt >= max_attempts:
if on_failure:
on_failure(exc, attempt)
raise
delay = base_delay * (backoff ** (attempt - 1))
jitter_amt = delay * jitter
actual_delay = max(
0.0, delay + random.uniform(-jitter_amt, jitter_amt)
)
if on_retry:
on_retry(exc, attempt, actual_delay)
else:
logger.warning(
"%s: %s (attempt %d/%d), retrying in %.2fs",
func.__name__, exc, attempt, max_attempts, actual_delay
)
await asyncio.sleep(actual_delay)
return async_wrapper if is_async else sync_wrapper
return decorator
# Demo
import random as _random
@retry(
exceptions={ConnectionError: 3, TimeoutError: 2},
base_delay=0.05, # Fast for demo purposes
on_retry=lambda exc, attempt, delay: print(
f" Retry after {type(exc).__name__} (attempt {attempt}), delay={delay:.3f}s"
),
on_failure=lambda exc, attempts: print(
f" FAILED after {attempts} attempts: {exc}"
),
)
def unstable_api():
"""Fetch data from an unreliable API."""
r = _random.random()
if r < 0.5:
raise ConnectionError("connection refused")
elif r < 0.7:
raise TimeoutError("request timed out")
return {"data": "success"}
for i in range(3):
try:
result = unstable_api()
print(f"Run {i+1}: Success: {result}")
except (ConnectionError, TimeoutError) as e:
print(f"Run {i+1}: Exhausted: {e}")
print()
Quick Reference
| Clause | When It Runs | Key Rule |
|---|---|---|
try | Always (first) | The guarded block |
except ExcType | If ExcType or subclass raised in try | First matching clause wins |
except (A, B) | If A or B raised | Tuple syntax, single handler |
except ExcType as e | If match; binds exception to e | e is deleted after block |
else | Only if try completed without exception | Exceptions here NOT caught by above |
finally | Always - even with return/break | Never return from here |
except Exception | All real errors; lets SystemExit through | Usually correct catch-all |
except BaseException | Everything including Ctrl+C | Almost always wrong |
bare except: | Same as except BaseException: | Avoid |
| Pattern | Use When | Approach |
|---|---|---|
| EAFP | Failure rare; external resources | try: op() except Exc: ... |
| LBYL | Cheap check; complex validation | if valid(x): op(x) |
| Retry | Transient failures (network, DB) | Loop with try/except and sleep |
| Context manager | Resource cleanup guarantees | with resource() as r: ... |
| Circuit breaker | Cascading failure prevention | State machine around try/except |
Key Takeaways
- Python's exception syntax has four clauses:
try,except,else,finally- andelseis the most overlooked elseruns only when thetryblock completes without raising; exceptions inelseare not caught by theexceptclauses above itfinallyalways runs, even withreturn,break, orcontinueinsidetry; areturninfinallysilently discards in-flight exceptions - never do it- Order
exceptclauses most-specific first - Python checks them in order and a parent class will catch all subclasses - The
as ebinding is deleted after the except block in Python 3; save what you need to a different variable name before the block ends try/excepthas near-zero overhead when no exception is raised - use EAFP freely for rare failures; avoid exceptions for normal control flow- The
withstatement is a cleanertry/finallyfor resource cleanup;contextlib.ExitStackhandles dynamic numbers of resources
