Project 01 - Publish an Internal Utility Package
Estimated time: 4–6 hours | Level: Intermediate → Engineering
The Scenario
Your team is tired of copy-pasting the same string manipulation functions, date formatting helpers, and retry decorators across five different services. You decide to extract them into a properly packaged, tested, versioned utility library - one that can be installed with a single pip install command and depended on with a version constraint.
This project takes you through that entire process: source layout, build configuration, testing, changelog, and publishing.
Requirements
R1 - src/ Layout with Proper Package Structure
The package must use the src/ layout. This is the recommended layout for Python packages because it prevents the package from being accidentally importable from the project root (which hides missing installation steps).
pyutils-engineersofai/
src/
pyutils_engineersofai/
__init__.py ← package version and public exports
py.typed ← PEP 561 typed marker (empty file)
strings.py
dates.py
retry.py
tests/
__init__.py
test_strings.py
test_dates.py
test_retry.py
pyproject.toml
poetry.lock
CHANGELOG.md
README.md
.gitlab-ci.yml
The py.typed marker file (PEP 561) tells type checkers (mypy, pyright) that this package provides inline type annotations and they should be checked.
R2 - pyproject.toml with Hatchling Backend
Use hatchling as the build backend (not setuptools, not poetry-core - hatchling specifically). The pyproject.toml must include:
- Full
[project]metadata: name, version, description, authors, license, readme, classifiers, keywords - Python
>=3.11requirement [project.optional-dependencies]withdevanddocsextras[tool.hatch.build.targets.wheel]specifying thesrc/layout[tool.pytest.ini_options]configuration[tool.mypy]configuration[tool.ruff]configuration[tool.coverage.run]and[tool.coverage.report]for coverage configuration
R3 - Three Utility Modules
Implement all three modules with full type annotations:
strings.py - String manipulation utilities:
slugify(text: str, separator: str = "-") -> str- converts "Hello World! Café" → "hello-world-cafe" (Unicode-aware, removes punctuation, normalizes separators)truncate(text: str, max_length: int, suffix: str = "...") -> str- truncates to max_length characters including the suffix; returns text unchanged if it fitscamel_to_snake(name: str) -> str- converts "CamelCase" and "camelCase" → "camel_case", handles acronyms correctly: "HTTPSClient" → "https_client"
dates.py - Date and time utilities:
humanize_delta(dt: datetime, *, now: datetime | None = None) -> str- returns "2 minutes ago", "in 3 hours", "yesterday", "3 days ago", "2 months ago", "last year" depending on delta;nowparameter allows testing without mockingparse_flexible(text: str) -> datetime- parses common date formats without requiring a format string: ISO 8601, "2024-01-15", "Jan 15 2024", "15/01/2024", "01/15/2024"; raisesValueErrorwith a helpful message listing supported formats if none match
retry.py - Retry decorator with exponential backoff:
retry(max_attempts: int = 3, exceptions: tuple[type[Exception], ...] = (Exception,), backoff_base: float = 1.0, backoff_factor: float = 2.0, jitter: bool = True) -> Callable- decorator that retries a function on specified exceptions with exponential backoff; logs each retry attempt with the exception message and attempt number; raises the last exception if all attempts fail
R4 - Full pytest Test Suite with ≥90% Branch Coverage
Write tests for all three modules. Requirements:
- All tests in
tests/directory - Use
pytest.mark.parametrizefor data-driven tests - Coverage must be ≥90% branch coverage (not just line coverage)
- The
retry.pytests must useunittest.mock.patchto controltime.sleep(so tests do not actually sleep) - The
dates.pytests must use thenowparameter to test different time deltas without mockingdatetime.now()
Run coverage with:
pytest tests/ --cov=src/pyutils_engineersofai --cov-branch --cov-report=term-missing --cov-fail-under=90
R5 - Lockfile Committed
poetry.lock (or pip-tools-generated requirements.txt) must be committed to the repository. The lockfile must be consistent with pyproject.toml (verify with poetry lock --check).
R6 - CHANGELOG.md Following Keep a Changelog Format
A CHANGELOG.md file following the Keep a Changelog format, at version 0.1.0:
## [0.1.0] - 2024-XX-XX
### Added
- `strings.slugify()` - Unicode-aware slugification
- `strings.truncate()` - smart truncation with suffix
- `strings.camel_to_snake()` - camelCase and PascalCase conversion
- `dates.humanize_delta()` - human-readable time deltas
- `dates.parse_flexible()` - multi-format date parsing
- `retry.retry()` - decorator with exponential backoff and jitter
R7 - Publish to TestPyPI and Verify Install
Publish the package to TestPyPI and verify it installs and works:
# After publishing, this must work in a fresh virtual environment:
pip install \
--index-url https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
pyutils-engineersofai
python -c "
from pyutils_engineersofai.strings import slugify, truncate, camel_to_snake
from pyutils_engineersofai.dates import humanize_delta
from pyutils_engineersofai.retry import retry
from datetime import datetime, timedelta
print(slugify('Hello World! Café')) # hello-world-cafe
print(truncate('A very long string', 10)) # A very...
print(camel_to_snake('HTTPSClient')) # https_client
delta = datetime.now() - timedelta(minutes=5)
print(humanize_delta(delta)) # 5 minutes ago
print('All imports OK')
"
R8 - GitLab CI Pipeline
A .gitlab-ci.yml that:
- Runs tests on every push and merge request
- Publishes to the GitLab Package Registry on
v*tags pushed tomain - Does not publish from feature branches or merge requests
- Caches the
.venv/directory keyed onpoetry.lock - Uses
CI_JOB_TOKENfor authentication (no stored credentials)
Complete pyproject.toml
[project]
name = "pyutils-engineersofai"
version = "0.1.0"
description = "Typed Python utility library: string manipulation, date helpers, and retry logic"
readme = "README.md"
license = {text = "MIT"}
authors = [
{name = "Engineers of AI", email = "[email protected]"},
]
keywords = ["utilities", "strings", "dates", "retry", "toolkit"]
classifiers = [
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Software Development :: Libraries :: Python Modules",
"Typing :: Typed",
]
requires-python = ">=3.11"
dependencies = [
"python-dateutil>=2.9", # for flexible date parsing
"unicodedata2>=15.1", # for Unicode normalization in slugify
]
[project.optional-dependencies]
dev = [
"pytest>=8.0",
"pytest-cov>=5.0",
"ruff>=0.4",
"mypy>=1.10",
"pre-commit>=3.7",
]
docs = [
"mkdocs>=1.6",
"mkdocs-material>=9.5",
"mkdocstrings[python]>=0.25",
]
[project.urls]
Homepage = "https://github.com/engineersofai/pyutils-engineersofai"
Documentation = "https://engineersofai.github.io/pyutils-engineersofai"
Repository = "https://github.com/engineersofai/pyutils-engineersofai"
"Bug Tracker" = "https://github.com/engineersofai/pyutils-engineersofai/issues"
Changelog = "https://github.com/engineersofai/pyutils-engineersofai/blob/main/CHANGELOG.md"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/pyutils_engineersofai"]
[tool.hatch.build.targets.sdist]
include = [
"/src",
"/tests",
"/CHANGELOG.md",
"/README.md",
]
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = [
"--tb=short",
"-q",
]
[tool.coverage.run]
source = ["src/pyutils_engineersofai"]
branch = true
[tool.coverage.report]
fail_under = 90
show_missing = true
skip_covered = false
exclude_lines = [
"pragma: no cover",
"if TYPE_CHECKING:",
"raise NotImplementedError",
"@overload",
]
[tool.mypy]
python_version = "3.12"
strict = true
files = ["src/"]
ignore_missing_imports = false
[tool.ruff]
target-version = "py311"
line-length = 88
[tool.ruff.lint]
select = ["E", "F", "B", "I", "UP", "N", "SIM", "TCH"]
ignore = ["E501"]
[tool.ruff.lint.isort]
known-first-party = ["pyutils_engineersofai"]
Starter Code
src/pyutils_engineersofai/__init__.py
"""
pyutils-engineersofai - Typed Python utility library.
Modules:
strings: slugify, truncate, camel_to_snake
dates: humanize_delta, parse_flexible
retry: retry decorator with exponential backoff
"""
__version__ = "0.1.0"
__all__ = ["__version__"]
src/pyutils_engineersofai/py.typed
This file must exist and be empty. It signals PEP 561 compliance - the package provides inline type annotations.
touch src/pyutils_engineersofai/py.typed
src/pyutils_engineersofai/strings.py
"""String manipulation utilities."""
from __future__ import annotations
import re
import unicodedata
def slugify(text: str, separator: str = "-") -> str:
"""
Convert text to a URL-safe slug.
Normalizes Unicode characters to ASCII equivalents, converts to lowercase,
removes non-alphanumeric characters, and joins words with the separator.
Args:
text: Input text to slugify.
separator: Character to use between words. Defaults to "-".
Returns:
Slug string containing only ASCII alphanumeric characters and separator.
Examples:
>>> slugify("Hello World")
'hello-world'
>>> slugify("Café au lait")
'cafe-au-lait'
>>> slugify("Python 3.12 Release!", separator="_")
'python_3_12_release'
"""
# Normalize Unicode: decompose accented characters
normalized = unicodedata.normalize("NFKD", text)
# Encode to ASCII, ignoring characters that cannot be converted
ascii_text = normalized.encode("ascii", "ignore").decode("ascii")
# Lowercase
lower = ascii_text.lower()
# Replace non-alphanumeric with separator
slug = re.sub(r"[^\w\s-]", "", lower)
# Replace whitespace and hyphens with separator
slug = re.sub(r"[-\s]+", separator, slug)
# Strip leading/trailing separators
return slug.strip(separator)
def truncate(text: str, max_length: int, suffix: str = "...") -> str:
"""
Truncate text to a maximum length, appending a suffix if truncated.
The total length of the returned string (including suffix) will not exceed
max_length. If text already fits within max_length, it is returned unchanged.
Args:
text: Input text to truncate.
max_length: Maximum length of the returned string, including suffix.
suffix: String appended when truncation occurs. Defaults to "...".
Returns:
Original text if it fits; truncated text with suffix otherwise.
Raises:
ValueError: If max_length is less than len(suffix).
Examples:
>>> truncate("Hello, World!", 8)
'Hello...'
>>> truncate("Hi", 10)
'Hi'
>>> truncate("A long sentence here", 15, suffix=" [more]")
'A long s [more]'
"""
if max_length < len(suffix):
raise ValueError(
f"max_length ({max_length}) must be >= len(suffix) ({len(suffix)})"
)
if len(text) <= max_length:
return text
return text[: max_length - len(suffix)] + suffix
def camel_to_snake(name: str) -> str:
"""
Convert CamelCase or camelCase to snake_case.
Handles acronyms correctly: consecutive uppercase letters are treated
as a single unit except when followed by a lowercase letter.
Args:
name: CamelCase or camelCase identifier.
Returns:
snake_case equivalent.
Examples:
>>> camel_to_snake("CamelCase")
'camel_case'
>>> camel_to_snake("camelCase")
'camel_case'
>>> camel_to_snake("HTTPSClient")
'https_client'
>>> camel_to_snake("parseHTTPResponse")
'parse_http_response'
>>> camel_to_snake("getURL")
'get_url'
"""
# Insert underscore before sequences: uppercase letter followed by lowercase
# e.g. "HTTPSClient" -> "HTTPS_Client"
s1 = re.sub(r"([A-Z]+)([A-Z][a-z])", r"\1_\2", name)
# Insert underscore before uppercase letter preceded by lowercase/digit
# e.g. "CamelCase" -> "Camel_Case"
s2 = re.sub(r"([a-z\d])([A-Z])", r"\1_\2", s1)
return s2.lower()
src/pyutils_engineersofai/dates.py
"""Date and time utilities."""
from __future__ import annotations
from datetime import datetime
from typing import TYPE_CHECKING
if TYPE_CHECKING:
pass
# Ordered from most specific to least specific for parse_flexible
_DATE_FORMATS = [
"%Y-%m-%dT%H:%M:%S", # ISO 8601 without timezone: 2024-01-15T14:30:00
"%Y-%m-%dT%H:%M", # ISO 8601 short: 2024-01-15T14:30
"%Y-%m-%d", # ISO date: 2024-01-15
"%d/%m/%Y", # European: 15/01/2024
"%m/%d/%Y", # American: 01/15/2024
"%d %b %Y", # Day Mon Year: 15 Jan 2024
"%b %d %Y", # Mon Day Year: Jan 15 2024
"%d %B %Y", # Day Month Year: 15 January 2024
"%B %d %Y", # Month Day Year: January 15 2024
"%B %d, %Y", # Month Day, Year: January 15, 2024
]
def humanize_delta(dt: datetime, *, now: datetime | None = None) -> str:
"""
Return a human-readable description of a time delta.
Args:
dt: The datetime to describe.
now: The reference datetime. Defaults to datetime.now() if not provided.
Providing this parameter makes the function testable without mocking.
Returns:
Human-readable string like "2 minutes ago", "in 3 hours", "yesterday".
Examples:
>>> from datetime import datetime, timedelta
>>> ref = datetime(2024, 6, 15, 12, 0, 0)
>>> humanize_delta(ref - timedelta(seconds=30), now=ref)
'just now'
>>> humanize_delta(ref - timedelta(minutes=5), now=ref)
'5 minutes ago'
>>> humanize_delta(ref + timedelta(hours=2), now=ref)
'in 2 hours'
"""
if now is None:
now = datetime.now()
delta = now - dt
total_seconds = delta.total_seconds()
is_past = total_seconds >= 0
abs_seconds = abs(total_seconds)
def _format(value: float, unit: str, past: bool) -> str:
n = int(value)
unit_str = f"{n} {unit}" if n != 1 else f"1 {unit.rstrip('s')}"
return f"{unit_str} ago" if past else f"in {unit_str}"
if abs_seconds < 60:
return "just now"
elif abs_seconds < 3600:
return _format(abs_seconds / 60, "minutes", is_past)
elif abs_seconds < 86400:
return _format(abs_seconds / 3600, "hours", is_past)
elif abs_seconds < 2 * 86400:
return "yesterday" if is_past else "tomorrow"
elif abs_seconds < 30 * 86400:
return _format(abs_seconds / 86400, "days", is_past)
elif abs_seconds < 365 * 86400:
return _format(abs_seconds / (30 * 86400), "months", is_past)
else:
return "last year" if is_past else "next year"
def parse_flexible(text: str) -> datetime:
"""
Parse a date string in one of several common formats.
Tries each supported format in order. Does not guess between ambiguous
formats (e.g., 01/02/2024 is always treated as DD/MM/YYYY first, then
MM/DD/YYYY).
Args:
text: Date string to parse.
Returns:
Parsed datetime object.
Raises:
ValueError: If text does not match any supported format. The error
message lists all supported formats.
Examples:
>>> parse_flexible("2024-01-15")
datetime.datetime(2024, 1, 15, 0, 0)
>>> parse_flexible("Jan 15 2024")
datetime.datetime(2024, 1, 15, 0, 0)
>>> parse_flexible("15/01/2024")
datetime.datetime(2024, 1, 15, 0, 0)
"""
text = text.strip()
for fmt in _DATE_FORMATS:
try:
return datetime.strptime(text, fmt)
except ValueError:
continue
supported = "\n ".join(_DATE_FORMATS)
raise ValueError(
f"Cannot parse date: {text!r}\n"
f"Supported formats:\n {supported}"
)
src/pyutils_engineersofai/retry.py
"""Retry decorator with exponential backoff."""
from __future__ import annotations
import logging
import random
import time
from collections.abc import Callable
from functools import wraps
from typing import Any, TypeVar
logger = logging.getLogger(__name__)
F = TypeVar("F", bound=Callable[..., Any])
def retry(
max_attempts: int = 3,
exceptions: tuple[type[Exception], ...] = (Exception,),
backoff_base: float = 1.0,
backoff_factor: float = 2.0,
jitter: bool = True,
) -> Callable[[F], F]:
"""
Decorator that retries a function on specified exceptions with exponential backoff.
The delay before attempt n is:
delay = backoff_base * (backoff_factor ** (n - 1))
If jitter=True, the delay is randomized: delay * random.uniform(0.5, 1.5)
Args:
max_attempts: Maximum number of attempts (including the first call).
Must be >= 1. Defaults to 3.
exceptions: Tuple of exception types to catch and retry on.
Defaults to (Exception,) - retries on any exception.
backoff_base: Base delay in seconds before the first retry.
Defaults to 1.0.
backoff_factor: Multiplier applied to delay after each attempt.
Defaults to 2.0 (exponential backoff).
jitter: If True, randomize the delay to avoid thundering herd.
Defaults to True.
Returns:
Decorator function.
Raises:
ValueError: If max_attempts < 1.
The last exception raised by the wrapped function if all attempts fail.
Examples:
>>> import httpx
>>> @retry(max_attempts=3, exceptions=(httpx.TimeoutException,))
... def fetch(url: str) -> str:
... return httpx.get(url).text
>>> @retry(max_attempts=5, backoff_base=0.5, jitter=False)
... def connect_to_db() -> None:
... db.connect()
"""
if max_attempts < 1:
raise ValueError(f"max_attempts must be >= 1, got {max_attempts}")
def decorator(func: F) -> F:
@wraps(func)
def wrapper(*args: Any, **kwargs: Any) -> Any:
last_exception: Exception | None = None
for attempt in range(1, max_attempts + 1):
try:
return func(*args, **kwargs)
except exceptions as exc:
last_exception = exc
if attempt == max_attempts:
logger.error(
"Function %s failed after %d attempts. "
"Last error: %s",
func.__name__,
max_attempts,
exc,
)
raise
delay = backoff_base * (backoff_factor ** (attempt - 1))
if jitter:
delay *= random.uniform(0.5, 1.5)
logger.warning(
"Function %s failed on attempt %d/%d: %s. "
"Retrying in %.2fs...",
func.__name__,
attempt,
max_attempts,
exc,
delay,
)
time.sleep(delay)
# This line is unreachable but satisfies the type checker
raise last_exception # type: ignore[misc]
return wrapper # type: ignore[return-value]
return decorator
Test Stubs
tests/test_strings.py
"""Tests for pyutils_engineersofai.strings module."""
import pytest
from pyutils_engineersofai.strings import camel_to_snake, slugify, truncate
class TestSlugify:
@pytest.mark.parametrize(
"text, expected",
[
("Hello World", "hello-world"),
("Café au lait", "cafe-au-lait"),
("Python 3.12 Release!", "python-3-12-release"),
(" leading and trailing ", "leading-and-trailing"),
("multiple spaces", "multiple-spaces"),
("", ""),
("---", ""),
],
)
def test_slugify_default_separator(self, text: str, expected: str) -> None:
assert slugify(text) == expected
def test_slugify_custom_separator(self) -> None:
assert slugify("Hello World", separator="_") == "hello_world"
def test_slugify_unicode_normalization(self) -> None:
# TODO: add more Unicode test cases
assert slugify("naïve résumé") == "naive-resume"
class TestTruncate:
def test_truncate_short_text_unchanged(self) -> None:
assert truncate("Hi", 10) == "Hi"
def test_truncate_exact_length_unchanged(self) -> None:
assert truncate("Hello", 5) == "Hello"
@pytest.mark.parametrize(
"text, max_length, suffix, expected",
[
("Hello, World!", 8, "...", "Hello..."),
("A long sentence here", 15, " [more]", "A long s [more]"),
("abcdefgh", 5, "..", "abc.."),
],
)
def test_truncate_applies_suffix(
self, text: str, max_length: int, suffix: str, expected: str
) -> None:
assert truncate(text, max_length, suffix) == expected
def test_truncate_raises_if_max_length_less_than_suffix(self) -> None:
with pytest.raises(ValueError, match="max_length"):
truncate("some text", 2, "...")
# TODO: add edge case for max_length exactly equal to len(suffix)
class TestCamelToSnake:
@pytest.mark.parametrize(
"name, expected",
[
("CamelCase", "camel_case"),
("camelCase", "camel_case"),
("HTTPSClient", "https_client"),
("parseHTTPResponse", "parse_http_response"),
("getURL", "get_url"),
("simpleword", "simpleword"),
("AlreadySnake", "already_snake"), # not snake, but single word
],
)
def test_camel_to_snake(self, name: str, expected: str) -> None:
assert camel_to_snake(name) == expected
tests/test_dates.py
"""Tests for pyutils_engineersofai.dates module."""
import pytest
from datetime import datetime, timedelta
from pyutils_engineersofai.dates import humanize_delta, parse_flexible
class TestHumanizeDelta:
"""Use the `now` parameter for all tests - never mock datetime.now()."""
def setup_method(self) -> None:
# Fixed reference point for all tests
self.now = datetime(2024, 6, 15, 12, 0, 0)
@pytest.mark.parametrize(
"delta_seconds, expected",
[
(30, "just now"),
(-30, "just now"),
(90, "1 minute ago"),
(150, "2 minutes ago"),
(3700, "1 hour ago"),
(7400, "2 hours ago"),
(86400 + 3600, "yesterday"),
],
)
def test_past_deltas(self, delta_seconds: int, expected: str) -> None:
dt = self.now - timedelta(seconds=delta_seconds)
assert humanize_delta(dt, now=self.now) == expected
def test_future_delta_minutes(self) -> None:
dt = self.now + timedelta(minutes=5)
assert humanize_delta(dt, now=self.now) == "in 5 minutes"
def test_tomorrow(self) -> None:
dt = self.now + timedelta(hours=25)
assert humanize_delta(dt, now=self.now) == "tomorrow"
# TODO: add tests for weeks, months, last year, next year
class TestParseFlexible:
@pytest.mark.parametrize(
"text, expected",
[
("2024-01-15", datetime(2024, 1, 15)),
("2024-01-15T14:30:00", datetime(2024, 1, 15, 14, 30, 0)),
("Jan 15 2024", datetime(2024, 1, 15)),
("15 Jan 2024", datetime(2024, 1, 15)),
("January 15, 2024", datetime(2024, 1, 15)),
],
)
def test_parse_valid_formats(self, text: str, expected: datetime) -> None:
assert parse_flexible(text) == expected
def test_parse_strips_whitespace(self) -> None:
assert parse_flexible(" 2024-01-15 ") == datetime(2024, 1, 15)
def test_parse_invalid_raises_value_error(self) -> None:
with pytest.raises(ValueError, match="Cannot parse date"):
parse_flexible("not a date")
def test_error_message_lists_supported_formats(self) -> None:
with pytest.raises(ValueError, match="%Y-%m-%d"):
parse_flexible("32/13/2024") # invalid day/month
tests/test_retry.py
"""Tests for pyutils_engineersofai.retry module."""
from unittest.mock import MagicMock, call, patch
import pytest
from pyutils_engineersofai.retry import retry
class TestRetryDecorator:
def test_succeeds_on_first_attempt(self) -> None:
mock_fn = MagicMock(return_value="success")
decorated = retry(max_attempts=3)(mock_fn)
result = decorated()
assert result == "success"
assert mock_fn.call_count == 1
def test_retries_on_exception_then_succeeds(self) -> None:
mock_fn = MagicMock(side_effect=[ValueError("fail"), "success"])
decorated = retry(max_attempts=3, exceptions=(ValueError,))(mock_fn)
with patch("pyutils_engineersofai.retry.time.sleep") as mock_sleep:
result = decorated()
assert result == "success"
assert mock_fn.call_count == 2
mock_sleep.assert_called_once()
def test_raises_after_max_attempts(self) -> None:
mock_fn = MagicMock(side_effect=ConnectionError("refused"))
decorated = retry(max_attempts=3, exceptions=(ConnectionError,))(mock_fn)
with patch("pyutils_engineersofai.retry.time.sleep"):
with pytest.raises(ConnectionError, match="refused"):
decorated()
assert mock_fn.call_count == 3
def test_does_not_retry_on_non_matching_exception(self) -> None:
mock_fn = MagicMock(side_effect=TypeError("type error"))
decorated = retry(max_attempts=3, exceptions=(ValueError,))(mock_fn)
with pytest.raises(TypeError):
decorated()
assert mock_fn.call_count == 1
def test_sleep_is_called_with_backoff(self) -> None:
mock_fn = MagicMock(side_effect=ValueError("fail"))
decorated = retry(
max_attempts=3,
exceptions=(ValueError,),
backoff_base=1.0,
backoff_factor=2.0,
jitter=False, # disable jitter for deterministic test
)(mock_fn)
with patch("pyutils_engineersofai.retry.time.sleep") as mock_sleep:
with pytest.raises(ValueError):
decorated()
# Attempt 1 fails → sleep(1.0); Attempt 2 fails → sleep(2.0); Attempt 3 raises
assert mock_sleep.call_count == 2
assert mock_sleep.call_args_list[0] == call(1.0)
assert mock_sleep.call_args_list[1] == call(2.0)
def test_raises_value_error_for_zero_max_attempts(self) -> None:
with pytest.raises(ValueError, match="max_attempts"):
retry(max_attempts=0)
def test_preserves_function_metadata(self) -> None:
@retry(max_attempts=2)
def my_function() -> str:
"""My docstring."""
return "result"
assert my_function.__name__ == "my_function"
assert my_function.__doc__ == "My docstring."
# TODO: test jitter randomizes the sleep value within expected bounds
# TODO: test that all attempts use args/kwargs correctly
.gitlab-ci.yml
# .gitlab-ci.yml for pyutils-engineersofai
stages:
- lint
- test
- build
- publish
variables:
POETRY_VIRTUALENVS_IN_PROJECT: "true"
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
POETRY_CACHE_DIR: "$CI_PROJECT_DIR/.cache/poetry"
.python-base:
image: python:3.12-slim
cache:
key:
files:
- poetry.lock
paths:
- .venv/
- .cache/pip/
- .cache/poetry/
before_script:
- pip install poetry --quiet
- poetry install --no-interaction --with dev
lint:
extends: .python-base
stage: lint
script:
- poetry run ruff check src/ tests/
- poetry run ruff format --check src/ tests/
- poetry run mypy src/
- poetry lock --check # verify lockfile is up to date
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == "main"
- if: '$CI_COMMIT_TAG =~ /^v[0-9]+\.[0-9]+\.[0-9]+/'
test:
extends: .python-base
stage: test
script:
- poetry run pytest tests/
--cov=src/pyutils_engineersofai
--cov-branch
--cov-report=xml
--cov-report=term-missing
--cov-fail-under=90
-v
coverage: '/TOTAL.+?(\d+\%)/'
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
expire_in: 1 week
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == "main"
- if: '$CI_COMMIT_TAG =~ /^v[0-9]+\.[0-9]+\.[0-9]+/'
build-package:
image: python:3.12-slim
stage: build
before_script:
- pip install build twine --quiet
script:
- python -m build
- twine check dist/*
- echo "Built distributions:"
- ls -la dist/
artifacts:
name: "$CI_PROJECT_NAME-$CI_COMMIT_TAG"
paths:
- dist/
expire_in: 30 days
rules:
# Only build on version tags on main
- if: '$CI_COMMIT_TAG =~ /^v[0-9]+\.[0-9]+\.[0-9]+/'
publish-gitlab-registry:
stage: publish
image: python:3.12-slim
needs:
- job: build-package
artifacts: true
- job: test
- job: lint
before_script:
- pip install twine --quiet
script:
- |
twine upload \
--repository-url "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi" \
--username "gitlab-ci-token" \
--password "${CI_JOB_TOKEN}" \
dist/*
environment:
name: gitlab-registry
rules:
# Publish ONLY on version tags - never on branches or MRs
- if: '$CI_COMMIT_TAG =~ /^v[0-9]+\.[0-9]+\.[0-9]+/'
Step-by-Step Publish Workflow
Follow these steps in order. Do not skip TestPyPI - it exists precisely to catch the errors listed in the next section.
Step 1 - Set Up the Project
# Clone or create the repository
mkdir pyutils-engineersofai && cd pyutils-engineersofai
git init
# Create the directory structure
mkdir -p src/pyutils_engineersofai tests
# Create all files listed in R1
touch src/pyutils_engineersofai/__init__.py
touch src/pyutils_engineersofai/py.typed
touch src/pyutils_engineersofai/strings.py
touch src/pyutils_engineersofai/dates.py
touch src/pyutils_engineersofai/retry.py
touch tests/__init__.py
touch tests/test_strings.py
touch tests/test_dates.py
touch tests/test_retry.py
Step 2 - Install Dependencies and Run Tests
# Install hatchling and dev dependencies
pip install hatchling build twine
pip install -e ".[dev]"
# Run tests with coverage
pytest tests/ --cov=src/pyutils_engineersofai --cov-branch --cov-report=term-missing
# Target: ≥90% branch coverage
# If below 90%: add test cases for uncovered branches (look at "Missing" column)
Step 3 - Validate the Build
# Build both distributions
python -m build
# Verify output
ls dist/
# pyutils_engineersofai-0.1.0-py3-none-any.whl
# pyutils_engineersofai-0.1.0.tar.gz
# Check metadata and README rendering
twine check dist/*
# Checking dist/pyutils_engineersofai-0.1.0.tar.gz: PASSED
# Checking dist/pyutils_engineersofai-0.1.0-py3-none-any.whl: PASSED
Step 4 - Upload to TestPyPI
# Upload to TestPyPI (requires account at test.pypi.org)
twine upload --repository testpypi dist/*
# Enter username: __token__
# Enter password: pypi-xxxxx... (your TestPyPI API token)
Step 5 - Verify the TestPyPI Install
# Create a fresh virtual environment - not the development one
python -m venv /tmp/test-install-env
source /tmp/test-install-env/bin/activate
# Install from TestPyPI
# --extra-index-url is needed because TestPyPI doesn't have all dependencies
pip install \
--index-url https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
pyutils-engineersofai
# Run the verification script
python -c "
from pyutils_engineersofai.strings import slugify, truncate, camel_to_snake
from pyutils_engineersofai.dates import humanize_delta
from pyutils_engineersofai.retry import retry
from datetime import datetime, timedelta
assert slugify('Hello World! Café') == 'hello-world-cafe'
assert truncate('A very long string', 10) == 'A very...'
assert camel_to_snake('HTTPSClient') == 'https_client'
ref = datetime(2024, 1, 1, 12, 0, 0)
assert humanize_delta(ref - timedelta(minutes=5), now=ref) == '5 minutes ago'
call_count = 0
@retry(max_attempts=2, exceptions=(ValueError,))
def flaky():
global call_count
call_count += 1
if call_count < 2:
raise ValueError('not yet')
return 'done'
import time
original_sleep = time.sleep
time.sleep = lambda x: None # don't actually sleep in this test
result = flaky()
time.sleep = original_sleep
assert result == 'done'
assert call_count == 2
print('All assertions passed - package works correctly from TestPyPI')
"
deactivate
Step 6 - Tag and Push
# Commit all files (do not commit .venv/, dist/, __pycache__)
git add src/ tests/ pyproject.toml poetry.lock CHANGELOG.md README.md .gitlab-ci.yml
git commit -m "feat: initial release 0.1.0"
# Create annotated tag
git tag -a v0.1.0 -m "Release 0.1.0: initial release with strings, dates, retry modules"
# Push to GitLab (triggers CI pipeline)
git push origin main
git push origin v0.1.0
# The CI pipeline will:
# 1. Run lint and test jobs
# 2. Build the package (triggered by v* tag)
# 3. Publish to GitLab Package Registry
Common Errors and Fixes
Error: HTTPError: 400 Bad Request - File already exists
HTTPError: 400 Bad Request from https://test.pypi.org/legacy/
File already exists.
Cause: You already uploaded version 0.1.0 to TestPyPI. Unlike regular files, package releases are immutable.
Fix: Bump the version to 0.1.1 (or 0.1.0.post1 for TestPyPI-only testing), rebuild, and upload:
# For TestPyPI testing iterations, use dev versions:
# 0.1.0.dev1, 0.1.0.dev2, etc.
# These signal "development release" and are excluded from stable install by default.
# Edit pyproject.toml: version = "0.1.0.dev2"
python -m build
twine upload --repository testpypi dist/*
Error: InvalidDistribution: Missing project metadata
InvalidDistribution: Missing metadata for required field: name
Cause: The [project] section in pyproject.toml is missing or malformed.
Fix: Verify pyproject.toml has a valid [project] table with at minimum name, version, and requires-python:
# Validate the config
python -m build --no-isolation 2>&1 | head -30
# Or use hatchling directly:
python -c "import hatchling.build; print('pyproject.toml is valid')"
Error: twine check fails on README
`long_description` has syntax errors in markup and would not be rendered on PyPI.
Cause: The README.md contains Markdown syntax that PyPI's renderer cannot handle, or the file has encoding issues.
Fix:
# Check what twine sees as the long description
twine check dist/* --strict
# Common causes:
# 1. Relative image links:  → use absolute GitHub URLs
# 2. HTML inside Markdown that's not standard: some extensions not supported
# 3. Missing blank line before code blocks in some parsers
# Verify locally with readme-renderer
pip install readme-renderer[md]
python -m readme_renderer README.md -o /tmp/readme-check.html
# Open /tmp/readme-check.html in browser to see what PyPI would render
Error: Package name already taken on PyPI
HTTPError: 403 Forbidden - The user 'yourname' isn't allowed to upload to project 'utils'.
Cause: A package named utils (or whatever you chose) already exists on PyPI and belongs to someone else.
Fix: Choose a more specific name. If you are publishing to TestPyPI for the first time, the error means someone has already reserved that name on TestPyPI too. Add a unique prefix:
# In pyproject.toml:
name = "pyutils-engineersofai" # organization-prefixed
# or:
name = "yourname-utils" # username-prefixed
# or:
name = "myproject-utils" # project-prefixed
Check if a name is available before building:
pip index versions pyutils-engineersofai 2>&1 | head -5
# If: "WARNING: No distributions found for..." → name is available on real PyPI
# Always check both test.pypi.org and pypi.org
Error: src/ package not found after install
from pyutils_engineersofai.strings import slugify
# ModuleNotFoundError: No module named 'pyutils_engineersofai'
Cause: The [tool.hatch.build.targets.wheel] section does not correctly specify the src/ layout, so hatchling does not include the src/ packages in the wheel.
Fix: Verify the hatchling configuration:
[tool.hatch.build.targets.wheel]
packages = ["src/pyutils_engineersofai"]
# ↑ Must match the actual path: src/<package_name>
Verify the wheel contents before publishing:
# List all files in the wheel
python -m zipfile -l dist/pyutils_engineersofai-0.1.0-py3-none-any.whl
# You should see:
# pyutils_engineersofai/__init__.py
# pyutils_engineersofai/py.typed
# pyutils_engineersofai/strings.py
# pyutils_engineersofai/dates.py
# pyutils_engineersofai/retry.py
# pyutils_engineersofai-0.1.0.dist-info/METADATA
# pyutils_engineersofai-0.1.0.dist-info/WHEEL
# pyutils_engineersofai-0.1.0.dist-info/RECORD
# If you only see dist-info/ and no package files: the packages path is wrong
Error: poetry.lock is not consistent with pyproject.toml
The lock file is not up to date with the latest changes in pyproject.toml.
Run poetry lock to update the lock file.
Cause: You edited pyproject.toml (added/changed a dependency) but did not regenerate poetry.lock.
Fix:
poetry lock # regenerate lockfile
git add poetry.lock
git commit -m "chore: update poetry.lock"
In CI, poetry lock --check catches this before publishing. The pipeline will fail with a clear error message if the lockfile is stale.
© 2026 EngineersOfAI. All rights reserved.