Module 01 - Scientific Python Stack Overview

Here is a real ML engineering pipeline condensed to its essentials:

import numpy as np
import pandas as pd
from scipy import stats, sparse
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
import torch
import torch.nn as nn

# 1. Load and inspect raw data (Pandas)
df = pd.read_parquet("user_events.parquet")
df["hour"] = df["timestamp"].dt.hour
df["is_weekend"] = df["timestamp"].dt.dayofweek >= 5

# 2. Feature engineering (Pandas + NumPy)
feature_cols = ["hour", "is_weekend", "session_length", "page_depth"]
X = df[feature_cols].to_numpy(dtype=np.float32)   # zero-copy to ndarray
y = df["converted"].to_numpy(dtype=np.int64)

# 3. Statistical validation before training (SciPy)
_, p_value = stats.mannwhitneyu(
    df.loc[df["converted"] == 1, "session_length"],
    df.loc[df["converted"] == 0, "session_length"],
)
print(f"Session length is a significant signal: {p_value < 0.05}")

# 4. Classical ML baseline (scikit-learn)
clf = Pipeline([
    ("scaler", StandardScaler()),
    ("lr", LogisticRegression(max_iter=500)),
])
clf.fit(X, y)

# 5. Deep learning refinement (PyTorch)
X_tensor = torch.from_numpy(X)
y_tensor = torch.from_numpy(y)

model = nn.Sequential(
    nn.Linear(4, 64), nn.ReLU(),
    nn.Linear(64, 1), nn.Sigmoid(),
)

Every line above corresponds to a different library. Each one was written because the previous tool in the ecosystem could not do the job well enough. Understanding why each library exists - what specific limitation it solves - is what separates an engineer who can use these tools from one who can compose them correctly under production constraints.

This module teaches you all of it.

Why This Matters

You will not be writing production ML systems in PyTorch alone. The moment you need to:

• Load a 20 GB Parquet file and join it to another table - you need Pandas
• Run a rolling 30-day feature over 500 million rows efficiently - you need NumPy strides
• Fit a Gaussian to residuals or run a KS test for data drift - you need SciPy
• Tune hyperparameters of a gradient boosting model without gradient information - you need scipy.optimize
• Visualize a confusion matrix or a learning curve for a stakeholder report - you need Matplotlib
• Build and cross-validate a preprocessing-plus-model pipeline - you need scikit-learn pipelines
• Train a transformer on GPUs with automatic differentiation - you need PyTorch
• Compile a neural network down to XLA and run it at TPU speed - you need JAX

These are not theoretical scenarios. They are the daily reality of ML engineering at any company running ML in production. This module maps the terrain.

The Stack Layers

Every library in the stack sits on top of NumPy's ndarray. PyTorch tensors can be zero-copy converted to NumPy arrays. Pandas DataFrames store their columns as NumPy arrays internally. SciPy takes NumPy arrays and returns NumPy arrays. This shared foundation is intentional - it means data can flow between libraries without serialisation or copying in most cases.

What Each Library Actually Does

NumPy - The Foundation

NumPy provides the ndarray: a typed, contiguous block of memory with a powerful abstraction layer on top. Its defining feature is that element-wise operations run in C, not Python. This eliminates Python's per-element interpreter overhead and is the reason a NumPy sum over 10 million floats is 50x faster than a Python list comprehension doing the same sum.

NumPy's other core contributions:

• Broadcasting: arithmetic on arrays of different shapes without explicit loops
• Views and strides: advanced memory manipulation that enables slicing, transposing, and reshaping without copying data
• Ufuncs: composable element-wise functions that support reduction, accumulation, and outer products
• Linear algebra: np.linalg wraps BLAS/LAPACK for matrix operations

Everything in the stack ultimately converts to NumPy arrays before feeding to numerical kernels.

Pandas - The Data Engineering Layer

Pandas wraps NumPy arrays in a DataFrame abstraction that adds:

• Labelled axes (string column names, DatetimeIndex)
• Heterogeneous column types (one column int32, next column string, next category)
• SQL-style operations: groupby, join, merge, pivot
• Time series functionality: resampling, rolling windows, shifting

The DataFrame is not a database and not a spreadsheet. It is an in-memory data structure optimised for column-wise operations on tabular data. In an ML pipeline, Pandas is where you spend most of your engineering time - loading data, joining feature tables, computing aggregates, and building the feature matrix.

SciPy - The Scientific Algorithms Library

SciPy is a collection of numerical algorithms that go beyond what NumPy provides:

• scipy.optimize: minimisation, root finding, curve fitting - the toolkit for hyperparameter search and model calibration
• scipy.stats: probability distributions, hypothesis tests, confidence intervals
• scipy.sparse: sparse matrix formats (CSR, CSC) - essential for NLP bag-of-words and collaborative filtering
• scipy.spatial.distance: efficient pairwise distance computation (used inside KNN, DBSCAN, t-SNE)
• scipy.signal: filtering and spectral analysis for time series features
• scipy.linalg: more complete linear algebra than np.linalg (LU decomposition, Cholesky, etc.)

SciPy fills the gap between "raw array arithmetic" and "full ML framework". When you need a Kolmogorov-Smirnov test for data drift, or L-BFGS-B to minimise a custom loss without gradients, SciPy is what you reach for.

Matplotlib - The Visualisation Layer

Matplotlib is the foundational plotting library. Everything else (Seaborn, Plotly, Pandas .plot()) builds on or wraps it. For ML engineering, its main roles are:

• Loss curves during training
• Confusion matrices and ROC curves
• Feature importance bar charts
• Distribution plots before and after preprocessing
• Residual plots for regression diagnostics

You do not need to master Matplotlib's API in depth. You need to know enough to produce clear, exportable plots without spending an hour fighting with axis labels.

scikit-learn - The Classical ML Framework

scikit-learn is the reference implementation of classical ML in Python. Its design philosophy - a consistent fit / transform / predict interface across all estimators - enables composable pipelines and makes cross-validation, hyperparameter search, and model evaluation straightforward.

In an ML engineering context, scikit-learn's highest-value components are:

• Pipeline: chain preprocessing and modelling steps, preventing data leakage
• ColumnTransformer: apply different preprocessing to different columns
• GridSearchCV / RandomizedSearchCV: systematic hyperparameter tuning
• Cross-validation utilities: cross_val_score, StratifiedKFold
• Preprocessing: StandardScaler, OneHotEncoder, OrdinalEncoder
• Metrics: every standard metric for classification, regression, clustering

For tree-based models, you will likely use XGBoost or LightGBM which follow the scikit-learn interface, allowing them to drop into any scikit-learn Pipeline.

PyTorch - The Deep Learning Framework

PyTorch is the dominant deep learning framework in research and increasingly in production. Its key design decisions:

• Dynamic computation graph (eager execution): the graph is built as Python executes, making debugging with print and pdb trivial
• Autograd: automatic differentiation tracked at the tensor level, not the graph level
• nn.Module: composable building block for neural network layers
• torch.optim: SGD, Adam, AdamW, and all standard optimisers
• CUDA integration: .to(device) moves tensors between CPU and GPU

In this module you learn the fundamentals that underpin everything else: what a tensor is, how gradient tracking works, how to build and train a network, and how data flows from NumPy through DataLoader into a training loop.

JAX - The Compiled ML Framework

JAX is NumPy-compatible but adds four transformations that change how you write numerical code:

• jax.jit: compile a Python function to XLA, which can target CPU, GPU, or TPU
• jax.grad: automatic differentiation that works on arbitrary Python-NumPy functions
• jax.vmap: vectorise a function over a batch dimension without writing a loop
• jax.pmap: parallelise a function across multiple devices

JAX is not a drop-in replacement for PyTorch - it requires functional programming style (no in-place mutation, explicit PRNG state). But for research code, custom gradient computations, and TPU deployment, it offers performance and composability that PyTorch cannot match.

How the Stack Composes in a Real Pipeline

Here is the complete data flow in a realistic production ML pipeline, showing which library handles each stage:

The critical insight: each arrow in this diagram represents a data handoff that can be zero-copy if you are careful. NumPy arrays become PyTorch tensors with torch.from_numpy (shared memory). Pandas columns become NumPy arrays with .to_numpy() (zero-copy for numeric dtypes). PyTorch tensors become NumPy arrays with .detach().cpu().numpy() (zero-copy for CPU tensors). Understanding this matters at scale - a single unnecessary copy of a 10 million row feature matrix is 800 MB of allocation.

A Concrete End-to-End Example

The following example intentionally touches every library in the stack to show the handoffs in practice. It is a simplified churn prediction pipeline.

import numpy as np
import pandas as pd
from scipy import stats
from scipy.sparse import csr_matrix
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.metrics import roc_auc_score
import matplotlib.pyplot as plt
import torch
import torch.nn as nn

# ------------------------------------------------------------------ #
# Step 1: Load and engineer features - Pandas
# ------------------------------------------------------------------ #
np.random.seed(42)
n = 50_000

df = pd.DataFrame({
    "user_id":        np.arange(n),
    "tenure_days":    np.random.randint(1, 730, n),
    "plan":           np.random.choice(["free", "basic", "pro"], n),
    "monthly_spend":  np.random.exponential(25, n).astype(np.float32),
    "support_tickets": np.random.poisson(1.5, n).astype(np.int16),
    "last_login_days": np.random.randint(0, 90, n).astype(np.int16),
})

# Target: users with high tickets + long absence tend to churn
churn_score = (
    df["support_tickets"] * 0.3 +
    df["last_login_days"] * 0.02 -
    df["tenure_days"] * 0.001
)
df["churned"] = (churn_score + np.random.randn(n) > 0.5).astype(np.int8)

# Feature: log-transform right-skewed spend (Pandas + NumPy)
df["log_spend"] = np.log1p(df["monthly_spend"])

# Feature: engagement bucket (Pandas categorical)
df["engagement"] = pd.cut(
    df["last_login_days"],
    bins=[0, 7, 30, 90],
    labels=["active", "lapsed", "dormant"],
).astype("category")   # memory-efficient representation

print(f"Churn rate: {df['churned'].mean():.1%}")
print(f"DataFrame memory: {df.memory_usage(deep=True).sum() / 1024 / 1024:.1f} MB")

# ------------------------------------------------------------------ #
# Step 2: Statistical validation - SciPy
# ------------------------------------------------------------------ #
churned = df.loc[df["churned"] == 1, "last_login_days"]
retained = df.loc[df["churned"] == 0, "last_login_days"]

stat, p = stats.mannwhitneyu(churned, retained, alternative="greater")
print(f"Last login days - churned > retained: p = {p:.4f}")
# p << 0.05 confirms the signal before we spend time training a model

# Check distribution shape (are we safe to use z-score normalisation?)
stat_ks, p_ks = stats.kstest(
    df["log_spend"], "norm",
    args=(df["log_spend"].mean(), df["log_spend"].std())
)
print(f"Log-spend normality KS p-value: {p_ks:.3f}")

# ------------------------------------------------------------------ #
# Step 3: Build scikit-learn Pipeline
# ------------------------------------------------------------------ #
numeric_features = ["tenure_days", "log_spend", "support_tickets", "last_login_days"]
categorical_features = ["plan", "engagement"]

preprocessor = ColumnTransformer([
    ("num", StandardScaler(), numeric_features),
    ("cat", OneHotEncoder(handle_unknown="ignore", sparse_output=False), categorical_features),
])

pipeline = Pipeline([
    ("prep", preprocessor),
    ("clf", LogisticRegression(max_iter=1000, C=1.0)),
])

X = df[numeric_features + categorical_features]
y = df["churned"].to_numpy()

# 5-fold CV AUROC
scores = cross_val_score(pipeline, X, y, cv=5, scoring="roc_auc", n_jobs=-1)
print(f"LR CV AUROC: {scores.mean():.4f} ± {scores.std():.4f}")

# ------------------------------------------------------------------ #
# Step 4: Convert to NumPy / PyTorch for a neural net baseline
# ------------------------------------------------------------------ #
pipeline.fit(X, y)
# Get preprocessed features (now a NumPy array - pure float64)
X_processed = pipeline.named_steps["prep"].transform(X).astype(np.float32)

# Zero-copy to PyTorch
X_tensor = torch.from_numpy(X_processed)   # shares memory
y_tensor = torch.from_numpy(y.astype(np.float32))

# Simple 3-layer net
input_dim = X_processed.shape[1]
net = nn.Sequential(
    nn.Linear(input_dim, 128), nn.ReLU(), nn.Dropout(0.2),
    nn.Linear(128, 64),        nn.ReLU(), nn.Dropout(0.2),
    nn.Linear(64, 1),          nn.Sigmoid(),
)

optimiser = torch.optim.Adam(net.parameters(), lr=1e-3)
criterion = nn.BCELoss()

dataset = torch.utils.data.TensorDataset(X_tensor, y_tensor.unsqueeze(1))
loader = torch.utils.data.DataLoader(dataset, batch_size=512, shuffle=True)

losses = []
for epoch in range(10):
    epoch_loss = 0.0
    for xb, yb in loader:
        optimiser.zero_grad()
        pred = net(xb)
        loss = criterion(pred, yb)
        loss.backward()
        optimiser.step()
        epoch_loss += loss.item() * len(xb)
    losses.append(epoch_loss / len(dataset))

# ------------------------------------------------------------------ #
# Step 5: Visualise - Matplotlib
# ------------------------------------------------------------------ #
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))

# Training loss curve
ax1.plot(losses)
ax1.set_xlabel("Epoch")
ax1.set_ylabel("BCE Loss")
ax1.set_title("Neural Net Training Loss")

# Churn rate by plan
churn_by_plan = df.groupby("plan")["churned"].mean() * 100
ax2.bar(churn_by_plan.index, churn_by_plan.values, color=["#339af0", "#51cf66", "#ff6b6b"])
ax2.set_ylabel("Churn Rate (%)")
ax2.set_title("Churn Rate by Plan")

plt.tight_layout()
plt.savefig("churn_analysis.png", dpi=150)
print("Saved churn_analysis.png")

Notice the data flow:

1. Pandas builds the feature table
2. SciPy validates the statistical assumptions before modelling
3. scikit-learn handles preprocessing and cross-validation
4. pipeline.transform() returns a NumPy array
5. torch.from_numpy() converts that array to a tensor with zero allocation
6. PyTorch trains the network
7. Matplotlib visualises results

This is not contrived. This is what production ML pipelines look like.

Module Lessons at a Glance

Lesson	Topic	What You Will Build
01	NumPy Internals	Custom rolling feature engine, stride-based sliding window extractor
02	Pandas for ML	Production feature pipeline for a tabular ML dataset
03	SciPy for ML	Hyperparameter optimiser using scipy.optimize, sparse TF-IDF encoder
04	Matplotlib and Seaborn	Full model diagnostic dashboard
05	scikit-learn Pipelines	A leakage-free preprocessing + model pipeline with custom transformers
06	PyTorch Fundamentals	Train a neural network from scratch, understanding every step
07	JAX and Functional ML	JIT-compiled training loop, custom gradient with jax.grad

Prerequisites

• Completed Python Foundation and Python Intermediate (or equivalent experience)
• Comfortable with Python classes, decorators, generators, and context managers
• Basic linear algebra: matrix multiplication, dot product, transpose
• Basic statistics: mean, variance, standard deviation, what a p-value means

You do not need prior ML theory. The lessons introduce theory as motivation for engineering decisions, not as prerequisites.

Key Takeaways

• The scientific Python stack is not a set of independent tools - it is a layered system where every library ultimately stores and exchanges data through NumPy ndarray objects.
• Each library fills a specific gap: NumPy for raw numerical computation, Pandas for tabular data manipulation, SciPy for scientific algorithms, Matplotlib for visualisation, scikit-learn for classical ML pipelines, PyTorch for deep learning, JAX for compiled functional ML.
• Understanding which library handles which concern - and how to pass data between them without unnecessary copies - is a core ML engineering skill.
• A real ML pipeline uses all of these libraries. A real ML engineer understands each one well enough to know when to use it and when to reach for the next tool.