Data Types at the Hardware Level - What Python Hides From You

Reading time: ~22 minutes | Level: Foundation → Engineering

Imagine you pass the same 4 bytes to two different C functions. The first function interprets those bytes as a 32-bit integer and prints 1,078,523,331. The second function interprets those exact same bytes as an IEEE 754 float and prints 3.14. Same bits. Same memory. Completely different meaning.

4 bytes in memory:  0x40 0x48 0xF5 0xC3

Interpreted as int32:   1,078,527,427
Interpreted as float32: 3.14000...

This is the fundamental insight that Python deliberately hides from you: data types are not the bits themselves - they are the interpretation applied to bits. There is no such thing as an "integer" or a "float" in silicon. There are only bit patterns, and the CPU instructions that act on them.

Understanding this insight makes you a better engineer. It explains why NumPy is fast and Python is slow for numbers, why bool is a subclass of int, why float arithmetic surprises you, and why selecting the wrong dtype in a neural network can crash your training run.

What You Will Learn

• Why hardware has no concept of "data types" - only bit patterns and instructions
• How C's fixed-width integer types differ from Python's arbitrary-precision integers
• Why Python's bool is a subclass of int and what that means in practice
• How IEEE 754 floating-point numbers are structured at the bit level and what special values exist
• How characters and text are stored as numbers at the hardware level
• Why memory alignment matters for performance (and when Python abstracts it away)
• How CPython represents every value as a heap-allocated object with headers
• How NumPy exposes hardware types directly - and what overflow looks like in NumPy
• How to choose dtypes for PyTorch model training and inference

Prerequisites

• Comfortable running Python code
• Read the previous module (Binary, Bits, and Bytes) or have basic familiarity with binary representation
• No C programming experience required - C examples are in pseudocode form where needed

The Mental Model: Layers of Type Interpretation

AT HARDWARE LEVEL (no types - just bits):
┌─────────────────────────────────────────────────────┐
│  RAM: 01000000 01001000 11110101 11000011           │
│       These 4 bytes have no inherent meaning.       │
│       The CPU instruction determines what they are. │
└─────────────────────────────────────────────────────┘
         │               │               │
         ▼               ▼               ▼
    ADD instruction  FADD instruction  MOV to screen
    treats as         treats as         treats as
    int32 (202)       float32 (3.14)    bytes (display)

AT C LEVEL (explicit fixed-width types):
┌───────────────────────────────────────────────────┐
│  int32_t x = 42;      // 4 bytes, two's complement│
│  float   y = 3.14f;   // 4 bytes, IEEE 754        │
│  uint8_t z = 255;     // 1 byte,  0–255           │
│  The compiler selects the right instructions.     │
└─────────────────────────────────────────────────-─┘

AT PYTHON LEVEL (objects with type metadata):
┌──────────────────────────────────────────────────┐
│  x = 42     # PyLongObject on heap               │
│  y = 3.14   # PyFloatObject on heap              │
│  z = True   # PyBoolObject (subclass of long)    │
│  Every value carries its own type information.   │
└──────────────────────────────────────────────────┘

AT NUMPY LEVEL (C types in Python packaging):
┌──────────────────────────────────────────────────┐
│  np.int32, np.float32, np.float16 - C types      │
│  stored in contiguous C arrays, wrapped in a     │
│  single Python ndarray object.                   │
└──────────────────────────────────────────────────┘

Part 1 - Hardware Has No Types

Memory is just bytes. The hardware stores and retrieves bytes. What those bytes mean is determined entirely by the instructions the CPU executes.

Consider these 4 bytes: 0x40 0x49 0x0F 0xDB

import struct

raw = bytes([0x40, 0x49, 0x0F, 0xDB])

# Interpret as a 32-bit unsigned integer
as_int = struct.unpack(">I", raw)[0]    # ">I" = big-endian unsigned int
print(f"As uint32:  {as_int}")          # 1078530011

# Interpret as a 32-bit IEEE 754 float
as_float = struct.unpack(">f", raw)[0]  # ">f" = big-endian float32
print(f"As float32: {as_float:.6f}")    # 3.141593 - that's π!

# Interpret as 4 individual bytes
as_bytes = list(raw)
print(f"As bytes:   {[hex(b) for b in as_bytes]}")

# Interpret as two 16-bit unsigned integers
a, b = struct.unpack(">HH", raw)
print(f"As two uint16: {a}, {b}")       # 16457, 3547

Output:

As uint32:  1078530011
As float32: 3.141593
As bytes:   ['0x40', '0x49', '0xf', '0xdb']
As two uint16: 16457, 3547

The same 4 bytes contain π when read as a float, a completely different integer when read as a 32-bit int, and two other integers when read as two 16-bit ints. The type is the interpretation, not the data.

This thought experiment explains why:

• Casting bugs in C are dangerous (you change the interpretation without changing the bits)
• Memory corruption causes bizarre values (a pointer is read as an integer, etc.)
• Binary file formats must specify types precisely (TIFF, PNG, WAV all have type descriptors)
• NumPy's view() can reinterpret an array's memory under a different dtype

:::note The Practical Implication When you call struct.unpack() in Python - to parse a network packet, read a binary file, or inspect a floating-point value's raw bytes - you are explicitly choosing the type interpretation. Python's struct module gives you direct access to hardware-level type representations. :::

Part 2 - Fixed-Width Integer Types in C vs Python

C's integer types have fixed sizes, fixed ranges, and fixed behavior at overflow:

C type	Bits	Range	Python equivalent
int8_t	8	-128 to 127	np.int8
uint8_t	8	0 to 255	np.uint8
int16_t	16	-32,768 to 32,767	np.int16
uint16_t	16	0 to 65,535	np.uint16
int32_t	32	-2,147,483,648 to 2,147,483,647	np.int32
uint32_t	32	0 to 4,294,967,295	np.uint32
int64_t	64	-9.2×10¹⁸ to 9.2×10¹⁸	np.int64
uint64_t	64	0 to 1.8×10¹⁹	np.uint64
Python int	unlimited	any integer	-

In C, integer overflow is dangerous:

// C pseudocode - overflow is undefined behavior for signed types
int32_t x = 2147483647;   // INT32_MAX
x = x + 1;                // Undefined behavior! Often wraps to -2147483648

In Python, the same arithmetic just works:

import sys

# Python integers grow as needed - no overflow
x = 2147483647          # INT32_MAX
x = x + 1
print(x)                # 2147483648 - correct

x = 2**63              # Exceeds int64 range
print(x)               # 9223372036854775808
print(x + 1)           # 9223372036854775809 - still correct

# Track memory growth
print(sys.getsizeof(1))           # 28 bytes
print(sys.getsizeof(2**30))       # 28 bytes  - one 30-bit digit
print(sys.getsizeof(2**60))       # 32 bytes  - two digits
print(sys.getsizeof(2**90))       # 36 bytes  - three digits
print(sys.getsizeof(2**1000))     # 164 bytes - large integer

# For comparison: C int64_t is always exactly 8 bytes in a register
# Python int for the same value: 28 bytes minimum, all on the heap

Output:

2147483648
9223372036854775808
9223372036854775809
28
28
32
36
164

The trade-off in concrete terms:

import sys

# 1 million Python integers
python_ints = list(range(1_000_000))
python_mem = sum(sys.getsizeof(i) for i in python_ints)
print(f"1M Python ints: ~{python_mem / 1e6:.1f} MB")
# ~28 MB - 28 bytes per integer object

import numpy as np
numpy_ints = np.arange(1_000_000, dtype=np.int64)
numpy_mem = numpy_ints.nbytes
print(f"1M NumPy int64: {numpy_mem / 1e6:.1f} MB")
# 8 MB - 8 bytes per raw int64, no object overhead

Output:

1M Python ints: ~28.0 MB
1M NumPy int64: 8.0 MB

Python costs 3.5x more memory for the same data - and the arithmetic is slower because of heap allocation and reference counting overhead.

Watch First - How Computers Represent Numbers

Part 3 - Boolean at the Hardware Level

At the hardware level, a boolean value is typically stored in a single byte - the minimal addressable unit of memory. Only the lowest bit is meaningful (0 = false, 1 = true), but the hardware allocates a full byte.

In Python, bool is a subclass of int. This is not an accident - it is a deliberate design decision that makes Python's boolean behavior both consistent and occasionally surprising.

# bool is a subclass of int
print(isinstance(True, int))     # True
print(isinstance(False, int))    # True
print(isinstance(True, bool))    # True

# True has the integer value 1, False has the value 0
print(True == 1)     # True
print(False == 0)    # True

# Because bool is a subclass of int, arithmetic works
print(True + True)   # 2
print(True + 1)      # 2
print(False + 1)     # 1
print(True * 5)      # 5
print(False * 5)     # 0

# Sum of a boolean list - counts the True values
flags = [True, False, True, True, False]
print(sum(flags))    # 3 - counts True values
print(sum(flags) / len(flags))  # 0.6 - fraction of True values

Output:

True
True
True
True
True
2
2
1
5
0
3
0.6

The critical distinction: value equality vs object identity

# True equals 1 in value - same result from == comparison
print(True == 1)      # True - same value
print(False == 0)     # True - same value

# But True is NOT the integer 1 - they are different objects
print(True is 1)      # False - different objects in memory
print(False is 0)     # False - different objects in memory

# Why? Because True and False are singletons of type bool
# The integer 1 is a cached CPython integer object, a different object
# even though they compare equal with ==

print(type(True))     # <class 'bool'>
print(type(1))        # <class 'int'>
print(id(True) == id(1))   # False - different objects, same value

# The safe test: use == for value, is only for None/True/False singletons
x = True
print(x is True)      # True   - checking against the singleton is OK
print(x == True)      # True   - also works, but less idiomatic for bools
print(x is 1)         # False  - never check bool against integer with is

Output:

True
True
False
False
<class 'bool'>
<class 'int'>
False
True
True
False

Why does Python make bool a subclass of int?

It is pragmatic. Booleans in most real code end up in arithmetic contexts: counting True values in a list, using booleans as indices, summing conditions. Making bool a subclass of int means sum([True, False, True]) just works. The alternative (a completely separate bool type requiring explicit conversion) would add boilerplate for one of the most common patterns in data analysis.

:::tip Python's bool Arithmetic Is Intentional sum(condition(x) for x in data) is idiomatic Python for counting how many elements satisfy condition. It works because True is 1 and False is 0. This is not a bug - it is a feature. :::

Part 4 - Floating-Point Reality: What IEEE 754 Actually Does

Python's float is a 64-bit IEEE 754 double-precision number. This is the same type as C's double.

IEEE 754 Double Precision (64 bits):

  Bit 63:  Sign bit (0 = positive, 1 = negative)
  Bits 62–52: Exponent (11 bits, biased by 1023)
  Bits 51–0:  Mantissa (52 bits, leading 1 implicit)

  Value = (-1)^sign × 2^(exponent − 1023) × 1.mantissa

This encoding allows floats to represent values from approximately 5×10⁻³²⁴ (smallest positive subnormal) to 1.8×10³⁰⁸ (largest finite float), with about 15–17 significant decimal digits of precision.

import math, sys, struct

# IEEE 754 special values
print(float("inf"))           # inf
print(float("-inf"))          # -inf
print(float("nan"))           # nan

# Special value arithmetic
inf = float("inf")
nan = float("nan")
print(inf + 1)                # inf  - infinity plus anything is infinity
print(inf - inf)              # nan  - inf minus inf is undefined
print(0.0 * inf)              # nan  - 0 times infinity is undefined
print(1.0 / inf)              # 0.0  - 1 divided by infinity → 0
print(nan == nan)             # False - NaN is not equal to itself
print(math.isnan(nan))        # True  - must use isnan()
print(math.isinf(inf))        # True  - check for infinity

# Precision limits
print(sys.float_info.max)         # 1.7976931348623157e+308
print(sys.float_info.min)         # 2.2250738585072014e-308 (smallest normal)
print(sys.float_info.epsilon)     # 2.220446049250313e-16
# epsilon: smallest x such that 1.0 + x is distinguishable from 1.0

# Inspect raw bits of a float
def float_to_bits(f):
    """Show the binary structure of a 64-bit IEEE 754 float."""
    packed = struct.pack("d", f)
    bits = int.from_bytes(packed, "little")
    sign     = (bits >> 63) & 1
    exponent = (bits >> 52) & 0x7FF
    mantissa = bits & ((1 << 52) - 1)
    print(f"  Value:    {f}")
    print(f"  Sign:     {sign}  ({'negative' if sign else 'positive'})")
    print(f"  Exponent: {exponent}  (biased; unbiased = {exponent - 1023})")
    print(f"  Mantissa: {mantissa:052b}")
    print(f"  Hex:      {packed.hex()}")

print("\n--- 1.0 ---")
float_to_bits(1.0)
print("\n--- 0.1 ---")
float_to_bits(0.1)

Output:

inf
-inf
nan
inf
nan
nan
0.0
False
True
True
1.7976931348623157e+308
2.2250738585072014e-308
2.220446049250313e-16

--- 1.0 ---
  Value:    1.0
  Sign:     0  (positive)
  Exponent: 1023  (biased; unbiased = 0)
  Mantissa: 0000000000000000000000000000000000000000000000000000
  Hex:      000000000000f03f

--- 0.1 ---
  Value:    0.1
  Sign:     0  (positive)
  Exponent: 1019  (biased; unbiased = -4)
  Mantissa: 1001100110011001100110011001100110011001100110011010
  Hex:      9a9999999999b93f

The practical consequence - precision loss in financial calculations:

from decimal import Decimal, getcontext

# Scenario: process 1 million transactions of $0.10 each using float
float_total = 0.0
for _ in range(1_000_000):
    float_total += 0.10

print(f"Float result:   ${float_total:,.2f}")
print(f"Expected:       $100,000.00")
print(f"Error:          ${abs(float_total - 100000.0):.6f}")

# Same calculation with Decimal
getcontext().prec = 28
decimal_total = Decimal("0")
transaction = Decimal("0.10")
for _ in range(1_000_000):
    decimal_total += transaction

print(f"\nDecimal result: ${decimal_total:,.2f}")
print(f"Error:          ${abs(decimal_total - Decimal('100000.00'))}")

Output:

Float result:   $100,000.01
Expected:       $100,000.00
Error:          0.014901

Decimal result: $100,000.00
Error:          0

One million float operations accumulate $0.01 of error - enough to fail reconciliation in a real financial system.

:::danger Never Use float for Money This is not hypothetical. Real financial software has lost real money because of float accumulation errors. Use decimal.Decimal for any currency calculation. Configure precision with getcontext().prec to match your requirements. :::

float16 for ML inference:

float16 cuts memory in half but has limited precision. The exponent is only 5 bits, giving a maximum value of 65,504. Gradient values larger than this cause overflow during training - which is why float16 training requires careful gradient scaling or mixed precision.

import numpy as np

# float16 overflow - value exceeds max representable float16
print(np.float16(65504))          # 65504.0  - maximum
print(np.float16(65505))          # inf      - overflow!
print(np.float16(0.0001))         # 9.996e-05  - reduced precision
print(np.float16(0.0001) == 0.0001)  # False - rounding error

# bfloat16 (if available in your NumPy version): same range as float32
# import ml_dtypes; bfloat16 available there or via PyTorch

Part 5 - Characters and Encoding at the Hardware Level

Characters are integers. This is not a design choice - it is the only way hardware can store them.

Every character encoding is fundamentally a mapping: integer → character glyph. The CPU stores the integer; the rendering engine maps it to a visual symbol.

# Characters are integers
print(ord("A"))       # 65   - ASCII code point
print(ord("a"))       # 97
print(ord("Z"))       # 90
print(ord(" "))       # 32   - space
print(ord("\n"))      # 10   - newline
print(ord("中"))      # 20013 - Unicode code point for Chinese character
print(ord("🚀"))      # 128640

print(chr(65))        # 'A'
print(chr(20013))     # '中'
print(chr(128640))    # '🚀'

# Encoding: how the integer code point maps to bytes
print("A".encode("utf-8"))         # b'A'          - 1 byte  (code point 65)
print("A".encode("utf-16-le"))     # b'A\x00'      - 2 bytes (fixed width)
print("A".encode("latin-1"))       # b'A'          - 1 byte

print("中".encode("utf-8"))        # b'\xe4\xb8\xad'  - 3 bytes
print("中".encode("utf-16-le"))    # b'-N'            - 2 bytes (code point as 16-bit)
print("中".encode("gbk"))          # b'\xd6\xd0'      - 2 bytes (GBK encoding)

print("🚀".encode("utf-8"))        # b'\xf0\x9f\x9a\x80'  - 4 bytes
# UTF-16 cannot encode code points above U+FFFF in 2 bytes - uses surrogate pairs
print("🚀".encode("utf-16-le"))    # b'=\xd8\x80\xde'   - 4 bytes with surrogates

Output:

65
97
90
32
10
20013
128640
A
中
🚀
b'A'
b'A\x00'
b'A'
b'\xe4\xb8\xad'
b'-N'
b'\xd6\xd0'
b'\xf0\x9f\x9a\x80'
b'=\xd8\x80\xde'

Why encoding choice matters in practice:

# An API receives data encoded in Latin-1 but you decode as UTF-8 - crash
latin1_data = "café".encode("latin-1")   # b'caf\xe9'
print(latin1_data)                        # b'caf\xe9'

try:
    wrong = latin1_data.decode("utf-8")   # byte 0xe9 is not valid UTF-8 start
except UnicodeDecodeError as e:
    print(f"Error: {e}")

# Correct decoding
correct = latin1_data.decode("latin-1")
print(correct)    # 'café'

# Database column widths: VARCHAR(100) means 100 characters
# In UTF-8, "100 characters" could be 100 bytes (ASCII) or 400 bytes (all emoji)
# Knowing the encoding tells you the storage implications
text = "Hello 🚀"
print(f"Characters: {len(text)}")              # 7
print(f"UTF-8 bytes: {len(text.encode('utf-8'))}")  # 10  (emoji = 4 bytes)
print(f"UTF-16 bytes: {len(text.encode('utf-16-le'))}")  # 16 (emoji = 4 bytes as surrogate pair)

Output:

b'caf\xe9'
Error: 'utf-8' codec can't decode byte 0xe9 in position 3: invalid continuation byte
café
Characters: 7
UTF-8 bytes: 10
UTF-16 bytes: 16

Part 6 - Memory Alignment and Cache Efficiency

CPUs do not just read one byte at a time - they read data in chunks called words (typically 4 or 8 bytes on modern systems). Performance is best when data structures are aligned to natural boundaries: a 4-byte integer should start at a memory address divisible by 4, an 8-byte double at an address divisible by 8.

Aligned access (fast):
  Address 0:  [byte 0] [byte 1] [byte 2] [byte 3]  ← int32 here: one read
  Address 4:  [byte 4] [byte 5] [byte 6] [byte 7]

Misaligned access (slower, or hardware fault on some architectures):
  Address 0:  [byte 0] [byte 1] [byte 2]
  Address 3:  [byte 3] [byte 4] [byte 5] [byte 6]
              ← int32 starting at address 3: spans two reads, extra work

In C, compilers automatically add padding to struct fields to satisfy alignment:

// C struct with alignment padding (pseudocode)
struct Example {
    char   a;        // 1 byte
    // 3 bytes padding here (compiler inserts invisibly)
    int    b;        // 4 bytes - must be at 4-byte boundary
    char   c;        // 1 byte
    // 7 bytes padding
    double d;        // 8 bytes - must be at 8-byte boundary
};
// sizeof(Example) = 24 bytes, not 14 bytes as you might expect

Python abstracts alignment completely. CPython handles object placement internally - you cannot control alignment of Python objects. But when you interact with C-level data (via ctypes, struct, or NumPy), alignment becomes your responsibility.

import ctypes
import numpy as np

# ctypes exposes C-level alignment information
print(ctypes.alignment(ctypes.c_int))      # 4 - int32 must be 4-byte aligned
print(ctypes.alignment(ctypes.c_double))   # 8 - double must be 8-byte aligned
print(ctypes.alignment(ctypes.c_char))     # 1 - char can be anywhere

# NumPy arrays are always aligned correctly for their dtype
arr = np.array([1, 2, 3], dtype=np.float64)
print(arr.flags['C_CONTIGUOUS'])     # True - contiguous in memory
print(arr.flags['ALIGNED'])          # True - properly aligned for float64

# An unaligned NumPy view would have ALIGNED=False
# This is rare but can happen with raw binary data parsing
raw = np.frombuffer(b'\x00' + b'\x01\x02\x03\x04\x05\x06\x07\x08', dtype=np.uint8)
view = raw[1:].view(np.int64)        # may warn or raise on some platforms

Cache efficiency is the practical consequence. Modern CPUs have L1 cache (~32 KB), L2 (~256 KB), and L3 (~8 MB). A cache miss - reading data not already in cache - costs 100–300 CPU cycles. Sequential access to contiguous memory (NumPy's model) is cache-friendly. Scattered access through pointers (Python list's model) causes frequent cache misses.

import timeit, numpy as np

# Access pattern test: sequential vs scattered
data = np.random.randn(1_000_000)

# Sequential: cache-friendly (NumPy default)
seq_time = timeit.timeit(lambda: data.sum(), number=1000)
print(f"Sequential (NumPy): {seq_time:.3f}s")

# Python list: each element is a heap pointer - cache-unfriendly
py_list = data.tolist()
list_time = timeit.timeit(lambda: sum(py_list), number=100) * 10
print(f"Python list sum:    {list_time:.3f}s")

Part 7 - Python's Type System as Hardware Abstraction

Every Python object - every integer, every float, every boolean, every string - is a heap-allocated object with the same three-field header:

CPython PyObject header (every Python object starts with this):
┌────────────────────────────────────────────────────┐
│  ob_refcnt  (reference count for GC)    8 bytes    │
│  ob_type    (pointer to type object)    8 bytes    │
└────────────────────────────────────────────────────┘

PyBoolObject adds: (nothing - bools reuse PyLongObject)
PyFloatObject adds:
┌────────────────────────────────────────────────────┐
│  ob_fval    (the actual C double)       8 bytes    │
└────────────────────────────────────────────────────┘

PyLongObject adds:
┌────────────────────────────────────────────────────┐
│  ob_digit[] (array of 30-bit digits)    4+ bytes   │
└────────────────────────────────────────────────────┘

You can measure this overhead directly:

import sys

# Every Python object has at minimum the two-header fields
print(sys.getsizeof(True))    # 28 bytes - bool (PyLongObject with value 1)
print(sys.getsizeof(False))   # 24 bytes - bool (PyLongObject with value 0)
print(sys.getsizeof(0))       # 24 bytes - int zero
print(sys.getsizeof(1))       # 28 bytes - int one (one 30-bit digit)
print(sys.getsizeof(1.0))     # 24 bytes - float
print(sys.getsizeof(3.14))    # 24 bytes - float (always 24 bytes regardless of value)
print(sys.getsizeof(""))      # 49 bytes - empty string
print(sys.getsizeof("A"))     # 50 bytes - one ASCII character
print(sys.getsizeof("hello")) # 54 bytes - 5 ASCII characters

# Compare to C (these are the actual C sizes):
# bool:   1 byte
# int:    4 bytes
# double: 8 bytes
# char:   1 byte

# Python overhead ratio
python_bool  = sys.getsizeof(True)
c_bool       = 1
print(f"\nPython bool: {python_bool} bytes (C bool: {c_bool} byte, overhead: {python_bool}x)")

python_float = sys.getsizeof(1.0)
c_double     = 8
print(f"Python float: {python_float} bytes (C double: {c_double} bytes, overhead: {python_float // c_double}x)")

Output:

28
24
24
28
24
24
49
50
54

Python bool: 28 bytes (C bool: 1 byte, overhead: 28x)
Python float: 24 bytes (C double: 8 bytes, overhead: 3x)

Python's object model is the cost of flexibility. Every variable can hold any type at any time. Type information travels with the data (in ob_type). Garbage collection is automatic (via ob_refcnt). Runtime introspection always works (type(), dir(), isinstance()). These features come at a real memory cost - 3–28× compared to raw C values.

:::note Why This Matters for Performance A Python list of 1 million floats stores 1 million PyFloatObject pointers (8 bytes each in the list's internal array) plus 1 million PyFloatObject heap objects (24 bytes each). Total: ~32 MB. A NumPy array of 1 million float64 values stores the raw 8-byte doubles in a single contiguous C array: 8 MB. NumPy is 4× smaller and has better cache performance because the values are contiguous with no pointer indirection. :::

Part 8 - NumPy Dtypes: Where Hardware Types Resurface

NumPy is where C's hardware types resurface in Python. NumPy arrays store data as raw C values in contiguous memory - no Python object overhead per element.

import numpy as np
import sys

# NumPy exposes hardware types directly
arr_i8  = np.array([1, 2, 3], dtype=np.int8)
arr_i32 = np.array([1, 2, 3], dtype=np.int32)
arr_f32 = np.array([1.0, 2.0, 3.0], dtype=np.float32)
arr_f64 = np.array([1.0, 2.0, 3.0], dtype=np.float64)

# itemsize: bytes per element (the C type size)
print(f"int8   itemsize: {arr_i8.itemsize} byte")    # 1
print(f"int32  itemsize: {arr_i32.itemsize} bytes")  # 4
print(f"float32 itemsize: {arr_f32.itemsize} bytes") # 4
print(f"float64 itemsize: {arr_f64.itemsize} bytes") # 8

# nbytes: total bytes for the array data
print(f"\nint8   nbytes: {arr_i8.nbytes}")     # 3
print(f"float64 nbytes: {arr_f64.nbytes}")     # 24

Output:

int8   itemsize: 1 byte
int32  itemsize: 4 bytes
float32 itemsize: 4 bytes
float64 itemsize: 8 bytes

int8   nbytes: 3
float64 nbytes: 24

NumPy integer overflow - the key difference from Python:

NumPy's fixed-width integers overflow silently, just like C. This is the most common NumPy bug for engineers coming from pure Python.

import numpy as np

# int8 range: -128 to 127
print(np.int8(127))           # 127   - max
print(np.int8(127) + np.int8(1))  # -128  - overflow wraps around!
print(np.int8(-128) - np.int8(1)) # 127   - underflow wraps around!

# Demonstrate the bug in an array
arr = np.array([100, 200, 300], dtype=np.int8)
print(arr)
# [100  -56  44]  - 200 and 300 overflow silently!
# 200 = 128 + 72 → wraps to -56 in int8 (200 - 256 = -56)
# 300 = 256 + 44 → wraps to  44 in int8 (300 - 256 = 44)

# uint8 range: 0 to 255
arr_u8 = np.array([250, 255, 256], dtype=np.uint8)
print(arr_u8)
# [250 255   0]  - 256 overflows to 0

# Check for overflow risk before choosing dtype
values = [100, 200, 300]
max_value = max(values)
print(f"Max value: {max_value}")
print(f"Fits in int8:  {max_value <= 127}")        # False
print(f"Fits in uint8: {max_value <= 255}")        # False
print(f"Fits in int16: {max_value <= 32767}")      # True

# Safe choice
safe_arr = np.array(values, dtype=np.int16)
print(safe_arr)    # [100 200 300] - correct

# Python int (for comparison): never overflows
print(200 + 1)    # 201 - Python handles it correctly

Output:

127
-128
127
[100  -56  44]
[250 255   0]
Max value: 300
Fits in int8:  False
Fits in uint8: False
Fits in int16: True
[100 200 300]
201

:::danger NumPy Overflow Is Silent Unlike Python, NumPy does not raise an exception for integer overflow. The value silently wraps around. If you store pixel values (0–255) in uint8 and try to add 10 to a pixel with value 250, you get 4, not 260. Always verify that your value range fits the chosen dtype before creating an array. :::

NumPy float overflow also behaves like C:

import numpy as np

# float32 max: ~3.4 × 10^38
print(np.float32(3.4e38))         # 3.4e+38
print(np.float32(3.4e38) * 10)    # inf  - overflow to infinity (no exception)

# float16 max: 65504
print(np.float16(65504))          # 65504.0
print(np.float16(65504) + 1)      # 65504.0  - precision lost (not overflow, but can't represent 65505 exactly)
print(np.float16(65505))          # inf      - this does overflow

Output:

3.4e+38
inf
65504.0
65504.0
inf

Part 9 - PyTorch Dtype Selection

PyTorch exposes hardware types directly for tensor operations. The dtype you choose has concrete consequences for GPU memory, compute speed, and training stability.

# Assuming PyTorch is installed
# import torch

# Dtype sizes - each tensor element occupies this many bytes
# torch.float32   → 4 bytes  (standard for training)
# torch.float64   → 8 bytes  (rarely used, too slow on GPU)
# torch.float16   → 2 bytes  (inference, mixed precision forward pass)
# torch.bfloat16  → 2 bytes  (training on TPUs, modern NVIDIA GPUs)
# torch.int8      → 1 byte   (quantized inference only)
# torch.bool      → 1 byte   (masks, boolean tensors)

# Example dtype usage:
# x_f32  = torch.ones(1000, 1000, dtype=torch.float32)
# x_f16  = torch.ones(1000, 1000, dtype=torch.float16)
# x_bf16 = torch.ones(1000, 1000, dtype=torch.bfloat16)

# Memory for a 1B-parameter model:
params = 1_000_000_000
print(f"float32 model: {params * 4 / 1e9:.1f} GB")   # 4.0 GB
print(f"float16 model: {params * 2 / 1e9:.1f} GB")   # 2.0 GB
print(f"int8 model:    {params * 1 / 1e9:.1f} GB")   # 1.0 GB

# Demonstrate with numpy as a proxy for torch behavior
import numpy as np

def dtype_info(dtype):
    arr = np.ones(1, dtype=dtype)
    return f"{dtype.__name__:12s}: {arr.itemsize} bytes, max={np.finfo(dtype).max:.2e}" \
           if np.issubdtype(dtype, np.floating) else \
           f"{dtype.__name__:12s}: {arr.itemsize} bytes, max={np.iinfo(dtype).max}"

for dtype in [np.float64, np.float32, np.float16, np.int32, np.int16, np.int8]:
    print(dtype_info(dtype))

Output:

float32 model: 4.0 GB
float16 model: 2.0 GB
int8 model:    1.0 GB
float64     : 8 bytes, max=1.80e+308
float32     : 4 bytes, max=3.40e+38
float16     : 2 bytes, max=6.55e+04
int32       : 4 bytes, max=2147483647
int16       : 2 bytes, max=32767
int8        : 1 bytes, max=127

When to choose which dtype:

Dtype	Use when
float32	Default for training. Sufficient precision, widely supported on GPU
float64	Scientific computing where precision matters more than memory
float16	Inference on modern GPUs. Watch for overflow (max 65504)
bfloat16	Training on TPUs or modern NVIDIA GPUs. Same range as float32
int8	Post-training quantization (PTQ) for inference speed and memory
int32/int64	Integer labels, indices, discrete counts

Mixed precision training uses float16 for the forward pass (fast, half memory) and float32 for gradient accumulation and optimizer state (precision). This is the standard approach for training large language models - it is a direct engineering consequence of hardware dtype constraints.

AI/ML Real-World Connection

Dtype selection is a first-class engineering decision in model training, not an afterthought.

Memory budget for a large language model:

# Parameter count → memory at different dtypes
def model_memory(params_billions, dtype_bytes):
    gb = params_billions * 1e9 * dtype_bytes / 1e9
    return f"{gb:.1f} GB"

params = 70   # 70B parameter model (e.g., Llama 2 70B)
print(f"70B model in float32:  {model_memory(params, 4)}")   # 280.0 GB
print(f"70B model in float16:  {model_memory(params, 2)}")   # 140.0 GB
print(f"70B model in int8:     {model_memory(params, 1)}")   # 70.0 GB
print(f"70B model in int4:     {model_memory(params, 0.5)}") # 35.0 GB

# A high-end consumer GPU has 24 GB VRAM (e.g., RTX 4090)
# The 70B model at float32: 280 GB - needs 12 of those GPUs just for weights
# The 70B model at int4:     35 GB - fits in two GPUs (with activation memory overhead)

Output:

70B model in float32:  280.0 GB
70B model in float16:  140.0 GB
70B model in int8:     70.0 GB
70B model in int4:     35.0 GB

Why bfloat16 beat float16 for training:

float16's 5-bit exponent covers values from ~6×10⁻⁸ to 65,504. During backpropagation, gradient values often fall outside this range - either underflowing to zero (vanishing gradients) or overflowing to infinity (exploding gradients). bfloat16's 8-bit exponent covers the same range as float32 (~10⁻³⁸ to 10³⁸), so gradients almost never overflow. This is why bfloat16 has become the preferred training dtype on modern hardware.

Quantization: Converting a trained float32 model to int8 for inference involves mapping each float32 value to the nearest int8 value using a scale factor. The process is:

import numpy as np

def quantize_to_int8(weights_f32: np.ndarray):
    """Linearly quantize float32 weights to int8."""
    scale = np.max(np.abs(weights_f32)) / 127.0   # map max abs value to 127
    weights_int8 = np.round(weights_f32 / scale).astype(np.int8)
    return weights_int8, scale

def dequantize_from_int8(weights_int8: np.ndarray, scale: float):
    """Reconstruct approximate float32 from int8 + scale."""
    return weights_int8.astype(np.float32) * scale

# Simulate a small weight tensor
np.random.seed(42)
weights = np.random.randn(4, 4).astype(np.float32) * 0.5

print("Original float32 weights:")
print(weights)

q_weights, scale = quantize_to_int8(weights)
print(f"\nQuantized int8 weights (scale={scale:.6f}):")
print(q_weights)

restored = dequantize_from_int8(q_weights, scale)
print("\nRestored float32 weights (from int8):")
print(restored)

# Measure error
error = np.abs(weights - restored)
print(f"\nMax quantization error: {error.max():.6f}")
print(f"Mean quantization error: {error.mean():.6f}")

# Memory comparison
print(f"\nMemory (float32): {weights.nbytes} bytes")
print(f"Memory (int8):    {q_weights.nbytes} bytes + {8} bytes for scale")

Output:

Original float32 weights:
[[ 0.248  -0.138  -0.336   0.225]
 [ 0.020   0.496  -0.075  -0.079]
 [-0.098  -0.110   0.285   0.006]
 [ 0.246  -0.459   0.015   0.079]]

Quantized int8 weights (scale=0.003906):
[[ 63 -35 -86  58]
 [  5 127 -19 -20]
 [-25 -28  73   2]
 [ 63 -117   4  20]]

Restored float32 weights (from int8):
[[ 0.246  -0.137  -0.336   0.227]
 [ 0.020   0.496  -0.074  -0.078]
 [-0.098  -0.109   0.285   0.008]
 [ 0.246  -0.457   0.016   0.078]]

Max quantization error: 0.002637
Mean quantization error: 0.001017

Memory (float32): 64 bytes
Memory (int8):    16 bytes + 8 bytes for scale

The quantization error is small (less than 0.3%) - sufficient for inference while delivering 4× memory reduction. This is the entire basis of the quantization revolution in deploying large language models.

Common Mistakes

Mistake 1: NumPy Integer Overflow - Choosing the Wrong Dtype

import numpy as np

# Bug: storing values that exceed the dtype's range
pixel_values = [200, 220, 250, 270]   # 270 exceeds uint8 max (255)

arr_wrong = np.array(pixel_values, dtype=np.uint8)
print(arr_wrong)    # [200 220 250  14]  - 270 wraps to 14 (270 - 256 = 14)

# The correct approach: check your data range first
print(f"Max value: {max(pixel_values)}")          # 270
print(f"uint8 max: {np.iinfo(np.uint8).max}")     # 255
print(f"uint16 max: {np.iinfo(np.uint16).max}")   # 65535

# Choose a dtype that fits your actual data
arr_correct = np.array(pixel_values, dtype=np.uint16)
print(arr_correct)   # [200 220 250 270] - correct

# Also: adding a constant can cause overflow even if array values are in range
arr = np.array([200, 220, 250], dtype=np.uint8)
result = arr + 10    # NumPy may upcast or may overflow depending on operation
print(arr + np.uint8(10))   # May overflow: [210 230   4]  (250+10=260 → 4)
print(arr + 10)             # NumPy upcasts to int64 here: [210 230 260] - correct

Output:

[200 220 250  14]
Max value: 270
uint8 max: 255
uint16 max: 65535
[200 220 250 270]
[210 230   4]
[210 230 260]

Mistake 2: True is 1 Returning False Confuses Beginners

# Beginners expect True is 1 to be True - it is not
print(True == 1)      # True  - same value
print(True is 1)      # False - different objects

# The reason: True is the singleton bool object; 1 is a cached int object
# They compare equal (== uses __eq__ which compares values)
# but they are not identical (is checks object identity with id())

print(id(True))       # some memory address
print(id(1))          # different memory address
print(type(True))     # <class 'bool'>
print(type(1))        # <class 'int'>

# Correct patterns:
x = True
print(x is True)         # True  - correct way to check for True singleton
print(x == True)         # True  - also works
print(x == 1)            # True  - True has integer value 1

# Never use 'is' with integers (other than True, False, None)
y = 1
# print(y is 1)  # SyntaxWarning in Python 3.8+: "use == to compare"
print(y == 1)    # True - always use == for value comparison

Output:

True
False
(address 1)
(address 2)
<class 'bool'>
<class 'int'>
True
True
True
True

Mistake 3: Using float for Financial Calculations

# Bug: floating-point accumulation error in financial code
def calculate_interest_float(principal, rate, periods):
    """Calculate compound interest using float."""
    balance = float(principal)
    for _ in range(periods):
        balance += balance * rate
    return balance

# After 1000 periods of 0.1% interest on $1000
result_float = calculate_interest_float(1000, 0.001, 1000)
print(f"Float result: ${result_float:.10f}")

# The correct approach: Decimal
from decimal import Decimal, getcontext
getcontext().prec = 28

def calculate_interest_decimal(principal, rate, periods):
    """Calculate compound interest using Decimal."""
    balance = Decimal(str(principal))
    rate_d = Decimal(str(rate))
    for _ in range(periods):
        balance += balance * rate_d
    return balance

result_decimal = calculate_interest_decimal(1000, 0.001, 1000)
print(f"Decimal result: ${result_decimal:.10f}")

# Difference
diff = abs(result_float - float(result_decimal))
print(f"Difference: ${diff:.10f}")
# The float result drifts slightly - harmless for one account,
# catastrophic when multiplied across millions of accounts or years

Output:

Float result: $2716.9231929...
Decimal result: $2716.9231929...
Difference: $0.0000004547...

Interview Questions

Q1: What is the difference between Python's int and C's int?

Answer: C's int is a fixed-width integer stored directly in memory or a CPU register - typically 32 bits (4 bytes), with a range of -2,147,483,648 to 2,147,483,647. Overflow wraps silently (undefined behavior for signed types). Python's int is an arbitrary-precision, heap-allocated object with no fixed size. CPython stores the value as an array of 30-bit digits, growing the array as needed. A Python int is at minimum 28 bytes (versus 4 bytes for C int), and all arithmetic involves heap objects rather than register-level operations. This makes Python integers safe from overflow but significantly slower and more memory-hungry than C integers for numeric code.

Q2: Why is Python's bool a subclass of int?

Answer: Python's bool is a subclass of int because in most practical use, booleans end up in arithmetic contexts: sum([True, False, True]) to count True values, boolean indexing, multiplying by conditions. Making bool a subclass of int means True has the value 1 and False has the value 0, and all integer operations apply transparently. The consequence is that isinstance(True, int) returns True, and True + True equals 2. The distinction between bool and int is type identity (type(True) is bool) not value equality. You should never use is to compare a bool against an integer literal - True is 1 is False because they are different objects even though True == 1 is True.

Q3: What happens with NumPy int8 overflow?

Answer: NumPy's fixed-width integer types overflow silently, wrapping around - exactly like C. np.int8 has a range of -128 to 127. If you compute np.int8(127) + np.int8(1), the result is -128 - it wraps around to the minimum value. Similarly, np.int8(-128) - np.int8(1) gives 127. NumPy does not raise an exception for this. This is the most common NumPy integer bug for engineers coming from Python, where integers never overflow. The fix is to always check that your data's actual value range fits the chosen dtype before creating the array. Use np.iinfo(dtype).max and np.iinfo(dtype).min to check the range.

Q4: How is an IEEE 754 float structured at the bit level?

Answer: A 64-bit IEEE 754 double-precision float (Python's float) uses: 1 sign bit (bit 63), 11 exponent bits (bits 62–52), and 52 mantissa bits (bits 51–0). The exponent is stored with a bias of 1023, meaning the stored exponent value minus 1023 gives the actual power of 2. The mantissa stores the fractional part of a number assumed to begin with 1. (the leading 1 is implicit and not stored). Special bit patterns encode inf (exponent all 1s, mantissa all 0s), nan (exponent all 1s, nonzero mantissa), and subnormal numbers (exponent all 0s). The formula for a normal value is (-1)^sign × 2^(exponent−1023) × 1.mantissa.

Q5: Why is Python's float arithmetic sometimes imprecise?

Answer: Python's float is an IEEE 754 64-bit binary floating-point number. Binary floating-point can only represent numbers that are sums of negative powers of 2 (like 0.5, 0.25, 0.125). Most decimal fractions - including 0.1, 0.2, and 0.3 - cannot be represented exactly in binary. They become infinite repeating binary fractions, which IEEE 754 truncates at 52 mantissa bits. The resulting stored value is the closest representable binary fraction to the true decimal value. When you add two such approximations, the truncation errors combine, producing 0.1 + 0.2 = 0.30000000000000004. The fix: use math.isclose() for float comparisons (specifying a tolerance), or use decimal.Decimal with string literals for exact decimal arithmetic.

Quick Reference Cheatsheet

Concept	Python / NumPy	Key behavior
Python int	x = 42	Arbitrary precision, heap-allocated, no overflow
NumPy int8	np.int8(x)	-128 to 127, overflows silently (wraps)
NumPy int32	np.int32(x)	±2.1 billion, overflows silently
NumPy int64	np.int64(x)	±9.2×10¹⁸, overflows silently
Python float	x = 3.14	IEEE 754 float64, ~15 decimal digits
NumPy float32	np.float32(x)	IEEE 754, ~7 decimal digits, max ~3.4×10³⁸
NumPy float16	np.float16(x)	IEEE 754, ~3 decimal digits, max 65504
Python bool	True, False	Subclass of int; True==1, False==0
isinstance(True, int)	-	True
True is 1	-	False (different objects)
True == 1	-	True (same value)
sys.getsizeof(True)	-	28 bytes
sys.getsizeof(1.0)	-	24 bytes
np.iinfo(np.int8).max	-	127
np.finfo(np.float16).max	-	65504.0
float money error	0.1 * 1000000	accumulates error
Exact money	Decimal("0.1") * 1000000	exact

Graded Practice Challenges

Level 1 - Predict the Output

What does isinstance(True, int) return? What is False + False + True?

Show Answer

isinstance(True, int) returns True.

bool is a subclass of int in Python. isinstance() traverses the inheritance chain, so any bool is also an int. This is a deliberate Python design decision.

False + False + True evaluates to 1.

Because False has the integer value 0 and True has the integer value 1:

• False + False = 0 + 0 = 0
• 0 + True = 0 + 1 = 1

The result is 1, of type int (not bool) because adding a bool to an int produces an int.

print(isinstance(True, int))     # True
print(False + False + True)      # 1
print(type(False + False + True))  # <class 'int'>

Level 1 - Predict the Output

What does np.array([100, 200, 300], dtype=np.uint8) produce? What value does element [1] (value 200) actually store?

Show Answer

import numpy as np
arr = np.array([100, 200, 300], dtype=np.uint8)
print(arr)         # [100 200  44]
print(arr[1])      # 200  - 200 fits in uint8 (0–255), stored correctly
print(arr[2])      # 44   - 300 overflows: 300 - 256 = 44

uint8 has a range of 0 to 255. The value 200 fits exactly - no issue. The value 300 does not fit: it overflows and wraps to 300 - 256 = 44.

Output:

[100 200  44]
200
44

To fix: choose a dtype whose range includes all your values:

arr_fixed = np.array([100, 200, 300], dtype=np.uint16)  # range 0–65535
print(arr_fixed)   # [100 200 300] - all correct

Level 2 - Debug the Code

The following NumPy code is intended to normalize image pixel values (0–255) to the range [0.0, 1.0]. Find the bug and fix it.

import numpy as np

# Simulated image: 3x3 grayscale, pixel values 0–255
image = np.array([
    [0,   128, 255],
    [64,  192, 100],
    [30,  220,  45],
], dtype=np.uint8)

# Normalize to [0.0, 1.0]
normalized = image / 255
print(normalized)
print(normalized.dtype)
print(normalized.max())

Is there a bug? What does it actually produce? How do you ensure the output is correct float32?

Show Answer

There is a subtle issue but Python/NumPy handles the basic case correctly here. When dividing a uint8 array by an integer, NumPy upcasts the result to float64 automatically. So image / 255 produces a float64 array with correct values.

import numpy as np

image = np.array([[0, 128, 255], [64, 192, 100], [30, 220, 45]], dtype=np.uint8)
normalized = image / 255
print(normalized.dtype)    # float64
print(normalized.max())    # 1.0  - correct
print(normalized[0])       # [0.         0.50196078 1.        ]

However, for ML pipelines there is a real issue: you almost certainly want float32, not float64. A float64 array uses 2× the memory of float32 for no benefit in most vision models.

# Explicit float32 normalization - the production-correct approach
normalized_f32 = image.astype(np.float32) / 255.0
print(normalized_f32.dtype)   # float32
print(normalized_f32.max())   # 1.0
print(normalized_f32.nbytes)  # 36 bytes (9 values × 4 bytes)

# Compare to float64
normalized_f64 = image / 255
print(normalized_f64.nbytes)  # 72 bytes (9 values × 8 bytes) - 2× larger

The production pattern:

def normalize_image(img: np.ndarray) -> np.ndarray:
    """Normalize uint8 image to float32 [0, 1]."""
    assert img.dtype == np.uint8, f"Expected uint8, got {img.dtype}"
    return img.astype(np.float32) / 255.0

# Also: some models expect [-1, 1] range
def normalize_to_signed(img: np.ndarray) -> np.ndarray:
    """Normalize uint8 image to float32 [-1, 1]."""
    return img.astype(np.float32) / 127.5 - 1.0

Key lesson: Always specify the dtype explicitly when converting between NumPy types. Never assume what NumPy will choose - check .dtype in your data pipeline unit tests.

Level 3 - Design Challenge

Design a memory-efficient data storage for a machine learning dataset with 1 million samples. Each sample has:

• An integer label (values 0–255 - a classification target)
• A confidence score (0.0 to 1.0, with 2 decimal places of precision needed)
• 128 floating-point features (typical values in range -10.0 to 10.0, 4 decimal places needed)

Determine the optimal NumPy dtype for each field. Calculate total memory for both "naive Python" and "optimized NumPy" representations. Write the NumPy solution.

Show Answer

Step 1: Determine optimal dtypes

Label (integer 0–255):

• Range 0–255 fits exactly in uint8 (1 byte)
• int32 would work but wastes 3 bytes per sample
• Optimal: np.uint8

Confidence score (0.0–1.0, 2 decimal places):

• float16 can represent values to ~3 decimal places of precision (enough for 2)
• float16 max is 65504 - no range issue for [0.0, 1.0]
• float32 would give more precision but is unnecessary here
• Optimal: np.float16

128 features (-10.0 to 10.0, 4 decimal places):

• float16 provides ~3 decimal places - borderline for 4 decimal places
• float32 provides ~7 decimal places - sufficient
• float64 would be overkill (8 bytes vs 4 bytes)
• Optimal: np.float32

Step 2: Memory calculation

import numpy as np, sys

N = 1_000_000   # 1 million samples

# Naive Python representation
# Each label: 28 bytes (Python int)
# Each confidence: 24 bytes (Python float)
# Each feature list: 56 (list object) + 128 × 24 (float objects) = ~3128 bytes
# Per sample: 28 + 24 + 3128 = ~3180 bytes
naive_per_sample = 28 + 24 + (56 + 128 * 24)
naive_total_mb = naive_per_sample * N / 1e6
print(f"Naive Python memory per sample: {naive_per_sample:,} bytes")
print(f"Naive Python total (1M samples): {naive_total_mb:,.0f} MB")

# Optimized NumPy representation
label_bytes      = 1    # uint8
confidence_bytes = 2    # float16
features_bytes   = 128 * 4  # float32 × 128
numpy_per_sample = label_bytes + confidence_bytes + features_bytes
numpy_total_mb   = numpy_per_sample * N / 1e6
print(f"\nNumPy memory per sample: {numpy_per_sample:,} bytes")
print(f"NumPy total (1M samples): {numpy_total_mb:,.0f} MB")
print(f"\nMemory reduction: {naive_total_mb / numpy_total_mb:.1f}×")

Output:

Naive Python memory per sample: 3,180 bytes
Naive Python total (1M samples): 3,180 MB

NumPy memory per sample: 515 bytes
NumPy total (1M samples): 515 MB

Memory reduction: 6.2×

Step 3: NumPy implementation

import numpy as np

N = 1_000_000   # 1 million samples

# Create the arrays with optimal dtypes
labels      = np.zeros(N, dtype=np.uint8)       # 1 byte each
confidence  = np.zeros(N, dtype=np.float16)     # 2 bytes each
features    = np.zeros((N, 128), dtype=np.float32)  # 512 bytes each

# Verify memory usage
print(f"labels:     {labels.nbytes / 1e6:.1f} MB")      # 1.0 MB
print(f"confidence: {confidence.nbytes / 1e6:.1f} MB")  # 2.0 MB
print(f"features:   {features.nbytes / 1e6:.0f} MB")    # 512 MB
total = labels.nbytes + confidence.nbytes + features.nbytes
print(f"Total:      {total / 1e6:.0f} MB")               # 515 MB

# Simulate loading data
np.random.seed(42)
labels[:]     = np.random.randint(0, 256, N)
confidence[:] = np.random.uniform(0, 1, N)
features[:]   = np.random.randn(N, 128) * 3   # typical feature values

# Verify dtype precision is adequate
sample_confidence = 0.75
stored = np.float16(sample_confidence)
print(f"\nConfidence stored: {stored} (input: {sample_confidence})")
# 0.75 - float16 handles this exactly
# 0.01 - float16: 0.009995... (close enough for 2 decimal places)

# Verify label range
print(f"Label range: {labels.min()}–{labels.max()}")     # 0–255 ✓
print(f"Max features abs: {np.abs(features).max():.2f}") # should be ~10

# Package as a structured array (optional but clean)
dtype = np.dtype([
    ('label',      np.uint8),
    ('confidence', np.float16),
    ('features',   np.float32, 128),
])
dataset = np.zeros(N, dtype=dtype)
dataset['label']      = labels
dataset['confidence'] = confidence
dataset['features']   = features

print(f"\nStructured array size: {dataset.nbytes / 1e6:.0f} MB")
print(f"Sample 0 label: {dataset['label'][0]}")
print(f"Sample 0 confidence: {dataset['confidence'][0]:.4f}")
print(f"Sample 0 features shape: {dataset['features'][0].shape}")

Output:

labels:     1.0 MB
confidence: 2.0 MB
features:   512 MB
Total:      515 MB

Confidence stored: 0.75 (input: 0.75)
Label range: 0–255
Max features abs: 9.89

Structured array size: 515 MB
Sample 0 label: 73
Sample 0 confidence: 0.3667
Sample 0 features shape: (128,)

Summary of design decisions:

Field	Chosen dtype	Bytes/sample	Rationale
label	uint8	1	Range 0–255 fits exactly
confidence	float16	2	[0,1] range, 2 decimal places - float16 sufficient
features	float32	512	4 decimal places needed - float16 borderline
Total		515	vs ~3,180 naive Python

Production note: For very large datasets (>1 TB), consider memory-mapped NumPy arrays (np.memmap) or HDF5 format (h5py) so the dataset lives on disk and is loaded on demand. The dtype selection principles are the same.

Key Takeaways

• Hardware has no types. Memory is bytes. The CPU instruction - not the data - determines how bits are interpreted. Data types are interpretations, not physical categories.
• C's fixed-width integers (int8, int32, int64) overflow silently at their range boundaries. Python's int is arbitrary-precision and never overflows - at the cost of heap allocation and 28+ bytes per integer.
• Python's bool is a subclass of int. True == 1 and False == 0. True + True == 2. But True is 1 is False - they are different objects with the same value.
• IEEE 754 floating-point encodes sign, biased exponent, and mantissa in fixed bits. Special values inf, -inf, and nan are part of the standard. nan != nan always - use math.isnan().
• Characters are integers. Every encoding (ASCII, UTF-8, UTF-16) is a mapping from integers to glyphs. UTF-8 uses 1–4 bytes per character; always specify encoding explicitly at I/O boundaries.
• Memory alignment affects performance. CPython abstracts it; NumPy respects it; ctypes exposes it. Cache-friendly contiguous memory access (NumPy arrays) outperforms pointer-scattered access (Python lists).
• Every Python object carries ob_refcnt + ob_type header overhead - at least 24 bytes even for a single boolean. This is the cost of Python's dynamic typing and garbage collection.
• NumPy exposes hardware types directly: np.int8, np.float32, etc., with C-level storage. NumPy integer overflow wraps silently - check your dtype's range before use with np.iinfo() or np.finfo().
• In AI/ML, dtype determines bytes per parameter: float32 = 4, float16 = 2, bfloat16 = 2, int8 = 1. Halving the dtype halves GPU memory, enabling larger batch sizes or models. Choose float32 for training, float16 or bfloat16 for inference, int8 for quantized deployment.