Master LLMs - From Transformers to Production Systems
Transformer internals, fine-tuning, RAG, agents, inference optimization, alignment, and reasoning models - the complete LLM engineering curriculum.
17 Modules. Full LLM Stack.
From transformer internals to alignment and safety. Every module links directly to the lessons.
Transformer Architecture
Self-attention, multi-head attention, positional encoding, tokenization, embeddings, and the scaling laws that govern modern LLMs.
What you'll master
- Attention Is All You Need - the 2017 paper
- Self-Attention Mechanism (Q, K, V)
- Multi-Head Attention
- Positional Encoding - Sinusoidal, RoPE, ALiBi
- Feed-Forward Layers & SwiGLU
- Layer Normalization & Residuals
- Encoder vs Decoder vs Encoder-Decoder
- Tokenization - BPE, WordPiece, SentencePiece
- Embedding Spaces
- Scaling Laws - Kaplan & Chinchilla
10 lessons
Start for Free →
Pretraining & Fine-Tuning
Language modeling objectives, BERT, GPT, LoRA, QLoRA, RLHF, DPO - the full training pipeline from pretraining to alignment.
What you'll master
- Language Modeling Objectives (MLM vs CLM)
- Masked Language Modeling - BERT
- Causal Language Modeling - GPT
- Pretraining at Scale (ZeRO, Flash Attention)
- Supervised Fine-Tuning (SFT)
- Instruction Tuning & FLAN
- LoRA - Low-Rank Adaptation
- QLoRA - 4-bit Fine-Tuning
- Full Fine-Tuning vs PEFT
- RLHF - Reward Model + PPO
- DPO - Direct Preference Optimization
- Modern Alignment Techniques
12 lessons
Start for Free →
Prompt Engineering
Zero-shot, few-shot, chain-of-thought, tree-of-thought, ReAct, structured outputs, prompt injection, and DSPy optimization.
What you'll master
- Zero-Shot Prompting
- Few-Shot Prompting & In-Context Learning
- Chain-of-Thought Reasoning
- Tree-of-Thought
- ReAct - Reasoning + Acting
- System Prompts & Context Design
- Prompt Injection & Security
- Structured Output & JSON Mode
- Prompt Optimization & DSPy
9 lessons
Start for Free →
RAG Systems
Chunking, embedding models, vector databases, hybrid search, reranking, Graph RAG, Agentic RAG, and production evaluation patterns.
What you'll master
- Why RAG and When Not To
- Document Chunking Strategies
- Embedding Models Deep Dive
- Vector Databases (Pinecone, Weaviate, pgvector)
- Retrieval Algorithms - ANN (HNSW, IVF)
- Reranking with Cross-Encoders
- Hybrid Search - Dense + Sparse (BM25)
- RAG Evaluation (RAGAS, TruLens)
- Advanced RAG Patterns
- Graph RAG
- Agentic RAG
11 lessons
Start for Free →
LLM Agents
Tool use, ReAct agents, planning, memory systems, multi-agent architectures, LangChain, LlamaIndex, and safety guardrails.
What you'll master
- Tool Use & Function Calling
- ReAct Agent Pattern
- Planning & Reasoning in Agents
- Memory Systems - Short & Long Term
- Multi-Agent Architectures
- Agent Evaluation
- LangChain Deep Dive
- LlamaIndex Deep Dive
- Agent Safety & Guardrails
9 lessons
Start for Free →
LLM Evaluation
Perplexity, BLEU, ROUGE, human eval, LLM-as-judge, MMLU, HumanEval, safety evaluation, and production monitoring.
What you'll master
- Perplexity & Language Model Metrics
- BLEU, ROUGE & Generation Metrics
- Human Evaluation Protocols
- LLM-as-Judge
- Benchmarks - MMLU, HumanEval, HELM
- Safety & Bias Evaluation
- RAG Evaluation Metrics
- Production Monitoring for LLMs
8 lessons
Start for Free →
LLM Inference & Optimization
KV cache, quantization, speculative decoding, continuous batching, tensor parallelism, vLLM, and inference cost optimization.
What you'll master
- Autoregressive Decoding
- KV Cache - Memory and Architecture
- Sampling Strategies (Temperature, Top-K, Top-P)
- Quantization - INT8 & INT4
- Speculative Decoding
- Continuous Batching
- Tensor & Pipeline Parallelism
- vLLM & Inference Servers
- Inference Cost Optimization
9 lessons
Start for Free →
Multimodal Models
Vision-language models, CLIP contrastive learning, diffusion models, audio-language models, and production multimodal systems.
What you'll master
- Vision-Language Models (LLaVA, GPT-4V)
- CLIP & Contrastive Learning
- Diffusion Models
- Audio-Language Models (Whisper, Gemini)
- Multimodal RAG
- Production Multimodal Systems
6 lessons
Start for Free →
LLM System Design
LLM product architecture, latency/cost trade-offs, context management, caching, LLM gateways, guardrails, observability, and real case studies.
What you'll master
- LLM Product Architecture
- Latency & Cost Trade-offs
- Context Window Management
- Caching Strategies
- LLM Gateway & Model Routing
- Guardrails & Safety Systems
- Observability for LLM Apps
- Case Studies (production at scale)
8 lessons
Start for Free →
Reasoning Models
Test-time compute, o1/o3 architecture, DeepSeek-R1, process reward models, MCTS for LLMs, and when to use reasoning models.
What you'll master
- Test-Time Compute Scaling
- Chain-of-Thought at Inference
- OpenAI o1 & o3 Architecture
- DeepSeek-R1
- Process Reward Models (PRM)
- Monte Carlo Tree Search for LLMs
- When to Use Reasoning Models
- Evaluating Reasoning Models
8 lessons
Start for Free →
Mixture of Experts
MoE architecture, router mechanisms, sparse vs dense models, Mixtral, DeepSeek-MoE, and inference optimization for sparse models.
What you'll master
- MoE Architecture
- Router Mechanisms (Top-K, Expert Choice)
- Sparse vs Dense Models
- Training MoE Models
- Mixtral Deep Dive
- DeepSeek-MoE
- Inference Optimization for MoE
7 lessons
Start for Free →
State Space Models
Attention limitations, SSM foundations, Mamba architecture, Mamba vs Transformer trade-offs, and hybrid architectures like Jamba.
What you'll master
- Limitations of Attention at Scale
- SSM Foundations (HiPPO, S4)
- Mamba Architecture
- Mamba vs Transformer
- Hybrid Architectures - Jamba
- When to Use SSMs
6 lessons
Start for Free →
Structured Generation
Constrained decoding, Outlines, Instructor, JSON mode, LMQL, and production patterns for reliable structured LLM outputs.
What you'll master
- Why Structured Output Matters
- Constrained Decoding
- Outlines Library
- Instructor Library & Pydantic
- JSON Mode & Tool Schemas
- LMQL & Guidance
- Production Patterns
7 lessons
Start for Free →
Model Merging
Model soup, TIES merging, DARE, SLERP, MergeKit, and the limits of frankenmodels - combining fine-tuned models without retraining.
What you'll master
- Why Model Merging
- Linear Interpolation & Model Soup
- TIES Merging
- DARE
- SLERP
- MergeKit
- Frankenmodels & Limitations
7 lessons
Start for Free →
Long Context Strategies
Attention at long contexts, RoPE/ALiBi, context window extension, lost-in-the-middle, context compression, and the 128k context guide.
What you'll master
- Attention Complexity at Long Contexts
- RoPE & ALiBi
- Context Window Extension (YaRN, NTK)
- Lost-in-the-Middle Problem
- Retrieval-Augmented Context
- Context Compression
- Practical 128k Context Guide
7 lessons
Start for Free →
Alignment & Safety
The alignment problem, RLHF deep dive, Constitutional AI, DPO, red teaming, jailbreaks, AI safety evals, and EU AI Act.
What you'll master
- The Alignment Problem
- RLHF Deep Dive
- Constitutional AI (Anthropic)
- DPO & Modern Alignment
- Red Teaming LLMs
- Jailbreaks & Adversarial Prompts
- AI Safety Evaluations
- EU AI Act & Regulation
8 lessons
Start for Free →
Embeddings Engineering
Embedding models, fine-tuning embeddings, Matryoshka embeddings, evaluation (MTEB), quantization, multimodal embeddings, and production systems.
What you'll master
- What Are Embeddings
- Embedding Models Overview (E5, BGE, GTE)
- OpenAI & API Embeddings
- Fine-Tuning Embedding Models
- Matryoshka Embeddings
- Embedding Evaluation (MTEB)
- Embedding Quantization
- Multimodal Embeddings
- Embeddings in Production
9 lessons
Start for Free →
Ready to understand the models that are changing everything?
From transformer internals to alignment and safety - the complete LLM engineering curriculum.
Start Learning Free →