Research Engineer: 10-Week Prep Path

Reading time: ~55 min | Interview relevance: Critical | Roles: Research Engineer, Research Scientist (Industry), Applied Research Engineer, ML Research Engineer

The Real Interview Moment

You are seated in a virtual interview room at a top AI lab. The researcher across the screen slides a paper onto the shared whiteboard -- one you have never seen before. "Take ten minutes to read the abstract and introduction," she says. "Then walk me through the key contribution, what assumptions the authors make, and how you would extend this work."

Your pulse quickens. This is not a LeetCode problem. There is no single correct answer. The interviewer wants to see how you think about research: Can you identify what matters in a paper? Can you spot methodological weaknesses? Can you propose a meaningful extension that is not just throwing more compute at the problem?

Twenty minutes later, she shifts gears. "Now implement the core algorithm from Section 3. You can use PyTorch." You open an empty code editor. You have forty-five minutes to go from a set of equations on a page to working, tested code.

This is the research engineer interview. It tests a unique combination of skills that no other role demands: mathematical maturity, coding fluency, research taste, and the ability to communicate ideas clearly. It is the role where you bridge the gap between "interesting idea in a paper" and "working system that advances the state of the art."

The good news: these skills are trainable. This 10-week plan will build each one systematically, from strengthening your math foundations through developing genuine research taste. The timeline is longer than other prep paths because research engineering demands depth, not just breadth.

Role Overview

What Research Engineers Do

Research Engineers work at the frontier of AI, embedded within or alongside research teams at organizations like Google DeepMind, Meta FAIR, Anthropic, OpenAI, Microsoft Research, and top university labs. They:

• Implement novel algorithms from papers, often from scratch without reference code
• Run large-scale experiments and manage compute infrastructure for training
• Prototype research ideas rapidly to test hypotheses
• Reproduce results from published papers to validate and build upon them
• Collaborate closely with research scientists on experiment design
• Develop custom tooling, training frameworks, and evaluation pipelines
• Contribute to writing papers and presenting findings
• Maintain and improve shared codebases and research infrastructure

How It Differs from MLE

Dimension	Research Engineer	Machine Learning Engineer
Primary goal	Advance the state of the art	Ship production ML systems
Code lifespan	Experimental, often short-lived	Production-grade, long-lived
Key metric	Research impact (papers, breakthroughs)	Business metrics (latency, revenue)
Math depth	Deep (proofs, derivations, novel methods)	Applied (enough to use and debug models)
Paper reading	Daily activity, core skill	Occasional, stays current
System design	Experiment infrastructure	Production serving infrastructure
Typical output	Paper, prototype, benchmark result	Deployed model, pipeline, API

Interview Format (Typical)

Round	Duration	Focus
Phone Screen	60 min	Coding + ML/math fundamentals
Coding Round	60 min	Algorithm implementation (often ML-specific)
Math/Theory Round	45-60 min	Probability, linear algebra, optimization, proofs
Paper Discussion	45-60 min	Read and critique a paper, propose extensions
Research Presentation	30-45 min	Present your past research or a paper you admire
System/Experiment Design	45-60 min	Design an experiment or research infrastructure
Behavioral/Culture Fit	45 min	Collaboration, research philosophy, motivation

:::tip Not All Labs Are the Same Interview formats vary significantly across labs. DeepMind emphasizes math and algorithmic coding. Anthropic focuses heavily on alignment thinking and careful reasoning. Meta FAIR values strong engineering alongside research. OpenAI tests practical implementation speed. Research your target lab's specific format early in your preparation. :::

Focus Area Allocation

Breakdown by Skill

Paper Reading and Critique (25% -- ~75 hours total)

• Reading and summarizing papers efficiently
• Identifying key contributions, assumptions, and limitations
• Proposing meaningful extensions and follow-up work
• Connecting papers to the broader research landscape
• Reproducing key results and figures

Math Foundations (20% -- ~60 hours total)

• Linear algebra: eigendecomposition, SVD, matrix calculus
• Probability and statistics: Bayesian inference, concentration inequalities, information theory
• Optimization: convex optimization, gradient methods, constrained optimization
• Analysis: convergence proofs, asymptotic notation, approximation theory

Algorithm Implementation (20% -- ~60 hours total)

• Implementing papers from scratch in PyTorch/JAX
• Translating mathematical notation to working code
• Writing clean, testable research code
• Debugging numerical issues (NaN gradients, precision errors)
• Efficient GPU utilization and memory management

Coding and DSA (15% -- ~45 hours total)

• Core data structures and algorithms
• ML-specific coding (custom layers, training loops, data loaders)
• Python proficiency and debugging
• Code quality and testing

Research Taste and Presentation (10% -- ~30 hours total)

• Developing intuition for what makes good research
• Presenting research clearly (both written and verbal)
• Asking good questions during discussions
• Formulating research directions

Behavioral (10% -- ~30 hours total)

• Research collaboration stories
• Handling negative results and pivots
• Mentoring and knowledge sharing
• Motivation and research philosophy

10-Week Schedule Overview

Week 1: Foundations -- Math Refresh and Coding Warmup

Goal: Rebuild mathematical fluency and warm up coding skills. Establish daily habits.

Daily time: 3.5 hours (weekdays), 5 hours (weekends)

Monday -- Linear Algebra I: Vectors and Matrices

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode easy/medium (arrays, strings)
Lunch (20 min)	Read	Coding Interviews overview
Evening (90 min)	Math study	Vector spaces, linear independence, basis, rank, null space
Night (15 min)	Review	Write down key definitions from memory

Math exercises:

• Prove that the set of solutions to Ax = 0 forms a vector space
• Compute rank and null space of a 4x3 matrix by hand
• Show that rank(A) + dim(null(A)) = number of columns

Tuesday -- Linear Algebra II: Eigendecomposition

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium (hash maps, two pointers)
Lunch (20 min)	Read	Gilbert Strang lecture notes on eigenvalues
Evening (90 min)	Math study	Eigenvalues, eigenvectors, diagonalization, spectral theorem, positive definite matrices
Night (15 min)	Review	Work through eigendecomposition of a 3x3 symmetric matrix

Math exercises:

• Compute eigenvalues and eigenvectors of a 3x3 matrix
• Prove that eigenvalues of a real symmetric matrix are real
• Show that eigenvectors of distinct eigenvalues are orthogonal

Wednesday -- Linear Algebra III: SVD and Matrix Calculus

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium (trees, recursion)
Lunch (20 min)	Read	ML Fundamentals overview
Evening (90 min)	Math study	SVD, low-rank approximation, matrix calculus (gradients of matrix expressions), Frobenius norm
Night (15 min)	Review	Derive the gradient of

:::tip Matrix Calculus Is Non-Negotiable Research engineer interviews at top labs will ask you to derive gradients of loss functions, sometimes with non-standard architectures. If you cannot differentiate through matrix operations, you will struggle. The Matrix Cookbook is a useful reference but understanding the derivations is more important than memorizing results. :::

Thursday -- Probability and Statistics I: Foundations

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium (BFS/DFS, graph traversal)
Lunch (20 min)	Read	Review Bayes' theorem and common distributions
Evening (90 min)	Math study	Probability axioms, conditional probability, Bayes' theorem, common distributions (Gaussian, Bernoulli, Poisson, Exponential)
Night (15 min)	Review	Solve 3 probability brain teasers

Math exercises:

• Derive the MLE for the parameters of a Gaussian distribution
• Compute the MAP estimate with a conjugate prior
• Prove that the sum of two independent Gaussians is Gaussian

Friday -- Probability and Statistics II: Estimation and Inference

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium (DP intro)
Lunch (20 min)	Read	MLE vs MAP vs Bayesian inference comparison
Evening (90 min)	Math study	Maximum likelihood, MAP estimation, expectation, variance, covariance, law of large numbers, CLT
Night (15 min)	Review	Write out MLE derivation for logistic regression

Saturday -- Coding Deep Dive: ML-Specific Coding

Time	Activity	Details
Morning (2 hrs)	Implementation	Implement linear regression with gradient descent from scratch (NumPy only)
Afternoon (2 hrs)	Implementation	Implement logistic regression with gradient descent, add L2 regularization
Evening (1 hr)	Review	Compare your implementations with scikit-learn; verify gradient computation numerically

:::note Numerical Gradient Checking Always verify your analytical gradients with numerical gradients: (f(x+h) - f(x-h)) / 2h. This is a fundamental debugging technique for research code. If your analytical and numerical gradients disagree, you have a bug in your derivation or implementation. :::

Sunday -- Week 1 Review and Planning

Time	Activity	Details
Morning (2 hrs)	Review	Re-derive all math from the week without notes
Afternoon (2 hrs)	Coding review	Re-solve any coding problems you struggled with
Evening (1 hr)	Plan	Read Week 2 plan; identify weakest math areas

:::note Week 1 Milestone Checkpoint By the end of Week 1, you should be able to:

• Compute eigendecomposition and SVD of small matrices by hand
• Derive the gradient of common loss functions using matrix calculus
• Explain MLE vs MAP estimation with examples
• Implement linear and logistic regression from scratch with correct gradients
• Solve LeetCode medium problems in under 25 minutes :::

Week 2: Foundations -- Optimization and Coding Patterns

Goal: Master optimization theory and solidify coding patterns for technical interviews.

Daily time: 3.5 hours (weekdays), 5 hours (weekends)

Monday -- Optimization I: Convex Optimization

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium (sorting, binary search)
Lunch (20 min)	Read	Boyd and Vandenberghe Chapter 1 summary
Evening (90 min)	Math study	Convex sets, convex functions, first-order and second-order conditions, Jensen's inequality
Night (15 min)	Review	Prove that the intersection of convex sets is convex

Tuesday -- Optimization II: Gradient Methods

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium (linked lists, stacks)
Lunch (20 min)	Read	Convergence rates of gradient descent
Evening (90 min)	Math study	Gradient descent convergence proof (for convex, strongly convex), learning rate selection, SGD, mini-batch SGD
Night (15 min)	Review	Derive the convergence rate of GD for L-smooth convex functions

Math exercises:

• Prove that gradient descent converges at O(1/T) rate for convex functions
• Show that SGD has O(1/sqrt(T)) convergence for convex functions
• Analyze the effect of batch size on gradient variance

Wednesday -- Optimization III: Advanced Methods

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium (heaps, priority queues)
Lunch (20 min)	Read	Adam optimizer paper (Kingma and Ba, 2014)
Evening (90 min)	Math study	Momentum, Nesterov acceleration, Adam, AdaGrad, RMSProp, learning rate schedules
Night (15 min)	Review	Implement Adam optimizer from scratch

:::warning Understanding vs. Memorizing Optimizers Do not just memorize the update rules. Understand why momentum helps (smooths noisy gradients), why adaptive methods help (per-parameter learning rates for sparse features), and why Adam sometimes fails to converge (non-convergence in certain settings, addressed by AMSGrad). Interviewers will test your understanding of the reasoning. :::

Thursday -- Information Theory

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium (backtracking)
Lunch (20 min)	Read	Cover page and Chapter 2 of Cover and Thomas
Evening (90 min)	Math study	Entropy, cross-entropy, KL divergence, mutual information, data processing inequality
Night (15 min)	Review	Prove that KL divergence is non-negative (Gibbs' inequality)

Math exercises:

• Prove KL(p||q) >= 0 using Jensen's inequality
• Show that H(X,Y) = H(X) + H(Y|X)
• Derive the cross-entropy loss from MLE for classification
• Compute the mutual information between two binary random variables

Friday -- Coding Patterns for Research Interviews

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard (DP)
Lunch (20 min)	Read	Common research coding interview patterns
Evening (90 min)	Implementation	Implement a simple neural network (2-layer MLP) from scratch using only NumPy; train on MNIST
Night (15 min)	Review	Verify gradients numerically; check that loss decreases

Saturday -- Implement Core ML Algorithms

Time	Activity	Details
Morning (2 hrs)	Implementation	Implement k-nearest neighbors and a decision tree from scratch
Afternoon (2 hrs)	Implementation	Implement k-means clustering and PCA from scratch using eigendecomposition
Evening (1 hr)	Review	Test all implementations on toy datasets; compare with scikit-learn

Sunday -- Week 2 Review

Time	Activity	Details
Morning (2 hrs)	Math review	Re-derive optimization convergence proofs
Afternoon (2 hrs)	Coding review	Refactor all implementations for code clarity; add proper docstrings
Evening (1 hr)	Planning	Select 5 papers for Week 3 reading list

:::note Week 2 Milestone Checkpoint

• Prove gradient descent convergence for convex functions
• Implement Adam optimizer from scratch with correct bias correction
• Derive cross-entropy loss from maximum likelihood
• Explain KL divergence and its connection to information theory
• Train a neural network from scratch (NumPy only) on MNIST
• Implement PCA using eigendecomposition from scratch :::

Week 3: Core Skills -- Paper Reading and Deep Learning Theory

Goal: Develop systematic paper reading skills. Deepen understanding of deep learning theory.

Daily time: 4 hours (weekdays), 6 hours (weekends)

Monday -- Paper Reading Technique + Attention Is All You Need

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium (graphs)
Lunch (20 min)	Read	Paper Discussion overview
Evening (120 min)	Paper study	Read "Attention Is All You Need" using the three-pass method: (1) skim for structure, (2) read for understanding, (3) read for critique
Night (15 min)	Write	One-paragraph summary: contribution, method, results, limitations

:::tip The Three-Pass Paper Reading Method Pass 1 (5 min): Read title, abstract, introduction, section headings, conclusion. Understand the high-level contribution. Pass 2 (30 min): Read the full paper, skipping dense proofs/derivations. Understand the method, experimental setup, and main results. Note questions. Pass 3 (30 min): Read carefully, work through key equations, examine figures and tables, think about assumptions and limitations.

This is the method used by working researchers. Practice it consistently and your paper reading speed will increase dramatically. :::

Tuesday -- Implement Self-Attention from Scratch

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium
Lunch (20 min)	Read	Re-read Section 3 of the Transformer paper (attention mechanics)
Evening (120 min)	Implementation	Implement scaled dot-product attention and multi-head attention in PyTorch from the equations in the paper
Night (15 min)	Review	Verify shapes at each step; test with random inputs

Wednesday -- Paper: BERT and Self-Supervised Learning

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Pre-training vs fine-tuning paradigm overview
Evening (120 min)	Paper study	Read BERT paper. Focus on: masked language modeling objective, next sentence prediction, fine-tuning strategy. Write a one-page critique
Night (15 min)	Compare	List differences between BERT, GPT, and T5 pre-training objectives

Thursday -- Backpropagation Theory and Computational Graphs

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard (DP, graphs)
Lunch (20 min)	Read	Computational graph and automatic differentiation overview
Evening (120 min)	Math study	Chain rule in computational graphs, reverse-mode autodiff, forward-mode autodiff, Jacobian-vector products
Night (15 min)	Implementation	Implement a tiny autograd engine (scalar-valued, supporting +, *, ReLU)

Friday -- Paper: ResNet and Training Deep Networks

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Vanishing gradient problem and historical context
Evening (120 min)	Paper study	Read ResNet paper. Focus on: skip connection motivation, identity mapping analysis, why depth helps. Write critique
Night (15 min)	Compare	Compare ResNet, DenseNet, and Highway Networks approaches to depth

Saturday -- Implement a Transformer Block

Time	Activity	Details
Morning (2.5 hrs)	Implementation	Implement a complete Transformer encoder block from scratch: multi-head attention, feed-forward network, layer normalization, residual connections
Afternoon (2 hrs)	Testing	Write unit tests for each component; verify against PyTorch's nn.TransformerEncoderLayer
Evening (1.5 hrs)	Paper	Read the Layer Normalization paper (Ba et al., 2016); understand why LayerNorm works better than BatchNorm in transformers

Sunday -- Week 3 Review

Time	Activity	Details
Morning (2 hrs)	Review	Re-derive self-attention equations; explain them to an imaginary audience
Afternoon (2.5 hrs)	Paper summaries	Write structured summaries of all 3 papers read this week (contribution, method, results, limitations, extensions)
Evening (1.5 hrs)	Plan	Select papers for Week 4; identify math areas needing more work

:::note Week 3 Milestone Checkpoint

• Read a new paper using the three-pass method in under 90 minutes
• Implement multi-head attention from scratch with correct masking
• Explain the Transformer architecture end-to-end with equations
• Articulate why residual connections help training deep networks
• Compare BERT, GPT, and T5 pre-training approaches clearly
• Implement a basic autograd engine :::

Week 4: Core Skills -- Algorithm Implementation and Advanced Papers

Goal: Build speed in implementing algorithms from papers. Read more diverse papers.

Daily time: 4 hours (weekdays), 6 hours (weekends)

Monday -- Paper: Variational Autoencoders

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Latent variable models overview
Evening (120 min)	Paper study	Read Kingma and Welling (2013) VAE paper. Focus on: ELBO derivation, reparameterization trick, relationship to EM algorithm
Night (15 min)	Math	Re-derive ELBO from scratch

Math exercises:

• Derive the ELBO (evidence lower bound) from Jensen's inequality
• Show why the reparameterization trick enables gradient-based optimization
• Prove that maximizing ELBO is equivalent to minimizing KL(q||p) + reconstruction error

Tuesday -- Implement a VAE

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	VAE implementation tips and common pitfalls
Evening (120 min)	Implementation	Implement a VAE in PyTorch: encoder, reparameterization, decoder, ELBO loss. Train on MNIST
Night (15 min)	Evaluate	Generate samples, interpolate in latent space, visualize

Wednesday -- Paper: GANs and Adversarial Training

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	GAN training dynamics overview
Evening (120 min)	Paper study	Read Goodfellow et al. (2014) GAN paper. Focus on: minimax formulation, theoretical analysis, mode collapse. Read WGAN for contrast
Night (15 min)	Math	Derive the optimal discriminator for fixed generator

Thursday -- Paper: Diffusion Models

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Deep Learning generative models section
Evening (120 min)	Paper study	Read DDPM (Ho et al., 2020). Focus on: forward process, reverse process, connection to score matching, simplified training objective
Night (15 min)	Compare	Create a comparison table: VAE vs GAN vs Diffusion (training, generation quality, diversity, speed)

:::warning Generative Models Are a Hot Interview Topic In 2025-2026, expect at least one interview question on generative models at any top AI lab. You do not need to implement diffusion models from scratch, but you must understand the forward and reverse process, the training objective, and how they relate to score matching and variational inference. :::

Friday -- Implement a Training Loop with Best Practices

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Research code best practices (logging, checkpointing, reproducibility)
Evening (120 min)	Implementation	Build a reusable training loop: gradient clipping, learning rate warmup, cosine annealing, mixed precision, checkpointing, W&B logging
Night (15 min)	Review	Compare your training loop with HuggingFace Trainer internals

Saturday -- Speed Implementation Challenge

Time	Activity	Details
Morning (2.5 hrs)	Timed challenge	Pick a paper you have read this week. Set a 2.5-hour timer. Implement the core algorithm from scratch. This simulates interview conditions
Afternoon (2 hrs)	Debugging	Fix any issues. Add tests. Document design decisions
Evening (1.5 hrs)	Paper reading	Read one bonus paper of your choice from a different subfield (RL, NLP, vision, optimization)

Sunday -- Week 4 Review

Time	Activity	Details
Morning (2 hrs)	Math review	Re-derive ELBO, GAN objective, diffusion forward process
Afternoon (2.5 hrs)	Implementation review	Clean up all code from the week; push to a personal GitHub repo
Evening (1.5 hrs)	Paper summaries	Write structured summaries of all papers read this week

:::note Week 4 Milestone Checkpoint

• Derive the ELBO and explain the reparameterization trick
• Implement a working VAE that generates recognizable MNIST digits
• Explain the GAN minimax objective and mode collapse
• Describe the diffusion forward and reverse process
• Implement a complete training loop with all standard bells and whistles
• Complete a timed implementation challenge in under 2.5 hours :::

Week 5: Depth -- Advanced Deep Learning and Math

Goal: Go deeper into topics that distinguish research engineers. Build mathematical maturity.

Daily time: 4 hours (weekdays), 6 hours (weekends)

Monday -- Optimization in Deep Learning: Landscape Analysis

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Loss landscape visualization research
Evening (120 min)	Math study	Non-convex optimization in deep learning, saddle points, sharp vs flat minima, generalization bounds, PAC-Bayes
Night (15 min)	Review	Summarize why SGD generalizes better than GD despite non-convexity

Tuesday -- Paper: Scaling Laws and Emergent Behavior

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	LLM Interviews scaling section
Evening (120 min)	Paper study	Read Kaplan et al. (2020) "Scaling Laws for Neural Language Models." Focus on: power-law relationships, compute-optimal training, Chinchilla findings
Night (15 min)	Analysis	Plot scaling law relationships; estimate compute requirements for a 10B parameter model

Wednesday -- Reinforcement Learning Foundations

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	MDP formulation and Bellman equations
Evening (120 min)	Math study	MDPs, value functions, Bellman equations, policy gradient theorem, REINFORCE algorithm
Night (15 min)	Implementation	Implement REINFORCE on CartPole

Thursday -- RLHF and Alignment

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	InstructGPT paper abstract and introduction
Evening (120 min)	Paper study	Read InstructGPT paper. Focus on: reward model training, PPO fine-tuning, KL penalty, human evaluation methodology. Also read DPO as an alternative
Night (15 min)	Compare	RLHF vs DPO vs RLAIF: trade-offs and assumptions

:::tip Alignment Is Core to Research Engineering Now If you are targeting Anthropic, OpenAI, or DeepMind, understanding RLHF, reward modeling, and alignment techniques is not optional. These organizations are hiring research engineers specifically to work on these problems. Be prepared to discuss alignment challenges, evaluation methodology, and the limitations of current approaches. :::

Friday -- Advanced Probability: Concentration Inequalities

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode hard
Lunch (20 min)	Read	Concentration inequality survey
Evening (120 min)	Math study	Markov, Chebyshev, Chernoff, Hoeffding bounds. Applications in ML: generalization bounds, sample complexity, bandit algorithms
Night (15 min)	Practice	Prove Hoeffding's inequality from Chernoff bound

Saturday -- Implement Policy Gradient + Paper Deep Dive

Time	Activity	Details
Morning (2.5 hrs)	Implementation	Implement PPO from scratch in PyTorch. Train on CartPole and LunarLander
Afternoon (2 hrs)	Paper reading	Read a recent paper from your target lab's publications (2024-2025). Write a detailed 2-page critique
Evening (1.5 hrs)	Coding	Solve 3 LeetCode problems (mix of medium and hard)

Sunday -- Week 5 Review

Time	Activity	Details
Morning (2 hrs)	Math review	Re-derive policy gradient theorem, ELBO, concentration inequalities
Afternoon (2.5 hrs)	Presentation prep	Prepare a 10-minute presentation on one paper from the past two weeks
Evening (1.5 hrs)	Plan	Curate your list of "most impactful papers" for interview discussion

:::note Week 5 Milestone Checkpoint

• Explain scaling laws and their implications for training large models
• Derive the policy gradient theorem from first principles
• Implement PPO and train a simple RL agent
• Explain RLHF pipeline end-to-end including reward modeling
• State and apply Hoeffding's inequality to bound sample complexity
• Deliver a clear 10-minute paper presentation :::

Week 6: Depth -- Specialized Topics and Experiment Design

Goal: Cover specialized topics relevant to your target area. Practice experiment design.

Daily time: 4 hours (weekdays), 6 hours (weekends)

Monday -- Paper: In-Context Learning and Prompting

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Few-shot learning and in-context learning survey
Evening (120 min)	Paper study	Read "A Survey on In-Context Learning" or "Why Can GPT Learn In-Context?" Focus on: theoretical explanations, connections to gradient descent, task vectors
Night (15 min)	Write	Propose an experiment to test one hypothesis about in-context learning

Tuesday -- Experiment Design for Research

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Ablation study methodology
Evening (120 min)	Study	Experiment design: controls, ablations, statistical significance, error bars, reproducibility, hyperparameter sweeps, compute budgets
Night (15 min)	Practice	Design an ablation study for a Transformer variant

:::danger The Ablation Study Question "How would you design experiments to validate this approach?" is one of the most common research engineer interview questions. A weak answer lists one experiment. A strong answer describes: (1) the hypothesis, (2) the baseline, (3) the ablations, (4) the metrics, (5) the expected results, and (6) what each outcome would tell you. :::

Wednesday -- Paper: Representation Learning and Contrastive Methods

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Self-supervised representation learning overview
Evening (120 min)	Paper study	Read SimCLR (Chen et al., 2020) or CLIP (Radford et al., 2021). Focus on: contrastive objective, data augmentation strategy, linear probe evaluation
Night (15 min)	Math	Derive the InfoNCE loss and its connection to mutual information

Thursday -- Implement Contrastive Learning

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Implementation details that matter in contrastive learning
Evening (120 min)	Implementation	Implement SimCLR in PyTorch: data augmentation pipeline, projection head, NT-Xent loss. Train on CIFAR-10
Night (15 min)	Evaluate	Run linear probe evaluation on learned representations

Friday -- Paper: Mechanistic Interpretability

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode hard
Lunch (20 min)	Read	Mechanistic interpretability overview (Anthropic research blog)
Evening (120 min)	Paper study	Read one interpretability paper (e.g., "A Mathematical Framework for Transformer Circuits" or "Toy Models of Superposition"). Focus on: methodology, what was discovered, limitations
Night (15 min)	Reflect	Write down 3 open questions in interpretability

Saturday -- Full Paper Reproduction

Time	Activity	Details
Morning (3 hrs)	Implementation	Choose a simple paper from weeks 3-6. Reproduce the main result from scratch. Time yourself
Afternoon (2 hrs)	Analysis	Compare your results with the paper's results. Write up discrepancies and potential causes
Evening (1 hr)	Paper reading	Read one more paper from your target lab

Sunday -- Week 6 Review

Time	Activity	Details
Morning (2 hrs)	Review	Create a "research landscape" diagram connecting all papers you have read
Afternoon (2.5 hrs)	Experiment design	Practice: given a vague research idea, formulate it into a concrete experiment plan
Evening (1.5 hrs)	Behavioral prep	Start drafting STAR stories for research collaboration scenarios

:::note Week 6 Milestone Checkpoint

• Explain in-context learning theories and their evidence
• Design a rigorous ablation study with proper controls
• Implement contrastive learning and evaluate with linear probes
• Reproduce a simple paper's main result independently
• Describe the mechanistic interpretability research agenda
• Connect 10+ papers you have read into a coherent research narrative :::

Week 7: Research Skills -- Developing Research Taste

Goal: Develop the ability to evaluate research quality, propose extensions, and articulate research vision.

Daily time: 4 hours (weekdays), 6 hours (weekends)

Monday -- What Makes Good Research?

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	"You and Your Research" by Richard Hamming (adapted for ML)
Evening (120 min)	Study and reflect	Analyze 5 highly cited papers: what made them impactful? Categorize: new method, new theory, new benchmark, surprising finding, practical breakthrough
Night (15 min)	Write	Your personal definition of "good research"

Tuesday -- Paper Critique Practice: Rapid Fire

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Common paper weaknesses checklist
Evening (120 min)	Practice	Read 3 recent workshop or rejected papers. For each: identify the main flaw, suggest how to fix it, rate the paper 1-10
Night (15 min)	Reflect	Compare your critiques with any publicly available reviews

:::tip Building Research Taste Research taste is not innate -- it develops through exposure. The more papers you read critically, the better your intuition becomes for what is interesting, what is sound methodology, and what is incremental. Aim to read at least 3 papers per week even after interview prep ends. :::

Wednesday -- Proposing Extensions and Follow-Up Work

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	How to write a research proposal
Evening (120 min)	Practice	Take 3 papers you have read. For each, propose a concrete, feasible extension. Write a one-paragraph "proposal" for each: motivation, method, expected results, evaluation
Night (15 min)	Self-critique	Evaluate your proposals: are they novel? Feasible? Impactful?

Thursday -- Research Presentation Skills

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Presentation tips for research talks
Evening (120 min)	Practice	Prepare and rehearse a 15-minute research presentation on your strongest paper. Focus on: motivation, intuition before math, clear figures, key results, honest limitations
Night (15 min)	Record	Record yourself giving the presentation; watch and critique

Friday -- Mock Paper Discussion

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode hard
Lunch (20 min)	Prep	Select a paper you know well for mock discussion
Evening (120 min)	Mock	Self-timed paper discussion mock: (1) 10 min to read a new paper, (2) 15 min to present key ideas, (3) 15 min to answer likely interviewer questions, (4) 10 min to propose extensions
Night (15 min)	Debrief	List questions you struggled with

Saturday -- Research Deep Dive: Target Lab Focus

Time	Activity	Details
Morning (2.5 hrs)	Research	Read 3 recent papers from your target lab. Understand their research agenda, recurring themes, and open problems
Afternoon (2 hrs)	Analysis	Write a 2-page brief: "What [Lab X] is working on, why it matters, and where I could contribute"
Evening (1.5 hrs)	Coding	Solve 3 LeetCode problems to maintain coding readiness

Sunday -- Week 7 Review

Time	Activity	Details
Morning (2 hrs)	Review	Compile all paper summaries into a personal research reference document
Afternoon (2.5 hrs)	Presentation	Refine your research presentation based on self-critique; rehearse twice more
Evening (1.5 hrs)	Behavioral	Prepare STAR stories: research collaboration, debugging a failed experiment, pivoting a research direction, mentoring

:::note Week 7 Milestone Checkpoint

• Articulate what makes research impactful with specific examples
• Critique a paper you have never seen in under 20 minutes
• Propose 3 feasible, non-trivial research extensions
• Deliver a clear 15-minute research presentation with Q&A
• Describe your target lab's research agenda and where you fit
• Have 6 polished STAR stories ready for behavioral rounds :::

Week 8: Research Skills -- Advanced Implementation and Systems

Goal: Practice implementing complex algorithms under time pressure. Understand research infrastructure.

Daily time: 4 hours (weekdays), 6 hours (weekends)

Monday -- Distributed Training Deep Dive

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Data parallelism vs model parallelism vs pipeline parallelism
Evening (120 min)	Study	Distributed training: AllReduce, gradient accumulation, mixed precision (FP16/BF16), FSDP, DeepSpeed ZeRO stages, activation checkpointing
Night (15 min)	Design	Sketch the training infrastructure for a 70B parameter model

Tuesday -- Paper: Efficient Transformers and Architecture Search

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Efficient attention mechanisms survey
Evening (120 min)	Paper study	Read one efficiency paper: Flash Attention (Dao et al., 2022) or Mixture of Experts (Switch Transformer). Focus on: the bottleneck identified, the solution, hardware-aware design
Night (15 min)	Compare	Compare standard attention, Flash Attention, and linear attention in a table (complexity, memory, quality)

Wednesday -- Timed Implementation: Novel Architecture

Time	Activity	Details
Morning (60 min)	Warm-up	1 LeetCode medium for focus
Afternoon (180 min)	Timed challenge	Choose a paper with a novel architecture component. Set a 3-hour timer. Implement it from scratch. Write tests. This simulates the interview coding round
Evening (15 min)	Debrief	Note: what slowed you down? What will you do differently?

:::danger Time Management in Implementation Interviews The biggest failure mode in research coding interviews is not bugs -- it is running out of time. Practice this pattern: (1) Read the spec (5 min), (2) Write pseudocode and identify key components (5 min), (3) Implement the core logic (25 min), (4) Test with simple cases (10 min), (5) Handle edge cases (5 min). Resist the urge to gold-plate. Working code that is slightly ugly beats beautiful code that is incomplete. :::

Thursday -- Evaluation and Benchmarking

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Benchmark design best practices
Evening (120 min)	Study	Evaluation methodology: benchmark design, human evaluation, contamination, leaderboard gaming, confidence intervals, effect sizes, statistical tests for comparing models
Night (15 min)	Critique	Pick a leaderboard result and list potential issues with the evaluation

Friday -- Research Code Quality

Time	Activity	Details
Morning (60 min)	Coding practice	2 LeetCode medium/hard
Lunch (20 min)	Read	Best practices for research repositories
Evening (120 min)	Refactor	Take your messiest implementation from the past 8 weeks. Refactor it: proper abstractions, configuration management (hydra/yaml), reproducibility (seed everything), logging, and a clean README
Night (15 min)	Review	Compare your code with a well-maintained research repo (e.g., HuggingFace, Meta's fairseq)

Saturday -- Full Mock: Coding + Paper Discussion

Time	Activity	Details
Morning (2.5 hrs)	Mock	Simulate two interview rounds: (1) 60-min coding round (implement an algorithm from a paper), (2) 60-min paper discussion (read and discuss a new paper)
Afternoon (2 hrs)	Review	Analyze performance, identify gaps, create action items
Evening (1.5 hrs)	Study	Fill the largest gap identified in the mock

Sunday -- Week 8 Review

Time	Activity	Details
Morning (2 hrs)	Review	Create cheat sheets: distributed training concepts, evaluation methodology, research code patterns
Afternoon (2.5 hrs)	Implementation	Timed mini-challenge: implement a simple attention variant in 60 minutes
Evening (1.5 hrs)	Planning	Finalize Week 9-10 schedule based on remaining gaps

:::note Week 8 Milestone Checkpoint

• Explain distributed training strategies (FSDP, ZeRO, pipeline parallelism)
• Implement a paper's core algorithm in under 2.5 hours
• Critique evaluation methodology in published work
• Write research-quality code with proper reproducibility practices
• Complete a mock coding + paper discussion session
• Have clean, well-documented implementations on GitHub :::

Week 9: Polish -- Mock Interviews and Presentation Refinement

Goal: Intensive mock interview practice. Refine all presentation skills.

Daily time: 4 hours (weekdays), 6 hours (weekends)

Monday -- Research Presentation Refinement

Time	Activity	Details
Morning (60 min)	Coding	2 LeetCode medium (maintain fluency)
Lunch (20 min)	Read	Behavioral interview guide
Evening (120 min)	Presentation	Refine your research presentation to 12 minutes. Practice until smooth. Prepare for 10 common follow-up questions
Night (15 min)	Record	Record final version; send to a friend for feedback

Tuesday -- Mock: Math and Theory Round

Time	Activity	Details
Morning (60 min)	Coding	2 LeetCode medium/hard
Lunch (20 min)	Review	Skim all math cheat sheets
Evening (120 min)	Mock	Self-timed math/theory round (60 min): probability, linear algebra, optimization, information theory questions. Then review (60 min)
Night (15 min)	Note	List every question you could not answer fluently

Wednesday -- Mock: Coding Round (Research-Style)

Time	Activity	Details
Morning (60 min)	Warm-up	1 LeetCode medium
Afternoon (180 min)	Mock	Full coding mock: (1) 60-min algorithm implementation from a paper, (2) 30-min review and debugging. Rest 30 min. (3) 60-min standard coding (2 LeetCode medium/hard)
Evening (15 min)	Debrief	Score yourself on: correctness, speed, communication, code quality

Thursday -- Mock: Paper Discussion Round

Time	Activity	Details
Morning (60 min)	Coding	2 LeetCode medium/hard
Lunch (20 min)	Prep	Select 2 papers you have NOT read before
Evening (120 min)	Mock	Simulate paper discussion: (1) 15 min to read Paper 1, (2) 20 min to discuss it. Rest 10 min. (3) Repeat with Paper 2. Focus on: summarizing the contribution, identifying weaknesses, proposing extensions
Night (15 min)	Debrief	Rate your performance on clarity, depth, and speed

Friday -- Behavioral and Research Philosophy

Time	Activity	Details
Morning (60 min)	Coding	2 LeetCode medium
Lunch (20 min)	Read	Company Guides for your target lab
Evening (120 min)	Practice	Prepare answers for research-specific behavioral questions: "What is the most interesting paper you have read recently?", "Describe a time a research direction failed", "How do you decide what to work on?", "What is your research philosophy?"
Night (15 min)	Polish	Refine your "research motivation" story

:::tip The "Research Philosophy" Question Many research labs ask: "What is your research philosophy?" or "How do you decide what problems to work on?" This is not about having a grand theory. It is about showing that you think carefully about research direction. A strong answer discusses: (1) what problems excite you and why, (2) how you evaluate whether a direction is promising, (3) how you balance exploration vs exploitation in research. :::

Saturday -- Full Loop Simulation

Time	Activity	Details
Morning (3 hrs)	Mock	Full interview simulation: coding (60 min) + math/theory (45 min) + paper discussion (45 min)
Afternoon (2 hrs)	Review	Detailed analysis of every round; create improvement plan
Evening (1 hr)	Relaxation	Light activity; let the day's practice consolidate

Sunday -- Week 9 Review

Time	Activity	Details
Morning (2 hrs)	Gap filling	Study your weakest topic identified from mocks
Afternoon (2.5 hrs)	Presentation	Final rehearsal of research presentation with Q&A
Evening (1.5 hrs)	Organization	Organize all cheat sheets, paper summaries, and notes into a quick-reference document

:::note Week 9 Milestone Checkpoint

• Complete a full loop simulation with passing scores in all rounds
• Deliver research presentation smoothly in 12 minutes with confident Q&A
• Read and critique an unseen paper in under 30 minutes
• Answer math/theory questions fluently (probability, optimization, linear algebra)
• Articulate your research philosophy and motivation clearly
• Have all behavioral stories polished and natural-sounding :::

Week 10: Final Week -- Confidence and Readiness

Goal: Final polishing, logistics, and confidence building. No new material.

Daily time: 3 hours (weekdays), 4 hours (weekends)

Monday -- Light Coding and Review

Time	Activity	Details
Morning (60 min)	Coding	2 LeetCode medium (patterns you know well)
Lunch (20 min)	Read	Negotiation and Offers overview
Evening (60 min)	Review	Skim all math cheat sheets and paper summaries
Night (15 min)	Relax	Mental break

:::warning Do Not Learn New Material This Week Week 10 is about sharpening and confidence. If you encounter an unfamiliar topic, note it but do not go deep. Focus on reinforcing what you already know. Cramming new material the week before interviews increases anxiety and reduces performance. :::

Tuesday -- Final Full Mock

Time	Activity	Details
Morning (60 min)	Warm-up	1 easy problem for flow
Afternoon (150 min)	Mock	Final full loop: coding (60 min) + paper discussion (45 min) + behavioral (30 min)
Evening (30 min)	Debrief	Celebrate progress. Note final confidence level for each area

Wednesday -- Targeted Review

Time	Activity	Details
Morning (60 min)	Coding	2 LeetCode medium (targeted at remaining weak spots)
Lunch (20 min)	Read	Skim your "research landscape" document
Evening (90 min)	Review	Walk through your top 3 paper critiques aloud. Practice explaining key math derivations
Night (15 min)	Mental rehearsal	Visualize a successful interview day

Thursday -- Presentation and Behavioral Final Polish

Time	Activity	Details
Morning (60 min)	Practice	Final rehearsal of research presentation
Lunch (20 min)	Read	Target lab culture and values
Evening (90 min)	Behavioral	Run through all STAR stories. Practice "tell me about yourself" and "why this lab?"
Night (15 min)	Prepare	Write 8-10 thoughtful questions to ask interviewers

Friday -- Rest and Logistics

Time	Activity	Details
Morning (30 min)	Logistics	Confirm interview schedule, test video/audio, prepare workspace
Afternoon	Rest	Exercise, walk, enjoyable activities
Evening (30 min)	Light review	Skim cheat sheets one final time

Saturday -- Optional Light Practice

Time	Activity	Details
Morning (2 hrs)	Light practice	2 easy/medium coding problems for confidence
Afternoon (1.5 hrs)	Review	Walk through one paper discussion slowly and thoroughly
Evening (30 min)	Mental prep	Confidence visualization; remind yourself of all you have accomplished

Sunday -- Complete Rest

Take the day completely off. Sleep well. Eat well. You have put in the work. You are prepared.

:::note Week 10 Final Assessment

• Can implement an algorithm from a paper in under 2 hours with clean, tested code
• Can read, critique, and discuss an unseen paper in 30 minutes
• Can derive key results in probability, optimization, and linear algebra on a whiteboard
• Can deliver a research presentation and handle tough questions
• Can articulate your research taste, motivation, and philosophy
• Feel confident (not perfect, but confident) about each interview round
• Have prepared thoughtful questions for each interviewer :::

Problem Sets by Week

Cumulative Practice Counts

Week	LeetCode	Papers Read	Implementations	Math Derivations	Presentations
1	12	0	2	8	0
2	12	1	5	6	0
3	12	3	3	4	0
4	12	3	3	4	0
5	12	3	2	5	1
6	12	3	2	2	0
7	12	6	0	0	2
8	12	2	3	2	0
9	10	2	1	3	2
10	6	0	0	1	1
Total	112	23	21	35	6

Recommended Paper Reading List

Foundational (Must Read):

1. Attention Is All You Need (Vaswani et al., 2017)
2. BERT (Devlin et al., 2018)
3. ResNet (He et al., 2015)
4. Adam Optimizer (Kingma and Ba, 2014)
5. Batch Normalization (Ioffe and Szegedy, 2015)
6. VAE (Kingma and Welling, 2013)
7. GAN (Goodfellow et al., 2014)

Modern (Highly Recommended): 8. GPT-3 / InstructGPT (Brown et al., 2020 / Ouyang et al., 2022) 9. DDPM (Ho et al., 2020) 10. Flash Attention (Dao et al., 2022) 11. Scaling Laws (Kaplan et al., 2020) 12. CLIP (Radford et al., 2021) 13. SimCLR (Chen et al., 2020) 14. LoRA (Hu et al., 2021) 15. DPO (Rafailov et al., 2023)

Specialized (Choose Based on Target Area): 16. Switch Transformer / Mixture of Experts 17. Mechanistic Interpretability (Anthropic circuits work) 18. Constitutional AI (Bai et al., 2022) 19. Chain-of-Thought Prompting (Wei et al., 2022) 20. Chinchilla (Hoffmann et al., 2022)

Mock Interview Schedule

Finding Mock Interview Partners

• Research group reading groups -- Participate in or start a paper reading group
• PhD friends -- Ask PhD students or postdocs to conduct mock paper discussions
• interviewing.io -- Anonymous technical interviews
• AI Discord communities -- Find research-focused study partners
• Twitter/X ML community -- Connect with researchers who share interests

Common Mistakes to Avoid

:::danger Top 7 Research Engineer Interview Mistakes

1. Not reading papers actively enough. Passive reading builds familiarity, not understanding. Annotate, question, and critique every paper. If you cannot explain the key contribution in 2 sentences, you did not read it carefully enough.

2. Weak mathematical foundations. "I use PyTorch so I do not need to know the math" will not fly. You will be asked to derive gradients, prove convergence, and manipulate probability distributions. The math is the job.

3. Implementing without understanding. Copying code patterns without understanding the underlying algorithm leads to brittle implementations. When the interviewer asks "why did you use X here?" you need a principled answer.

4. Forgetting to test and debug. Research code has bugs. Interviewers want to see that you write tests, check shapes, verify gradients numerically, and have a systematic debugging process.

5. Presenting research without clarity. Technical depth without communication skills is insufficient. Practice explaining complex ideas simply. If a smart non-expert cannot follow your explanation, it is not clear enough.

6. Having no research opinion. "I think all research areas are interesting" signals that you have not thought deeply. Have opinions about what directions are promising, what is overhyped, and what needs more attention. Be able to justify your views.

7. Neglecting behavioral preparation. Research engineers collaborate intensively. Stories about conflict resolution, handling failure, and supporting teammates matter more than you think. :::

Essential Resources

Handbook Chapters to Prioritize

Priority	Chapter	When to Study
Critical	ML Fundamentals	Weeks 1-4
Critical	Deep Learning	Weeks 3-6
Critical	Paper Discussion	Weeks 3-9
High	Coding Interviews	Weeks 1-6
High	LLM Interviews	Weeks 5-7
Medium	Behavioral	Weeks 7-9
Medium	Company Guides	Weeks 7-8
Low	ML System Design	Week 8
Low	Negotiation	Week 10

External Resources

Textbooks:

• "Deep Learning" by Goodfellow, Bengio, and Courville (the deep learning bible)
• "Convex Optimization" by Boyd and Vandenberghe (free online)
• "Information Theory, Inference and Learning Algorithms" by David MacKay (free online)
• "Pattern Recognition and Machine Learning" by Christopher Bishop
• "The Matrix Cookbook" by Petersen and Pedersen (reference for matrix calculus)

Courses:

• Stanford CS229 (ML theory and math)
• Stanford CS231n (computer vision and deep learning)
• Stanford CS224n (NLP and transformers)
• MIT 18.065 (matrix methods in data analysis)
• Stanford CS330 (meta-learning and multi-task learning)

Paper Sources:

• arXiv (daily new papers)
• Papers With Code (implementations alongside papers)
• Semantic Scholar (paper recommendations and citations)
• Target lab publication pages (Google Research, Meta FAIR, Anthropic, OpenAI)

Implementation Practice:

• Papers With Code benchmarks (reproduce results)
• HuggingFace Transformers source code (learn from production research code)
• JAX/Flax examples (alternative to PyTorch, valued at DeepMind)

Next Steps

You now have a complete 10-week roadmap for Research Engineer interview preparation. This is the most mathematically demanding and intellectually rigorous prep path in the handbook. If this path does not quite fit your target role, consider:

• MLE Prep Path -- If your target role emphasizes production ML systems over research
• AI Engineer Prep Path -- If your target role focuses on building applications with LLMs and foundation models
• Data Scientist Prep Path -- If your target role emphasizes statistical analysis and experimentation

Start today. Pick up a paper. Read the abstract. Ask yourself: what is the key contribution, and how would I implement it? The 10-week clock starts now.