The Two Categories of ML Skills

Named entity recognition was a specialist skill in 2020. Today, GPT-4o extracts entities from arbitrary domains with a single prompt — often more accurately than domain-specific models trained for weeks. The value was in the task. The task is gone.

TF-IDF + SVM was the state of the art for text classification in 2018. It worked. But GPT-3.5 beats it zero-shot on most benchmarks. The technique didn't become wrong — it became unnecessary. Engineers who only knew this skill lost their use case.

Regex pipelines for entity extraction, custom tokenizers, handcrafted preprocessing rules — these were real engineering work. Foundation models handle the same tasks through learned representations. The rules are still useful for edge cases; as a core skill, they're commoditized.

Fine-tuning BERT on a classification dataset was a differentiator in 2019. Now it's a tutorial exercise. The skill of fine-tuning still matters enormously — but at the foundation model level with LoRA and domain-specific data. "BERT fine-tuner" is not a job title with a premium anymore.

Unless you work at a frontier lab, this skill has no production application. The compute cost alone (GPT-3 training: ~$4.6M in 2020 dollars) makes it inaccessible. And even if you could afford it, a fine-tuned open-weight model will outperform your scratch-trained model on any practical task.

Careful grid search over learning rates, dropout, layer freezing strategies for BERT — this was a real skill. With foundation models, hyperparameter tuning is largely replaced by prompt iteration and data quality work. The effort got redistributed to what you put in, not how you tune the knobs.

Every RAG system depends on embeddings. Choosing the wrong model costs 15–25 points in retrieval accuracy (MTEB data). Detecting embedding drift as your corpus changes prevents silent degradation. Understanding cosine vs dot product, late interaction vs bi-encoder — these distinctions matter in production in ways they didn't in 2020.

"Lost in the Middle" (Liu et al., 2023): 30% accuracy drop when the answer is in the middle of a 20-document context. You cannot fix this without understanding how attention score distributions work. KV cache sizing, RoPE vs ALiBi position encodings, long-context reliability — all require this foundational knowledge.

Llama-3 8B fine-tuned on domain-specific instruction data routinely beats GPT-4 on that domain. The skill premium is in knowing when to fine-tune (vs prompt), how to construct quality training data, how to evaluate the result, and how to keep training stable. LoRA made it accessible — understanding made it powerful.

Most AI teams have no systematic way to know if their model improved or regressed after an update. Engineers who can build eval suites — automated test cases, LLM-as-a-judge, human preference benchmarks — become the gatekeeper for every model update. Almost nobody does this well. That's the opportunity.

Hybrid search (BM25 + dense retrieval), reranking, chunk sizing strategies, metadata filtering — the fundamentals of information retrieval are now core AI engineering skills. The engineers who understand why BM25 still outperforms dense retrieval on keyword-exact queries, and when to combine them, are the ones fixing broken RAG systems.

KV cache management, speculative decoding, batching strategies, quantization tradeoffs (4-bit vs 8-bit precision loss), latency vs throughput budgets — serving LLMs in production at acceptable cost and latency requires systems knowledge that no framework abstracts away. This skill barely existed 3 years ago and is now scarce.