AI Letters #07 · Interactive Timeline

Memory in AI Systems: From Lookup Tables to Vector Stores

70 years of how AI systems learned to remember. Click any event to explore.

1956
First symbolic memory in AI
1B+
Vector scale FAISS supports
1M+
Tokens in long-context models 2025
12
Milestones charted
Symbolic Era
Statistical Era
Neural Era
Agent Era
1956
Logic Theorist: First Symbolic Memory in AI
Newell, Shaw, and Simon build an AI that stores and retrieves logical theorems as symbolic structures.

The Logic Theorist, developed for the Dartmouth Conference, was the first program to manipulate stored symbolic knowledge. It memorized axioms as explicit logical expressions and searched them via pattern matching to prove mathematical theorems — proving 38 of the first 52 theorems in Whitehead and Russell's Principia Mathematica.

The memory model was simple: a flat list of rules and facts. Retrieval was exhaustive search. No similarity, no ranking. But the core insight was profound: memory could be structured, stored, and queried programmatically. Every rule-based AI system for the next 40 years inherited this architecture.

Foundational
1980s
Expert Systems: Rule-Based Knowledge Bases
MYCIN, DENDRAL, and others encode domain expertise as thousands of IF-THEN rules stored in curated knowledge bases.

Expert systems like MYCIN (medical diagnosis) and DENDRAL (molecular structure analysis) scaled the Logic Theorist's approach to real domains. MYCIN's knowledge base contained ~600 rules about infectious diseases and antibiotics. Retrieval was via forward or backward chaining inference engines.

The key problem that emerged: knowledge maintenance. Every fact in the knowledge base had to be written, verified, and updated by human experts. The memory was authoritative but brittle. Adding new knowledge required expensive manual effort. This was the fundamental limit of symbolic memory — it did not scale.

High Impact
1997
LSTM: Learned Sequential Memory
Hochreiter and Schmidhuber publish Long Short-Term Memory — the first architecture to learn what to remember and what to forget.

LSTM introduced the gating mechanism: separate neural components for deciding whether to store information (input gate), retain it over time (cell state), and expose it to downstream processing (output gate). The forget gate was added by Gers et al. in 1999.

This was a conceptual shift. Previous neural networks had fixed memory structure. LSTM learned the memory policy from data. The network discovered, through training, that certain patterns were worth remembering and others should be discarded. This differentiated it from both symbolic systems (which stored everything explicitly) and vanilla RNNs (which could not selectively retain long-range context).

LSTM dominated sequence modeling until 2017 and remains widely used in production systems.

Foundational
2003
Latent Semantic Analysis: Distributional Semantic Memory
LSA demonstrates that meaning can be encoded as position in a high-dimensional semantic space derived from co-occurrence statistics.

Latent Semantic Analysis (Deerwester et al. 1990, widely applied 2000-2010) applied Singular Value Decomposition to term-document co-occurrence matrices to produce dense semantic representations. Words with similar meanings ended up near each other in the latent space — not because anyone specified this, but because they appeared in similar contexts.

This was the intellectual precursor to word embeddings and vector search. The key insight: semantic similarity can be computed as geometric distance in a learned vector space. You don't need to curate knowledge rules. You let statistical patterns in text encode semantic relationships implicitly.

High Impact
2013
word2vec: Dense Vector Representations at Scale
Mikolov et al. (Google) show that shallow neural networks trained on large corpora produce vectors with remarkable semantic arithmetic.

word2vec (Mikolov et al., 2013) trained two-layer networks on billions of words using two objectives: predict surrounding words from a center word (Skip-gram), or predict a center word from surrounding words (CBOW). The resulting 300-dimensional embeddings captured semantic and syntactic relationships: king - man + woman = queen.

The practical implication for memory: for the first time, you could encode documents as dense vectors and measure semantic similarity with a dot product or cosine distance. This made large-scale semantic retrieval computationally tractable. word2vec directly inspired the vector store architecture that now underlies all agent memory systems.

Foundational
2017
Transformer Attention: Dynamic Contextual Memory
Vaswani et al. introduce attention mechanisms that let every token attend to every other token — memory as dynamic weighted retrieval.

The Transformer's self-attention mechanism is, architecturally, a soft retrieval system. Each token computes a query vector and attends to key-value pairs from all other tokens. High dot-product scores mean high relevance — the model "remembers" relevant context from anywhere in the sequence.

The profound consequence: memory became entirely within the context window. The transformer had no external state, no recurrent connections, no explicit memory cells. Its "memory" was purely the set of tokens in its input, with attention providing the retrieval mechanism. This is why context window size became the central constraint in modern LLMs, and why external vector stores became necessary for anything requiring memory beyond a single context.

Foundational
2020
FAISS Released by Meta: Billion-Scale Vector Search
Facebook AI Research releases FAISS, enabling approximate nearest neighbor search over a billion vectors in milliseconds on a single machine.

FAISS (Facebook AI Similarity Search) provided production-grade ANN search with support for IVF (Inverted File Index) and HNSW (Hierarchical Navigable Small World) index types. IVF partitions the vector space into Voronoi cells; search only checks the most relevant cells. HNSW builds a navigable graph structure that enables logarithmic-time search.

Before FAISS, approximate nearest neighbor search at scale required specialized infrastructure. FAISS made it a library call. This was the enabling technology for practical agent memory — you could now store millions of embedded documents and retrieve the most relevant ones in under 50ms. Most modern vector stores (ChromaDB, Weaviate, Pinecone) use FAISS or FAISS-inspired algorithms internally.

Foundational
2021
Pinecone Launches: First Managed Vector Database
Pinecone abstracts vector storage and retrieval behind a managed API, making semantic memory a cloud service.

Pinecone's launch marked the transition from vector search as a library (FAISS) to vector search as infrastructure. The key contributions: managed indexing, automatic scaling, metadata filtering, and a REST/SDK API that developers could call without managing index files.

The business impact was significant: retrieval-augmented generation (RAG) became practical for teams without dedicated ML infrastructure. By 2023, Pinecone reported hundreds of millions of vectors managed across thousands of customers. The managed vector DB model directly enabled the RAG and agent memory patterns that became standard practice.

High Impact
2022
ChromaDB, Weaviate: Open-Source Vector Stores Proliferate
A wave of open-source vector databases launches, making local and self-hosted semantic memory a developer-accessible primitive.

ChromaDB launched in late 2022 offering an embedded, serverless vector store designed specifically for LLM applications. Weaviate (originally 2019, widely adopted 2022) provided a self-hosted option with GraphQL API and built-in vectorization. Qdrant, Milvus, LanceDB, and others followed rapidly.

The consequence: vector memory became a zero-infrastructure setup. A developer could run chromadb.PersistentClient(path="./db") and have a production-capable semantic memory store running locally in under a second. This democratized agent memory architecture, removing the need for managed cloud services during development and enabling local-first agent prototyping.

High Impact
2023
MemGPT Paper: OS-Inspired Hierarchical Memory for LLMs
Packer et al. (arXiv:2310.08560) propose a tiered memory architecture where the LLM itself manages its own memory hierarchy, inspired by OS virtual memory.

MemGPT (Packer et al., 2023) frames the LLM context window as CPU registers: fast, limited, and expensive. External storage is disk: slower, unlimited, cheap. The key contribution: instead of a fixed retrieval pipeline, the agent itself decides when to read from and write to memory via explicit function calls: core_memory_append, archival_memory_search, archival_memory_insert.

MemGPT demonstrated that a GPT-4-based agent could maintain coherent long-term conversations over hundreds of turns by actively managing its own memory budget. It could summarize and archive old context, retrieve relevant past information on demand, and maintain a running "working memory" of the most important current facts. This self-directed memory management is qualitatively different from passive RAG retrieval.

Foundational
2024
pgvector 0.7: Production Vector Search in PostgreSQL
pgvector reaches maturity with HNSW index support, bringing vector similarity search into the database tier most teams already operate.

pgvector 0.7 added HNSW indexing (previously only IVFFlat was supported), dramatically improving query performance for high-dimensional vectors. A single Postgres instance could now serve both structured relational queries and semantic similarity searches with sub-100ms latency on millions of vectors.

The architectural implication: teams no longer needed a separate vector database. Agent memory could live in the same Postgres instance as the rest of their application data, sharing transactions, backups, and monitoring. Supabase, Neon, and Timescale all added pgvector support as a first-class feature. For many production deployments, pgvector became the recommended default over specialized vector stores.

High Impact
2025
Long-Context Models vs. Retrieval: The Architecture Debate
Models with 1M+ token context windows (Gemini 1.5 Pro, Claude 3.5) challenge the assumption that external retrieval is always necessary.

Gemini 1.5 Pro's 1-million-token context window (Google, 2024-2025) and Claude's 200K-token context sparked serious debate: if you can stuff an entire codebase or document corpus into context, why maintain a vector store at all?

The evidence so far: long-context models excel at "needle in a haystack" recall within a single session but degrade on tasks requiring precise extraction from very large inputs (the "lost in the middle" phenomenon). Retrieval remains superior when: (1) the corpus exceeds even the largest context window, (2) you need sub-100ms per-query latency, (3) you need to update the knowledge base without reloading context, (4) cost matters at scale. The debate will continue as both context windows and retrieval systems improve. Most production systems in 2026 use a hybrid: long context for working state, retrieval for large knowledge bases.

Evolving
www.engineersofai.com · AI Letters #07 · Agentic AI A-Z Series