70 years of how AI systems learned to remember. Click any event to explore.
The Logic Theorist, developed for the Dartmouth Conference, was the first program to manipulate stored symbolic knowledge. It memorized axioms as explicit logical expressions and searched them via pattern matching to prove mathematical theorems — proving 38 of the first 52 theorems in Whitehead and Russell's Principia Mathematica.
The memory model was simple: a flat list of rules and facts. Retrieval was exhaustive search. No similarity, no ranking. But the core insight was profound: memory could be structured, stored, and queried programmatically. Every rule-based AI system for the next 40 years inherited this architecture.
FoundationalExpert systems like MYCIN (medical diagnosis) and DENDRAL (molecular structure analysis) scaled the Logic Theorist's approach to real domains. MYCIN's knowledge base contained ~600 rules about infectious diseases and antibiotics. Retrieval was via forward or backward chaining inference engines.
The key problem that emerged: knowledge maintenance. Every fact in the knowledge base had to be written, verified, and updated by human experts. The memory was authoritative but brittle. Adding new knowledge required expensive manual effort. This was the fundamental limit of symbolic memory — it did not scale.
High ImpactLSTM introduced the gating mechanism: separate neural components for deciding whether to store information (input gate), retain it over time (cell state), and expose it to downstream processing (output gate). The forget gate was added by Gers et al. in 1999.
This was a conceptual shift. Previous neural networks had fixed memory structure. LSTM learned the memory policy from data. The network discovered, through training, that certain patterns were worth remembering and others should be discarded. This differentiated it from both symbolic systems (which stored everything explicitly) and vanilla RNNs (which could not selectively retain long-range context).
LSTM dominated sequence modeling until 2017 and remains widely used in production systems.
FoundationalLatent Semantic Analysis (Deerwester et al. 1990, widely applied 2000-2010) applied Singular Value Decomposition to term-document co-occurrence matrices to produce dense semantic representations. Words with similar meanings ended up near each other in the latent space — not because anyone specified this, but because they appeared in similar contexts.
This was the intellectual precursor to word embeddings and vector search. The key insight: semantic similarity can be computed as geometric distance in a learned vector space. You don't need to curate knowledge rules. You let statistical patterns in text encode semantic relationships implicitly.
High Impactword2vec (Mikolov et al., 2013) trained two-layer networks on billions of words using two objectives: predict surrounding words from a center word (Skip-gram), or predict a center word from surrounding words (CBOW). The resulting 300-dimensional embeddings captured semantic and syntactic relationships: king - man + woman = queen.
The practical implication for memory: for the first time, you could encode documents as dense vectors and measure semantic similarity with a dot product or cosine distance. This made large-scale semantic retrieval computationally tractable. word2vec directly inspired the vector store architecture that now underlies all agent memory systems.
FoundationalThe Transformer's self-attention mechanism is, architecturally, a soft retrieval system. Each token computes a query vector and attends to key-value pairs from all other tokens. High dot-product scores mean high relevance — the model "remembers" relevant context from anywhere in the sequence.
The profound consequence: memory became entirely within the context window. The transformer had no external state, no recurrent connections, no explicit memory cells. Its "memory" was purely the set of tokens in its input, with attention providing the retrieval mechanism. This is why context window size became the central constraint in modern LLMs, and why external vector stores became necessary for anything requiring memory beyond a single context.
FoundationalFAISS (Facebook AI Similarity Search) provided production-grade ANN search with support for IVF (Inverted File Index) and HNSW (Hierarchical Navigable Small World) index types. IVF partitions the vector space into Voronoi cells; search only checks the most relevant cells. HNSW builds a navigable graph structure that enables logarithmic-time search.
Before FAISS, approximate nearest neighbor search at scale required specialized infrastructure. FAISS made it a library call. This was the enabling technology for practical agent memory — you could now store millions of embedded documents and retrieve the most relevant ones in under 50ms. Most modern vector stores (ChromaDB, Weaviate, Pinecone) use FAISS or FAISS-inspired algorithms internally.
FoundationalPinecone's launch marked the transition from vector search as a library (FAISS) to vector search as infrastructure. The key contributions: managed indexing, automatic scaling, metadata filtering, and a REST/SDK API that developers could call without managing index files.
The business impact was significant: retrieval-augmented generation (RAG) became practical for teams without dedicated ML infrastructure. By 2023, Pinecone reported hundreds of millions of vectors managed across thousands of customers. The managed vector DB model directly enabled the RAG and agent memory patterns that became standard practice.
High ImpactChromaDB launched in late 2022 offering an embedded, serverless vector store designed specifically for LLM applications. Weaviate (originally 2019, widely adopted 2022) provided a self-hosted option with GraphQL API and built-in vectorization. Qdrant, Milvus, LanceDB, and others followed rapidly.
The consequence: vector memory became a zero-infrastructure setup. A developer could run chromadb.PersistentClient(path="./db") and have a production-capable semantic memory store running locally in under a second. This democratized agent memory architecture, removing the need for managed cloud services during development and enabling local-first agent prototyping.
MemGPT (Packer et al., 2023) frames the LLM context window as CPU registers: fast, limited, and expensive. External storage is disk: slower, unlimited, cheap. The key contribution: instead of a fixed retrieval pipeline, the agent itself decides when to read from and write to memory via explicit function calls: core_memory_append, archival_memory_search, archival_memory_insert.
MemGPT demonstrated that a GPT-4-based agent could maintain coherent long-term conversations over hundreds of turns by actively managing its own memory budget. It could summarize and archive old context, retrieve relevant past information on demand, and maintain a running "working memory" of the most important current facts. This self-directed memory management is qualitatively different from passive RAG retrieval.
Foundationalpgvector 0.7 added HNSW indexing (previously only IVFFlat was supported), dramatically improving query performance for high-dimensional vectors. A single Postgres instance could now serve both structured relational queries and semantic similarity searches with sub-100ms latency on millions of vectors.
The architectural implication: teams no longer needed a separate vector database. Agent memory could live in the same Postgres instance as the rest of their application data, sharing transactions, backups, and monitoring. Supabase, Neon, and Timescale all added pgvector support as a first-class feature. For many production deployments, pgvector became the recommended default over specialized vector stores.
High ImpactGemini 1.5 Pro's 1-million-token context window (Google, 2024-2025) and Claude's 200K-token context sparked serious debate: if you can stuff an entire codebase or document corpus into context, why maintain a vector store at all?
The evidence so far: long-context models excel at "needle in a haystack" recall within a single session but degrade on tasks requiring precise extraction from very large inputs (the "lost in the middle" phenomenon). Retrieval remains superior when: (1) the corpus exceeds even the largest context window, (2) you need sub-100ms per-query latency, (3) you need to update the knowledge base without reloading context, (4) cost matters at scale. The debate will continue as both context windows and retrieval systems improve. Most production systems in 2026 use a hybrid: long context for working state, retrieval for large knowledge bases.
Evolving