Andrej Karpathy Proposes 'LLM Knowledge Base' Architecture Using Evolving Markdown Libraries as RAG Alternative

Summary

Andrej Karpathy has shared a new architecture concept called 'LLM Knowledge Base' that proposes replacing traditional Retrieval-Augmented Generation (RAG) pipelines with a curated, evolving library of markdown documents maintained by AI. The approach envisions LLMs continuously organizing, updating, and distilling knowledge into structured markdown files that can be fed directly into model context windows, eliminating the need for vector databases, embedding models, and retrieval infrastructure