llms-full.txt

llms-full.txt is an extended variant of the llms.txt convention proposed in 2024 by Answer.AI co-founder Jeremy Howard, intended to provide large language models with a comprehensive, machine-readable representation of an entire website’s content in a single file. While the standard llms.txt file contains a brief site description and an index of important URLs, llms-full.txt is designed to include the full text of all significant pages concatenated together, so that an LLM can ingest the site’s content in one context window without needing to crawl individual pages. This is particularly useful for AI assistants that support document upload or for retrieval-augmented generation pipelines that need an up-to-date snapshot of a site’s information. Publishers can generate llms-full.txt programmatically from a CMS or static site generator and serve it at the root of their domain alongside the standard llms.txt file.