The /llms.txt Standard: An Elegant Solution Nobody's Using
There’s something beautifully ironic happening on the web right now. Hundreds of websites have implemented a new standard called /llms.txt—a carefully crafted markdown file designed to help AI systems understand their content. Developers have built tools to generate these files. Community directories catalog implementations. SEO platforms flag sites that don’t have one.
There’s just one problem: not a single major AI platform actually uses it.
No, really. Not OpenAI. Not Google. Not Anthropic. Not Meta. The very systems that /llms.txt was designed to serve don’t even check if the file exists. Server logs confirm it: when AI crawlers visit your website, they sail right past your lovingly crafted llms.txt file without a second glance.
This isn’t just a story about a failed web standard. It’s a revealing case study in the power dynamics of the AI era, the challenges of grassroots standardization, and the growing tension between publishers and the platforms that increasingly control how their content reaches users. The /llms.txt saga tells us something important about who holds power in the AI/web ecosystem—and it’s not the people creating content.