$ grep -r "web standards" ~/blog/

There’s something beautifully ironic happening on the web right now. Hundreds of websites have implemented a new standard called /llms.txt—a carefully crafted markdown file designed to help AI systems understand their content. Developers have built tools to generate these files. Community directories catalog implementations. SEO platforms flag sites that don’t have one.

There’s just one problem: not a single major AI platform actually uses it.

No, really. Not OpenAI. Not Google. Not Anthropic. Not Meta. The very systems that /llms.txt was designed to serve don’t even check if the file exists. Server logs confirm it: when AI crawlers visit your website, they sail right past your lovingly crafted llms.txt file without a second glance.

This isn’t just a story about a failed web standard. It’s a revealing case study in the power dynamics of the AI era, the challenges of grassroots standardization, and the growing tension between publishers and the platforms that increasingly control how their content reaches users. The /llms.txt saga tells us something important about who holds power in the AI/web ecosystem—and it’s not the people creating content.

Every web developer has encountered the frustration of inconsistent metadata discovery across different websites and services. Where do you find a site’s security contact information? How do you discover OAuth endpoints? What about password change URLs for password managers? The web’s decentralized nature, while powerful, has historically led to fragmented approaches for exposing essential service metadata.

The Well-known URI standard, formalized in RFC 8615 by the Internet Engineering Task Force (IETF), provides an elegant solution to this fundamental problem. By establishing a standardized location for service metadata at /.well-known/, this specification enables consistent, predictable discovery of critical information across the entire web ecosystem.