# Cameron Rye - Complete Content Archive > This file contains the complete text of all blog posts and projects from rye.dev. > Generated for LLMs with large context windows per the llms.txt specification. > See also: https://rye.dev/llms.txt for a concise overview with links. Total blog posts: 28 Total projects: 15 --- # Blog Posts ## No Way to Say No > The small web spent a year with no way to tell my MCP server no. The criticism of gopher-mcp was mostly right, and this is what shipped because of it. Date: 2026-09-01 Tags: mcp, gopher, gemini, small-web, ai, open-web URL: https://rye.dev/blog/no-way-to-say-no/ There's a Gemini capsule out there with an imaginary plant on it. The plant is over 400 days old, its name is Beelzebub, and the plan is to keep watering it until day 666. I know this because the person who tends it wrote [a post this week](https://sava.rocks/blog/when-the-small-web-gets-an-ai-door/) about a tool I built, and reading it sent me wandering around their capsule for half an hour. That half hour of wandering is, honestly, the whole argument against my tool. The tool is [gopher-mcp](https://github.com/cameronrye/gopher-mcp), an MCP server that lets an AI assistant fetch Gopher and Gemini pages. I built it a year ago because wiring a 1991 protocol into a 2024 one seemed interesting. That's the whole origin story: no grand plan, no business model. This week the small web found it. A thread titled ["We're doomed"](https://portal.mozz.us/gemini/bbs.geminispace.org/s/Gemini/47887) went up on the Gemini bulletin board (natively at `gemini://bbs.geminispace.org/s/Gemini/47887`), then a Mastodon thread, then sava's post, and the reaction ranged from thoughtful to furious. I want to answer the thoughtful part. Some of it's right, and I'd rather say so in public than hope it blows over. ## What the critics got right before I did When someone filed [an issue](https://github.com/cameronrye/gopher-mcp/issues/89) asking whether the server respected robots.txt, the honest answer was that it didn't. There was no robots.txt handling anywhere in the codebase. Worse, when I went in to fix that, I found the politeness controls that did exist (per-host rate limiting and a concurrency cap) shipped switched off. Here's the sentence I wrote in [the fix](https://github.com/cameronrye/gopher-mcp/pull/90) once I understood what I was looking at: > Gopherspace and Geminispace are served largely by individuals on small machines, and this server had no way for them to say no. That was true for a year. From September 2025 until this week, a model looping over the fetch tools was unthrottled out of the box, pointed at small hand-run servers, the machines least able to absorb it. Nothing bad happened that I know of, but honestly, we just got lucky. The person who filed that issue spotted the risk in my code before I did, and the fix shipped the same day because of them.  There's also a version of this criticism I can't dodge, because I wrote it myself. A year ago, in [a post about llms.txt](/blog/llms-txt-standard-elegant-solution-nobody-using/), I said this about Anthropic publishing consent files it doesn't promise to read on other people's sites: > It's the equivalent of putting up a sign in your window while ignoring everyone else's signs. This site has markdown mirrors, token-count headers, rate limits that fail closed, a small pile of machine-readable courtesies for any agent that visits. I care a lot about how agents behave on my server. Then I shipped a tool that ignored the signs on everyone else's. I'd written the criticism of my own project a year early, aimed at somebody else, and didn't notice it applied to me. ## The part with no patch Sava's post makes a second argument, the more serious one, because no config flag answers it. A capsule isn't a database. It's blog posts, terrible jokes, homelab notes, CGI experiments, an imaginary plant. You visit one, follow a link to another weird little page, maybe say hello, maybe eventually start your own. That chain is how the small web reproduces, and an AI summarizing capsules from a distance breaks it. Sava has a blunt list of what goes missing, and the item that sticks with me is "nobody says hello." I have no rebuttal, and I'm suspicious of anyone who claims one. Rate limiting answers the load concern. Nothing answers "nobody says hello." What I can offer is an observation about who's on the other end. This tool isn't run by a crawler company; every session starts because a person asked their assistant about the small web, and that person now knows the small web exists. I found sava's capsule through criticism of this very tool and stayed for the plant. Some fraction of the people who first meet Geminispace through an assistant will install [Lagrange](https://gmi.skyjake.fi/lagrange/) and go look for themselves. I can't prove that fraction is large. I can only say the door swings in as well as out.  ## What the tool actually is Some of the reaction assumed a scraper. The details matter here, because they bound what damage is even possible. gopher-mcp doesn't crawl anything on its own. It fetches a page when a conversation asks for one, caches it for five minutes, caps responses at a megabyte, and gives up after thirty seconds. The content goes to one place: the conversation of the person who asked. The server stores nothing else and trains nothing. What the assistant on the other end does with its own conversation is its provider's policy, and I can't promise anything on their behalf, which is exactly why robots.txt rules naming ClaudeBot and GPTBot get honored here. Since the fix, the server also won't exceed 60 requests a minute to any single host, five in flight, no matter what the model driving it decides. The worst case from one running instance is a sustained one request per second against your server. That's the ceiling, not the average. Several people in the "We're doomed" thread made the sharper version of the capability point themselves. Acidus noted that assistants can already fetch gemini:// URLs with `echo` and `openssl s_client`, and that an MCP server is a middleman, easier to ban and block than raw protocol access. skyjake pointed out the Gemini support had been sitting there for almost a year before the sky fell. They're right, and their point cuts the opposite way from how some of the thread used it. You can't teach a raw socket to check robots.txt. You can teach this thing to, which is the only reason I think it should keep existing. One more voice from that thread, jecxjo, drew the most useful line: indexers are welcome, archivers are welcome, it's the group that's neither that needs to go away. By that taxonomy this tool is neither, and that's the uncomfortable part. What I can do is make it behave the way the welcome categories behave: identified, throttled, obedient to every robots.txt it meets. If that still isn't enough, then the verdict on the third group applies here too, and it's a fair verdict.  ## What shipped, and what ships now The immediate fix landed the same day as the issue: rate limiting and the concurrency cap are on by default now, and robots.txt support exists for both protocols. For Gemini it follows the [companion spec](https://geminiprotocol.net/docs/companion/robots.gmi) rather than RFC 9309, deliberately: the modern RFC honors `Allow:` lines the 1994 grammar ignores, so a capsule author who wrote a plain Disallow deserves the strict old reading, not the permissive new one. Rules naming AI crawler agents get honored too. They aren't part of either protocol's convention, but it's obvious what someone means by writing one. I made the robots.txt check itself opt-in, and argued in the PR that it spared an extra fetch per host and avoided silently breaking existing deployments. The person who filed the issue flagged that choice, politely, and they were right about that too. So: the release going out alongside this post turns robots.txt checking on by default, for both protocols. And if you want this tool specifically gone from your server, it answers to its own name: a `User-agent: gopher-mcp` rule is honored. Neither protocol carries a user-agent on the wire, so robots.txt is the one place you can single this tool out, and now you can. ## What I'm not building My [original post about gopher-mcp](/blog/gopher-mcp-server/) ended with a list of other protocols that "could benefit AI assistants": Finger, NNTP, IRC. I've deleted that list. Reading it after this week, it describes the small web as a feedstock in several flavors, and that framing was the real offense, more than any socket the code ever opened. I'm not building those servers. While I'm at it: my homepage said I connect "legacy protocols" to modern AI. Gemini came out in 2019 and was built as a deliberate refusal of the web's direction. Calling it legacy described my interests, not the thing itself. The wording's gone. Two housekeeping notes. I closed the robots.txt issue with a sarcastic comment, and it landed on the one person who'd done everything right. I've deleted it. And to the corner of the reaction that was mostly anatomical suggestions: I read it, I'm fine, and I'm not answering it, because the people worth answering wrote actual arguments. ## Small for a reason Sava's post ends: "The small web is small for a reason. Let's keep it that way." I agree with that more after this week than before it. The open question, and it's a real one, is whether a tool like this can exist inside that sentence. Maybe a throttled, identified version that obeys every sign gets to exist. Or maybe no version does, and the right number of users is zero. If you run a capsule and have an answer, my inbox is open. I'll listen to a no this time. Either way, I hope the plant makes it to 666. --- ## The Test Wasn't Flaky. The Server Was Quitting. > A prod-smoke suite failed for weeks and the error I spent that time chasing was never the cause. The real one was an upgrade that made a dropped connection fatal. Date: 2026-08-06 Tags: cloudflare, wrangler, testing, debugging, ci URL: https://rye.dev/blog/the-test-wasnt-flaky-the-server-was-quitting/ For most of July, the last gate before production on this site failed about four times in ten. It is a Playwright suite I call the prod smoke: it builds the real Worker bundle, serves it with `wrangler dev --local`, and drives a browser at it. When it goes red, the deploy job is skipped and nothing ships. The failures looked like this. Four specs would go down together — CSP headers, front-door cache, agent terminal, search — all with `ECONNREFUSED`. Four unrelated parts of the site, broken at once, in a suite that had passed an hour earlier on the same commit. That shape is worth learning to recognize, because it is not four defects. It is one. When everything downstream of a server fails at the same instant, you do not have four bugs; you have a server that stopped answering. Every minute I spent looking at the CSP test was a minute spent on a symptom. What follows is the part I got wrong for three weeks, and the thing that turned out to be true. ## The wrong answer, written down and committed I had a diagnosis early. Buried in the run logs was this: ``` kj/async-io-unix.c++:263: disconnected: ::write(...): Broken pipe ``` That is workerd, the runtime under `wrangler dev`, complaining about a write to a socket the client had already closed. It appeared in failing runs. It looked exactly like a crash. I believed it, and on 19 July I committed a supervisor script to restart the server around it, in `c376ab9`, with a message that stated the cause plainly: *"workerd intermittently dies mid-run on a client disconnect."* The supervisor worked. The failures continued. The problem with a wrong diagnosis that ships with a partial fix is that it stops looking like a hypothesis. It becomes the thing everybody knows. It went into a commit message, then into comments in three files, and every time I came back to the flake I started from it. Here is what finally killed it. I stopped reading the logs for the error I expected and started counting: | Run | Broken pipe lines | Server exits | Result | |---|---|---|---| | CI, 3 Aug | 2 | 0 | **passed** | | CI, 4 Aug | 1 | 8 | failed | | 46 local runs | **0** | 12 | 4 failed | A passing run had two Broken pipe lines. A failing run had one. Forty-six instrumented runs on my own machine produced twelve failures and not a single Broken pipe line. In one run, the first two exits both happened *before* the first Broken pipe line appeared. The two events were unrelated. Broken pipe is noise that a dev server emits when a browser closes a tab, and it appears in healthy runs. I had spent three weeks on a coincidence. The number that did correlate was the one I had not been counting: **every run where the server process exited failed, and every run where it did not, passed.** Restart count was not predictive either — one CI run restarted the server eight times and still went green. What mattered was whether the process was gone at the moment a test needed it.  ## An error with no message So why had I not noticed the server exiting? Because of how it announced itself: ``` ✘ [ERROR] ``` That is the whole line. No message, no stack, no code. `wrangler dev` prints an empty error and the Node process exits 1. In a CI log with thousands of lines of build output, a bare glyph is invisible, and it is not something you would ever think to grep for. The stack existed, but only in wrangler's own debug log, which nothing was capturing. The fix was two lines in the workflow: set `WRANGLER_LOG: debug` on the prod-smoke job, and upload the log directory as an artifact whether the job passes or fails. Do that before you theorize. I had been reasoning about a process whose last words I had never read. With the log in hand the sequence was legible, and it was not a crash at all.  ## What was happening `wrangler dev` runs your code in workerd, but it puts a small Worker in front of it called the ProxyWorker, which forwards each request to the UserWorker — your actual code — and pipes the response back. When Playwright tears down a browser context, it closes sockets. If a response is still being written when that happens, the ProxyWorker's `fetch` to the UserWorker rejects with a disconnect: `Network connection lost.` The ProxyWorker reports that to the controller, where it arrives as an error named **`Error inside ProxyWorker`**. Wrangler keeps an allowlist of error strings that are known to be harmless during development. There are two on it: `Failed to send message to…` and `Could not connect to InspectorProxyWorker`. `Error inside ProxyWorker` is not one of them. So it falls through to `emit('error')`, and with no handler attached to that event, Node does what Node does. Note what is *not* happening here. workerd does not die. It survives the whole thing, which is why it goes on holding port 4321 after its parent is gone — and why the naive cleanup step I will get to below made everything worse. The thing that quits is wrangler's own Node process. There is no crash to catch and no flag that makes this non-fatal. ## The one-line change that caused it None of this was new code on my side. The suite broke because I upgraded wrangler from 4.110 to 4.114 in `d63dea6`, in a routine batch of dependency updates, and deploys started failing three days later. The relevant code is in the ProxyWorker's error handler. It has to decide whether an error it just caught is still worth reporting, because by the time a network error surfaces, the UserWorker may already have been replaced by a hot reload — in which case the error is stale and should be dropped. In 4.113, that check is one line: ```ts // only report errors if the downstream proxy has NOT changed if (userWorkerUrl.href === newUserWorkerUrl?.href) { ``` Now look at what those two URLs are. `userWorkerUrl` is built from the incoming request, so it carries that request's path: ```ts const userWorkerUrl = new URL(request.url); Object.assign(userWorkerUrl, proxyData.userWorkerUrl); ``` `newUserWorkerUrl` is rebuilt from the connection parts alone, so its path is `/`. For any request to a real page, those two hrefs are never equal. `http://localhost:4321/blog/some-post` does not equal `http://localhost:4321/`. The comparison is false, the error is judged stale, and it is silently dropped. **On every path except the root, this check swallowed the error by accident.** In 4.114 the comparison became origin-based, and wrangler's own source says why: > `isSameUserWorkerOrigin` compares origin (not href) so a genuine error on a non-root path isn't misread as a reload That is a correct fix for a real bug. Origins now match on every path, so the error is reported instead of dropped — and because `Error inside ProxyWorker` was never on the non-fatal allowlist, "reported" means the dev server exits. My suite had been depending on a bug. When upstream fixed it, a dropped connection on any page went from silently ignored to fatal. Both of those behaviors are wrong; I had just been on the comfortable side of the trade. ## It was never one bad test While the diagnosis was wrong I kept trying to find the test responsible. There wasn't one. I suspected my heaviest spec, which loads about 1.3 MB of Mermaid chunks and has one case that aborts requests mid-flight. Running it alone: zero failures in eight attempts. The agent-terminal spec alone: zero in five. The full suite: about half. Then I stopped bisecting and measured the actual trigger — a page torn down while responses were in flight: - A page with no subresources survived **80** teardowns. - A normal page died within **6 to 22**. - Letting the page settle before teardown cut the rate by roughly **6×**. It is dose-response. Every teardown with something in flight is a roll of the dice, and a browser test suite does that hundreds of times. That also explains why concurrency looked like a lever and wasn't: running with `--workers=1` failed at the same rate over eight runs. Serializing the tests does not reduce the number of teardowns, only their overlap. If you are hunting a flake and single-spec runs come back clean, consider that you may be looking for a rate rather than a culprit. ## A real bug that fixed nothing One detour is worth publishing because the code looked obviously correct. The supervisor needed to free the port before restarting, since workerd outlives its parent. It did this: ```bash lsof -ti:4321 | xargs kill -9 ``` `lsof -i:PORT` matches a socket if **either** endpoint uses that port — local or foreign. A browser connected to the dev server has 4321 as its *remote* port, so it matches. That command was killing Chromium's connections during every restart. The correct filter is `-sTCP:LISTEN`, which returns only the listener; I confirmed it with a listener and one client, where the bare form returns both PIDs and the filtered form returns one. The same mistake was in the CI cleanup step. It was a genuine bug, I was right to fix it, and it changed nothing: four of six runs still failed afterward. Fixing a real bug you found while looking for a different one is not evidence you found the right one — which is the same error as the Broken pipe, wearing better clothes.  ## The fix, and why it is three things The fix is a version pin: ```json "wrangler": "4.113.0" ``` Exact, not a caret. A caret floats straight back to 4.114 on the next install and undoes everything. The pin alone is not enough, and this is the part that would have cost someone else an afternoon. Two more entries are required: - **`@cloudflare/vite-plugin` held at `~1.46.0`.** The plugin pins `miniflare` and `workerd` exactly in its own dependencies, and 1.46 is the line whose pins match wrangler 4.113. Let it float to 1.47 and the tree resolves two copies of miniflare and two of workerd. The tell is a pnpm warning naming two workerd versions at install time. - **`sharp` overridden to `^0.35.3`.** Plugin 1.46.0 carries sharp 0.34.5, which has a high-severity advisory. I had raised the audit gate from `critical` to `high` a few days earlier, which is the only reason this surfaced before it shipped rather than after. Measured across full runs of the suite: | wrangler | server exits | outcome | |---|---|---| | 4.114.0 | 7 in 6 runs | 4 red | | 4.113.0 | **0 in 14 runs** | 12 green | The one non-green run under 4.113 was an unrelated timeout, with zero exits. ## A pin is a debt Pinning to an older version to dodge an upstream fix is not a good place to end up. It is a deliberate decision to stay on the far side of a change that was, on its own terms, correct. So the pin carries its reasoning where the next person will hit it — in the Dependabot ignore entry, alongside the measurements and an explicit condition for removing it: not "when a newer version exists," but when upstream restores non-fatal handling for a client disconnect. A newer version number is not evidence. 4.118 was still reported failing, and it also pins a miniflare 5 alpha. The whole-suite retry stays in CI as a backstop, because a pin is a bet and I would rather not re-learn this from a red deploy. On the first green run after the pin landed, the retry never fired — and the run before it had restarted the server four times and still passed. Three things I would keep from this: **When everything fails at once, suspect the thing underneath all of it.** Four specs failing together was never four problems. **An error you can explain is not the same as the error that is happening.** Broken pipe had a plausible story, appeared in the right logs, and was a coincidence. The number that mattered was one I was not counting until I stopped assuming. **Capture the logs before you theorize.** The actual cause printed a bare `✘ [ERROR]` with no message, and its stack existed the whole time in a debug log nothing was uploading. Three weeks of guessing, and the evidence was one workflow line away. --- ## Making My Portfolio Agent-Readable: From Files to an Interface Agents Can Act On > I stopped publishing files for agents to discover and built an interface they can act on: markdown mirrors, an A2A agent card, and verifiable skills. Date: 2026-06-15 Tags: ai, agents, web-standards, mcp, protocols URL: https://rye.dev/blog/making-my-portfolio-agent-readable/ A few months ago I published [an essay](/blog/llms-txt-standard-elegant-solution-nobody-using/) arguing that `llms.txt` is an elegant solution to a problem nobody important is willing to solve. No major AI platform reads it. Not OpenAI, not Google, not Anthropic, not Meta. The standard, I wrote, "sits unused, waiting for a problem that the powerful have chosen not to solve." Then I went back to my portfolio and built more of it. I added an agent card. I added an A2A endpoint that agents can actually POST to. I wrote markdown twins of every page on the site. And I added a manifest of skills an agent is told to *verify*, not just trust, before acting on them. If you only read the first essay, this looks like a man losing an argument with himself. It is not. There is one distinction that makes building all of this the opposite of insane. ## Discovery is dead; usability is the live question The thing I declared dead was **discovery**. The idea that publishing the right file would make an AI crawler find me, rank me, surface me. That bet has not paid off and there is no sign it will. In [my follow-up on AI slop](/blog/ai-slop-is-a-search-problem-now/) I went further and reframed the entire category: these standards are valuable "not as a way to be discovered, but as a way to be verified once a user has found you." Hold onto that sentence. It is the hinge of this whole post. Because there is a completely different scenario the discovery debate ignores. A human is sitting in front of an agent (Claude, ChatGPT, some autonomous research thing) and they say: *go look at Cameron's site and tell me what he's done with MCP.* The agent is already here. Nobody had to discover me. The question is no longer "will it find me." The question is: When it gets here, can it actually do anything? That is a usability question, not an SEO question. And usability for an agent has a spectrum: - **Can it read me?** Most sites hand an agent a tag-soup DOM full of nav chrome, cookie banners, and analytics noise, then make it guess which `
### Polar Coordinate Spirals
The simpler spirals translate directly from mathematical formulas:
```typescript
// Archimedean: constant spacing between turns
r = a * theta;
// Fibonacci (Golden): self-similar, found in nature
r = a * Math.pow(PHI, (2 * theta) / Math.PI) * 0.1;
// Logarithmic: equiangular, seen in hurricanes
r = a * Math.exp(0.1 * theta);
// Fermat: parabolic, used in optics
r = a * Math.sqrt(Math.abs(theta)) * 2;
```
### Construction Spirals
Other spirals require iterative construction rather than simple formulas:
```typescript
// Theodorus: built from right triangles
for (let n = 1; n <= numSteps; n++) {
angle += Math.atan(1 / Math.sqrt(n));
x += Math.cos(angle);
y += Math.sin(angle);
}
// Vogel: phyllotaxis pattern (sunflower seeds)
const GOLDEN_ANGLE = Math.PI * (3 - Math.sqrt(5)); // ~137.5 degrees
for (let n = 0; n < numSteps; n++) {
const theta = n * GOLDEN_ANGLE + rotation;
const r = scale * Math.sqrt(n) * 2;
}
```
The Curlicue fractal produces particularly striking patterns by accumulating angles based on the golden ratio squared.
## Web Performance: Workers and TypedArrays
Generating thousands of points per frame while maintaining 60fps required moving computation off the main thread. Web Workers handle spiral generation, but the real performance gain came from TypedArrays.
```typescript
export function generateSpiralTyped(params: SpiralParams): TypedSpiralPoints {
const points = createTypedPoints(numSteps); // Float32Array
const rotation = time * spinRate;
for (let i = 0; i < numSteps; i++) {
const theta = i * stepSize + rotation;
const r = calculateRadius(i * stepSize, params);
setPoint(points, i, r * Math.cos(theta), r * Math.sin(theta));
}
return points;
}
```
TypedArrays are transferable between the main thread and Web Workers without copying, eliminating serialization overhead. The interleaved `[x0, y0, x1, y1, ...]` format maps directly to canvas drawing operations.
## Swift Parity: SIMD Vectorization
The Swift implementation needed matching performance. Apple's SIMD framework enables vectorized math operations that process multiple points simultaneously:
```swift
func generatePolarSpiral(params: SpiralParams) -> [SIMD2