Nothing Improves a Framework Like Building on It

I maintain three pieces of software that all my apps share:

• TechieFlow, the framework that runs my AI coding agents: it gives each agent its task, its checks and its rules.
• TrBlazeUI, a Blazor component library: buttons, menus, dialogs, drop-downs, the screens are made of it.
• TechieRag, a retrieval library: reading documents and talking to models.

On top of those three I have built five public apps with agents: Chatur, an AI-first code editor; TechieBlog, the engine this blog runs on; TfLens, a dashboard that reads the agents' run records; and the two libraries themselves, which are built under the framework and so count as apps of it.

This post is about one thing: what happens when the apps find bugs in the three shared pieces. Short version: the apps write the bug down, the shared piece gets fixed, and in five weeks that has happened 133 times. Here is one of them, start to finish.

One bug, start to finish: TechieBlog finds a crash in TrBlazeUI

On 2 November I pasted the handoff command into TechieBlog. The handoff is the last step of a build: the agent runs the app, clicks through every screen the way a user would, and tells me whether it is ready for me to test. I expected "ready". I got "not ready".

The agent had signed in as the admin, opened the posts list and typed four letters into the search box. Instead of a filtered list, it got the error page: "Something went wrong". Five tries, five crashes. It clicked Delete on a post and crashed again, before the confirm dialog could even open. I had not opened that page myself. I never got the chance to.

The bug was not in TechieBlog. It was in Select, the drop-down component in TrBlazeUI. When a page redraws while the value bound to the drop-down is an empty string, the component throws. TrBlazeUI's own demo page has used Select for months and never crashed once, because the demo never binds an empty string and never redraws a list around it. The library's own tests never hit that case either. It took an app, TechieBlog, built on the library by an agent that had never seen the demo, to hit it.

What happened next is the loop this post is about:

1. TechieBlog filed it. Its agent wrote an entry in a file called TechieBlog-TrBlazeUI-Feedback.md, which lives in TechieBlog's own repository. Every app keeps one such file per shared piece. The entry has an id (TR-077), the library version, how bad it is, whether it blocks a requirement, how to reproduce it down to the test id of the search box and the two lines of the page that bind the value, what was expected, what happened with the exception from the log, and what the agent did in the meantime. It is written so that TrBlazeUI's agent, which has never seen TechieBlog, can reproduce the crash without asking me anything.
2. TrBlazeUI fixed it. The same morning I ran TrBlazeUI's fix command. Its agent read TechieBlog's file, found the cause in 16 minutes, fixed and verified it in 21 more, and wrote under the entry: fixed, ships in 2.2.2. I committed and tagged 2.2.2 that evening.
3. TechieBlog closed it. Three days later TechieBlog's agent upgraded to 2.2.2, re-ran the same search and the same Delete, and wrote the proof under the entry: "re-checked on 2.2.2, the search box filters, Delete deletes." Only then did the entry close.

Filed by the app, fixed by the library, closed by the app. The app owns the first step and the last.

133 bugs in five weeks

That loop runs for all three shared pieces. Here is what the five apps filed between 1 October and 8 November, and how many releases came out of it.

133 entries. 31 releases. I planned none of them. Every release exists because an app asked for something, and I know what is in each one because the libraries' agents never commit or publish anything: I read the reply, I commit, I tag.

One number stands out. TfLens, the newest app and the one built fastest, filed 56 bugs against the framework, more than the other four apps put together. A new app walks the framework in the order a new builder would, and trips over everything the older apps learned to step around. If you want to know where your framework is weak, build something new on it.

What Chatur asked TrBlazeUI for

Not everything filed is a crash. Chatur, the code editor, needed things from TrBlazeUI that a component library built for blogs and dashboards did not have: a tree control for files and folders, a side-by-side diff viewer, an editable code block, a sortable list, submenus in the menu bar, a disabled menu item that a screen reader actually announces as disabled. Seven of those were filed on one day at the start of October and closed within the week, across four TrBlazeUI releases, each with the proof written under the entry by Chatur's own agent: "TrBlazeUI 2.1.3, each run step uses StepperItem.Icon."

Three bugs in TechieFlow that its own tests missed

TechieFlow, the framework, has eleven test suites and runs them in two time zones on every push. Here are three bugs from this month that those suites did not find and an app did. All three were found by TechieRag, the retrieval library, while it was being built under the framework.

The verifier that believed one passing test. TechieFlow's verifier grades each requirement from the tests that carry its id in their name. One of TechieRag's requirements had been marked Blocked, waiting on a decision only I could make, and one of its tests happened to pass. The verifier wrote Verified over Blocked, and a requirement I had deliberately parked was reported done. TechieFlow's own suites could never have found this: none of their fixture projects has a parked row with a passing test. TechieRag's agent filed it. The fix is a new verdict that keeps a Blocked row where it is and records the passing test in the remark, so only the person who parked it can un-park it.

The test that could only pass alone. One of TechieRag's tests loads a model and needs more memory than a full test run leaves it. Run on its own, it passes. Run with the others, it is skipped, and a skipped test is not a pass. TechieFlow's verifier had no way to run one test by itself, so for three weeks that requirement was never measured in either direction: not red, not green, invisible. TechieRag filed it; TechieFlow's verifier now runs a single test on request and merges the result into the full run.

The installer that drifted. TechieFlow has three shell scripts and one Node installer that are all supposed to write the same ignore list into every new project. One night the scripts gained a line and the installer did not. Nothing broke anywhere, because every existing project had been set up by the scripts. TechieFlow's own CI caught it the next morning, red on both jobs, because its installer test compares all four against each other. The fix is one shared list that all four read.

None of these is visible from inside TechieFlow. The first two needed a library with a parked requirement and a heavy test. The third needed four installers and a project set up by each. A test suite tests what its author imagined. An app tests what a builder actually does, in an order the author never imagined, with data the author never had. With agents as the builders this gets stronger, not weaker: an agent follows the framework's instructions to the letter, so every ambiguity in the framework becomes a wrong turn in some app, and the wrong turn gets filed.

The two times the loop broke

It waited on my calendar. In mid-October I made a rule that TrBlazeUI and TechieRag fixes ship once a week, on one evening, because the libraries' test runs and the apps' test runs share one browser and I did not want them colliding. TechieBlog's Select crash arrived on a Monday, which was not the day. TechieBlog would have sat unusable for its admin for three days because of a rule I wrote to protect a browser. I changed the rule that morning: the weekly slot is for improvements; a bug that blocks a requirement ships the same day.

It read one file. That same morning's instruction to TrBlazeUI named only TechieBlog's entry, because TechieBlog's handoff was the one I had just read. Chatur had filed a TrBlazeUI bug a few days earlier, a focus ring standing off its border and running into the label above it, and it was sitting in Chatur's file, unread. TrBlazeUI 2.2.2 would have shipped without it. The rule now: a library's fix run starts by reading every app's feedback file for that library and listing every open entry, before it does anything else. Both fixes went into 2.2.2.

Both rules exist for the same reason. The loop is only as good as what the library's agent is told to read, and I had been telling it to read what I happened to remember.

Why only the app closes a bug

Chatur asked TrBlazeUI for submenus in the menu bar in mid-October. TrBlazeUI shipped them within days. Chatur re-checked three weeks later, on a later release, when its own fix run next touched the menu bar: File, Recent projects and View, Zoom are now submenus, and the entry closed. For those three weeks the entry said "fixed upstream" and nothing more, and that was the truth.

TrBlazeUI's agent can show that a component now does a thing, in TrBlazeUI's own demo. Only Chatur can show that Chatur's menu bar works, on the version Chatur actually runs. Those are different claims. If the file used one word for both, the app's "fixed" would quietly come to mean the library's, and you would find out which one it meant on a screen, probably in front of someone. So the two words stay different, and only the app writes the second one.

A small note on who dogfoods

Everything above is built on Microsoft's stack: Blazor for the screens, .NET MAUI with Blazor Hybrid for the desktop apps. I am not special in that. Plenty of companies ship real products on it, and a great deal of the software the world runs on every day is .NET. I like the stack and I am not leaving it.

What I find funny is this. As far as I can tell, Microsoft's own big apps are not built on MAUI or Blazor Hybrid. The people building on those two frameworks are everyone except the company that makes them: other companies, consultancies, and people like me in the evenings. If this post shows one thing, it is that a framework gets better fastest when its owner has to live on it. For MAUI and Blazor Hybrid, the ones living on it are us, and the bug reports only go one way.

What it costs me

I want to be straight about this, because the version of this post I almost published said "about ten minutes a release", and that is the kind of number that makes dogfooding sound free. It is not free. The agent runs are on a flat monthly plan, so the tokens do not show up as a bill. My time does.

Every one of those 133 entries went through me. I read the app's handoff that found it. I wrote the instruction for the library's run. I read the library's reply. I committed, I tagged, I pushed. Then I read the app's re-check. Then, more often than I would like, I fixed the CI by hand because the agent's fix was the wrong one. TechieFlow's CI went red three times in the last fortnight and I was the one up at midnight making it green. Thirty-one releases is thirty-one evenings with a terminal open. Two of last week's ended after one in the morning, waiting on runs. My time tracker says the terminal is where more of my own hours go than anything else: seven hours last week, on top of a 40-hour office week.

Dogfooding is the right thing to do. It is also the most expensive thing on this page, and the price is paid in evenings.

If you maintain something other people build on

Build something real on it yourself, more than one thing. Give each app a file where it can write down what broke, in enough detail that the library can reproduce it without asking. Make the library read every app's file first, every time, before it looks at its own roadmap. Ship a bug that blocks someone the same day, and batch the rest. Let the library say "fixed", and let only the app say "closed".

And count your own evenings in the cost, because that is where the real cost is. The 133 fixes in my three shared pieces were all found by the apps. Not one of them was my idea. Every one of them was my evening.