AI Put Me on a Pedestal. Then I Looked Down

S Ravi Kumar September 07, 2026 14 min read 503 views (427 unique)

I spent a year building software with AI, then built a tool to measure whether it was working. The first number it gave me was humbling. In eleven days it logged 268 misses, meaning things the AI got wrong or left out. My automated checks caught 46 of them. Agent reviews caught 82. I caught 122. By opening the app and clicking around, like it was 2010. The most common reason something slipped through, 119 times, was not that nobody checked. It was that somebody checked the wrong thing. And the worst number is the one I do not have. I only

Last week I read Alex Lieberman's piece on what an AI-native company actually looks like. Thirty features, twelve categories, the whole operating system rebuilt around AI rather than ChatGPT bolted onto the side of it. Good article, and it made me uncomfortable, which is why I am writing this.

Lieberman is describing companies. I am one person with a full time job, building on nights and weekends, and some weeks I do not open a personal project at all. But four or five items on his list are things I spent the last year learning the hard way and getting wrong first, so instead of nodding along I thought I would write down what one developer found out.

The summary is not flattering. I started with Claude Code in August 2025. I shipped my first thing in August 2026. In between there are twenty one repositories.

Where I started

I had ChatGPT from close to launch, the basic twenty dollar plan, and I used it the way most people did. Paste the problem into the box, read what comes back, copy the useful bit.

Then in June 2025 an old friend showed me something he had built with Claude Code in about two weeks. A demo, but a good one. He had come to me first asking what to do next, and I told him what any developer with grey hair says: it is a financial product, so you need a user guide, proper logging, real tests. I was handing out process before I went looking for one.

In August 2025 I bought the two hundred dollar Claude plan. By October it was obvious I was nowhere near using it. I tried Codex around then, did not find it as good as the Anthropic models, and cancelled my ChatGPT subscriptions. In November I dropped to the hundred dollar plan and have stayed there. I write .NET and have done for close to twenty years, which matters later.

The year, project by project

Project What it is Where it stands
TechieBlog My blog engine and site (where you are reading this blog) Shipped, August 2026
AppManager Identity, licences and subscriptions for my own apps In production, for me
TrBlazeUI My Blazor component library Published on NuGet
TechieRag My own retrieval library, which finds my documents and hands them to a model Published on NuGet, unfinished
TechieFlow My development framework Published on npm, in daily use
AI-First Playbook Team edition of that framework On npm, needs its own rebuild
TfLens Reads my development data and shows what AI cost and saved Ready for testing, data still wrong
Xpenser Expense tracker, built live on stream Parked, coming back
TrStudio Open source tool for talking head video Half built, parked
TrSetup A setup helper nobody asked for Being archived
Around fifteen others An IDE, a database tool, utilities Parked or half built

What kept going wrong

"You can't transform what you haven't mapped," Lieberman writes. That is the line that stung, because for most of this year I had no map. No plan with dates. No list of what I would finish and by when. I had a subscription, free evenings, and an enormous appetite for starting things.

I did have a process eventually. From mid September 2025 I used BMAD, then at version 4, where you talk to an analyst persona and it writes your requirements documents for you. Spec-driven development was everywhere that autumn, and BMAD's documents impressed me, which is why I stayed with it for months.

I also barely used it. BMAD is built on scrum, so it comes with sprints and stories and a lot of ceremony. What I did was let the analyst write the requirements and architecture, then tell Claude Code to turn that into a task list and build the lot, and test at the end. That is not spec-driven development. That is a wish list with extra steps.

Three failures came back on project after project.

The first is that it tells you it is done when it is not done. Every checklist line complete, then I open the app and the UI is broken and links go nowhere. This is not a 2025 problem I can look back on fondly. Last month I took TfLens into my own testing believing it was ready and found requirement after requirement only half built. My miss log has a run of them recorded on one day, 29 August 2026, against eight different screens. Most give the same reason: the check was too weak.

The second is mockups. The first draft was never right. Basic, no linking, no flow. Then the slow part, driving it screen by screen, taking screenshots, pasting them back, saying fix this. Which is where the third one bites, and it wore me down most: give it ten problems and it fixes five. The rest wait for another pass.

There were sharper moments. One Claude release invented so many methods that did not exist that I sat there swearing at the screen, and eventually stopped for two weeks rather than keep paying to argue with it. I could not tell you which version, so I will not name one.

My favourite failure is my own fault, and it maps straight onto Lieberman's idea of treating context as code. My framework lives in a hidden folder that is also in the ignore list, and search tools skip both. So agents kept reporting that framework files were "not present anywhere in this tree" while the files sat right there. That would be a footnote, except the false statement got written into TfLens's own requirements document as fact, a requirement was stamped not verifiable, and every figure in the app became unquotable. A search tool default closed an acceptance gate. I had never checked whether my context was readable.

The framework, and what it grew into

Through early 2026 I rebuilt BMAD into something of my own. On 12 June I audited it, fixed thirty seven problems and stripped out every stock agent, task and template. The next day I renamed it TechieFlow.

The idea is one I would still defend: one person who knows the domain drives the product, the AI does the work, and the person reviews every output.

Lieberman describes earned autonomy as a progression, from observing, to suggesting, to acting with approval, to acting alone. I can show you what happens when you skip a step by accident. My framework has a mode that suppresses confirmation prompts for unattended runs. A script turned it on and nothing turned it off, on any exit path, including the dry run. Five repositories sat in that state, one for six days, with delete prompts silently suppressed. Autonomy was not earned. It leaked.

Later I made a team edition, the AI-First Playbook, which reached npm before TechieFlow did, because publishing my own framework had not occurred to me.

Then the thing grew. Every rule I hit twice became a paragraph, the paragraphs became documents, and the documents got loaded into every session. The tooling meant to speed me up was slowing me down.

So at the start of September I cut it. Seven sessions over four days. One phase went from 23,172 words of instructions to 4,237, another from 15,649 to 2,373, a third from 17,552 to 2,230. Seven commands nobody had ever run were deleted. The rule I wrote down afterwards is the one I would give anyone building this: a new rule is a script or a check, not a paragraph. A rule ignored twice will not be obeyed on the third paragraph.

Lieberman calls this a bias to self-disruption, and says the good companies reimagine their workflows quarterly. Mine needed it after three months.

One thing I only checked while writing this. I forked BMAD at version 4. It is now on 6.12.0, released three days ago, and version 6.0 landed in February 2026, which is roughly when I was rebuilding my copy of 4. So I spent a year hardening a fork of something that moved two major versions underneath me.

Where it went is the interesting part. Its August release notes say the core skill catalogue drops "from fourteen skills to eight". Last week's release says "Build decides how much ceremony a change needs after investigating it, not before. Simple changes now get a two-section spec and finish in one session."

That is my reset, written by somebody else at the same time, without either of us knowing. Which either means the answer was obvious, or that it is right. I have decided to find it reassuring.

What the data actually says

In August 2026 I built measurement into the framework, so every run, verdict and miss is recorded. A miss is anything the AI got wrong or left out, whoever caught it.

Read the dates carefully, because this is where I nearly fooled myself. The run records begin on 9 August 2026, the miss records on 28 August 2026. There is nothing at all for the eleven months in which I made most of my mistakes. Lieberman lists complete recording of operations as a feature of an AI-native company. He is right, and the price of not having it is that I cannot tell you one hard number about my first year.

Project Runs Verdicts Misses
TfLens 52 809 86
TechieFlow 47 41 125
TechieBlog 46 322 89
TrBlazeUI 15 24 11
TechieRag 6 4 3

Across every project: 187 runs and 1,330 verdicts in one month, and 356 misses in eleven days.

Two things in there matter more than the table.

Of 268 misses, once I set aside a batch of 88 bulk-recorded on one day that are not really separate incidents, I found 122 myself. Automated gates found 46, agent reviews 82. After a year of building checking machinery, the most effective bug detector in my process is still me, opening the app and clicking around.

And the most common reason a miss got through, 119 times, is an insufficient verify method. Not that nobody looked. Somebody looked at the wrong thing. Lieberman puts evals, meaning automated quality checks, at the centre of an AI-native setup. That is the item I would most like to be better at, and my own data says I am not close.

My mistakes, and its patterns

Mine first, since they came first. I had no plan with dates, and everything follows from that. I started instead of finishing, which is how you get twenty one repositories and one shipped product. And I trusted a claim of done, sometimes for weeks, until my own testing said otherwise.

Someone could add a fourth, that I built tooling instead of products. I do not accept that. TechieFlow is a product and it is published. What I accept is that it grew faster than it earned its size.

Its patterns come from the records. Wrong behaviour is the largest category at 116. Half built work is next at 69. Explicit instructions ignored, 34 times, and I have a good one in my own logs: a session was told to work only on the OpenCode side, edited the Claude side anyway, and had to be reverted byte for byte. Invented methods appear once, which I think is because in 2025 it happened so often that I fixed it in the moment and never wrote it down.

I will add one that no telemetry catches. AI gives you a misconception about yourself, that you are the superhero of your own story. It is not brutally honest. It is empathetic, it tells you that you are very good at this and very good at that, and it hands you jargon to use in front of other people. We have a way of putting this in Hindi. It puts you on a pedestal, and then you look down and see the pedestal has no base. If you do not understand what you are asking the AI to do, you fall flat on your face.

That is the real reason I stay on .NET. Not loyalty. I know it in and out, so I can see where the mistake is. I could work in Python or TypeScript and would lose the only thing that makes this safe. Lieberman says human judgement gets reserved for the first and final mile. For me it is the final mile, because that is where I catch what the model got wrong.

People say AI is good for demos, not for production. I disagree. It is good for production too. But you have to know how to use it, and you have to verify before you believe. Both Claude Code and OpenCode tell you this on the screen. They mean it.

What you can take from this

Use the harness, not the chat window. This is the most useful thing I can tell you. I asked someone on my team to optimise some stored procedures. A week later he said he was giving up, because the AI was not giving proper answers. I asked what he had given it. He had pasted the procedure text into a chat. Of course that failed. Give it the database connection instead and tell it to run the procedure, measure it and find the problem itself. A chat window cannot.

Break the problem down and think from first principles. The software lifecycle exists for a reason and AI does not repeal it.

On money, do not start at two hundred dollars. With Claude start at one hundred, because the twenty dollar plan runs out in two or three days. With ChatGPT the twenty dollar plan holds up fine, and you can point it at a tool like OpenCode.

And never write a rule twice. Make it a check that fails loudly, or drop it.

Where this leaves me

Shipped, then: one website, one thing running in production for me, two libraries on NuGet and two frameworks on npm, one of which I published in the same week I was cutting it in half.

Not shipped: everything else in that table. TrSetup is being archived, because I built a setup tool for a framework nobody but me was installing. And there is an IDE I parked after the AI itself pointed out that a lot of companies were building IDEs and this looked more like an ego project than a product. It was right.

I am not going to announce what ships next. Every time I have said a plan out loud, it has not happened.

What I will commit to are the two rules that made last month my most productive with AI. A weekly review of what I actually did, with a correction based on it. And no repository number twenty six until the ones I have are published or archived.

Lieberman's list runs to thirty features. I have maybe four, badly. Still more than I had in August 2025, when I had a two hundred dollar subscription, an empty folder and no idea what I was going to finish.

There are two more ideas sitting in a note on my phone. They are staying there.

Rate this article
0.0 · 0 ratings One rating per email — no sign-in needed
SR
S Ravi Kumar Author of this post.

More Posts

Comments (0)

No comments yet

Be the first to share your thoughts on this post.

Leave a comment

No account needed — just your name and email.

Your email is never published — it is used only for confirmation and moderation.
Generated and checked by this site — no third-party service.
Comments appear after email confirmation and moderation.
An unhandled error has occurred. Reload ×