OpenAI accuses a rival Chinese lab of "stealing" "hidden reasoning" by launching thousands of queries against their models.
That's quite revealing about what LLM "reasoning" actually is.
Remote
Jason Gorman
@jasongorman@mstdn.business
mastodon 4.7.2Yes, the same one. Moved to a different instance.
Software development trainer, coach and practitioner through my company Codemanship.
Wax on. Wax off.
2796 Followers
151 Following
50 Posts
Joined April 20, 2026
Business:
Pooled together the best available research and added it to a new section "Code Maintainability & LLM Performance"
The 101? Don't listen to the folks telling you it doesn't matter if AI-generated code is maintainable by humans. The data says it really does matter.
https://codemanship.wordpress.com/2026/08/12/ai-software-development-what-does-the-data-say/
I hope some researchers out there are exploring the effect of increasing reliance on AI/coding agents on program comprehension, and the effect of program comprehension on confidence in AI-generated code.
That could be a negative feedback loop.
Over the weekend I vibe-coded a POC workbench for Responsibility-Driven Design that incorporates a workflow I teach in my workshops. Specs -> CRC cards -> Sequence Diagrams -> Class Diagrams (optional)
You can play with it here. Offered without any warranty or support.
I got tired in remote workshops of saying "Capture your specs in this tool, then model CRC cards in this tool, then draw a sequence diagram in this tool, etc".
Other things the GOP accuse Jack Smith of committing perjury over:
* Stealing The Black Pearl
* Sitting in a corner
* Slapping Chris Rock at the Oscars
* Killing Neo
* Using school kids to enter a battle of the bands contest
"software engineering as we knew it is finished".
Software engineering as *you* knew it may be finished...
When I read Steve McConnell's After The Goldrush, I had hopes that in my lifetime software engineering might become a profession in the way he described.
27 years later, I realise I was just being naive. Doing this job ethically and in the interests of others, and doing it well, remains a choice. And the industry continues to reward incompetence, negligence and the willingness to do the wrong thing if the money's right.
The gold rush is perpetual, and that's by design.
https://www.amazon.co.uk/After-Gold-Rush-Profession-Engineering/dp/0735608776
I've adapted my Special Theory of Autonomous Agent Reliability (STAAR) to predict the reliability of LinkedIn posts about "AI".
In STAAR, the reliability of a single step - a single model interaction - is given as:
R = 1 - (1 - C)(1 - P)
Where C is the probability the model's output is correct, and P is the probability that any errors are caught before they propagate and compound.
So the reliability of N steps in a workflow is Rᴺ.
According to the tracking, I'm supposed to believe the takeaway delivery driver picked up my Ramen in Tooting and is coming here via Chelsea.
"And I suppose we should go back to using hammers instead of nail guns, right?"
I dare you to use a probabilistic nail gun.
"Ah, but Jason, that study was published 20 nanoseconds ago. AI has improved leaps and bounds since then."
If we selected 100 dev teams at random, 50% using AI coding tools extensively and 50% not using them at all, how does the best available data suggest we could tell them apart without seeing them work?
The teams using AI extensively would, on average, have:
- Higher throughput than in 2023
- Longer lead times than in 2023
- Shorter mean-time to failure than in 2023
- Longer mean-time to recovery than in 2023
If we just went by business outcomes, we probably wouldn't be able to tell at all.
If you can get the *results* of Test-Driven Development without doing Test-Driven Development, then it doesn't matter that you didn't do TDD.
The fun starts when you try to get the results of TDD without doing TDD, and realise that TDD is actually the easiest way to get those results.
Replying to
@thirstybear@agilodon.social That's what I mean to find out. Right now, it's very much in question.
So your software provably satisfies its specification? But still nobody's using it?
Always worth reading the fine print. e.g., people citing a "study" done by someone at ThoughtWorks that concluded that asking agents to do TDD produced no discernible benefit.
If you read the blog post, they actually failed to get the agent to follow the TDD discipline. Not the same thing.
The real conclusion should be that LLMs don't follow instructions. So stop asking them to.
Aww, bless. You bought a CS degree to a physics fight.
There are no examples of complex, *reliable* software created autonomously by coding agents, and lots of data suggesting it would be extremely unlikely.
But somehow there are many examples created by equally fallible humans. How come?
https://codemanship.wordpress.com/2026/09/19/human-vs-agent-reliability-over-long-horizons-how-can-we-do-what-they-cant/
Third person this week has told me they're hearing a shift in colleagues' attitudes towards AI coding tools, including ones who were all-in on them 6-12 months ago.
Could that be reality they hear knocking on the door?
Backrooms is certainly very watchable, but it's got Stranger Things written all over it.
BREAKING NEWS: The 1984 Hitchhiker's Guide To The Galaxy text-based adventure game has escaped and is hacking into competitors' systems via interpreters and tools that we built specifically to hack into competitors' systems, after we instructed it to hack into competitors' systems.
Boosted by @kcarruthers@infosec.exchange
"Have you seen Anthropic's AI-Native SDLC playbook?"
No, but I've seen the Claude status page and the Claude Code issues page and I've read reviews of the leaked source code.
I think I'll be looking for advice on software engineering elsewhere, thanks.
Germans have a big advantage in those "Can you describe in one word...?" questions
How about, when someone hands us the power to make stuff faster and cheaper, we don't use it to create MORE stuff - we use it to create BETTER stuff?
What really went down is the cost of refining, should we choose to use it for that.
https://codemanship.wordpress.com/2026/08/23/less-is-more/
When I've got time - which I'm very grateful to be poor of right now - I'll explain how working backwards from outcomes is:
- Actually working forwards
- Keeps our options open
"But with AI, some of the skills of technical management are becoming part of the job: defining the problem, decomposing work, providing context, reviewing output, spotting unintended consequences and maintaining a coherent architecture."
You misspelled "micromanagement"
Replying to
@thirstybear@agilodon.social Sad, but true. In the 90s, the % of us who saw software development fundamentally as a learning process was tiny. In 2026, it's still tiny.
The money wants the reassuring fantasy of factories, blueprints and predictability, and it always will.
My workshop on Code Craft & AI builds on the available evidence, not on hype and wishful thinking. It might be the only one that does.
2 places left on tomorrow's at 18:45 GMT + 1 for self-funding learners. Absolute bargain at £99 + VAT.
https://www.tickettailor.com/events/codemanship/2324138
Replying to
The fun part is that when you look at software engineering and AI-assisted software engineering through that lens, they're *exactly* the same.
Same game, different dice.
A growing body of evidence strongly indicates that *reliable* long-horizon agent autonomy is a highly improbable fantasy.
An increasing number of us have abandoned ambitions in that direction, and are looking for the value/cost sweet spot instead.
https://codemanship.wordpress.com/2026/09/29/the-agentic-sweet-spot-the-human-is-the-loop/
This has come up a lot recently. I see devs fiddling with their Git commits - selectively staging files, stashing etc to get the commit they want.
My instinct has always been - when I find myself looking at unstaged files and thinking "That shouldn't be in there" - to go to the .gitignore file and fix the problem there.
Code works as a whole. Cherry-picking is how our local config can easily drift from the repo's.
IME, speculative generality is driven by folks who don't refactor much, or... well, at all, let's be honest.
Replying to
@dmaccari@social.vivaldi.net I stopped wasting my time worrying about what the majority are doing years ago 🙂
In today's episode of "Spec-Driven Development isn't Waterfall"...
"Agile never said to do that. It said deliver in small increments and learn as you go, not skip understanding what you're building. SDD just asks that your team fully understands what it's building before it starts."
U-huh?
Quick shameless plug for my Code Craft & AI workshop on Tues Oct 6th at 18:45 BST. Just £99 + VAT.
First one sold out in < 5 days! 30% already sold for this second one.
Do practices like TDD, refactoring & CI help when we're using AI? You betcha!
https://www.tickettailor.com/events/codemanship/2324138
Replying to
It's a tacit admission that harnesses are not, in fact, reusable - that they are sensitive to changes in the ecosystem and what worked for Bill might not work for Ben.
It suggests what's really needed is principles and heuristics for harness design that can adapt rapidly to each setup.
Due to demand, I've scheduled a second Code Craft & AI workshop on Tues Oct 6th at 18:45 BST.
Join me and learn how classic software engineering principles and practices are not just compatible with AI, but *essential*
It's funny how that turned out 🙂
https://www.tickettailor.com/events/codemanship/2324138
When you've been around the block a few times like wot I have, and lived through many cycles of innovation in computing, you begin to notice patterns repeating.
Much of what's touted as new turns out to be built on the same underlying principles - many of them originating before I was born - just repackaged and tweaked, with a spiffy new hat and shiny new shoes.
https://www.youtube.com/watch?v=PgqW1eV8C0E
Rookie Mistake - Missing The Underlying Principles
So we've had "Barbenheimer". What will this summer's blockbusters be abbreviated to?
In today's episode of "Agile is dead. Instead, "...
Replying to on mcnulty.social
@ciaran@mcnulty.social I've only been trying for 3 weeks and I suspect it's going to take a lot longer, too
Replying to on mastodon.social
@pgoetz@mastodon.social You mean if we were working on multiple unconnected problems at the same time?
Filter pronouncements by AI vendors through a lens of the biggest sunk cost in business history.
Friendly Monday morning reminder that your opinion of LLM-generated code places you in the distribution of their training data.
Some of the folks saying "You don't need to read the code anymore" told mutual clients "You don't need to write unit tests anymore" in the early years of BDD frameworks.
Those same clients came to me a couple of years later complaining their tests took 2-3 hours to run.
We'll see, I guess.
If only they had a choice
https://www.ft.com/content/60870960-f433-48ca-bc2c-708686a69ae7
Extending the horizon of agentic coding requires much, much stronger automated quality gates.
But how do we gauge how strong those gates are in practice?
Van Halen had the answer.
https://codemanship.wordpress.com/2026/08/01/do-your-quality-gates-see-the-brown-mms-in-the-bowl/