Mark Levison

The bottleneck was never writing code. It's understanding what to build. If you're using AI coding tools, focus on: • Smaller features (if it's 1000 lines, it's too big to review) • Clear acceptance criteria before you prompt • Tests first, AI-generated code second • Security audits (AI can't do this) More code isn't the goal. Solving real problems is.

Full analysis: https://agilepainrelief.com/blog/ai-generated-code-quality-problems/?utm_source=mastodon&utm_campaign=archive-reshare #AI #AIinSoftwareDevelopment

@anna_lillith nice sentiment. It turns out the claims about trees and plants don’t validate. See: https://pca.st/episode/84021c1d-fdd3-4cd8-9842-f16684be9494

@pcdevil@mastodon.social @deborahh@cosocial.ca It’s complicated. I think that medical transcription will get much much better. It’s a highly specialized language. The models in question aren’t good enough yet, but this is a time problem.

@jasongorman@mastodon.cloud how long are the workshops. I’m thinking of joining your SBE workshop.

@WTL More like there might be family obligations, wherein being a parent takes priority. Somedays I need parenting.

@WTL I like both, the only question will be, am in parent mode?

@ccgus I really like working in Zed. I only miss the refactoring tools from JetBrains. I’m might even put in the effort to port a set over to Zed. The obvious one to pattern off of is Abracadabra in VSCode.

@weddige agreed and the evidence is that you can review 200-400 lines of code in an hour, with 300 lines being sane. Realistically, you can do that once or twice a day.

@georgetakei it brings me so much joy to see you participating here. May the next year bring you many smiles. I look forward to seeing the snark around the corrupt orange man.

I still link out. I also cite with some rigour.

@carnage4life@mas.to productive or technical debt? Read: https://agilepainrelief.com/blog/ai-generated-code-quality-problems/ for details on the increase in defect density etc

@davidbisset but my LLM is trained on 2015

@ceremus@hachyderm.io I played it on a Sunday Sparc workstation so even more variants. @cmconseils@mastodon.social

@imcdowall @jose Early days. We're still learning. I mostly use: https://github.com/PaulDuvall/claude-code --- I use /xspec to create the core acceptance criteria. I read and edit. Then use /xtdd to attempt to write the code. I keep all changes small and only commit what I can read and understand.

@oschonrock I just skimmed the paper the obvious difference seems to be around code base size. This paper seems to be focused on small tasks and code bases. I think the papers I'm citing are more focused on larger code bases.

@oschonrock well I wouldn’t cite them if they didn’t do solid research. I'm very selective about who I reference. So good++ ++ ++

@jrp Interesting. I've never done Auido related development. I think i would fail harder than the GenAI tools.

@yon I have lots of theories as well. Some of my friends have experience that contradicts the data. Ideally we would measure the quality of the code from people doing this well and compare to the baseline. Until we do that we have to assume we’re no better than the baseline.

@rhold the links are in the blog post: https://agilepainrelief.com/blog/ai-generated-code-quality-problems/

@project1enigma these might help The bigger issue is that GenAI has no judgment. No understanding of correctness; readability etc. Better refactoring is great, but not enough. Curious what languages are you referring to?

@mhoye Apparently people like working for Bending Spoons proper. However if your company is bought by Bending Spoons, good luck.

@marcoarment@mastodon.social …corked.app pulls a hold my beer on you. I’ve been using Corked unsuccessfully for most of a year. Every time it updates the casks a few of them require I finish from the command line. So really it just exists to yell run a command.

@jose@mastodon.gamedev.place I like your friend already. Yes we can try TDD/BDD, the problem even with good gaurd rails, it will sometimes do more than you want.

@crackhappy Most of the time I call it GenAI. LLMS is a better choice, but I need to use the language of the audience. If I say LLM then I have to explain it, @thirstybear

@stonedonkey@mastodon.social I did the digging: https://agilepainrelief.com/blog/ai-generated-code-quality-problems/

@oschonrock I highly recommend the book: "Your Code as a Crime Scene" - Adam Tornhill. https://codescene.com/hubfs/web_docs/Business-impact-of-code-quality.pdf this is the paper that I recommend to Scrum Masters and Developers.

@robpike@hachyderm.io just found your account here. Thanks for building the platforms I’ve relied on for forty years

@project1enigma I do refactoring all the time. I still have a copy 1st edition of Refactoring - Martin Fowler. They tools exist, but they don't necessarily help the genai tools. Maybe a Claude Plugin for refactoring?

@elly@donotsta.re I don’t love the tools, however they exist. In some cases, they help. I've written large chunks of personal automation that would never have been possible without LLMs. I'm building an app, part-time for my wife's business, in TypeScript + SvelteKit + Postgres. Without ClaudeCode, I wouldn’t have a chance - I’m not a TypeScript developer, my last professional work was in Java and C#. In addition, these tools are going to sweep over our industry, I want to understand where they are going

@sambeaven Makes sense

@oschonrock As I've said to a few other people, the best teams with exceptional discipline may get better results. In which case, I want data and I want measured by the CodeRabbit people so we have a basis for comparison.

@mhoye@cosocial.ca I’ve looked and the reviews are great idea, only adequate execution.

@stonedonkey@mastodon.social this has been my reading for most of the past month. FWIW some in the Agile community claim they’re doing better work with these tools. I just wish these claims were tested. Also in the department of honesty. I do use these tools to automate work that wouldn’t otherwise get done.

@steltenpower@social.edu.nl LOL

Would you consider taking the time to write an analysis of the code of ClaudeCodr from a maintainablity, quality perspective? I promote that on repeat

@GossiTheDog@cyberplace.social gross but not surprising

@dtemme related I’m blown away by the emotional reaction I’m getting to research backed posts that examine the challenges we face with these tools. Foot instance this post https://agilepainrelief.com/blog/ai-generated-code-quality-problems/ has got many reactions about dated research @samir

@remi@ruby.social what does failing to keep ssl up to date have to do with GenAI?

It’s more complicated than the claim in the article. They’re not really scanning your computer. They’re fingerprinting your web browser. However, this is a common bad practice.

@remi@ruby.social people were making the same mistake before AI. Others have pointed out that GitHub has gone down hill since the MS acquisition. No AI required.

@mhoye@cosocial.ca I commented on the proposal

@oschonrock Street cred: The first author of the paper is associated with CodeScene; see, I cite one of their papers for students wanting to understand where to tackle technical debt/mess.

Congrats

@AnarchoCatgirlism ...the same file apparently has 12 layers of indentation. If you have that depth of analysis in a blog post, it is very easy to reshare with the world. And it makes a nice job ad - because it shows your skills and depth.

@celesteh But those resources wanted to be consumed. Debugging vibe code is entertaining.

@pcdevil@mastodon.social I'm not an advocate. I study how they work and how they fail. The failure mode in this case appears to be the language not recognizing they words for medications and other medical terms. Medical use of LLMs, isn't something I've studied. However, I would expect they need 100-1000s of doctors saying the names of different medications etc. Then someone needs to do the reinforcement learning. None of this is trivial and shouldn't violate privacy. @deborahh@cosocial.ca