A while ago someone posted a claim like that on LinkedIn again.
And of course there was the usual herd of LinkedIn sheep who were full of compliments and wows about the claim he was making: a 10x speedup of his daily work.
The difference with the zillion others who did the same, is that he attached a link to a live stream where he was going to show his 10x speedup on a real life problem.
Credits to him for doing that! So I decided to go have a look.
What I then saw was him struggling for one hour with some simple extension to his project. He didn't manage to finish in the hour what he was planning to. And when I had some thought about how much time it would have cost me by hand, I found it would have taken me just as long.
So I answered him in his LinkedIn thread and asked where the 10x speed up was. What followed was complete denial. It had just been a hick up. Or he could have done other things in parallel while waiting 30 seconds for the AI to answer. Etc etc.
I admit I was sceptic at the start but I honestly had been hoping that my scepticism would be proven wrong. But not.
I'm going to try and be honest with you because I'm where you were at 3 months ago
I honestly don't think there's anything I can say to convince you because from my perspective that's a fools errand and the reason for that has nothing to do with the kind of person either of us are, but what kind of work we're doing and what we're trying to accomplish
The value I've personally been getting which I've been valuing is that it improves my productivity in the specific areas where it's average quality of response as one shot output is better than what I would do myself because it is equivalent to me Googling an answer, reading 2 to 20 posts, consolidating that information together and synthesising an output
And that's not to say that the output is good, that's to say that the cost of trying things as a result is much cheaper
It's still my job to refine, reflect, define and correct the problem, the approach etc
I can say this because it's painfully evident to me when I try and do something in areas where it really is weak and I honestly doubt that the foundation model creators presently know how to improve it
My personal evidence for this is that after several years of tilting those windmills, I'm successfully creating things that I have on and off spent the last decade trying to create successfully and have had difficulty with not because I couldn't do it, but because the cost of change and iteration was so high that after trying a few things and failing, I invariably move to simplifying the problem because solving it is too expensive, I'm now solving a category of those problems now, this for me is different and I really feel it because that sting of persistent failure and dread of trying is absent now
That's my personal perspective on it, sorry it's so anecdotal :)
>The value I've personally been getting which I've been valuing is that it improves my productivity in the specific areas where it's average quality of response as one shot output is better than what I would do myself because it is equivalent to me Googling an answer, reading 2 to 20 posts, consolidating that information together and synthesising an output
>And that's not to say that the output is good, that's to say that the cost of trying things as a result is much cheaper
But there's a hidden cost here -- by not doing the reading and reasoning out the result, you have learned nothing and your value has not increased. Perhaps you extended a bit less energy producing this output, but you've taken one more step down the road to atrophy.
Seeing the code that the LLM generates and occasionally asking it to explain has been an effective way to improve my understanding. It's better in some ways than reading documentation or doing tutorials because I'm working on a practical project I'm highly motivated by.
I agree that there is benefit in doing research and reasoning, but in my experience skill acquisition through supervising an LLM has been more efficient because my learning is more focused. The LLM is a weird meld of domain expert/sycophant/scatterbrain but the explanations it gives about the code that it generates are quite educational.
I think there's a potential unstated assumption here, though forgive me if it was made explicit elsewhere and/or I missed it.
LLM-assisted can be with or without code review. The original meaning of "vibe coding" was without, and I absolutely totally agree this rapidly leads to a massive pile of technical debt, having tried this with some left-over credit on a free trial specifically to see what the result would be. Sure, it works, but it's a hell of a mess that will make future development fragile (unless the LLMs improve much faster than I'm expecting) for no good reason.
Before doing that, I used Claude Code the other way, with me doing code reviews to make sure it was still aligned with my ideas of best practices. I'm not going to claim it was perfect, because it did a python backend and web front end for a webcam in one case and simultaneously on a second project a browser-based game engine and example game for that engine and on a third simultaneous project a joke programming language, and I'm not a "real" python dev or "real" web dev or any kind of compiler engineer (last time I touched Yacc before this joke language was 20 years earlier at university). But it produced code I was satisfied I could follow, understand, wasn't terrible, had tests.
I wouldn't let a junior commit blindly without code review and tests because I know what junior code looks like from all the times I've worked with juniors (or gone back to 20 year old projects of my own), but even if I was happy to blindly accept a junior's code, or even if the LLM was senior-quality or lead quality, the reason you're giving here means code review before acceptance is helpful for professional development even when all the devs are at the top of their games.
Yes, but I'm talking about more then code review -- there is a ton of value in discovering all of the ways not to solve a problem. When reading 25 forum posts or whatever in trying to write some function, you're learning more then just the answer. You're picking up a ton of context about how these sorts of problems are solved. If all you're doing is reviewing the output of some code generator, your mental context is not growing in the same way.
I'm curious if you think the same thing was lost with the transition from reading man pages and first-party documentation to going to stackoverflow or google first (at least, I assume the former was more common a couple decades ago)
What was lost in that transition was the required quality of that first-party documentation decreased; generally that first party documentation simply didn't contain enough information, so you needed to determine things empirically or read source code to get more information. I do think the culture of "copy-and-paste from stackoverflow" harmed the general competency of programmers, but having more third-party information available was only a positive thing.
Merely choosing lines to copy and paste from one file of your own code to another is a learning experience for your brain. AI is excellent for removing a lot of grunt work, but that type of work also reinforces your brain even if you think you are learning nothing. Something can still be lost even if AI is merely providing templates or scaffolding. The same can be said of using Google to find examples, though. You should try to come up with the correct function name or parameter list yourself in your head before using a search engine or AI. And that is for the moist simple examples, e.g. "SQL table creation example". These should be things we know off the top of our heads, so we should first try to type it out before we go to look for an answer.
I suppose the way I approach this is, I use libraries which solve problems that I have, that in principle understand, because I know and understand the theory, but in practice I don't know the specific details, because I've not implemented the solution myself
And honestly, it's not my job to solve everything, I've just got to build something useful or that serves my goals
I basically put LLM's into that category, I'm not much of a NIH kinda person, I'm happy to use libraries, including alpha ones on projects if they've been vetted over the range of inputs that I care about, and I'm not going to go into how to do that here, because honestly it's not that exciting, but there's very standard boring ways to produce good guarantees about it's behaviour, so as long as I've done that, I'm pretty happy
So I suppose what I'm saying is that isn't a hidden cost to me, it's a pragmatic decision I made that I was happy with the trade off :)
When I want to learn, and believe me I do now and again, I'll focus on that there :)
No I agree with you, there are area's where AI is helping amazingly. Every now and then it helps me with some issue as well, which would have cost me hours earlier and now it's done in minutes. E.g. some framework that I'm not that familiar with, or doing the scaffolding for some unit test.
However this is only a small portion of my daily dev work. For most of my work, AI helps me little or not at all. E.g. adding a new feature to a large codebase: forget it. Debugging some production issue: maybe it helps me a little bit to find some code, but that's about it.
And this is what my post was referring to: not that AI doesn't help at all, but to the crazy claims (10x speedup in daily work) that you see all over social media.
Example for me: I am primarily a web dev today. I needed some kuberntes stuff setup. Usually that’s 4 hours of google and guess and check. Claude did it better in 15 minutes.
Even if all it does is speed up the stuff i suck at, that’s plenty. Oh boy docker builds, saves my bacon there too.
And you learned nothing and have no clue if what it spit out is good or not.
How can you even assume what it did is "better" if you have no knowledge of kubernetes in the first place? It's mere hope.
Sure it gets you somewhere but you learned nothing in the way and now depend on the LLM to maintain it forever given you don't want to learn the skill.
I use LLMs to help verify my work and it can sometimes spot something I missed (more often it doesn't but it's at least something). I also automate some boring stuff like creating more variations of some tests, but even then I almost always have to read its output line by line to make sure the tests aren't completely bogus. Thinking about it now it's likely better if I just ask for what scenarios could be missing, because when they write it, they screw it up in subtle ways.
It does save me some time in certain tasks like writing some Ansible, but I have to know/understand Ansible to be confident in any of it.
These "speedups" are mostly short term gains in sacrifice for long term gains. Maybe you don't care about the long term and that's fine. But if you do, you'll regret it sooner or later.
My theory is that AI is so popular because mediocrity is good enough to make money. You see the kind of crap that's built these days (even before LLMs) and it's mostly shit anyways, so whether it's shit built by people or machines, who cares, right?
Unfortunately I do, and I rather we improve the world we live in instead of making it worse for a quick buck.
IDK how or why learning and growing became so unpopular.
> Sure it gets you somewhere but you learned nothing in the way and now depend on the LLM to maintain it forever given you don't want to learn the skill.
The kind of person who would vibe code a bunch of stuff and push it with zero understanding of what it does or how it does it is the kind of person who’s going to ruin the project with garbage and technical debt anyway.
Using an LLM doesn’t mean you shouldn’t look at the results it produces. You should still check it results. You should correct it when it doesn’t meet your standards. You still need to understand it well enough to say “that seems right”. This isn’t about LLMs. This is just about basic care for quality.
But also, I personally don’t care about being an expert at every single thing. I think that is an unachievable dream, and a poor use of individual time and effort. I also pay people to do stuff like maintenance on my car and installing HVAC systems. I want things done well. That doesn’t mean I have to do them or even necessarily be an expert in them.
I think it is more accurate to say some skills are declining (or not developing) while a different set of skills are improving (the skill of getting an LLM to produce functional output).
Similar to if someone started writing a lot of C, their assembly coding skills may decline (or at least not develop). I think all higher levels of abstraction will create this effect.
I agree with both of your points since I use LLMs for things I am not good at and dont give a single poop about. The only things i did with LLMs are three examples from the last two years:
- Some "temporary" tool I built years ago as a pareto-style workaround broke. (As temporary tools do after some years). Its basically a wrapper that calls a bunch of XSLs on a bmecat.xml every 3-6 months. I did not care to learn XSL back then and I dont care to do it now. Its arcane and non-universal - some stuff only works with certain XSL processors. I asked the LLM to fix stuff 20 times and eventually it got it. Probably got that stuff off my back another couple years.
- Some third party tool we use has a timer feature that has a bug where it sets a cookie everytime you see the timer once per timer (for whatever reason... the timers are set to end a certain time and there is no reason to attach it to a user). The cookies have a life time of one year. We run time limited promotions twice a week so that means two cookies a week for no reason. Eventually our WAF got triggered because it has a rule to block requests when headers are crazy long - which they were because cookies. I asked an LLM to give me a script that clears the cookie when its older than 7 days because I remember the last time i hacked together cookie stuff it also felt very "wtf" in a javascript kinda way and I did not care to relive that pain. This was in place until the third party tool fixed the cookie lifetime for some weeks.
- We list products on a marketplace. The marketplace has their own category system. We have our own category system. Frankly theirs kinda suck for our use case because it lumps a lot of stuff together, but we needed to "translate" the categories anyway. So I exported all unique "breadcrumbs" we have and gave that + the categories from the marketplace to an LLM one by one by looping through the list. I then had an apprentice from another dept. that has vastly more product knowledge than me look over that list in a day. Alternative would have been to have said apprentice do that stuff by hand, which is a task I would have personally HATED so I tried to lessen the burden for them.
All these examples are free tier in whatever I used.
We also use a vector search at work. 300,000 Products with weekly updates of the vector db.
We pay 250€ / mo for all of the qdrant instances across all environments and like 5-10 € in openai tokens. And we can easily switch whatever embedding model we use at anytime. We can even selfhost a model.
> I'm going to try and be honest with you because I'm where you were at 3 months ago
> I honestly don't think there's anything I can say to convince you
> The value I've personally been getting which I've been valuing
> And that's not to say that the output is good
> My personal evidence for this is that after several years of tilting those windmills
It sounds to me like you're rationalizing and your opening sentences embed your awareness of the fallibility of what you say and clearly believe about your situation later.
I feel there are two types of programmers who use AI:
Type A who aren't very good but AI makes them feel better about themselves.
Type B who are good with or without AI and probably slightly better with it but at a productivity cost due to fixing AI all the way through, rather than a boost; leading to their somewhat negative but valid view of AI.
It's great when the terrain is unfamiliar to the user but extremely familiar to the LLM. And it's useless in the opposite.
The best programmers are going to be extremely familiar with terrains that are unfamiliar to the LLMs, which is why their views are so negative. These are people working on core parts of complex high performing highly scalable systems, and people with extreme appreciation for the craft of programming and code quality.
But the most productive developers focused on higher level user value and functionality (e.g pumping out full stack apps or features), are more likely to be working with commonly used technologies while also jumping around between technologies as a means to a functionality or UX objective rather than an end of skill development, elegant code, or satisfying curiosity.
I think this explains a lot of the difference in perspectives. LLMs offer value in the latter but not the former.
It's a shame that so many of the people in one context can't empathize with the people in the other.
I think people get into a dopamine hit loop with agents and are so high on dopamine because its giving them output that simulates progress that they don't see reality about where they are at. It is SO DAMN GOOD AT OUTPUT. Agents love to output, it is very easy to think its inventing physics.
Ironic that I’m going to give another anecdotal experience here, but I’ve noticed this myself too. I catch myself trying to keep on prompting after an llm has not been able to solve some problem in a specific way. While I can probably do it faster at that point if I switch to doing it fully myself. Maybe because the llm output feels like its ‘almost there’, or some sunken cost fallacy.
Not saying this is you, but another way to look at it is that engaging in that process is training you (again, not you, the user) -- the way you get results is by asking the chat bot, so that's what you try first. You don't need sunk cost or gambling mechanics, it's just simple conditioning.
I also think that's the case, but I'm open to the idea that there are people that are really really good at this and maybe they are indeed 10x.
My experience is that for SOME tasks LLMs help a lot, but overall nowhere near 10x.
Consistently it's probably.... ~1X.
The difference is I procrastinate a lot and LLMs actually help me not procrastinate BECAUSE of that dopamine kick and I'm confident I will figure it out with an LLM.
I'm sure there are many people who got to a conclusion on their to-do projects with the help of LLMs and without them, because of procrastination or whatever, they would not have had a chance to.
It doesn't mean they're now rich, because most projects won't make you rich or make you any money regardless if you finish them or not
You nailed it - like posting on social media and getting dopamine hits as you get likes and comments.
Maybe that's what has got all these vibe coders hooked.
> What I then saw was him struggling for one hour with some simple extension to his project. He didn't manage to finish in the hour what he was planning to. And when I had some thought about how much time it would have cost me by hand, I found it would have taken me just as long.
For all who are doing that, what is the experience of coding in a livestream? It is something I never attempted, the simple idea makes me feel uncomfortable. A good portion of my coding would be rather cringe, like spending way too long on a stupid copy-paste or sign error that my audience would have noticed right away. On the other hand, sometimes, I am really fast because everything is in my head, but then I would probably lose everyone. I am impressed when looking at live coders by how fluid it looks compared to my own work, maybe there is a rubber duck effect at work here.
All this to say that I don't know how working solo compares to a livestream. It is more or less efficient, maybe it doesn't matter that much when you get used to it.
Have done it, never enough of an audience to be totally humiliated.
It's never going to be more efficient.
But as for your cringe issue that the audience noticed, one could see that to be a benefit -- prefer to have someone say e.g. "you typed `Normalise` (with an 's') again, C++ is written in U.S. English, don't you know / learn to spell, you slime" upfront than waiting for compiler to tell you that `Normalise` doesn't exist, maybe?
I suspect livestream coding, like programming competition coding and whiteboard coding for interviews, is a separate skill that's fairly well correlated with being able to solve useful problems, but it is not the same thing. You can be an excellent problem solver without being good at doing so while being watched, under time pressure.
I feel like I've been incredibly productive with AI assisted programming over the past few weeks, but it's hard to know what folks' baselines are. So in the interest of transparency, I pushed it all up to sourcehut and added Co-Authored-By footers to the AI-assisted commits (almost all of them).
Everything is out there to inspect, including the facts that I:
- was going 12-18 hours per day
- stayed up way too late some nights
- churned a lot (+91,034 -39,257 lines)
- made a lot of code (30,637 code lines, 11,072 comment lines, plus 4,997 lines of markdown)
- ended up with (IMO) pretty good quality Ruby (and unknown quality Rust).
I don't really know Ruby, so maybe I'm missing something major, but your commit messages seem extremely verbose yet messy (I can't make heads or tails of them) and I'm seeing language like "deprecated" and a stream of "releases" within a period of hours and it just looks a bit like nonsense.
Don't take "nonsense" negatively, please -- I mean it looks like you were having fun, which is certainly to be encouraged.
Copy-pasting the code would have been faster than their work, and there were several problems with their results. But they were so convinced that their work is quick and flawless, that they post a video recording of it.
> LLM marketers have succeeded at inducing collective delusion
That's the real trick & one I desperately wish I knew how to copy.
I know there's a connection to Dunning Kruger & I know that there's a dopamine effect of having a responsive artificial minion & there seems to be some of that "secret knowledge" sauce that makes cults & conspiracies so popular (there's also the promise of less effort for the same or greater productivity).
Add the list grows, I see the popularity, but I doubt I could easily apply all these qualities to anything else.
IMO algorithmically generated "social" media feeds combined with the lack of adequate mass-media alternatives have supercharged cult recruitment in the last approximately 10 years.
Stupid people in my life have been continually and recklessly joining harebrained cults for the last 5 years.
Really I think it's probably much, much easier to start a cult these days than it has ever been. Good news for tech company founders I guess, bad news for American culture, American society, and the American people.
One way to help stop it is to get off social media and stop giving these tech billionaires so much money.
The less people on social media, the less real the network effect is, the less people who join in the first place, the less money the billionaires have to throw hundreds of millions into politics, the less inadvertent cult members.
I've gotten to the point where I just leave my phone at home at this point, and it has been incredibly nice. Before that I deleted most apps that I found to be time wastes, deleted all social media (HN and two small discords are my exception).
It's very nice, I'm less stressed, I feel more in the moment, I respond to my friends when I check my phone every few hours on the speaker in the other room.
I encourage others to try it, add it to your dry January.
And ya know what I ain't doing a lick of? Sending money and reams of data to these billionaires I think are really lame individuals with corrupted moral compasses.
Now it ain't perfect, I'm sure Google's still getting reams of info about me from my old Gmail account that I still use sometimes, and Apple too from a few sources. But... getting closer!
So many folk sit here and recognize the same problems I do, the way it warps your attention, the addictiveness of the handheld devices, the social media echo chambers, the rising influence of misinformation, the lack of clarity between real and fake...
> So I answered him in his LinkedIn thread and asked where the 10x speed up was. What followed was complete denial. It had just been a hick up. Or he could have done other things in parallel while waiting 30 seconds for the AI to answer. Etc etc.
So I’ve been playing with LLMs for coding recently, and my experience is that for some things, they are drastically faster. And for some other things, they will just never solve the problem.
Yesterday I had an LLM code up a new feature with comprehensive tests. It wasn’t an extremely complicated feature. It would’ve taken me a day with coding and testing. The LLM did the job in maybe 10 minutes. And then I spent another 45 minutes or so deeply reviewing it, getting it to tweak a few things, update some test comments, etc. So about an hour total. Not quite a 10x speed up, but very significant.
But then I had to integrate this change into another repository to ensure it worked for the real world use case and that ended up being a mess, mostly because I am not an expert in the package management and I was trying to subvert it to use an unpublished package. Debugging this took the better part of the day. For this case, the LLM may be saved me maybe 20% because it did have a couple of tricks that I didn’t know about. But it was certainly not a massive speed up.
So far, I am skeptical that LLM’s will make someone 10x as efficient overall. But that’s largely because not everything is actually coding. Subverting the package management system to do what I want isn’t really coding. Participating in design meetings and writing specs and sending emails and dealing with red tape and approvals is definitely not coding.
But for the actual coding specifically, I wouldn’t be surprised if lots of people are seeing close to 10x for a bunch of their work.
I've noticed a similar trend. There seems to be a lot of babysitting and hand holding involved with vibe-coding. Maybe it can be a game changer for "non-technical founders" stumbling their way through to a product, but if you're capable of writing the code yourself, vibe coding seems like a lot of wasted energy.
Shopify's CEO just posted the other day that he's super productive using the newest AI models and many of the supportive comments responding to his claim were from CEOs of AI startups.
I think there is also some FOMO involved. Once people started saying how AI was helping them be more productive, a lot of folks felt that if they didn't do the same, they were lagging behind.
This mentioning of 'Wall Street' with investors if typical for the kind of populist argument that is used to argue that banning investors from buying houses is a good thing.
What would this 'Wall Street' even mean? Would it mean that companies listed on the stock exchange are banned but privately owned companies not?
Anyhow, I argue that investors are positive for the the house market. They shouldn't be banned. Investors provide enough liquidity to the market so that the building companies have enough certainty to invest in large housing projects, because they know that their properties will be sold quickly.
If investors would be banned they would sell their houses eventually as well but it would take much longer.
Similarly, investors improve mobility and throughput. An individual putting his house for sale will find a buyer much faster when investors are in the buying market, who are willing to buy up a house when nobody else takes it and sell it for a better price later.
So: sellers sell faster, so they can move out and buy a new home faster as well: mobility in the house market increases.
> it's holding prices steady at some point without the concurrent pressure to sell
Earlier you were arguing that investors were acting as marketmakers and now you say this. Marketmakers make their profit from the difference between buying and selling some asset. They don't want to hold prices, they want turnover.
If investors really are acting as marketmakers it's actually a good thing because marketmakers have the effect of adding liquidity to a market.
This works great when you're on the receiving end of e-mails/ messages.
However it is a pain to deal with when you are on the sending side and your issue is urgent.
I see the author fighting really hard against a paradigm that was already well known to be suboptimal 20 years ago ..
"Favor composition over inheritance" is a phrase that became common knowledge in e.g. the Java world half way during the 2000's.
It was even already mentioned in the famous Design Patterns book from the GOF in 1994.
Can't help but wonder if that isn't more the fault of Java's OOP design. Favoring composition over inheritance doesn't seem like something Smalltalkers and Rubyists had to worry as much about.
The GoF book predates Java, and many of the examples in the book are in Smalltalk - so I think it's quite likely that composition over inheritance came from the Smalltalk community.
Problems with inheritance, such as the fragile base case problem, are intrinsic to inheritance, and not any specific language's implementation of it.
Ruby does have language features for composition, and “composition over inheritance” has popped up in the community. I have also looked into the use of traits in Smalltalk.
Your post is absolutely golden and it doesn't apply only at Google.
I can't remember seeing so many deeply true statements together in one post.
I'm sure many here will start to contest some of them but with enough experience they will also realize that the points were true.
> Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
It's worse than that. It might not be you who has to debug it, but someone else. Maybe after you left the company already. Maybe at 3AM after a pager alert in production ..
The correlation seems to point to usage ground water that is contaminated with pesticides. So people living close to the golf courses have higher Parkinson risk.
Probably golfers and employees less so.
> I've had Claude Code write an entire unit/integration test suite in a few hours (300+ tests) for a fairly complex internal tool.
And what do you have then? 300 tests that test the behavior that's exposed by the implementations of the api. Are they useful? Probably some are, probably some are not. The ones that are not will just be clutter and maintenance overhead.
Plus, there will be lots of use-cases for which you need to look a little deeper than just the api implementation, which are now not covered. And those kind of tests, tests that test real business use cases, are by far the most useful ones if you want to catch regressions.
So if your goal is to display some nice test coverage metrics on SonarQube or whatever, making your CTO happy, yes AI will help you enormously. But if your goal is to speed up development of useful test cases, less so. You will still gain from AI, but nowhere near 90%.
The difference with the zillion others who did the same, is that he attached a link to a live stream where he was going to show his 10x speedup on a real life problem. Credits to him for doing that! So I decided to go have a look.
What I then saw was him struggling for one hour with some simple extension to his project. He didn't manage to finish in the hour what he was planning to. And when I had some thought about how much time it would have cost me by hand, I found it would have taken me just as long.
So I answered him in his LinkedIn thread and asked where the 10x speed up was. What followed was complete denial. It had just been a hick up. Or he could have done other things in parallel while waiting 30 seconds for the AI to answer. Etc etc.
I admit I was sceptic at the start but I honestly had been hoping that my scepticism would be proven wrong. But not.
reply