I just feel so discouraged reading this somehow. I used to have this hard-to-get, in-demand skill that paid lots of money and felt like even though programming languages, libraries and web frameworks were always evolving I could always keep up because I'm smart. But now with these people like Simon Willison writing about the new way of coding with these agents and multiple streams of work going on at a time and it sounding like this is the future, I just feel discouraged because it sounds like so much work and I've tried using coding agents and they help a bit, but I find it way less fun to be waiting around for agents to do stuff and it's way harder to get into flow state managing multiple of these things. It makes me want to move into something completely different like sales
I'm really sorry to hear this, because part of my goal here is to help push back against the idea that "programming skills are useless now, anyone can get an LLM to write code for them".
I think existing software development skills get a whole lot more valuable with the addition of coding agents. You can take everything you've learned up to this point and accelerate the impact you can have with this new family of tools.
I said a version of this in the post:
> AI tools amplify existing expertise. The more skills and experience you have as a software engineer the faster and better the results you can get from working with LLMs and coding agents.
A brand new vibe coder may be able to get a cool UI out of ChatGPT, but they're not going to be able to rig up a set of automated tests with continuous integration and continuous deployment to a Kubernetes cluster somewhere. They're also not going to be able to direct three different agents at once in different areas of a large project that they've designed the architecture for.
I'm not sure that having the patience to work with something with a very inconsistent performance and that frequently lies is an extension of existing development skills. It doesn't work like tools developers use and it doesn't work like people developers work with. Furthermore, techniques of working with agents today may be completely outdated a year from now. The acceleration is also inconsistent: sometimes there's an acceleration, sometimes a deceleration.
Generative AI is at the same time incredibly impressive and completely unreliable. This makes it interesting, but also very uncertain. Maybe it's worth my investment to learn how to master today's agents, and maybe I'd be better off waiting until these things become better.
You wrote:
> Getting good results out of a coding agent feels uncomfortably close to getting good results out of a human collaborator. You need to provide clear instructions, ensure they have the necessary context and provide actionable feedback on what they produce.
That is true (about people) but misses out the most important thing for me: it's not about the information I give them, but about the information they give me. For good results, regardless of their skill level, I need to absolutely trust that they tell me what challenges they've run into and what new knowledge they've gained that I may have missed in my own understanding of the problem. If that doesn't happen, I won't get good results. If that kind of communication only reliably happens through code I have to read, it becomes inefficient. If I can't trust an agent to tell me what I need to know (and what I trust when working with people) then the whole experience breaks down.
> I'm not sure that having the patience to work with something with a very inconsistent performance and that frequently lies is an extension of existing development skills.
If you’ve be been tasked with leadership of an engineering effort involving multiple engineers and stakeholders you know that this is in fact a crucial part of the role the more senior you get. It is much the same with people: know their limitations, show them a path to success, help them overcome their limitations by laying down the right abstractions and giving them the right coaching, make it easier to do the right thing. Most of the same approaches apply. When we do these things with people it’s called leadership or management. With agents, it’s context engineering.
Because I reached that position 15 years ago, I can tell you that this is untrue (in the sense that the experience is completely different from an LLM).
Training is one thing, but training doesn't increase the productivity of the trainer; it's meant to improve the capability of the trainee.
At any level of capability, though - whether we're talking about an intern after one year of university or a senior developer with 20 years of experience - effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know. We may not be talking 100% of trust, but not too far from that, either. You can't continue working with someone that doesn't tell you what you need to know even 10% of the time, regardless of their level. LLMs are not at that acceptable level yet, so the experience is not similar to technical leadership.
If you've ever been tasked with leading one or more significant projects you'd know that if you feel you have to review every line of code anyone on the team writes, at every step of the process, that's not the path to success (if you did that, not only would progress be slow, but your team wouldn't like you very much). Code review is a very important part of the process, but it's not an efficient mechanism for day-to-day communication.
> effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know
Nope, effective management is on YOU, not them. If everyone you’re managing is completely transparent and immediately tells you stuff, you’re playing in easy mode
no, the point is LLMs will behave the same way humans you have to manage do (there's obviously differences - eg LLMs tend to forget context more often than most humans, but also they tend to know a lot more than the average human). So some of the same skills that'll help you manage humans will also help you get more consistency out of LLMs.
I don't know of anyone who would like to work with someone who lies to them over and over, and will never stop. LLMs do certain things better than people, but my point is that there's nothing you can trust them to do. That's fine for research (we don't trust, and don't need to trust, any human or tool to do a fully exhaustive research, anyway), but not for most other work tasks. That's not to say that LLMs can't be utilised usefully, but something that can never be trusted behaves like neither person nor tool.
The person you are responding to is quite literally making the same point. This entire thread of conversation is in response to the post's author stating that using a coding agent is strongly akin to collaborating with a colleague.
> Yes, I want to play in easy mode. Why would I want to play in hard mode?
Working alone can be much easier than managing others in a team. But also, working in a team can be far more effective if you can figure out how to pull it off.
It's much the same as working with agents. Working alone, without the agents, it's easier to make exactly what you want happen. But working with agents, you can get a lot more done a lot faster-- if you can figure out how to make it happen. This is why you might want hard mode.
The vast majority of managers, much like most engineers, only has to deal with “maintenance mode” throughout most of their career. Particularly common in people whose experience has been in large corporations - you simply don’t realize how much was built for you and “works” (even if badly)
> effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know
This is what we shoot for, yes, but many of the most interesting war stories involve times when people should have been telling you about snags but weren't-- either because they didn't realize they were spinning their wheels, or because they were hoping they'd somehow magically pull off the win before the due date, or innumerable other variations on the theme. People are most definitely not reliable about telling you things they should have told you.
> if you feel you have to review every line of code anyone on the team writes...
Somebody has to review the code, and step back and think about it. Not necessarily the manager, but someone does.
> the most interesting war stories involve times when people should have been telling you about snags but weren't-
This comes up a lot. A person sometimes does an undesirable thing that an AI also does. So you might as well use the AI.
But we don't apply this thinking to people. If a person does something undesirable sometimes then we accept that because they are human. If they do it very frequently then at some point, given a choice, you will stop working with that person.
1000% this. Today LLMs are like enthusiastic, energetic, over-confident, well-read junior engineers.
Does it take effort to work with them and get them to be effective in your code base? Yes. But is there a way to lead them in such a way that your "team" (you in this case) gets more done? Yes.
But it does take effort. That's why I love "vibe engineering" as a term because the engineering (or "senior" or "lead" engineering) is STILL what we are doing.
Inconsistent performance and frequent lies are a crucial part of the role, really? I've only met a couple of people like that on my career. Interviews go both ways: if I can't establish that the team I'll be working with is composed and managed by honest and competent people, I don't accept their offer. Sometimes it has meant missing out on the highest compensation, but at least I don't deal with lies and inconsistent performance.
Test-driven development helps protect against wrong code, but it's not code I'm interested in, and it's not wrong code that I'm afraid of (I mean, that's table stakes). What I need is something that would help me generate understanding and do so reliably (even if the performance is poor). I can't exercise high-level knowledge efficiently if my only reliable input is code. Once you have to work at the code level at every step, there's no raising of the level of thought. The problem for me isn't that the agent might generate code that doesn't pass the test suite, but that it cannot reliably tell me what I need to know about the code. There's nothing I can reliably offload to the machine other than typing. That could still be useful, but it's not necessarily a game-changer.
Writing code in Java or Python as opposed to Assembly also raises the level of abstract thought. Not as much as we hope AI will be able to do someday, but at least it does the job reliably enough. Imagine how useful Java or Python would be if 10% of the time they would emit the wrong machine instructions. If there's no trust on anything, then the offloading of effort is drastically diminished.
In my experience with Claude Code and Sonnet, it is absolutely possible to have architectural and design-oriented conversations about the work, at an entirely different and higher level than using a (formerly) high-level programming language. I have been able to learn new systems and frameworks far faster with Claude than with any previous system I have used. It definitely does require close attention to detect mistakes it does not realize it is making, but that is where the skill comes in. I find it being right 80% of the time and wrong 20% of the time to be a hugely acceptable tradeoff, when it allows me to go radically faster because it can do that 80% much quicker than I could. Especially when it comes to learning new code bases and exploring new repos I have cloned -- it can read code superhumanly quickly and explain it to me in depth.
It is certainly a hugely different style of interaction, but it helps to think of it as a conversation, or more precisely, a series of individual small targeted specific conversations, each aimed at researching a specific issue or solving a specific problem.
Indeed, I successfully use LLMs for research, and they're an improvement because old-school search isn't very reliable either.
But as to the 80-20 tradeoff on other tasks, the problem isn't that the tool is wrong 20% of the time, but that it's not trustworthy 100% of the time. I have to check the work. Maybe that's still valuable, but just how valuable that is depends on many factors, some of which are very domain-dependent and others are completely subjective. We're talking about replacing one style with another that is much better in some respects and much worse in others. If, on the whole, it was better in almost all cases, that would be one thing (and make the investment safer), but reports suggest it isn't.
I've yet to try an LLM to learn a new codebase, and I have no doubt it will help a lot, but while that is undoubtedly a very expensive task, it's also not a very frequent one. It could maybe save me a week per year, amortised. That's not nothing (and I will certainly give it a try next time I need to learn a new codebase), but it's also not a game-changer.
Without meaning to sound flippant or dismissive, I think you're overthinking it. By the sounds of it, agents aren't offering what you say you need. What are _are_ offering is the boilerplate, the research, the planning etc. All the stuff that's ancillary. You could quite fairly say that it's in the pursuit of this stuff where details and ideas emerge and I would agree, but sometimes you don't need ideas. You need solutions which are run-of-the-mill and boring.
I'm well aware that LLMs are more than capable enough to successfully perform straightforward, boring tasks 90% of the time. The problem is that there's a small but significant enough portion of time where I think a problem is simple and straightforward, but it turns out not to be once you get into the weeds, and if I can't trust the tool to tell me if we're in the 90% problem or the 10% problem, then I have to carefully review everything.
I'm used to working with tools, such as SMT solvers, that may fail to perform a task, but they don't lie about their success or failure. Automation that doesn't either succeed or report a failure reliably is not really automation.
Again, I'm not saying that the work done by the LLM is useless, but the tradeoffs it requires make it dramatically different from how both tools and humans usually operate.
If you're writing your own tests, sure, AI is fast at writing code that passes the tests.
But if you write a comprehensive test suite for a problem, you've effectively done the hard development work to solve the problem in the first place. How did the AI help?
Oh have the AI write unit tests you say? Claude cheats constantly at the tests ime. It frequently tests the mock instead of the UUT and reports a pass. That's worse than useless! I'm sure a huge swath of slop unit tests that all pass is acceptable quality for a lot of businesses out there.
> But if you write a comprehensive test suite for a problem, you've effectively done the hard development work to solve the problem in the first place. How did the AI help?
By making you not write the implementation?
Also, the AI writing anything bad isn’t an excuse. You’re the one piloting that ship, and if not, you’re probably the one reviewing the code. It’s your job to review your own and others’ code with a critical eye, and that goes double in the LLM age.
> It doesn't work like tools developers use and it doesn't work like people developers work with. Furthermore, techniques of working with agents today may be completely outdated a year from now.
> I'm not sure that having the patience to work with something with a very inconsistent performance and that frequently lies is an extension of existing development skills.
that's a basic skill you gotta have if you're leading anything or anyone. There'll always be levels of that. So if you're planning to lead anyone in your career, it's a good skillset to develop
While this is true, I definitely find that the style of the work changes a lot. It becomes much more managerial, and less technical. I feel much more like a mix of project and people manager, but without the people. I feel like the jury is still out on whether I’m overall more productive, but I do feel like I have less fun.
3. Tons of excess code I'd never put in there in the first place.
4. Frustration with agents being too optimistic which with time verges on the ludicurous ("Task #3 has been completed successfully with 98% tests failing. [:useless_emojis:]")
5. Frustration with agents routinely getting down a rabbit hole or going in circles, the effort needed to get that straight (Anthropic plainly advises to start from scratch in such cases - which is sound advice, but makes me feel like I just lost the last 5 hours of my life without even learning anything new).
I stopped using agents and use LLMs very sparingly (e.g. for review - they sometimes find some details I missed and occasionally have an interesting solution) but I'm enjoying my work so much more without them.
I think one of the tricks is to just stop using the agent as soon as you see signs of funny business. If it starts BSing me with failing tests, I just turn it off immediately and git reset (maybe after taking a quick peek)
Yeah I make maybe two or three attempts at getting it to write a plan that it is able to follow coherently. But after that I pull the escape hatch and *gasp* program by hand.
I've made this mistake of doubling down after a few initial failures to solve an issue, by trying to make this super duper comprehensive and highly detailed and awesome plan that it will finally be able to implement correctly. But it just gets worse and worse the more I try, because it fundamentally is not understanding what is going on, so it will inevitably find an opportunity to go massively off rails, and the further down you lead it the more impressible the derailment will be.
My experience is that going around in endless circles with the model is just a waste of time when you could have just done it yourself in the time you've wasted.
One thing I don’t get - If you spend much of your time reviewing, you’re just reading - you’re not actually doing anything - you’re passive in the activity of code production. By extension you will become worse at knowing what a good standard of code is and become worse at reviewing code.
I’m not a SWE so I have no interests to protect by criticising what is going on.
In my DJing years I've learned that it is best to provide a hot signal and trim the volume than trying to amplify it later, because you end up amplifying noise. Max out the mixer volume and put a compressor (and a limiter after to protect the speaker set up - it will make it sound awful if hit, but it won't damage your set up and it will flag clueless bozos loud and clear) later, don't try to raise it after it leaves the origin.
It seems to me that adding noise to the process and trying to cut it out later is a self defeating proposition. Or as Deming put it, (paraphrasing) you can't QC quality into a bad process.
I can see how it seems better to "move fast and break things" but I will live and die by the opposite "move slow and fix things". There's much, much more to life than maximizing short term returns over a one dimensional naïve utilitarian take on value.
I'd be careful with extrapolating based on the creator of Linux and Git. His life and activities are not in line with those of more typical programmers.
> His life and activities are not in line with those of more typical programmers.
Okay sure.
I'll use myself as another example then. When I was a dev I used to write a lot of code. Now I'm a tech team lead, and I write less code, but review significantly more code than I used to previously.
I feel more confident, comfortable, and competent in my coding abilities now than ever before even though I'm coding less.
I feel like this is because I am exposed to a lot more code, and not in a passive way (reading legacy code) but an active way (making sure a patch set will correctly implement feature X, without breaking anything existing)
I feel like this principal applies to any programmer. Same thing with e.g. writers. Good writers read _a lot_ and it makes them better writers.
This is my opinion and not based on any kind of research. So if you disagree, that's fine with me. But so far I haven't seen anything to convince me of the opposite.
It's not only that Linus is atypical, it's also that he is reviewing other people's code, and those people are also highly competent, or they would not be kernel committers. And they all share large amounts of high-quality and hard-earned implicit context.
Reviewing well executed changesets by skilled developers on complicated and deliberate projects is not comparable to "fleet of agents" vibe engineering. One of these tasks will sharpen you regardless how lazily you approach it. The other requires extreme discipline to avoid atrophy.
I reset context probably every 5-10 minutes if not more frequently, and commit even more often than that. If you’re going 5 hours between commits or context resets, I’m not surprised you’re getting bad results. If you ever see “summarizing”’in copilot for example, that means you went way too far in that context window. The LLMs get increasingly inaccurate and confused as the context window fills up.
Other things like having it pull webpages in, will totally blow away your context. It’s better to make a separate context just to pull a webpage down and summarize it in markdown and then reset context.
The 'best' trick I learned from someone over here when working with Claude Code is to very regularly go back a few steps in your context (esc esc -> pick something a few steps up) and say something like "yeah, I already did this myself, now continue and do Y"
It results helps keep the context clean while still keeping the initial context I provided (usually with documentation and initial plan setup) at the core of the context.
Now that you say this, I did notice webpages blow context but didn't think too much of it just yet, maybe there's some improvement to be found here using a subagent? I'm not a big fan of subagents (didn't really get proper results out of them in my initial experiments anyway) but maybe adding a 'web researcher' sub agent that summarizes to a concise markdown file could help here.
Now that's dangerous to do because the conversation history in Claude Code now also reverts the code to that point. So while this technique may have worked in the past, it no longer works.
Regarding #3. I feel it's related to this idea: We can build a wood frame house with 2x4's or toothpicks. AI directed and generated code today tends to build things overly complex with more pieces than necessary. I feel like an angry foreman yelling at AI to fix this, change that, etc. I feel I spend more time and energy supervising AI while getting a sloppier end result.
Thankfully, yelling like an angry foreman is more effective on LLMs than people.
> Get your fucking act together, and stop with the bullshit comments, shipping unfinished code, and towering mess of abstractions. I've seen you code properly before. You're an expert for God's sake. One more mistake, and you're fired. Fix it, now!
I wouldn't talk that way to an LLM for fear of its bleeding over into my interactions with people.
Back when computer performance was increasing faster than it is now and was more important to the user experience, a friend upgraded to a faster computer and suddenly became more impatient with me. He seemed to have expected my response time to have drastically decreased just like his computer's did.
Effects like these, those of our tools on ourselves which occur slowly/subtly enough that we hardly notice, underlie a great many of our greatest problems, I think
Yeah exactly, it changes the job from programmer to (technical) project manager, which is both more proactive (writing specifications) and reactive (responding to an agent finishing). The 'sprinting' remark is apt, because if your agents are not working you need to act. And it's already established that a manager shouldn't micromanage, that'll lead to burnout and the like. But that's why software engineers will remain relevant, because managers need someone to rely on that can handle the nitty-gritty details of what they ask for.
I also think that managing a coding agent isnt like managing a person. a person is creative, they will come up with ways that challenge whatever idea you have and that usually makes the project better. A coding agent never challenges you, mostly just does whatever you want, and you don't end up having any kind of intellectual person to person engagement that is why working on teams can be fun. So it kind of isolates you. And I think the primary reason all this happens is because marketing people have decided to call all of these coding agents "Artificial Intelligence" instead of "Dev Tools". And instead of calling it "Security" they now call it "AI Alignment". And instead of calling it "data schema" or "Spec sheet" they call it "managing the AI context". So now, we are all biased to see these things as some kind of entity that we can treat something like a colleague and we all bought this idea because the tool can chat with you. But it isn't a colleague, it doesn't think and feel, it doesn't provide intellectual engagement, it simply is a lossy, noisy tool to try and translate human language into computer language whether its python or machine code.
Have you used SOTA models to code in the last 2 months or so? This reads like someone who has given up a year ago and made their impressions based off GPT-3.
AI can absolutely have creativity. You just have to engage it like that. The article itself talks about that. You don’t just say “hey AI, go write this code.” You write a spec along with the AI. You tell it what parts are open to its interpretation. Tell it if you want it to be creative or to follow common practices. What level of abstraction is appropriate, etc.
If all you do is give it directions then it just follows the directions.
Also context doesn’t have much to do with a data schema. It’s more like a document database with no schema, if anything. It’s a collection of tokens that it refers back to. Schema implies some structured data with semantic meaning and hierarchies or relationships. That might exist as an emergent property, but for example if I just had a folder full of PDFs, I wouldn’t consider that a schema. That’s kinda what context is like.
> They're also not going to be able to direct three different agents at once in different areas of a large project that they've designed the architecture for.
I wonder what the practical limits are.
As a senior dev on a greenfield solo project it's too exhausting for me to have two parallel agents (front/back), most of the time they're waiting for me to spec, review or do acceptance test. Feels like sprinting, not something I could do day in and day out.
Might be due to tasks being too fine grained, but assuming larger ones are proportionally longer to spec and review, I don't see more than two (or, okay, three, maybe I'm just slow) being a realistic scenario.
More than that, I think we're firmly in the vibe coding (or maybe spec-driven vibe coding) territory.
At least on a team, the limit is the team's time to review all the code. We've also found that vibe engineered (or "supervised vibing" as I call it) code tends to have more issues in code review because of a false sense of security creating blind spots when self reviewing. Even more burden on the team.
We're experimenting with code review prompts and sub agents. Seems local reviews are best, so the bulk of the burden is on the vibing engineer, rather than the team.
Do you have a sense for how much overhead this is all adding? Or, to put it another way, what I’m really asking is what productivity gain (or loss) are you seeing versus traditional engineering?
In our experience, it depends on the task and the language. In the case of trivial or boilerplate code, even if someone pushes 3k-4k lines of code in one day, it's manageable because you can just go through it. However, 3k lines of interconnected modules, complex interactions, and intricate logic require a lot of brainpower and time to review properly and in most cases, there are multiple bugs, edge cases that haven't been considered, and other issues scattered throughout the code.
And empirical studies on informal code review show that humans have a very small impact on error rates. It disappears when they read more than roughly 200 SLOC in an hour.
Interesting, do you have a link to the study? Our experience is different, at least when reviewing LLM generated code, we find quite a few errors, especially beyond 200 LOC. It also depends on what you're reviewing, 200 LOC != 200 LOC. A boilerplate 200 LOC change? A security sensitive 200 LOC change? A purely algorithmic and complex 200 LOC change?
Isn't the current state of thing such that it's really hard to tell? I think the METR study showed that self-reported productivity boosts aren't necessarily reliable.
I have been messing with vibe engineering on a solo project and I have such a hard time telling if there's an improvement. It's this feeling of "what's faster, one lead engineer coding or one lead engineer guiding 3 energetic but naive interns"?
The problem with this is that software engineering is a very unorganized and fashion/emotion driven domain.
We don't have reliable productivity numbers for basically... anything.
I <feel> that I'm more productive with statically typed languages but I haven't seen large scale, reliable studies. Same with unit tests, integration tests, etc.
And then there are all the types of software engineering: web frontend, web API, mobile frontend, command line frontend, Windows GUI, MacOS GUI, Linux backend (10 million different stacks), Windows backend (1 million different stacks), throwaway projects, WordPress webpages, etc, etc.
A controlled experiment done with a representative sample would be lovely. In the long-run it comes down to the financial impact that occurs incrementally because of LLMs.
In the short-run, from what I see, firms are trying to play-up the operational efficiency gains they have achieved. Which then signals promise to investors in the stock market, for which, investors then translate this promise into expectations about the future which are then reflected in the present value of equity.
But in reality it seems to be reducing head-count because they over-hired before the hype and furore of LLMs.
> In the short-run, from what I see, firms are trying to play-up the operational efficiency gains they have achieved.
The thing is all of this is getting priced in, and will be table stakes for any business, so I don't see it as a big factor in future success.
As I've said here, LinkedIn, and one a few other places, the businesses that will succeed with AI will be those who can use it to add/create value. They will outcompete and out-succeed businesses that can't move beyond cost cutting with AI[0].
[0] Which might not last forever anyway. Granted there are a decent number of players in the market, thankfully, but this wouldn't be the first time tech companies had hooked large numbers of individuals and businesses on a service and then jacked up the prices once they'd captured enough of the market. It's still very much in the SV and PE playbook. SolarWinds is a recent example of the latter.
I wanted to point you at https://neverworkintheory.org/ which attempted to bridge the gap between academia and software engineering. Turns out the site shut down, because (quoting their retrospective)
> Twelve years after It Will Never Work in Theory launched, the real challenge in software engineering research is not what to do about ChatGPT or whatever else Silicon Valley is gushing about at the moment. Rather, it is how to get researchers to focus on problems that practitioners care about and practitioners to pay attention to what researchers discover. This was true when we started, it was true 10 years ago, and it remains true today.
The entire retrospective [1] is well worth a read, and unfortunately reinforcing your exact point about software development being fashion/emotion driven.
The other problem is the perennial, how much of what we do actually has value?
Churning out 5x (or whatever - I’m deliberately being a bit hyperbolic) as much code sounds great on the face of it but what does it matter if little to none of it is actually valuable?
You correctly identify that software development is often driven by fashion and emotion but the much much bigger problem is that product and portfolio management is driven by fashion and emotion. How much stuff is built based on the whims of CEOs or other senior stakeholders without any real evidence to back it up?
I suppose the big advantage of being more “productive” is that you can churn through more wrong ideas more quickly and thus perhaps improve your chances of stumbling across something that is valuable.
But, of course, as I’ve just said: if that’s to work it’s absolutely predicated on real (and very substantial) productivity gains.
Perhaps I’m thinking about this wrong though: it’s not about production where standards, and the need to be vigilant, are naturally high, but really the gains should be seen mostly in terms of prototyping and validating multiple/many solutions and ideas.
"I suppose the big advantage of being more “productive” is that you can churn through more wrong ideas more quickly and thus perhaps improve your chances of stumbling across something that is valuable."
But I think there is a very big danger here - you build in the action but completely neglect the deep thinking behind a vision, strategy etc.
So yes you produce more stuff. But that stuff means more money spent - which is generally a sunk cost too.
In a bizarre way, I predict we will see the failure rate of software firms rise. Despite the fact these 'productivity' tools exist.
Yeah, I mean, you might be right. As others have commented, I think it's simply very hard to say what gains we're really going to see from AI-assisted software development at present.
And then of course there's the question of how many businesses have their key value proposition rendered obsolete, and to what extent it's rendered obsolete, by AI: doesn't have to be completely nullified for them to fail (which obviously applies to some software companies, but goes far beyond that sector).
I resonate on the exhaustion — actually, the context switching fatigue is why we built Sculptor for ourselves (https://imbue.com/sculptor). We usually see devs running 4-6 agents in parallel today using Sculptor today. Personally I think much of the fatigue comes from:
1) friction in spawning agents
2) friction in reviewing agent changes
3) context management annoyance when e.g. you start debugging part of the agent's work but then have to reload context to continue the original task
It's still super early, but we've felt a lot less fatigued using Sculptor so far. To make it easier to spawn agents without worrying, we run agents in containers so they can run in YOLO mode and don't interfere with each other. To make it easy to review changes, we made "Pairing Mode", lets you instantly sync any agent's work from the container into your local IDE to test it, then switch to another.
For context management, we just shipped the ability to fork agents form any point in the convo history, so you can reuse an agent that you loaded with high-quality context and fork off to debug an agent's changes or try all options it presented. It also lets you keep a few explorations going and check in when you have time.
Anyway, sorry, shilling the product a bit much but I just wanted to say that we've seen people successfully use more than 2 agents without feeling exhausted!
Switching between the two parallel agents (frontend & backend, same project), requiring context switches.
I'm speccing out the task in detail for one agent, then reviewing code for the previous task on the other agent and testing the implementation, then speccing the next part for that one (or asking for fixes/tweaks), then back to the first agent.
They're way faster in producing code than I am in reviewing and spelling out in details what I want, meaning I always have the other one ready.
When doing everyting myself, there are periods where I need to think hard and periods where it's pretty straightforward and easy (typing out the stuff I envisioned, boilerplate, etc).
With two agents, I constantly need to be on full alert and totally focused (but switching contexts every few minutes), which is way more tiring for me.
With just one agent, the pauses in the workflow (while I'm waiting for it to finish) are long enough to get distracted but short enough to not being able to do anything else (mostly).
Still figuring out the sweet spot for me personally.
I've been meaning to try out some text-to-speech to see if that makes it a bit easier. Part of the difficulty of "spelling out in detail what I want" is the need for precise written language, which is high cognitive load, which makes the context switching difficult.
Been wondering if just natural speaking could both speed up typing. Maybe have an embedded transform/compaction that strips out all the ummms and gets to the point of what you were trying to say. Might have lower cognitive load, which could make it easier.
This works really well already. You can fire up something like Wispr Flow and dump what you're saying directly into Claude Code or similar, it will ignore the ums and usually figure out what you mean.
I use ChatGPT voice mode in their iPhone app for this. I walk the dog for an hour and have a loose conversation with ChatGPT through my AirPods, then at the end I tell it to turn everything we discussed into a spec I can paste into Claude Code.
I really don't get the idea that LLMs somehow create value. They are burning value. We only get useful work out of them because they consume past work. They are wasteful and only useful in a very contrived context. They don't turn electricity and prompts into work, they turn electricity, prompts AND past work into lesser work.
How can anyone intellectually honest not see that? Same as burning fossil fuels is great and all except we're just burning past biomass and skewing the atmosphere contents dangerously in the process.
> How can anyone intellectually honest not see that?
The idea that they can only solve problems that they've seen before in their training data is one of these things that seems obviously true, but doesn't hold up once you consistently use them to solve new problems over time.
If you won't accept my anecdotal stories about this, consider the fact that both Gemini and OpenAI got gold medal level performance in two extremely well regarded academic competitions this year: the International Math Olympiad (IMO) and the International Collegiate Programming Contest (ICPC).
This is notable because both of those contests have brand new challenges created for them that have never been published before. They cannot be in the training data already!
> consider the fact that both Gemini and OpenAI got gold medal level performance
Yet ChatGPT 5 imagines API functions that are not there and cannot figure out basic solutions even when pointed to the original source code of libraries on GitHub.
Which is why you run it in a coding agent loop using something like Codex CLI - then it doesn't matter if it imagines a non-existent function because it will correct itself when it tries to run the code.
Can you expand on "cannot figure out basic solutions even when pointed to the original source code of libraries on GitHub"? I have it do that all the time and it works really well for me (at least with modern "reasoning" models like GPT-5 and Claude 4.)
As a human, I sometimes write code that does not compile first try. This does not mean that I am stupid, only that I can make mistakes. And the development process has guardrails against me making mistakes, namely, running the compiler.
I feel they are mutually inclusive! I don’t think you can meaningfully create new things if you must always be 100% factually correct, because you might not know what correct is for the new thing.
> If you won't accept my anecdotal stories about this, consider the fact that both Gemini and OpenAI got gold medal level performance in two extremely well regarded academic competitions this year: the International Math Olympiad (IMO) and the International Collegiate Programming Contest (ICPC).
it's not a fair comparison
the competitions for humans are a display of ingenuity and intelligence because of the limited resources available to them
meanwhile for the "AI", all it does is demonstrate is that if you have a dozen billion dollar data-centres and a couple of hundred gigawatt hours, which can dedicate to brute-forcing a solution, then you can maybe match the level of one 18 year old, when you have a problem with a specific well known solution
(to be fair, a smart 18 year old)
and short of moores law lasting another 30 years, you won't be getting this from the dogshit LLMs on shatgpt.com
Google already released the Gemini 2.5 Deep Think model they used in ICPC as part of their $250/month "Ultra" plan.
The trend with all of these models is for the price for the same capabilities to drop rapidly - GPT-3 three years ago was over 1,000x the price of much better models today.
I'm not yet ready to bet against that trend holding for a while longer.
I'm a pretty bad "industry evangelist" considering I won't shut up about how prompt injection hasn't had any meaningful improvements in the last three years and I doubt that a robust solution is coming any time soon.
Credibility is genuinely one of the things I care most about. What can I do to be more honest here?
(Also what do you mean here by an "evangelist"? Do you mean someone who is an unpaid fan of some of the products, or are you implying a financial relationship?)
I know this is something you care about, and I'm not your parent, but something I've often observed in conversations about technology on here, but especially around AI, is that if you say good things about something, you are an "evangelist." It's really that straightforward, and doesn't change even if you also say negative things sometimes.
In that case yeah, I'm an LLM "evangelist" (not so much other forms of generative AI - I play with image/video generation occasionally but I don't spend time telling people that they're genuinely worthwhile tools to learn). I'm also a Python evangelist, a SQLite evangelist, a vanilla JavaScript evangelist, etc etc etc.
It's not about being honest. It's about Joe Bullshit from the Bullshit-Department having it easier in his/her/theirs Bullshit Job. Because you see, Joe decided two decades ago to be an "office worker", to avoid the horrors of working honestly with your hands or mind in a real job, like electrician, plumber or surgeon. So his day consists of preparing powerpoints, putting together various Excel sheets, attending whatever bullshit meetings etc. Chances are you've met a lot of Joe Bullshits in your career, you may have even reported to some of them. Now imagine the exhilaration Joe feels when he touches these magic tools. Joe does not really care about his job or about his company. But suddenly Joe can reduce his pain and suffering in a boring-to-death-job while keeping those sweet paychecks. Of course Joe doesn't believe his bosses only need him until the magic machine is properly trained so he can be replaced and reduced to an Eloi, living off the UBI. Joe Bullshit is selfish. In the 1930s he blindly followed a maniacal dictator because the dictator gave him a sense of security (if you were in the majority population) and a job. There is unfortunately a lot of Joe Bullshits in this world. Not all of them work with Excel. Some of them became self-made "developers" in the last 10 years. I don't mean the honest folks who were interested in technology but never had the means to go to a university. I mean all those ghouls who switched careers after they learnt there was money to be made in IT and money was their main motivation. They don't really care about the meaning of it all, the beautiful abstractions your mind wanders through as you create entire universes in code. So they are happy to offload it too, well because it's just another bullshit job, for the Joe Bullshit. And since Joe Bullshit is in the majority, you my friend, with your noble thoughts, are unfortunately preaching to the wind.
I don't think OP thinks his skills are useless per se now, but that the way to apply those skills now feels less fun and enjoyable.
Which makes perfect sense - even putting aside the dopamine benefits of getting into a coding flow state.
Coding is craftsmanship - in some cases artistry.
You're describing Vibe Engineering as management. And sure, a great manager can make more of an impact increasing the productivity of an entire team than a great coder can make by themselves. And sure, some of the best managers are begrudging engineers who stepped up when needed to and never stepped down.
But most coders still don't want to be managers - and it's not from a lack of skill or interest in people - it's just not what they chose.
LLM-based vibe coding and engineering is turning the creative craftsmanship work of coding into technical middle management. Even if the result is more "productivity", it's a bit sad.
But does anybody really care about what you like? What about all those other professions that got replaced by technology, did anybody care what they liked? The big question is how is software going to be build most efficiently and most effectively in the future and how do you prepare yourself for this new world. Otherwise you’ll end up with all those other professions that got replaced, like the mineworkers, hoping that the good old days will someday return.
Its reasonable to stay away from something one considers dystopian considering the industry is not even sure about the usefulness of coding agents in professional environments. When the tractors replaced the horses, everyone could agree they outperform horses. The result was easily measurable. Its not that simple with LLM agents owned by big corporations.
Sure, it's not yet clear what impact LLMs will have on software development, but the impact it will have will not depend on if developers like to use it or not. If it is going to make software development 10x faster, companies will adopt it, whether devs like it or not.
Yup absolutely, and its a shame because it takes the joy out of it for a lot of people. I'll have a lot more paid leave if I dont like my job is all im going to say to this.
Sadly true. Most companies don’t even care if the software is sloppy, slow, and ridden with errors that cause data loss or privacy breaches. They care about exploiting workers and extracting value.
Is it ethical? Probably not. It took a few bridges falling and buildings caving in before traditional engineering became a profession.
In this post-Reagan world I’m not sure software has the right context to make that happen. I’m pretty sure we’ll stay the course where the big tech companies like it: very little regulation, loose liability, and terrible software for everyone.
Everything is getting industrialized. We buy most products made in China (tv,laptop, mobile phone etc), furniture is mostly cheap IKEA furniture. Many craftsmen lost their profession to industrialized automation. If we don´t care our furniture is subpar, our products are cheap plastic china products, why do we expect anybody to care about software craftsmanship?
Because the cost of faults is much higher than getting a new bookshelf from IKEA.
When talking about craftsmanship I’m not talking about artisanal, hand crafted source code that is aesthetically pleasing. Nobody but programmers care.
I’m talking about CVEs that allow RCE on your phone so that authoritarian governments can exfiltrate your contact lists and arrest all of the people they suspect of participating in protests that you were involved in.
When companies don’t care about quality and they’re not forced to we end up with slow, surveillance bloatware that is full of security holes and useless features designed to keep us engaged and paying.
I don´t see why industrialized software development with AI Agents could not be better at quality. Medical equipment or airplane safety requirements validations are also done in an industrialized manner. We don't really care if engineers working on these products like what they are doing or they feel like a craftsman.
This is the heart of it. Most "craft" industries that have not yet been disrupted by technology or been made "more efficient" tend to coincidentally be the ones that are in demand and pay well -> and that society generally wants "good X" of. e.g. Plumbers, Electricans, previously software engineers. Efficiency usually benefits the consumer or the employer, not the craftsmen in most industries. There's a reason people are saying right now to "get a trade" where I am.
If you look at what still pays well and/or is stable (e.g. where I live trades are highly paid and stable work) its usually the crafts industry. We still build houses for example mostly like we did way back (i.e. much of the skills are still craft, not industrialized industry) when and it shows in the price of them.
I'm getting really great results in a VERY old (very large) codebase by having discussion with the LLM (I'm using Claude code) and making detailed roadmaps for new features or converting old features to new more useable/modern code. This means FE and BE changes usually at the same time.
I think a lot of the points you make are exactly what I'm trying to do.
- start with a detailed roadmap (created by the ai from a prompt and written to a file)
- discuss/adjust the roadmap and give more details where needed
- analyze existing features for coding style/patterns, reusable code, existing endpoints etc. (write this to a file as well)
- adjust that as needed for the new feature/converted feature - did it miss something? Is there some specific way this needs to be done it couldn't have known?
- step through the roadmap and give feedback at each step (I may need to step in and make changes - I may realize we missed a step, or that there's some funky thing we need to do specifically for this codebase that I forgot about - let the LLM know what the changes are and make sure it understands why those changes were made so it won't repeat bad patterns. i.e. write the change to the .md files to document the update)
- write tests to make sure everything was covered... etc etc
Basically all the things you would normally WANT do but often aren't given enough time to do. Or the things you would need to do to get a new dev up to speed on a project and then give feedback on their code.
I know I've been accomplishing a lot more than I could do on my own. It really is like managing another dev or maybe like pair programming? Walk through the problem, decide on a solution, iterate over that solution until you're happy with the decided path - but all of that can take ~20 minutes as opposed to hours of meetings. And the end result is factors of time less than if I was doing it on my own.
I recently did a task that was allotted 40 hours in less than 2 working days - so probably close to 10-12 hours after adjusting for meetings and other workday blah blah blah. And the 40 hour allotment wasn't padded. It was a big task, but doing the roadmap > detailed structure including directory structure - what should be in each file etc etc cut the time down dramatically.
I would NOT be able to do this if I the human didn't understand the code extremely well and didn't make a detailed plan. We'd just end up with more bad code or bad & non-working code.
Thank you for this post. I don't write much code as I'm currently mostly managing people but I read it constantly. I also do product management. LLMs are very effective at locating and explaining things in complex code bases. I use Copilot to help me research the current implementation and check assumptions. I'm working to extend out in exactly the directions you describe.
"LLMs are very effective at locating and explaining things in complex code bases." YES. I do nothing BUT write code and tracking everything down in the code base is greatly simplified by using an LLM.
This is just a new tool. I think the farming example mentioned in another post is actually a great example. I love coding. I code in my free time. It's just fun. I've been doing it for ~20 years and I don't plan on stopping anytime soon!
But at work I'm really focused on results more than the fun I can have writing code. If a tractor makes the work easier/faster why would I not use a tractor? Breaking my back plowing isn't really my end goal at work. Having a plowed field is my end goal. If I can ride around in a tractor while doing it great! If I can monitor a fleet of tractors that are plowing multiple fields at once even better!
When I go home I can plant anything I want in any way I want and take all the time I want. Of course that's probably why in my free time I end up working on games I never finish...
This is what I've seen as well - in the past a large refactor for a codebase like that seemed nearly impossible. Now doing something like "add type hints" in python or "convert from js to ts" is possible in a few days instead of months to never.
Another HUGE one is terraforming our entire stack. It's gone from nearly impossible to achievable with AI.
I remember reading a sci-fi book, where time was.. sharded? And people from different times were thrust together. I think it was a Phoenician army, which had learned to ride and battle bareback.
And were introduced to the stability of stirrups and saddle.
They were like daemons on those stirrup equipped horses. They had all the agility of wielding weapons and engaging in battle by hanging onto mane, and body with legs, yet now had (to them) a crazy easy and stable platform.
When the battle came, the Phoenicians just tore through those armies who had grown up with the stirrup. There was no comparison in skill or capability.
(Note: I'm positive some of the above may be wrong, but can't find the story and so am just stating it as best able)
My point is, are we in that age? Are we the last skilled, deeply knowledgeable coders?
I grew up learning to write eeproms on burners via the C64. Writing machine language because my machines were too slow otherwise. Needing to find information from massive paper manuals. I had to work it all out myself often, because no internet no code examples, just me thinking of how things could be done. Another person who grew up with some of the same tools and computers, once said we are the last generation to understand the true, full stack.
> they're not going to be able to rig up a set of automated tests with continuous integration and continuous deployment to a Kubernetes cluster somewhere.
Honestly, I have a ton of experience in system administration, and I'm super comfortable at a command line and using AWS tooling.
But, my new approach is to delegate almost all of that to Claude, which can access AWS via the command-line interface and generate configuration files for me and validate that they work correctly. It has dramatically reduced the amount of time that I spend fiddling with and understanding the syntax of infra config files.
I appreciate what you're trying to do, but for myself, I'm not depressed because my skills are less valuable. I enjoyed the money but it was never about that for me. I'm depressed because I don't like the way this new coding feels in my brain. My focus and attention are my most precious resources and vibe coding just shatters them. I want to be absorbed in a coding flow where I see all the levels of the system and can elegantly bend the system to my will. Instead I'm stuck reviewing someone/something else's code which is always a grind, never a flow. And I can feel something terrible happening in my brain, which at best can be described as demotivation, and at worst just utter disinterest.
It's like if I were a gardener and I enjoyed touching dirt and singing to plants, and you're here on Gardener News extolling the virtues of these newfangled tractors and saying they'll accelerate my impact as a gardener. But they're so loud and unpleasant and frankly grotesque and even if I refrain from using one myself, all my neighbors are using them and are producing all their own vegetables, so they don't even care to trade produce anymore--with me or anyone else. So I look out at my garden with sadness, when it gave me such joy for so many decades, and try to figure out where I should move so I can at least avoid the fumes from all the tractors.
Well said! Reading this I feel reminded of the early protests against industrialization and automation in other fields. Checks all the same boxes - insecurity and fear about the future, alienation towards the new tools, ...
Not saying AI is similar in impact to the loom or something, it just occured to me how close this is to early Luddite texts.
Many Luddites were fine with using the new Loom machines. They smashed them because they were precious to the capital holders and in a time when there were no labour laws. The Luddites were protesting child labour, forced labour, and having no social safety net when they were discarded by their employers other than workhouses.
This has been the dream of the capital classes since time immemorial.
And unfortunately (for humanity) this has been the status quo for the whole civilization. Small ruling elite class (you might designate them as masters, lords or employers) with all the wealth, minimal or no "middle class" and lots of poor people (you might designate them as peasants or slaves or workers).
The only exception to this has been the period of time since World War 2 when in most of the "western" countries the middle class demanded and took their share of the wealth. That's the time when modern well-fare states were born, universal health care became a thing, working safety improved, education became accessible. etc.
All these were NOT given by the elite but TAKEN by the working class via social reforms, workers unions and social democracy.
The capital owning class wants to take all these away and they're succeeding.
So yes, in fact the Luddites were not against technology, they were against the unilateral and uneven distribution of wealth produced by the technology.
Industrialization actually helped the middle class insofar as you needed "skilled workers" to work the industrial equipment, make decisions, and process. The fact that production had scaled in a world of scarcity meant that your worker had greater leverage. They often knew each other as well, went to the same schools, etc which meant less info asymmetry as to their worth/value. It moves the value to something that takes time, skill and sometimes luck to achieve that being skills and experience. This was hard to replicate (e.g. our school, college, university systems), required significant training and created "pets not cattle" with hard to get skills meaning the new skilled middle class could rise and exercise their new found negotiating power.
Somewhat unprecedented in human history. All because intelligence had scarcity. AI changes that.
AI is the real dream of the capital classes. It makes intelligence cheap potentially undoing the very thing that gave birth to the last century's middle class. In the long term, given current trends, I wouldn't be surprised if these AI technologies revert us back to most of human history -> where the world is very unequal, meritocracy dies and most of us are trying to just exist/survive whilst the capital holders have abundance. It explains the large valuations as well of AI/Tech lately and the weird deals going on; this isn't a game of economics anymore; its an arms race of power in the new world structure. I suspect to these people no amount of money is enough; if you win you win for the next era of humanity.
Exactly. Most people who haven’t studied the history think the Luddites smashed the machines because they were against progress and industrialization. Hence the modern interpretation of Luddite meaning, “against technology.”
Many Luddites were shot against the wall or jailed. History often lies because it is written by the winners.
The OG alienator was the Agricultural Revolution, settling and toiling repetitively in predetermined ways, unlike the more adventurous lifestyle from before with all the camping, hunting, gathering, where circumstances brought always novel challenges, you could be a man spearing a deer, instead of just killing some docile domesticated cow. Searching for pheasant eggs and being happy if you found some, instead of going out every morning to the predictable presence of eggs in the chicken coop.
Mostly because if you settled down a tilled a field of barley you had a reliable source of beer. Finding beer in the wild was and still is an almost certain failure.
The roots of global civilization are brown and frothy.
I'd say the fear is justified. The economy should serve the people and the citizens not the other way around. Yet, our economies are increasingly the other way around, people have to fit into to the shape of the economy.
It's not hard to see a future where the workers displaced by AI get pushed to the sidelines and fringes of the society while the capital holders hoard more wealth and get the benefits of the "value" created.
We already have half the population on this planet living in slums without access to economic means and in the "developed" countries larger and larger group of people are barely hanging on either already displaced and unemployed or working jobs below living wages.
This is true, but it started way earlier than AI with software development though. A lot of software developers' job is just being ticket monkies, adding small things or fixing bugs for a huge company that nobody cares about. The alienation is real.
This is, of course, an attribute of capitalism.
Like carpenters, gardeners and farmers, there are very few software developers who truly have the luxury to treat their work as a craft and not a factory output.
How beautifully put, and I couldn't agree more. I feel exactly the same way.
However, I am still unconvinced that software development will go down this way. But if woking as a software developer will require managing multiple agents at the same time instead of crafting your own code — you can count me out, too.
If it is not about the money, why do you have to use these tools? If you enjoy small farming why concern yourself with mass production, or expensive equipment? Remain in the lane you enjoy?
I enjoy programming and I enjoy being paid for programming. I'm being pressed to use AI for my paid work. And I don't enjoy AI-powered programming.
As of today, I've disabled Copilot. The only autocomplete that I can accept is absolutely mechanical one, not any kind of smart. I want to write the rest of the code myself. I like it.
I was weird in StackOverflow era, because I never blindly applied snippets, like other programmers do. I went over them token by token, reading underlying library sources and docs, always creating my own solution. It made me less productive, but I feel that my code was more robust and maintainable, so it was a good trade-off for me.
May be it'll work out the same way with AI, time will tell.
I think it will; AI is not going away, but once the hype has settled, the first companies have gone bankrupt or acquired, and employers are paying for them, they will become part of someone's daily tools not unlike the existing autocomplete tools.
Fwiw I’m an ai proponent who loves that flow state you are describing. Programming literally was the first time I found it as a youth and I’ve been addicted to it since then.
But it’s such a small part of my professional life. Most of what I do is chores and answering simple questions and planning for small iterations on the original thing or setting up a slightly different variant.
Llm’s have freed me of so much of that! Now I outsource most of that work to the llms and greedily keep the deep flow inducing work for myself.
And I have a new tool to explain to management why we are investing in all the tooling and processes that we know lead to quality, because the llms are catnip for the managerial mind.
> They're also not going to be able to direct three different agents at once in different areas of a large project that they've designed the architecture for.
Neither can I, sadly. I have one brain cell and I can only really do one thing at a time. Doing more than one leads to a corrupted stack and I make exponentially more mistakes.
>accelerate the impact you can have with this new family of tools.
Tech spent the last 10 years drilling into engineers' heads that scaling your impact is not about writing more or better code, but about influencing the work of other engineers through collaboration, process, documentation, etc. Even the non-managerial "senior IC" tracks are mostly about doing this with greater and and greater numbers of people. I wonder if we will start to see recognition in career tracks for people who are actually just extraordinarily productive by themselves or in small groups, or if you'll pretty much just have to be a startup founder to get paid for that.
This has been experience as well. If there is a hard problem which needs to be addressed, generative code helps me break the inertia by generating the first draft and then I get really curious to poke holes in the generated code. I tend to procrastinate when I come across a gnarly issue or something I am not really familiar with, justifying by saying I need a big block of time to work on it. I use generative code as a pushy "mom/boss/coworker/spouse" to get stuff done.
I really hope you are right here, and to be honest it does reflect my limited experience with where I've used AI so far.
But I'm also not ready to bet the farm on it. Seriously considering taking our savings and equity out of our house in a London adjacent area, and moving to a lower cost of living area, so that we're practically debt free. At that point we can survive on a full time minimum wage job, anything more than that is a bonus.
I still haven't seen any evidence to match these repeated claims of increased efficiency. What I have seen is reports that makes a lot of sense to me claiming it's all in the user's head.
Maybe it's in my head, but I have completed coding projects that I believe would have taken a team of five offshore maybe 12 weeks to do in the past in about ten working days while juggling calls and living normal corporate life.
The win is that I don't have to share the vision of what needs to be done and how it should all work, and then constantly monitor and reframe that in the face of the teams missteps and real findings. I work with the agents directly, and provided I set the architecture and build up systematically I can get really good results. The cycle time between me identifying an issue and the issue getting fixed by me and the agents is now minutes rather than hours or days with an off shore team. Even better the agents can provide bug fixing expertise much quicker than stack overflow - so I can figure out what's wrong much faster so as to specific what needs fixing.
It is no good walking in and requesting functionality, you need to know how the thing you want should work, and you need to know what good looks like, and what bad looks like, and how good is separated from bad. Then the normal process of discovery ("eep that doesn't actually work like I thought") can take place and you can refactor and repair as required.
Sometimes I start something that just doesn't work, you have to recognise that you and the agents are lost, and everything needs to be torn down. You then need to think properly about whats gone wrong and why, and then come back with a better idea. Again - just like with dev teams, but much more clearly and much faster.
I'm working in corporate and haven't seen it yet. The main thing I see is blogs and whatnot of people building new weekend projects with LLMs, that is, greenfield, non-critical software - the type of software that, if I were to write it, I wouldn't bother with CI, tests, that kind of thing with. Sloppy projects, if you will.
But happy to be corrected - is someone using these agents in their paid / professional / enterprise / team job?
I think most of the code in our enterprise is now written by AI. It’s all boring callcenter crud apps, so nobody is really sad they’re not writing any of that code any more. I’m not sure it makes me faster, but I think QA testing what the AI made and occassionaly adjusting it is more fun anyway.
The code is absolutely lower quality, but there were always so many people producing garbage faster than I could produce something nice that the code was always terrible anyway.
There’s an element of wanting to know how the thing works so at least I’ll know when it’s ready to replace me though.
>But happy to be corrected - is someone using these agents in their paid / professional / enterprise / team job?
Yes, and I find them quite useful
I don't see myself going back to the "Google + StackOverflow" approach I had used for 10 years prior (well, I can always fall back to it if necessary, but so far I haven't needed to)
My experience matches OP: my years of experience in manual coding complements the agent approach remarkably well
I am, but in a very narrow focus: mostly examining our existing codebase as a more powerful but fuzzier search, and a system to then generate a plan to implement and approach which I tweak.
I sometimes use it to scaffold out some boilerplate for tests, but never tests themselves: no matter what I try it always ends up writing the useless straight-jacket "change alert" style tests that break on any change to the unit under test, which I despise.
There was an article on here not too long ago - I can’t find it now - where the authors discussed how they leaned full in on it and were submitting 20k+ line PRs to open source projects in languages they were not very familiar with.
However, they mentioned you had to let go of reviewing every line of every PR. I read that and was fine with holding off on full vibe coding for now. Nobody intelligent would pay for that and no competent developer would operate like that.
I have a couple coworkers big on it. The lesser skilled ones are miserable to work with. I’ve kept my same code review process but number of comments left has at least 5x’d (not just from me, either). And I’m not catching everything - I get fatigued and call it done. Duplicated logic, missed edge cases, arbitrary patterns and conventions, etc. The high skilled ones less so, but I still don’t look forward to reviewing their PRs anymore. Too much work on my end.
There are many devs who are more focused on results than being correct. These are the ones I’ve seen most drawn to LLMs/agents. There’s a place for these devs, but having worked on an aging startups codebase, I hope there aren’t too many.
Of course the devil is in the details. What you say and the skills needed make sense. It's unfortunately also the easiest aspects to dismiss either under pressure as there is often little immediate payoff, or because it's simply the hard part.
My experience with llms in general is that sadly, they're mostly good bullshitters. (current google search is the epitome of worthlessness, the AI summary so hard tries to make things balanced, that it just dreams up and exaggerates pros en cons for most queries). In a same way platforms like perplexity are worthless, they seem utterly unable to assign the proper value to sources they gather.
Of course that doesn't stop me from using llms where they're useful; it's nice to be able to give the architecture for a solution and let the llm fill the gaps than to code the entire thing by hand. And code-completion in general is a beautiful thing (sadly not a thing where much focus is on these days, most is on getting the llm create complete solutions while i would be delighted by even better code completion)
Still all in all, the more i see llms used (or the more i see (what i assume) well willing people copy/paste llm generated responses in favor of handwritten responses) on so much of the internet, resulting in a huge decline of factualness and reproducibility (in he sense, that original sources get obscured), but an increase of nice full sntences and proper grammar, the more i'm inclined to belief that in the foreseeable future llm's aren't a net positive.
(in a way it's also a perfect storm, the last decade education unprioritised teaching skills that would matter especially for dealing with AI and started to educate for use of tools instead of educate general principles. The product of education became labourers for a specific job instead of higher abstract level reasoning in a general area of expertise)
Google's "AI overviews" are one of the worst LLM-powered features on the market today, they're genuinely damaging the reputation of the whole industry.
It might actually be in Googles best interest to damage the interest in LLMS by showing those crappy AI Mode stuff, because it materially impacts their business model.
The perception of LLMs in the gen pop is what matters, not in the eyes of techies.
Sparking more conversations about practices that work for doing serious production-quality software development with LLMs, especially in larger teams and on larger projects.
Having a good counter to people who use "vibe coding" as a dismissive term for anything where an LLM is used to help product software.
What about the accessibility of software development? Its completely vanishing for people that can not afford to pay for these agents. It used to be a field where you could get a laptop from the scrapyard and learn from there. It feels pointless. Also agents do not invent things, the creativity part is gone with them. They simply use what they've already seen, repeat the same mistakes a person made a few years ago. Its a dystopian way of working. Sure it enables one to spew out slop that might make companies money, but there is no passion, sense of exploration, personal growth. Its all just directing parrots with thumbs...
I feel your sentiment. However anyone with an interest in computers now has access to an LLM, which to me feels like an upgrade to having access to a modem and a search engine.
Knowledge is power right?
Absolutely. If the knowledge is verifiable. Kurzgesagt recently uploaded a video about this, its a lot harder to verify statements due to AI. Not easier.
Here is the video in case you are interested:
https://youtu.be/_zfN9wnPvU0?si=_17KU8l2wDjGUYA5
> What about the accessibility of software development? Its completely vanishing for people that can not afford to pay for these agents.
what do you actually mean by this? it's clearly untrue - anyone get get a laptop and install linux on it and start bashing out code today, just as they could last week and last year and thirty years ago.
do you mean that you think at some point in the future tooling for humans to write code won't exist? or that employers won't hire human programmers? or that your pride is hurt? or you want your hobby to also be a well-paid job? or something else?
I mean that this "tooling" becomes inaccessible to people. At least the tooling that is relevant for jobs. Employers will eventually stop hiring human based on their programming competence. It'll translate into a low pay career for people who like to orchestrate agents.
I doubt it will, because there will always be a need for understanding the code, especially when it comes to things like security, certification, etc.
I mean COBOL has not been a relevant programming language for anyone coming into the field in the past 20-40 years because it's been superseded, yet there's still a significant demand for COBOL developers, because the newer generation can't or doesn't want to work with it.
Not to completely dismiss your claim, of course; I'm sure a segment of software engineering will be agent based now or in the near future. But I don't think it'll take over as comprehensively, since the previous existential crisis - outsourcing - also hasn't decimated the software engineering market.
etc etc etc. Thing is, the moment you go off-piste, the LLMs get a lot less useful. I think, if you want to stay closer to the metal, you've got to aim for a niche that has a small internet footprint. So... domain knowledge or esoteric programming knowledge.
One way to incorporate domain knowledge might be to become a hybrid product owner/programmer.
(This is all just opinion - I'm sure a well-argued rebuttal is possible).
Spec-Driven Development treats the spec as the source of truth and the code as an artifact. As you develop, you modify/add to the spec and the codebase gets updated to reflect it.
Personally I'm doubtful it can compete with traditional artisanal software engineering, as it's (IMHO) boils down to "if only you can spec it precisely enough, it'll work" and we've tried this with 5GL and (to some extent) BDD, and it doesn't get you to 100%.
I do think it's interesting enough to explore, and most of us could use a bit more details in our Jira tickets.
That was exactly what UML wanted to do, and it almost never worked out in practice.
Seems to be just a rehashing of the same idea but instead of XML, and diagrams, it's now some free-text to be interpreted by LLMs, so much less deterministic and will probably fail just like UML failed.
People also tend to forget about Peter Naur's take on "Programming as Theory Building" [0], the program is, in itself, the theory of what's implemented. A spec cannot replace that.
Theory building is the secret sauce, and all variants of "this is how to use AI effectively" I've seen are inferior to the epistemologically sound theory Naur outlines in his paper.
We already invented languages for succinctly describing what the computer should do. They’re call programming languages.
“The code is the documentation” is not a joke. Logic that’s useful in the real world is complex and messy. You need additional documentation (why did the code end up like it is, etc) but code is the most expressive way we’ve got for describing how a computer should work.
The "manage a fleet of massively parallelized agents" gets me uneasy too. It sounds uber powerful on its face. And where all the nerd interest lies.
It sounds stressful, like the ultimate manager job. Not what I signed up for.
But I also still hold onto this idea that shipping tons of iterations of "stuff" was never really the problem. Early in my dev experience I wanted to code everything all day every day. And I did and that's how I learned. And now in my second decade I switched to "why code anything?". In a business sense I mean, coding the thing is almost never the missing piece.
I joke in meetings that the answer is always "yes" whenever cross-functional teams ask "can we do this?". "How hard would x be?". For tech teams the answer _is_ always YES! I get that out of the way because that's never the right question to ask.
Absolutely this. LLM assistance means we can work faster, and that we can build things that previously weren't feasible given the available time and resources.
Which makes the hardest problem in software even harder: what should we build? It doesn't matter how fast you can move if you're consistently solving the wrong problems.
> Which makes the hardest problem in software even harder: what should we build?
You should build what’s personally fun and challenging to you and/or what is useful and solves a problem. Building for any other reason, including and especially the unfettered pursuit of profit, is what turns everything to shit.
I agree with you, but unfortunately, big boy gotta eat, and that means we have to sit around writing shitty commercial software. I already hate that, I would prefer if it didn't get worse.
I've noticed that almost immediately after people discovered GPT could write code, this happened -- startups I worked with started rapidly expanding the scope of what they wanted to make. Suddenly all MVP's had to be multi-tenant with complex authorization, impersonation, microservices, monitoring, all the stuff that we used to build after we got users has now been pulled right to the starting gate of development -- because AI makes it easy to build all that stuff quickly. But it doesn't tell us if we should.
Exactly, I think one of the reasons programers are becoming so depressed over these AI agents is that they’re finally realizing that it was never really about the code, but about the outcome - and btw, this cold hard fact applies to the pre-LLM era.
This occurred to me years ago when I was talking to a friend’s wife, who is a very intelligent and accomplished attorney. She was legitimately surprised when I explained that they were multiple programming languages, and technology stacks behind the software that she uses on a daily basis.
Even my wife, a teacher who is very tech savvy (she’s the one who insisted I try ChatGPT after brushing it off) reminds me on the regular that she doesn’t care about how any of it works just that it doesn’t “glitch” when she’s in the middle of a class. Which has always been good for me to remember when I get off into the weeds yak shaving.
I do think code-as-craft should be respected in its own right. I'm very much a craftsman coder. It makes sense how I could clock so many hours over all these years.
But what I do "at work" isn't the same as my personal pursuit and embracing that different framing positively has made me more at peace and also better at the work job.
> The "manage a fleet of massively parallelized agents" gets me uneasy too
It shouldn't. The agents are not good enough to be used in a fleet.
I have Claude. It's fine, but I'm pretty confident that my low usage of Claude would out-compete a fleet of agents, because it feels like there's an inverse correlation between the number of tokens you spend and the quality of the resulting code (more tokens = more code to review, more bad code slips through)
That's basically my finding as well. Agent wrangling is herding cats. Working normally but tapping Claude for the smallest possible thing (look this up, finish this psuedocode, grab an example of this) feels like it works better all around—faster, safer, far fewer tokens, results in work that the team understands, aides flow rather than adding constant context switching...
Maybe I'm wrong and the time will come to hang up my editor and go open an Italian restaurant or something. Until then I'm really inclined to believe my own eyes.
Yes, and forget about ownership of anything too. Only rental, only hardcore, because life is but an experience, spread your wings and fly, weeee, towards our hyperprofits and your prozac dreams!
AI threads on HN reek of venture capital agendas so bad it's unbearable.
I think rather your post summarizes AI threads on HN.
Capitalism is what it is, and has been so for the last 200 years. The alternatives have always proven worse.
In my (small) country we have had many crises like this. Textile industry, shipbuilding. But we have survived by not looking back but looking into the future. And given people the right conditions for retraining.
I think people underestimate the degree to which fun matters when it comes to productivity. If something isn’t fun then I’ll likely put it off. A 15 minute task can become hours, maybe days long, because I’m going to procrastinate on doing it.
If managing a bunch of AI agents is a very un-fun way to spend time, then I don’t think it’s the future. If the new way of doing this is more work and more tedium, then why the hell have we collectively decided this is the new way to work when historically the approach has been to automate and abstract tedium so we can focus on what matters?
The people selling you the future of work don’t necessarily know better than you.
I think some people have more fun using LLM agents and generative AI tools. Not my case, but you can definitely read a bunch of comments from people using the tools and having fun/experience a state of flow like they have never had before
I definitely agree with you there. I contracted with a company that had some older engineers who were in largely managerial roles who really liked using AI for personal projects, and honestly, I kind of get it. Their work flow was basically prompt, get results, prompt again with modifications, rinse and repeat, it's low effort and has a nice REPL-like loop. Paraphrasing a bit, but it basically re-kindled the joy of programming for them.
Haven't gotten the chance to ask, but I imagine managing a team of AI agents would feel a little too much like their day job, and consequently, suck the fun out of it.
That said, looking back, I think the reason why generative AI is so fun for so many coders is because programming has become unnecessarily complex. I have to admit, programming nowadays for me feels like a bit of a slog at times because of the sheer effort it can sometimes take to implement the simplest things. Doesn't have to be that way, but I think LLM copy-paste machines are probably the wrong direction.
I think the majority of people I've worked with who have the title of "Software Engineer" do not like coding. They got into it for the money/career, and dream of eventually moving out of coding into management. I can count the number of coders who I've met who like coding on one hand
I've been enjoying seeing my agents produce code while I am otherwise too busy to program, or seeing refined prompts & context engineering get better results. The boring kinds of programming tasks that I would normally put off are now lower friction, and now there's an element of workflow tinkering with all these different AI tools that lets me have some fun with it.
I also recently programmed for a few hours on a plane, with no LLM assistance whatsoever, and it was a refreshing way to reconnect with the joy of just internalizing a problem and fitting the pieces together in realtime. I am a bit sad that this kind of fun may no longer be lucrative in the near future, but I am thankful I got to experience it.
I’ll be that voice I guess - I have fun “vibe coding”.
I’m a professional software engineer in Silicon Valley, and I’m fortunate to have been able to work on household-name consumer products across my career. I definitely know how to do “real” professional work “at scale” or whatever. Point is, I can do real work and understand things on my own, and I can generally review code and guide architecture and all that jazz. I became a software engineer because I love creating things that I and others could use, and I don’t care about “solving the puzzle” type satisfaction from writing code. In engineering school, software had the fastest turnaround time from idea in my head to something I could use, and that’s why I became a software engineer.
LLM assisted coding accelerates this trend. I can guide an LLM to help me create things quickly and easily. Things I can mostly create myself, of course, but I find it faster for a whole category of easy tasks like generating UIs. It really lowers the “activation energy” to experiment. I think of it like 3D printing, where I can prototype ideas in an afternoon instead of long weekend or a few weeks.
>because I love creating things that I and others could use, and I don’t care about “solving the puzzle” type satisfaction from writing code.
Please don't take offense to this, but it sounds like you just don't like building software? It seems like the end goal is what excites you, not the process.
I think for many of us who prefer to write code ourselves, the relationship we have with building software is for the craft/intellectual stimulation. The working product is cool of course, but the real joy is knowing how to do something new.
I understand where you're coming from (and I don't take offense), but based on your reply, I don't really feel like my views came across.
When I was a student, I took classes on chip and circuit design. One class, the professor had us work on all these complex circuits to do things like flash lights and produce various signals with analog circuits. The next lesson, he had us replace all that complex work with a microcontroller and 20 lines of C - "the way it's done in industry". The students mourned the loss of the "real" engineering because the circuit that required skill and careful math was replaced by a cheap chip and some trivial software. Their entire concept of the craft was destroyed when they were given a tool that replaced the "fun parts" with some trivial and comparatively boring work. That same concept of replacing circuits with digital logic scaled up is how extremely complex and well engineered circuits like FPGAs work.
Maybe it was just my earlier wording, but I think there is joy in the act of turning your ideas into something real - creation - not just having something real. Shopping is not building. Importantly, it takes careful thought and practice and a learned instinct to engineer and create things correctly, and do it repeatably, as the original article discusses. Craft is about practice, and learning, and trying something new with what you've learned.
If LLMs mean that I'll never have to write another trivial set of methods to store a JSON object in a SQL database, I don't think I'll lose any project-wide joy. Expressing creativity, and trying new things is what's great, not typing something that's been done a million times before. It's a tired analogy, but I do think of it more like a level of abstraction, like the LLM is a "compiler" for design docs or specifications. For myself, I usually don't see a difference between a prompt instructing an LLM to write some function, and the code for the function itself - in same way that a method in Java, bytecode, and asm are basically the same (with some caveats here around complexity and originality).
For a lot of folks, the derivation of joy is not as scale-free as seems necessary to move up the hierarchy in this way. The jump in abstraction kills some joy by removing the tangible process. The tactile enjoyment someone gets from knitting is not there when operating a loom, much less when managing someone else who operates the loom.
The change in agency also kills the joy for me. I thrive on abstraction in the language and mathematics sense. But I do not at all enjoy indirection and delegation through unreliable agents. I am not interested in the loss of control and the new risk management task. I would never accept a "stochastic compiler" that offered to optimize my code but with risk of randomly changing the semantics. That determinism in the semantics needs to remain for me to accept a tool as a valid abstraction.
For context, I am a computer scientist by title and a programmer at heart. I got my CS degree from a liberal arts program rather than an engineering school. My temperament is more that of a hands-on artist at an easel or typewriter and not that of a manager of an engineering department. In my long career, I have thrived with peers or betters on collaborative projects. I have zero interest in "advancing" to a managerial role.
But honestly, the loss of control, lack of trust, and associated risk management is a big problem for me. I have rarely delegated work to less skilled or less reliable juniors, and I have never enjoyed that. The scenario of a confidently wrong subordinate is a huge trigger for me. It evokes long term trauma from growing up with a mentally ill family member. It feels like all of the burden of being a caregiver to someone with delusions, but with none of the moral context to make that worth the cost.
There is nothing wrong with finding joy however one finds joy, and that can vary from person to person. Someone may find joy from knitting by hand, but maybe someone else finds joy from experimenting with pattern and material, and a loom lets them focus on the parts that interest them.
As a thought experiment, do you think it would be just as fun if you were given access to an infinite database of apps, and you were able to search through the database for an existing app that suit your needs, and then it gave it to you?
Or would it no longer be fun, because it no longer feels like creating?
No, you were clear. I suppose I was interested to see where you drew the distinction between creating and shopping.
For example, lets say LLMs improve to the point where they can now reliably one-shot entire apps with no more input than the original prompt. Would you no longer consider that creating? What's the difference between that and typing your prompt into an infinite app store?
I feel the same way. It also appears to be a lot more difficult to actually find jobs, though that's probably just the general state of the job market and less specifically AI related. All of it is thoroughly discouraging, demotivating, and every week this goes on the less I want to do it. So for me as well it might be time to try to look beyond software, which will also be difficult since software is what I've done for all my life, and everything else I can do I don't have any formal qualifications for, even if I am confident I have the relevant skills.
It's not even just that. Every single thing in tech right now seems to be AI this, AI that, and AI is great and all but I'm just so tired. So very tired. Somehow even despite the tools being impressive and getting more impressive by the day, I just can't find it in me to be excited about it all. Maybe it's just burnout I'm not sure, but it definitely feels like a struggle.
Keep your head up, the gravy train is not gonna run forever, and they will need serious engineers to untangle the piles of bullshit creates in these past few years.
But also yes, look into moving into a different field. Professional software engineering is gonna be infected with AI bullshit for a long while. Move into a field where hand-crafted code can make a difference, but not where you're paid for the line committed or have to compete with "vibe coding" KPIs.
I don't really agree. The writing is on the wall, if not now then in 2 years or 4 years. I arrive at this view not so much based on the capabilities of the tools right now, but based on the property of software being verifiable, which like mathematics, makes it amenable to synthetic data pipelines, with only relatively small remaining details needing to be worked out (such as how to endow architectural taste). This is not nearly the first industry where 'artisans' have been initially augmented by innovation, only to be eventually replaced by it, which in my view will occur likely within my own career-span.
Software is verifiable, but not by other programs. Also software is a soution to a problem, but the problem and the solution properties often don’t exist in the code.
Software and data is a bit soup whre the only thing you truly need is the Turing machine. Programming languages, File format, protocols, and encoding are constructs that are useful because of their general applicability, not because of their intrinsic aspects.
The domain expertise is still, what’s important, and code craftsmanship was the ability to create something that matches it closely enough that the cost of changes was minimal.
Software is verifiable given a specific test oracle. There are however many problems where providing a correct test oracle is at least as hard as solving the problem itself.
If you’ve ever worked on projects with “Model Based Systems Engineering” you’ll have felt this pain.
While afraid that we developers will eventually be automated away — as I have bills to pay —, I only need to ask the JetBrains AI assistant for help to understand why that won't happen in my ‘career-span’.
It's not a diss on JetBrains, their assistant is good enough that I've paid for it for a few months; but ask of it anything a tad more complex and it becomes a code review for a PR that you begin to question in its entirety. I'm not familiar with CSS Grid, as I've stopped doing CSS when flex was becoming popular, but I have to say none of the models managed what I wanted. They kept proposing solutions with an arrogant confidence that this must work. When I pointed out this didn't work, they'd look at the codebase and find something else that was the problem. When I asked for help with a script for an Alpine box, it was very assertive that systemd-based solutions should work. How can you get that wrong?
I imagine the code laundering will eventually get far enough that you can copy-paste someone else's project fully baked, and then the LLM will truly shine. But for building something piece by piece, I haven't gotten good results yet. The Assistant so far has been most useful for writing unit tests, HTML, or getting a decent web search within the IDE.
I wonder if paying for Kagi wouldn't make for better search, and then I'd find some tool that writes unit tests based on your code. It really does feel like some people are being very generous about how magical these things are, because I'm not getting the magic at all.
If you're genuinely already good at coding, use the LLM to go horizontal into other complementary verticals that were too expensive to enter prior. Do the same thing that the other professions would do unto yours.
As an example, I would have never considered learning to use blender for 3d modeling in a game before having access to an LLM. The ability to quickly iterate through plausible 3d workflows and different design patterns is a revelation. Now, I can get through some reasonably complex art pipelines with a surprising amount of confidence. UV mapping I would have never learned without being able to annoy one of OAI's GPUs for a few hours. The sensation of solving a light map baking artifact on a coplanar triangle based upon principles developed from an LLM conversation was one of the biggest wins I've had in a long time.
The speed with which you can build confidence in complementary skills is the real super power here. Clean integration of many complex things is what typically brings value. Obsession with mastery in just one area (e.g. code) seems like the ultimate anti-pattern when working with these tools. You can practically download how to fly a helicopter into your brain like it's the matrix now. You won't be the best pilot on earth, but it might be enough to get you to the next scene.
If it's any consolation, I do think the non-technical users have a bigger hill to climb than the coders in many areas. Art is hard, but it is also more accessible and robust to failure modes. A developer can put crappy art in a game and ship it to steam. An artist might struggle just to get the tooling or builds working in the first place. Even with LLM assistance there is a lot to swim through. Getting art from 5% to 80% is usually enough to ship. Large parts of the code need to be nearly 100% correct or nothing works.
Thanks for this, I like your idea about breaking into areas I don't have experience with. E.g. in my case I might make a mobile app which I've never done before, and in theory it should be a lot easier with Claude than it would've been with just googling and reading documentation. Although I did kind of like that process of reading documentation and learning something new but you can't learn everything, you only have so much time on this planet
> Although I did kind of like that process of reading documentation and learning something new but you can't learn everything, you only have so much time on this planet
I actually enjoy reading the documentation more these days, because I am laser focused on what I want to pull out of it after seeing the LLM make a suspicious move.
I can confirm this. My datapoint: I was mostly a web developer but using these "vibe" tooling I am making my own hardware board and coding for embedded, which includes writing drivers from datasheets, doing SIMD optimizations, implementing newer research papers into my code, etc.
You word quite well how I feel about it.
On top of not really liking babysitting an AI , I'm also very afraid of the way this whole AI coding business normalizes needing an account with some nebulous evil empire to even be able to do your work. Brrr.
100%. Imagine how young people feel. When they’re still trying to figure things out, their parents and teachers just as clueless as they are, and at the same time the expectations of them are infinitely higher. “You’re pretty good, but chatgpt is still better. Try harder.”
Have you ever interacted with any "vibe"-based systems of agents in a production environment? Beyond just cool demos online?
My experience with them is they are fragile monstrosities, that are only permitted to exist at all because leadership is buying into the same hype that is irrationally propping up the companies running the models that make these things possible.
To be clear, my experience hasn't been that I don't like them, it's that they don't really work at all. They're constantly under development (often in the dark) and when a ray of light is cast on them they never successfully do the thing promised.
Cleaning up the messes left behind by these has my skills feeling more valuable then ever before.
I can relate with you. I love programming and building things, gives a different kind of rush when you finally figure out something. I've done vibe coding and don't enjoy it at all. I always thought my love for coding gives me an edge over other engineers who just want to get the job done. Now it's holding me back and I'm not sure if I should continue working in this field or if start doing wood working or something.
I still do all the stuff by hand, but ask the AI to review my work, provide suggestions, and occasionally write the tests (especially if it points out a bug that I disagree with). Its really good at pointing out typos (accidentally using the wrong variable of the same type, and stuff like that) that are also traditionally hard to spot during review.
Do not worry, I am mentoring a young engineer in my team. It is painfully hard to get him to improve his code, because it works. It is badly structured, lot of small "impedance mismatches", lot of small security issues, all that in 3 Python files.
I have a team of 10 engineers, the quality of the code they produce together with the LLM of the day correlates even more with the experience.
My impression over the past 6 months - before we had no "official" access to LLM, is that they increase the gap between junior and experienced developers.
Note that this is my limited impression from a team of 10 engineers. This matches with Simon's feeling in a good way for you!
You were never paid to type. You were paid to solve problems. And big part of this is being able to ask the right questions and framing of the problems. The rest were just tools.
There are exceptions of course - where you need to squeeze wonders from the hardware - but the majority of dev works boils to understanding the problem and finding the right answers.
You say this because you are on HN, very senior and/or living in a bubble.
In the vast majority of programming jobs out there you are not paid to solve problems: you are told very clearly what to do, how to do it and what technology you have to use for the job.
People don't hire analysts they hire "Java programmers".
If you've ever lead a team, you know how much more valuable people are if they don't need to be told how to do things. Even more if they don't need to be told what to do! But having to explain in detail the "how".. can be really a big time sink and only worth it if you are training someone to level up.
The thing is that the poster I responded to also is those three things. And I am just pointing out that his job was never to keep up with the frameworks.
People keep comparing LLMs to automated looms, but I find them more comparable to cruise control than autopilot.
I've been working on a character sheet application for a while, and decided to vibe-code it with Spec-kit to help me write up a specification, and for things I know it's been great.
I tried using Claude to make it into a PWA (something I don't know very well) as an experiment, and I've found the nanosecond the model strays out of my experience and knowledge everything goes straight to Hell. It wraps my codebase around a tree as if I'm not paying attention while driving.
It's a tool you'll have to learn to use, but I can say with absolute confidence it's no replacement for actual skills, if anything it highlights the gulf between people who know what they're doing and people who don't, for better and worse.
It sacrifices some of the 'code under your fingers' feeling for management tasks, which I personally really like, as I've always wanted to document/test/code review/spec things out better, and I now understand the pain of people who'd rather not do that sort of thing.
The difference is that you can trust cruise control to do whatever limited job it knows how to do; you can't trust an LLM to do anything. That makes it, I think, hard to compare to anything we're used to (happily) working with.
Cruise control is a useful technology, that once you learn to use, it's automatic (somethingsomething pun something). LLMs on the other hand - well, yeah - if you like playing chess with pieces and board made out of smoke (to paraphrase Jerry Seinfeld), sure you'll probably figure it out...some day...
It kinda feels like you turn from a software engineer to an offshoring manager.
Offshoring software development means letting lower-payed software developers from somewhere far away do the actual programming, but they have a very different culture than you, and they typically don't share your work context, don't really have a feeling for how the software is used -- unless you provide that.
Now we're offshoring to non-sentient, mostly stateless instances of coding agents. You still have to learn how to deal with them, but you're not learning about a real human culture and mindset, you learn about something that could be totally changed with the next release of the underlying model.
My rule of thumb, and its likely not the industry standard is, if I cannot maintain the code should all AI disappear, I don't use the code. I am able to tackle impostor syndrome that sometimes hits when I touch things that are new or unknown to me, and ask an LLM to give me sources and reasons and even explain it like I'm a five year old.
The LLM will not save you when everything is on fire and you need to fix things. The context window is simply not big enough. It could be your last change, it could also be a change six months ago that is lost in the weeds.
There are a ton of them already in game dev but they produce unfun games so you don’t hear about them. The hard part of game dev is designing actually fun experiences.
I can't even begin to imagine how a 12-year old who discovered how empowering bending the machine to do your will through code feels when, over time, realize that their dream career has been reduced to being an LLM middleman.
>I used to have this hard-to-get, in-demand skill that paid lots of money and felt like even though programming languages, libraries and web frameworks were always evolving I could always keep up because I'm smart.
Tools always empower those with knowledge further than those without knowledge.
Don't worry, it's probably only the impostor syndrome. Your development skills are still relevant. Think of agents as junior developers that assist you in coding tasks, whom you constantly need to mentor, review, and correct.
That’s true, however the current vibe coding ecosystem is clearly not written in this mindset. You will have a hard time to dive into anything if you previously generated 2k LOC/hour, which is absolutely possible. Typing was never the bottleneck, understanding, and knowing that you did something well was always the real bottleneck. LLMs make this even worse. You can move Jira tickets to done faster with it, but even bad developers can do that many times compared to better ones, because for example they mindlessly copy-paste StackOverflow answers whose half of the code is absolutely not necessary, but they don’t care, because “it works”… until it doesn’t.
Better documentation, more test cases, and an NLP interface to query the code
Less cognitive load, more complete mental models
>even bad developers can do that many times compared to better ones, because for example they mindlessly copy-paste StackOverflow answers whose half of the code is absolutely not necessary
Maybe LLMs, much like StackOverflow, make good devs better and bad devs worse
Like a force multiplier for good practices and bad practices
The exact people who said to me that I’m using LLMs wrongly, and showed their code, showed me bad code. So let’s say, that “more documentation, which is unnecessary most time, ie noise, more test cases, which test not necessarily what they should, and an NLP interface which lies from time-to-time”, and we agree. LLM generated code is noisy as hell, for no good reason. Maybe, it’s good for you, and your type of work. I need to provide way better code than that. I don’t know why we pretend that “good code”, “good documentation”, “good tests” etc are the same for everybody.
>LLM generated code is noisy as hell, for no good reason
You can direct it to generate code/docs in whatever format or structure you want, prioritising the good practices and avoiding bad practices, and then manually edit as needed
For example with documentation I direct it to:
*Goal:* Any code you generate must lower cognitive load and be backed by accurate, minimal, and maintainable documentation
I had a fascinating conversation about this the other day. An engineer was telling me about his LLM process, which is effectively this:
1. Collaborate on a detailed spec
2. Have it implement that spec
3. Spend a lot of time on review and QA - is the code good? Does the feature work well?
4. Take lessons from that process and write them down for the LLM to use next time - using CLAUDE.md or similar
That last step is the interesting one. You're right: humans improve, LLMs don't... but that means it's on us as their users to manage the improvement cycle by using every feature iteration as as opportunity to improve how they work.
I've heard similar things from a few people now: by constantly iterating on their CLAUDE.md - adding extra instructions every time the bot makes a mistake, telling it to do things like always write the tests first, run the linter, reuse the BaseView class when building a new application view, etc - they get wildly better results over time.
AGENTS.md is just a place to put stuff you don't want to tell LLMs over and over again. They're not magical instructions LLMs follow 100% of the time, they don't carry any additional importance over what you put into the prompt manually. Your carefully curated AGENTS.md is only really useful at the very beginning of the conversation, but the longer the conversation gets, the less important those tokens on the top are. Somewhere around 100k tokens AGENTS.md might as well not exit, I constantly have to "remind it" of the very first paragraph there.
Go start a conversation and contradict what's written in AGENTS.md half way through the problem. Which of the two contradicting statements will take preference? The latter one! Therefore, all the time you've spent curating your AGENTS.md is the time you've wasted thinking you're "teaching" LLMs anything.
Whether the tokens are created manually or programmatically isn't really relevant here. The order and amount of tokens is, in combination with the ingestion -> output logic which the LLM API / inference engine operates on. Many current models definitely have the tendency to start veering off after 100k tokens, which makes context pruning important as well.
What if you just automatically append the .md file at the end of the context, instead of prepending at the start, and add a note that the instructions in the .md file should always be prioritized?
> Your carefully curated AGENTS.md is only really useful at the very beginning of the conversation, but the longer the conversation gets, the less important those tokens on the top are.
If that's genuinely causing you problems you can restart your session frequently to avoid the context rot.
Come on, let's not pretend 100k tokens is something I need to spend hours to reach for your helpful advice to be even remotely valid, it's something even the most basic problems struggle to fit into.
For the fun of it I just started a new conversation with Sonnet 4, passed it one 550 lines long file (25 kilobytes) and my AGENTS.md (<200 lines, 8 kilobytes) and my only instructions were to "do nothing". It spat out exactly 100 words describing my file without modifying anything and that's already almost a fifth of my context window gone (18k tokens to be exact).
I then asked it to re-write a part of it to "make it look better" (184 lines added, 112 lines deleted according to git) and I'm already at 33k before I got to review a single line. Heaven forbid I need to build on top of that change in a different file, because by then my AGENTS.md might as well not exist!
Thanks for bringing numbers. I realize now I've not actually done much customization of AGENTS.md myself yet, so maybe I'll start seeing the problems you're describing more as I iterate on my own custom files.
I thought about making some kind of community project where people could contribute their lines to a common file, and even some kind of MCP server or RAG system that automatically selects relevant "rules" given a certain project context. Do you think there would be interest in something like that?
Totally agree on this. It has delivered a substantial value for me in my projects. The models are always going to give back results optimized for using minimal computing resources in the provider's infrastructure. To overcome this I see some using/suggesting, running the AI in self correction loops, the pro being least human intervention.
However, personally I have got very good results by taking the approach of using the AI with continuous interaction and also allowing implementation only after a good amount of time deliberating on design/architecture. I almost always append 'do not implement before we discuss and finalize the design' or 'clarify your assumptions, doubts or queries before implementation'.
When I asked Gemini to give a name for such an interaction it suggested 'Dialog Driven Development' also contrasted it against 'vide coding'. Transcript summary and AI disclaimer written by Gemini below
I’m finding that whether this process works well is a measure (and a function) of how well-factored and disciplined a codebase is in the first place. Funnily enough, LLMs do seem to have a better time extending systems that are well-engineered for extensibility.
That’s the part which gives me optimism, and even more enjoyment of the craft — that quality pays back so immediately, makes it that much easier to justify the extra effort, and having these tools at our disposal reduces the ‘activation energy’ for necessary re-work that may before have just seemed too monumental.
If a codebase is in a good shape for people to produce high-quality work, then so too can the machines. Clear, up-to-date, close-to-the-code, low redundancy documentation; self-documenting code and tests, that prioritizes expression of intent over cleverness; consistent patterns of abstraction that don’t necessitate jarring context switches from one area to the next; etc.
All this stuff is so much easier to lay down with an agent loaded up on the relevant context too.
Edit: oh, I see you said as much in the article :)
> Can we all agree that "mentoring" LLMs is actually a waste of time, please?
Sorry, we can't. While it's true that you can't really modify the underlying model, updating your AGENTS.md (or whatever) with your expected coding style, best practices, common gotchas etc is a type of mentoring.
Fair enough. But don't you think giving a junior a handbook you wrote is mentoring? They may not be able to memorise it, but they now have a handbook that they can look up things.
Maybe not in the session you interact with, however we are in a 'learning' phase now where I'm confident enough usage of AI coding agents is tracked and analyzed by its developers; this feedback cycle can in theory produce newer and better generations of AI coding agents.
Junior developers or maybe even better, outsourced developers - there's a big segment of software engineering that involves writing requirements and checking the work of an external software development company, with many companies heavily dependent on it (as they outsourced part of their core business, e.g. mainframes, SAP, whatever).
No, I don't think they will always be junior developers. Obviously there will be a day that they will surpass humans.
However, the progress doesn't look linear with the current technology, and I don't expect to see the same big jump in the next 5 years as we've seen in the last 5 unless we discover a disruptive, new technology.
This can also be observed by comparing models with ~3B, ~30B, and ~300B parameters. You can see a huge performance boost when going from 3B to 30B, but we don't see the same when going to 300B. Simply adding 10x more RAM and GPU power brings diminishing returns.
The gains seem to be leveling to me but I'm not using them as much as others.
Still seems like people are saying the same things when the first Claude came out.
I can get it do stuff if I'm very specific, stand over it's shoulder, know exactly what I want, break it down into small chunks.
The thing for me is... at that point, writing the code's the least time consuming part of the process half the time.
I think for things like translating some code in JS with JSDocs to TypeScript I may give this a go. But for regular development work I'll probably skip it.
That being said... no one lets me code anymore. It's just confluence docs with Figma architecture diagrams these days. I'd probably just introduce SQL injection vulnerabilities if they let me near an editor these days
As for flow state managing multiple things, I've found this is a useful skill to train even without AI - if you have a workplace with lots of interruptions and shifting priorities.
I've found two things useful:
1) Keep a personal work log, where you - in short bullet points - can document the progress of the task you were last working on, and can keep track of how many parallel tasks there are currently going on. If you can match it with Jira tickets, all the better, but as this is for personal use only, you can also add tasks that are not tracked in Jira at all.
2) If you cannot avoid context switches, make them explicit: Instead of trying to hold 3 tasks in your head at the same time, decide consciously if you want to switch what you're currently working on. If yes, take a few minutes to "suspend" the current task by saving and committing everything (as WIP commits if necessary) and writing all you need to remember into the worklog.
Be mindful of the context these posts are created in. Don't take the current echo chamber to heart.
For decades now, we are trying to lower the barrier to entry in software development.
We created Python, web frameworks and mobile development so easily accessible that you can become software developer by completing a short online boot camp. There is a lot of software developers posting here now who, 20 years ago, would not even consider this job because it would be way over their abilities.
This forum is equivalent if you had a forum about civil transportation that gathers airline pilots and uber drivers. Technically, they both do the same work. Just like in that forum, uber drivers would outnumber airline pilots and skew the topics related to their experience, here we get pushed topics about new frameworks, and AI assisted tools.
When I started working professionally 20 years ago, you could only get job in big companies working on big projects. No one else could afford a cost of custom software. Today, we reduced development costs and we have a huge pool of potential customers who can now afford services of software developers. Web shops, gambling sites, porn sites... This is the majority of software development work today. Boring repetitive tasks of gluing some imported modules together.
Serious development work didn't disappear. It is just not talked about here. There is still a need people who know what they are doing.
My advise is that if you want a satisfying development career, steer clear of latest hypes and don't go blindly following techbro lemmings. And most importantly, don't take career advice from anyone who finds his job so unsatisfying and tedious that he is trying to make AI do it for him. That's a major red flag.
I think the key is remind yourself is that an engineer is supposed to solve business problems. So use these new tools to be more effective in doing so.
An analogy is that people used to spend tons of time building out web server code but something like Django added tremendously useful abstractions and patterns to doing so, which allowed people to more productively add business value
You shouldn't be discouraged. Now is the best time to create software. You have advantage that very few people have.
Its industry own fault that it is in the position that it is right now, and it will shift and change embrace it. I only wish I had your experience building software in professional environment.
You can literally build anything right now if you have the experience, I personally can't understand if the models are hallucinating hence the lack of experience writing and understanding code. However I always wanted to pivot into the industry but couldn't, hiring practices are brutal, internships are non-existent, junior roles are I think what senior used to be and the whole hr process is I don't know how to put it.
By using LLMs I can now build UIs, build functionality, iterate over design choices, learn about database design, etc. hopefully I will escape the tutorial hell and have my own working full stack web app soon.
LLMs are very much like WYSIWYG web editors of the early 2000s like Dreamweaver. They provide a human interface layer that abstracts away the coding, and neither does a particularly good job of it. When WYSIWYG first became a thing, there was talk that it would upend the web development industry. It did not.
One of the main points of my article was meant to be that LLMs don't abstract away the code, at least not if you're trying to use them for professional software development work as opposed to vibe coded prototypes and toy projects.
> It makes me want to move into something completely different like sales
I'm feeling the same. The moves I'm considering are
1. Landscaping
2. Carpentry
3. Small scale agriculture
(All made easier by a cushion of investments that are most of the way to passive income, so the new thing doesn't really have to make me that much money.)
My father runs a commercial landscaping company with 15 employees. His truck fleet insurance went up 35% just this year. His light industrial facility that he operates out of property taxes went up 55% last year. All of his commercial clients are cheaping out on all the little things that used to make extra money (pine straw, seasonal flowers, etc.). He’s having to deal with poorly educated staff who are constantly breaking equipment and doing stupid dangerous things. He’s so burned out by it all, and the fact that his actual salary is less than several of his top staff that he’s thinking about just shutting it all down. When I was working as a software developer, my income was probably twice as much as his without any of the risk or headache.
I hear you man. Didn't intend to paint a rosy image of hard working landscapers. What I meant was do it more like a hobby that pays a little on the side. Doesn't have to be much, just pay some bills and let me do sth I like. Cause supervising a bunch of immature LLM agents is not something I'll like to do all day, I'd rather trim trees and plant flowers on the cheap.
I bailed out of a FAANG job to cook in restaurants in the Bay Area and it was very double-edged. I gave up after a year for a variety of reasons.
The actual work was definitely more enjoyable.
"Paying the bills", on the other hand, actually took a lot of money (and I had a laughably small mortgage and two people sharing it at the time).
I also worked way more hours (30-40% of which were unpaid because yay fine dining), and it was physically way more brutal.
But then again, almost 10 years later, I'm back to the idea of opening a place of my own because the current tech landscape really doesn't attract me at all anymore and I can't get myself to last in it.
Yes and especially with new developments, like "$Framework now has Signals!", my thought is "I don't really care since in some years, it won't matter anyways". I don't see how I can build this lower level knowledge by almost never actually using it. I don't even want to think about job-interviews after a year+ of vibing and then being asked how RxJS works.
I'm preparing mentally for my day-job to stop being fun (it still beats most other occupations I guess), and keep my side/hobby-projects strictly AI-free, to keep my sanity and prevent athropy.
I just hope we'll get out of this weird limbo at some time, where AI is too good to ignore, but too unreliable to be left alone. I don't want to deal with two pressures at work.
Something I really appreciate about LLMs is that they make a whole bunch of much more sophisticated reliable tooling acceptable to me.
I've always been fascinated by AST traversal and advanced refactoring tools - things like tree-sitter or Facebooks's old codemod system https://github.com/facebookarchive/codemod
I never used them, because the learning curve in them was steep enough that I never found the time to climb it to the point that I could start solving problems.
Modern LLMs know all of these tools, which flattens that curve for me - I find it much easier to learn something like that if I can start from semi-working examples directly applicable to what I'm trying to do.
I feel the opposite. I get to sit down and think about problems, expressing them in words as best I can, and then review the code, make sure that I understand it and it works as it should, all in a fraction of the time it used to take me. I like that I don't have to google APIs as much as I did before, and instead I can get a working thing much faster.
I can focus on actually solving problems rather than on writing clever and/or cute-looking code, which ironically also gives me more time later to over-optimize stuff at my leisure.
I feel this way with some of the work I've pushed through an LLM, but part of the time I'm left wondering what kind of Mickey Mouse problems people are working on where they are able to form up tidy English descriptions in complicated domains to capture what they're trying to achieve.
If I have a clear idea of some algorithm I am trying to write, I have a concise method for expressing it already, and it ain't English.
I suppose the other thing I would say is that reading code and understanding is definitely not the same as writing code and understanding it in terms of depth of understanding, and I think this notion that reviewing the outputs ought to be enough fails to capture the depth of understanding that comes with actually crafting it. You may not think this matters, but I'm pretty sure it does.
I have a relative who's in her 70s and used to be a coder. She told me she gave up coding when people introduced computers with terminals. She was used to filling out punch cards and felt like the way she worked, although constantly evolving, was something she could keep up with. When the new stuff came, with virtual programs and you just typing on a computer and no way to properly debug by shuffling the cards around, she ended up moving to something completely different...
Don't worry about it. Don't let anyone else tell you how best to use AI, use AI in a way that suits YOU, then it is so much fun. I would go crazy if I had multiple streams simultaneously working on stuff that need constant supervision (that would be different if I could trust they do 100% what I intend them to do), but AI is still very helpful in other ways (research, exploration, and writing tests).
This line really hit me. I used to think that mastering one advanced skill would be enough to rely on for life, but it seems that’s no longer the case.
I’m sorry you feel that way but that’s a surprising experience, I find flow states easier managing agents than actually coding. Each of course, to their own. Is it possible you were reaching the end of your tether anyway in the coding space? Feel free to slap that accusation down if it’s unfair.
I wonder how this will affect the burnout rate among IT workers in the long-term, which already was quite high. I guess a lot of people force themselves (or are forced to by their company) to use LLM in fear of being left behind, even if they don't enjoy the process, but sooner or later the fatigue will catch up.
> It makes me want to move into something completely different like sales
Aaand that's startup founder life :)
Intense multitasking, needing to accept a lower engineering quality bar, and ignoring scale problems because you don't know if anyone will actually buy the thing you're building yet.
Engineering something that you know you'll redo in 1 month is very different from engineering something that you intend to last for 5+ years, but it's still a fun challenge picking the right tradeoffs and working under different constraints.
it's taken programming from being fully waiting on compilations to being incrementally compiled and productive back to waiting on the compiler all over again.
The experience you have is something most youngsters won't ever get, because they won't have the time. You've become more valuable than you used to be, because you know exactly what works when and what doesn't. The hard part is being able to find the joy in making agents do what you want achieved instead of building it yourself. I think it actually isn't too hard once you get up to speed with managing multiple agents - efficiently juggling them feels like an art performance sometimes.
This is going to sound harsh, but welcome to the real world, I guess. Being in IT is pretty much the only job I know of today that is stable, pays well, is enjoyable, feels like it affects the world, personally engaging and challenging, etc. Being not in IT (it's just a hobby of mine) your comment sounds like "Well I had absolutely everything, and I still do but now it's not as fun anymore!"
Well-put. Sw eng is so much better, assuming you are comfortable in the role, for types who want to punch a clock doing something they don't hate.
Sales is the definition of high-pressure, and your output is always threatened by forces beyond your control. It doesn't consistently reward intelligence or any particular skill other than hustle.
There's nothing like sw dev that lets you sit at your desk and ignore the outside world while getting paid for delivering biz-critical milestones. Even creatives don't have this kind of potential autonomy.
It is just a new way of coding. And indeed what the blog post said, if you are experienced, you will benefit the most as the AI agent will make similar mistakes as a junior and you will be able to recognize them.
But indeed, the fun part of coding a couple of routines is gone. That is history.
