A few days back, I tried to implement a PDF reader by pure vibe coding. I used all my free Antigravity, Cursor, and Co-pilot tokens to create a half-baked, but working Next.js PDF-reader that (to be honest) I wouldn't have glued together without 2 weeks of work. As an MLE, I have done negligible web development using JavaScript and have mostly worked with Python and C.
But the struggle actually started after the free tokens were exhausted. I was feeling anxious to even look into those Next.js files. I am not able to describe, but it was probably some kind of fear - fear of either not being able to debug/implement a new feature, or not willing to put in precious hours (precious because of FOMO that I could do something cool with AI-paired vibe coding) to understand and build the feature myself.
I abandoned that project since that day. Haven't opened it yet - partly because I am waiting for the renewal of free tokens.
> If everything passes, it ships, automatically. If something fails, the agent fixes it. A human only gets involved when the system genuinely doesn’t know what to do.
This right here, he conveniently skipped elaborating how will a human fix a system he hasn’t genuinely built but just prompted.
Writing code by hand helps you build a mind-map that is helpful while debugging. Without it, a human needs to go through all of the complex code.
This is not efficiency, but delaying the heavy lifting to a later date, because system fail and they fail often (nowadays).
And based on how its phrased, it looks like AI has written this article.
> You can write 10x the code - good code. You can review and edit it before committing it. Nothing changes from a code quality perspective. Only speed.
I agree, but this is an oversimplification - we don't always get the speed boosts, specifically when we don't stay pragmatic about the process.
I have a small set of steps that I follow to really boost my productivity and get the speed advantage.
(Note: I am talking about AI-coding and not Vibe-coding)
- You give all the specs, and there are "some" chances that LLM will generate code exactly required.
- In most cases, you will need to do >2 design iterations and many small iterations, like instructing LLMs to properly handle error gracefully recover from errors.
- This will definitely increase speed 2x-3x, but we still need to review everything.
- Also, this doesn't take into account the edge cases our design missed. I don't know about big tech, but when I have to do the following to solve a problem
1. Figure out a potential solution
2. Make a hacky POC script to verify the proposed solution actually solves the problem
3. Design a decently robust system as a first iteration (that can have bugs)
4. Implement using AI
5. Verify each generated line
6. Find out edge cases and failure modes missed during design and repeat from step3 to tweak the design, or repeat from step4 to fix bug.
WHENEVER I jump directly from 1 -> 3 (vague design) -> 5, Speed advantages become obsolete.
I partially agree, but I don’t think “design ways to test the code without inspecting it manually line by line” is a good strategy.
Tests only cover cases you already know to look for. In my experience, many important edge cases are discovered by reading the implementation and noticing hidden assumptions or unintended interactions.
When something goes wrong, understanding why almost always requires looking at the code, and that understanding is what informs better tests.
Instead, just learning concepts with AI and then using HI (Human Intelligence) & AI to solve the problem at hand—by going through code line by line and writing tests - is a better approach productivity-, correctness-, efficiency-, and skill-wise.
I can only think of LLMs as fast typists with some domain knowledge.
Like typists of government/legal documents who know how to format documents but cannot practice law. Likewise, LLMs are code typists who can write good/decent/bad code but cannot practice software engineering - we need, and will need, a human for that.
Instead of "vending machine", I see many people calling generative AI "slot machine", which more aptly describes current genAI tools.
Yes, we can use it 10,000 times to refine our recipes, but "did we learn from it"? I am doubtful about that, given that even after running with the same prompt 10 times, it will give different answers in 8/10 responses.
But I am very confident that I can learn by iterating and printing designs on a 3D printer.
Previous title: "Anthropic: AI Coding shows no productivity gains; impairs skill development"
The previous title oversimplified the claim to "all" developers. I found the previous title meaningful while submitting this post because most of the false AI claims of "software engineer is finished" has mostly affected junior `inexperienced` engineers. But I think `junior inexperienced` was implicit which many people didn't pick.
The paper makes a more nuanced claim that AI Coding speeds up work for inexperienced developers, leading to some productivity gains at the cost of actual skill development.
Agreed. AI-generated code is "mid" by nature. You won’t feel amazed seeing AI-generated code because its "reasoning" competes with that of a potato.
I have recently started exploring AI-coding -- note that I said AI coding and not vibe-coding because that is for the brain-dead.
By AI coding, I mean I know the inputs, outputs, structures the code should have and the necessary context to write the code. Then, articulating the requirements in English as best as I can and feeding it to agents.
Needless to say, the code is pathetic; it chooses to implement meaningless abstractions even after explicitly providing the design and plan to follow.
I don’t understand how we, as a collective species, agreed to believe the criminally wrong lies of tech CEOs that, instead of implementing a “reliable system by hand,” we choose to convey our ideas and instructions in an “ambiguous,” “inconsistent,” and “context-dependent” language (English), which is then passed through a probabilistic system to generate the reliable system.
A few days back, I tried to implement a PDF reader by pure vibe coding. I used all my free Antigravity, Cursor, and Co-pilot tokens to create a half-baked, but working Next.js PDF-reader that (to be honest) I wouldn't have glued together without 2 weeks of work. As an MLE, I have done negligible web development using JavaScript and have mostly worked with Python and C.
But the struggle actually started after the free tokens were exhausted. I was feeling anxious to even look into those Next.js files. I am not able to describe, but it was probably some kind of fear - fear of either not being able to debug/implement a new feature, or not willing to put in precious hours (precious because of FOMO that I could do something cool with AI-paired vibe coding) to understand and build the feature myself.
I abandoned that project since that day. Haven't opened it yet - partly because I am waiting for the renewal of free tokens.
reply