Let's say I'm a business owner and I'm frustrated with the current state of the on-call system. I have options.
I can try negotiating with my clients to lessen the load in some way. Obviously this isn't always possible but it often is. I once had a freelance project that required 24 hours of on-call after a release. I negotiated release days that were convenient for me (never Fri/Sat/Sun). One time the client pushed back, I pushed back harder, and I won. In order for my push back to work I ensured that I had enough negotiating strength to do that which I planned for ahead of time.
I can upgrade my systems. For example if my current paging system is insufficient I can choose to pay $10/month more for another system that makes my life easier. I can set aside time to refactor my alerts code to make my life easier and I don't need to justify it to anyone but myself.
I can straight up refuse to do on-call and deal with the consequences to my business. Freelancer developers do this all of the time. We choose which client work to do and not to do. We can make these choices arbitrarily. Sometimes it's seasonal. Sometimes it's just based on vibes. Doesn't matter; it's our company.
Meanwhile the average on-call engineer at a large company has none of these freedoms. The underlying systems are chosen for them and they just have to deal with it.
This story is from an indian company, the on-call "expectations" might be different in your country. We were responsible for a 400k RPM service. It handled ads, so it was fairly important to the business. Whenever I had to go out for a night, or go out for a family event, or whatever, I was always able to hand over on-call to a team member for that duration. Of course, this also happened during other's on call, where I would take over. In fact, this happened daily! From ~7pm to ~9pm every day I would play football or whatever. I would always hand over on-call to another team member during this time. I usually wake up earlier than others, so I used to respond to alerts during those hours regardless of who was on call. The nights where I was staying up to watch champions league football matches or some other reason, I would take on-call as well. We just set up pagerduty's escalation order appropriately. Probably helped that there were just 5 of us in the team - easy coordination. Of course, this was my first job, and I messed up quite a bit, but I noticed the others following a similar system without me as well.
It also depends on the nature of the alerts I suppose. For us, the majority of the alerts could be checked and resolved from a mobile phone (they are alerts that could strictly speaking be resolved in an automated fashion, but the automation would get complex enough with dependencies on other service's metrics that we wouldn't be sure of not having bugs in _that_ code). About once a week or two weeks we would get an alert that needs us to look at the logs and so on.
> Meanwhile the average on-call engineer at a large company has none of these freedoms. The underlying systems are chosen for them and they just have to deal with it.
In most cases they have all of those freedoms, and the only barrier is one that's shared with the self-employed person: not liking the consequences of choosing those options.
They could negotiate with their manager to lessen the load. They could upgrade the systems. They could straight-up refuse on call.
They don't because they don't like the consequences of taking these options—and neither does the self-employed person!
> not liking the consequences of choosing those options
Correct. "Shove it" is usually preceded by not liking something.
> They could negotiate with their manager to lessen the load.
Most of the time the manager will simply refuse. As a business owner it's my decision.
> They could upgrade the systems.
At big companies this is usually outside the scope of an on-call engineer. The on-call engineer often doesn't even have commit rights to that repository.
The specific example I gave was paying $10/month more. That can be a very hard sell at a large company because their service contracts are much more complicated/expensive.
> They could straight-up refuse on call.
A business owner has much more negotiating power than an employee does.
> They don't because they don't like the consequences of taking these options—and neither does the self-employed person!
In the vast majority of cases making changes to the on-call infrastructure has very little (if any) measurable impact on the business. Like spending a week making the systems better. Or changing deploy/release dates to be more convenient.
As a business owner I can take advantage of this and make my life easier.
As an employee I have layers of bureaucracy to wade through and will probably be refused. Not because it affects the business but for other reasons.
Do others not generally get extra pay for the time on call?
I have the enviable situation where I am on call for half of every month, I get paid significantly extra for this, and there's maybe 1 emergency call per year.
> A function with well-constrained inputs and outputs is easy to reason about.
It's quite easy to imagine a well factored codebase where all things are neatly separated. If you've written something a thousand times, like user authentication, then you can plan out exactly how you want to separate everything. But user authentication isn't where things get messy.
The messy stuff is where the real world concepts need to be transformed into code. Where just the concepts need to be whiteboarded and explained because they're unintuitive and confusing. Then these unintuitive and confusing concepts need to somehow described to the computer.
Oh, and it needs to be fast. So not only do you need to model an unintuitive and confusing concept - you also need to write it in a convoluted way because, for various annoying reasons, that's what performs best on the computer.
Oh, and in 6 months the unintuitive and confusing concept needs to be completely changed into - surprise, surprise - a completely different but equally unintuitive and confusing concept.
Oh, and you can't rewrite everything because there isn't enough time or budget to do that. You have to minimally change the current uintuitive and confusing thing so that it works like the new unintuitive and confusing thing is supposed to work.
Oh, and the original author doesn't work here anymore so no one's here to explain the original code's intent.
> Oh, and the original author doesn't work here anymore so no one's here to explain the original code's intent.
To be fair, even if I still work there I don't know that I'm going to be of much help 6 months later other than a "oh yeah, I remember that had some weird business requirements"
So even if comments are flawlessly updated they are not a silver bullet. Not everyone are good at explaining confusing concepts in plain English so worst case you have confusing code and a comment that is 90% accurate but describe one detail in a way that doesn't really match what the code says. This will make you question if you have understood what the code does and it will take time and effort to convince yourself that code is in fact deterministic and unsurprising.
(but most often the comment is is just not updated or updated along with the code but without full understanding, which is what caused the bug that is the reason you are looking at the code in question)
I don't think comments need to be perfect to have value. My point was that if a certain piece of code is solving a particularly confusing problem in the domain, explaining it in a comment doesn't _necessarily_ mean the code will be less confusing to future dev if the current developer is not able to capture the issue in plain English. Future dev would be happier I think with putting more effort into refactoring and making the code more readable and clear. When that fails, a "here be dragons" comment is valuable.
They can write a very long comment explaining why it is confusing them in X, Y, Z vague ways. Or even multilingual comments if they have better writing skills in another lanaguage.
And even if they don’t know themselves why they are confused, they can still describe how they are confused.
And that time spent writing a small paper in one's native language would be better spent trying to make the code speak for itself. Maybe get some help, pair up and tackle the complexity. And when both/all involved is like, we can't make this any clearer and it's still confusing af. _Then_ it's time to write that lengthy comment for future poor maintainers.
You can only do the “what” with clearer code. The “why” needs some documentation. Even if it is obvious what the strange conditionals do, someone needs to have written down that this particular code is there because the special exemption from important tariffs of cigarettes due to the trade agreement between Serbia and Tunis that was valid between the years years 1992 and 2007.
This is where a good comment really can help! And in these types of domains I would guess/hope that there exists some project master list to crossref that will send both developers and domain experts to the same source for "tariff-EU-92-0578" specifically the section 'exemptions'. So the comment is not not just a whole paragraph copied in between a couple of /*/
Thing is, good documentation has to be part of the company's process. eg, a QA engineer would have to be responsible for checking the documentation and certifying it. Costs money and time.
You can't expect developers, already working 60 hour weeks to meet impossible deadlines, to spend another 15 hours altruistically documenting their code.
How about old, out of date documentation that is actively misleading? Because that’s mostly what I run into, and it’s decidedly worse that no documentation.
Give me readable code over crappy documentation any day. In an ideal world the docs would be correct all of the time, apparently I don’t live in that world, and I’ve grown tired of listening to those who claim we just need to try harder.
Every line of documentation is a line of code and is a liability as it will rot if not maintained. That’s why you should be writing self documenting code as much as possible that’s obviates the need for documentation. But unlike code, stale/wrong doc will not break tests.
Spending 15 hours documenting the code is something no leader should be asking of engineering to do. You should not need to do it. Go back and write better code, one That’s more clear at a glance, easily readable, uses small functions written at a comparable level of abstraction, uses clear, semantically meaningful names.
Before you write a line of documentation, you should ask yourself whether the weird thing you were about to document can be expressed directly in the name of the method of the variable instead. Only once you have exhausted all the options for expressing the concept in code, then, only then, are you allowed to add the line of the documentation regarding it.
> Only once you have exhausted all the options for expressing the concept in code, then, only then, are you allowed to add the line of the documentation regarding it.
But that's what people are talking about when talking about comments. The assumption is that the code is organized and named well already.
The real world of complexity is way beyond the expressiveness of code, unless you want function names like:
Or you know, work the devs 40 hour weeks and make sure documentation is valued. Everything costs one way or another, it's all trade-off turtles all the way down.
An outdated comment is still a datapoint! Including if the comment was wrong when it was first written!
We live in a world with version history, repositories with change requests, communications… code comments are a part of that ecosystem.
A comment that is outright incorrect at inception is still valuable even if it is at least an attempt by the writer to describe their internal understanding of things.
This. I have argued with plenty of developers on why comments are useful, and the counter arguments are always the same.
I believe it boils down to a lack of foresight. At some point in time, someone is going to revisit your code, and even just a small `// Sorry this is awful, we have to X but this was difficult because of Y` will go a long way.
While I (try to) have very fluid opinions in all aspects of programming, the usefulness of comments is not something I (think!) I'll ever budge on. :)
> // Sorry this is awful, we have to X but this was difficult because of Y
You don’t know how many times I’ve seen this with a cute little GitLens inline message of “Brian Smith, 10 years ago”. If Brian couldn’t figure it out 10 years ago, I’m not likely going to attempt it either, especially if it has been working for 10 years.
But knowing what Brian was considering at the time is useful, both due avoiding redoing that and for realising that some constraints may have been lifted.
IMO the only thing you can assume is that the person who wrote the comment wasn't actively trying to deceive you. You should treat all documentation, comments, function names, commit messages etc with a healthy dose of scepticism because no one truly has a strong grip on reality.
Right, unlike code (which does what it does, even if that isn't what the writer meant) there's no real feedback loop for comments. Still worth internalizing the info based on that IMO.
"This does X" as a comment when it in fact does Y in condition Z means that the probability you are looking at a bug goes up a bit! Without the comment you might not be able to identify that Y is not intentional.
Maybe Y is intentional! In which case the comment that "this is intentional" is helpful. Perhaps the intentionality is also incorrect, and that's yet another data point!
Fairly rare for there to be negative value in comments.
It just occurred to me that perhaps this is where AI might prove useful. Functions could have some kind of annotation that triggers AI
to analyze the function and explain it plain language when you do something like hover over the function name in the IDE, or, you can have a prompt where you can interact with that piece of code and ask it questions. Obviously this would mean developer-written comments would be less likely to make it into the commit history, but it might be better than nothing, especially in older codebases where the original developer(s) are long gone. Maybe this already exists, but I’m too lazy to research that right now.
But then could you trust it not to hallucinate functionality that doesn't exist? Seems as risky as out-of-date comments, if not more
What I'd really like is an AI linter than noticed if you've changed some functionality referenced in a comment without updating that comment. Then, the worst-case scenario is that it doesn't notice, and we're back where we started.
Comments that explain the intent, rather than implementation, are the more useful kind. And when intent doesn't match the actual code, that's a good hint - it might be why the code doesn't work.
I mean it's easy to say silly things like this, but in reality most developers suck in one way or another.
In addition companies don't seem to give a shit about straightforward code, they want LOC per day and the cheapest price possible which leads to tons of crap code.
Each person has their own strengths, but a worthwhile team member should be able to meet minimum requirements of readability and comments. This can be enforced through team agreements and peer review.
Your second point is really the crux of business in a lot of ways. The balance of quality versus quantity. Cost versus value. Long-term versus short term gains. I’m sure there are situations where ruthlessly prioritizing short term profit through low cost code is indeed the optimal solution. For those of us who love to craft high-quality code, the trick is finding the companies where it is understood and agreed that long-term value from high-quality code is worth the upfront investment and, more importantly, where they have the cash to make that investment.
>I’m sure there are situations where ruthlessly prioritizing short term profit through low cost code is indeed the optimal solution
This is mostly how large publicly traded corps work, unless they are ran by programmers that want great applications or are required by law, they tend to write a lot of crap.
>In addition companies don't seem to give a shit about straightforward code, they want LOC per day and the cheapest price possible which leads to tons of crap code.
Companies don't care about LOC, they care about solving problems. 30 LOC or 30k LOC doesn't matter much MOST of the time. They're just after a solution that puts the problem to rest.
If a delivery company has four different definitions of a customer’s first order, and the resulting code has contents that are hard to parse - does the Blake lie with the developer, or the requirements?
If the developer had time to do it, with him. Otherwise with the company
I'm sure there's some abysmal shit that's extremely hard to properly abstract. Usually the dev just sucks or they didn't have time to make the code not suck
Business requirements deviate from code almost immediately. Serving several clients with customisation adds even more strain on the process. Eventually you want to map paragraphs of business req to code which is not a 1:1 mapping.
Aging codebase and the ongoing operations make it even harder to maintain consistently. eventually people surrender.
Then in 3 months someone in between came changing the code slightly that makes comment obsolete but doesn’t update the comment. Making all worse not better.
Issue trackers are much better because then in git you can find tickets attached to the change.
No ticket explaining why - no code change.
Why not in repo? because business people write tickets not devs. Then tickets are passed to QA who also does read the code but also need that information.
So we just fire all the employees and hire better ones only because someone did not pay attention to the comment.
Of course it is an exaggeration - but also in the same line people who think "others are failing at their jobs" - should pick up and do all the work there is to be done and see how long they go until they miss something or make a mistake.
Solution should be systematic to prevent people from failing and not expecting "someone doing their job properly".
Not having comments as something that needs a review reduces workload on everyone involved.
Besides, interfaces for PRs they clearly mark what changed - they don't point what hasn't been changed. So naturally people review what has changed. You still get the context of course and can see couple lines above and below... But still I blame the tool not people.
A code tends to be reused. When it happens jira is not likely to travel alongside the code. All 'older' jira tickets are useless broken links. All you have in practice is jira name. It usually happen with 'internal documentation' links as well.
Git blame often lies when big merge was squashed. I mostly had these in Perforce so I might be wrong. Also when code travels between source version control servers and different source version control software it also loses information.
I would say in my gamedev practical experience the best comments I saw are
TODO implement me and (unit) test code that still runs. First clearly states that you have reached outside of what was planned before and 2nd allows you to inspect what code meant to do.
One of my favorite conventions is ‘TODO(username): some comment’. This lets attribution survive merges and commits and lets you search for all of someone’s comments using a grep.
// TODO: <the name of some ticket>: <what needs to happen here>
e.g.
// TODO: IOS-42: Vogon construction fleet will need names to be added to this poetry reading room struct
I've not felt my name is all that important for a TODO, as the ticket itself may be taken up by someone else… AFAICT they never have been, but they could have been.
Jira entries get wiped arbitrarily. Git blame may not lie, but it doesn't survive larger organizational "refactoring" around team or company mergers. Or refactoring code out into separate project/library. Hell, often enough it doesn't survive commits that rename bunch of files and move other stuff around.
Comments are decent but flawed. Being a type proponent I think the best strategy is lifting business requirements into the type system, encoding the invariants in a way that the compiler can check.
Thank god we’re held to such low standards. Every time I’ve worked in a field like pharmaceuticals or manufacturing, the documentation burden felt overwhelming by comparison and a shrug six months later would never fly.
We are not engineers. We are craftsmen, instead of working with wood, we work with code. What most customers want is an equivalent of "I need a chair, it should look roughly like this."
If they want blueprints and documentation (e.g. maximum possible load and other limits), we can supply (and do supply, e.g. in pharma or medicine), but it will cost them quite a lot more. By the order of magnitude. Most customers prefer cobbled up solution that is cheap and works. That's on them.
Edit: It is called waterfall. There is nothing inherently wrong with it, except customers didn't like the time it took to implement a change. And they want changes all the time.
Same difference. Both appellations invoke some sort of idealized professional standards and the conversation is about failing these standards not upholding them. We're clearly very short of deserving a title that carries any sort of professional pride in it. We are making a huge mess of the world building systems that hijack attention for profit and generate numerous opportunities for bad agents in the form of security shortfalls or opportunities to exploit people using machines and code.
If we had any sort of pride of craft or professional standards we wouldn't be pumping out the bug ridden mess that software's become and trying to figure out why in this conversation.
Hmm, thinking back, think most companies I worked (from the small to the very large tech companies) had on average pretty good code and automated tests, pretty good processes, pretty good cultures and pretty good architectures. Some were very weak with one aspect, but made up for it others. But maybe I got lucky?
> Both appellations invoke some sort of idealized professional standards
The key point of the comment was that engineers do have standards, both from professional bodies and often legislative ones. Craftsmen do not have such standards (most of them, at least where I am from). Joiners definitely don't.
Edit: I would also disagree with "pumping out bug ridden mess that software's become."
We are miles ahead in security of any other industry. Physical locks have been broken for decades and nobody cares. Windows are breakable by a rock or a hammer and nobody cares.
In terms of bugs, that is extraordinary low as well. In pretty much any other industry, it would be considered a user error, e.g. do not put mud as a detergent into the washing machine.
Whole process is getting better each year. Version control wasn't common in 2000s (I think Linux didn't use version control until 2002). CI/CD. Security analyzers. Memory managed/safe languages. Automatic testing. Refactoring tools.
We somehow make hundreds of millions of lines of code work together. I seriously doubt there is any industry that can do that at our price point.
> We are miles ahead in security of any other industry. Physical locks have been broken for decades and nobody cares. Windows are breakable by a rock or a hammer and nobody cares.
That is not such a great analogy, in my opinion. If burglars could remotely break into many houses in parallel while being mostly non-trackable and staying in the safety of their own home, things would look differently on the doors and windows front.
The reason why car keys are using chips is because physical safety sucks so much in comparison with digital.
The fact is we are better at it because of failure of state to establish the safe environment. Generally protection and safe environment is one of reason for paying taxes.
> The reason why car keys are using chips is because physical safety sucks so much in comparison with digital.
Not the reason. There is no safe lock, chip or not. You can only make it more inconvenient then the next car to break in.
> The fact is we are better at it because of failure of state to establish the safe environment. Generally protection and safe environment is one of reason for paying taxes.
Exactly backwards. The only real safety is being in a hi-sec zone protected by social convention and State retribution. The best existing lock in a place where bad actors have latitude won't protect you, and in a safe space you barely need locks at all.
OTOH, the level of documentation you get for free from source control would be a godsend in other contexts: the majority of the documentation you see in other processes is just to get an idea of what changed when and why.
Most software work in pharma and manufacturing is still CRUD, they just have cultures of rigorous documentation that permeates the industry even when it's low value. Documenting every little change made sense when I was programming the robotics for a genetic diagnostics pipeline, not so much when I had to write a one pager justifying a one line fix to the parser for the configuration format or updating some LIMS dependency to fix a vulnerability in an internal tool that's not even open to the internet.
Well, a hand watch or a chair cannot kill people, but the manufacturing documentation for them will be very precise.
Software development is not engineering because it is still relatively young and immature field. There is a joke where a mathematician, a physicist and a engineer are given a little red rubber ball and asked to find its volume. The mathematician measures the diameter and computes, the physicist immerses the ball into water and sees how much was displaced, and an the engineer looks it up in his "Little red rubber balls" reference.
Software development does not yet have anything that may even potentially grow into such a reference. If we decide to write it we would not even know where to start. We have mathematicians who write computer science papers; or physicists who test programs; standup comedians, philosophers, everyone. But not engineers.
Difference is that code is the documentation and design.
That is problem where people don’t understand that point.
Runtime and running application is the chair. Code is design how to make “chair” run on computer.
I say in software development we are years ahead when it comes to handling complexity of documentation with GIT and CI/CD practices, code reviews and QA coverage with unit testing of the designs and general testing.
So I do not agree that software development is immature field. There are immature projects and companies cut corners much more than on physical products because it is much easier to fix software later.
> Oh, and in 6 months the unintuitive and confusing concept needs to be completely changed into - surprise, surprise - a completely different but equally unintuitive and confusing concept.
But you have to keep the old way of working exactly the same, and the data can't change, but also needs to work in the new version as well. Actually show someone there's two modes, and offer to migrate their data to version 2? No way - that's confusing! Show different UI in different areas with the same data that behaves differently based on ... undisclosed-to-the-user criteria. That will be far less confusing.
In many problem spaces, software developers are only happy with interfaces made for software developers. This article diving into the layers of complex logic we can reason about at once perfectly demonstrates why. Developers ‘get’ that complexity, because it’s our job, and think about GUIs as thin convenience wrappers for the program underneath. To most users, the GUI is the software, and they consider applications like appliances for solving specific problems. You aren’t using the refrigerator, you’re getting food. You’re cooking, not using the stove. The fewer things they have to do or think about to solve their problem to their satisfaction, the better. They don’t give a flying fuck about how software does something, probably wouldn’t bother figuring out how to adjust it if they could, and the longer it takes them to figure out how to apply their existing mental models UI idioms to the screen they’re looking at, the more frustrated they get. Software developers know what’s going on behind the scenes so seeing all of the controls and adjustments and statuses and data helps developers orient themselves save figure out what they’re doing. Seeing all that stuff is often a huge hindrance to users that just have a problem they need to solve, and have a much more limited set of mental models and usage idioms they need to use figuring how which of those buttons to press and parameters to adjust. That’s the primary reason FOSS has so few non-technical users.
The problem comes in when people that aren’t UI designers want to make something “look designed” so they start ripping stuff out and moving it around without understanding how it works affect different types of users. I don’t hear too many developers complain about the interface for iMessage for example despite having a fraction of the controls visible at any given time, because it effectively solves their problem, and does so easier than with a visible toggle for read receipts, SMS/iMessages, text size, etc etc etc. It doesn’t merely look designed, it it’s designed for optimal usability.
Developers often see an interface that doesn’t work well for developers usage style, assume that means it doesn’t work well, and then complain about it among other developers creating an echo chamber. Developers being frustrated with an interface is an important data point that shouldn’t be ignored, but our perspectives and preferences aren’t nearly as generalizable some might think.
I'm not particularly bothered by non-developer UI. I'm bothered by the incessant application of mobile UI idioms to desktop programs (remember when all windows programs looked somewhat similar?), by UI churn with no purpose, by software that puts functionality five clicks deep for no reason other than to keep the ui 'minimal', by the use of unclear icons when there's room for text (worse, when it's one of the bare handful of things with a universally-understood icon and they decided to invent their own), by UIs that just plain don't present important information for fear of making things 'busy'. There's a lot to get mad about when it comes to modern UIs without needing to approach it from a software developer usage style perspective.
You're making a lot of assumptions about who's doing what, what problems they're trying to solve by doing it, and why. The discipline of UI design is figuring out how people can solve their problems easily and effectively. If you have advanced users that need to make five mouse clicks to perform an essential function, that's a bad design and the chance of that being a UI design decision is just about zero. Same thing with icons. UI design, fundamentally, is a medium of communication: do you think it's more likely a UI designer-- a professional and likely educated interactivity communicator-- chose those icons, or a developer or project manager grabbing a sexy looking UI mockup on dribble and trying to smash their use case into it?
Minimalism isn't a goal-- it's a tool to make a better interface and can easily be overused. The people that think minimalism is a goal and will chop out essential features to make something "look designed" are almost always developers. Same thing with unclear icons. As someone with a design degree that's done UI design but worked as a back-end developer for a decade before that, and worked as a UNIX admin off and on for a decade before that, I am very familiar with the technical perspective on design and it's various echo-chamber-reinforced follies.
It's not like all UI designers are incredibly qualified or don't underestimate the importance of some particular function within some subset of users, and some people that hire designers don't realize that a graphic designer isn't a UI designer and shouldn't be expected to work as one. But 700 times out of 1000, that's something dev said "this is too annoying to implement" or some project manager dropped it from the timeline. Maybe 250 of those remaining times, the project manager says "we don't need designers for this next set of features, right? Dev can just make it look like the other parts of the project?"
Developers read an edward tufte book, think they're experts, and come up with all sorts of folk explanations about what's happening with a design and why people are doing it, then talk about it in venues like this with a million other developers agreeing with them. That does a whole lot more damage to UIs in the wild than bad design decisions made by designers.
You seem to think I'm attacking UI designers. I'm not. I think software would be a lot better with professional UI designers designing UIs.
edit: I am making a lot of assumptions. I'm assuming that most UIs aren't really designed, or are 'designed' from above with directions that are primarily concerned about aesthetics.
+1 to all this. And when did it become cool to have icons that provide no feedback they've been clicked, combined with no loading state? I'm always clicking stuff twice now because I'm not sure I even clicked it the first time.
> That’s the primary reason FOSS has so few non-technical users.
Yeah, citation needed. If your argument that 'non-technical users' (whatever that is - being technical is not restricted to understanding computers and software deeply) don't use software that exposes a lot of data on its internals as exemplified by FOSS having few 'non-technical users' meaning people who are not software developers, this is just false. There are entire fields where FOSS software is huge. GIS comes to mind.
Normally in this rant I specifically note that non-software technical people are still technical. For genuinely non-technical software, what are the most popular end-user facing FOSS-developed applications? Firefox, signal, blender, Inkscape, Krita maybe… most of those are backed by foundations that pay designers and in Mozilla’s case, actually do a ton of open usability research. I don’t believe Inkscape does but they do put a ton of effort into thinking about things from the user workflow perspective and definitely do not present all of the functionality to the user all at once. Blender, at first, just made memorize a shitload of shortcuts but they’ve done a ton of work figuring out what users need to see in which tasks in different workflows and have a ton of different purpose-built views. For decades, Gimp treated design, workflow and UI changes like any other feature and they ended up with a cobbled-together ham fisted interface used almost exclusively by developers. You’ll have a hard time finding a professional photographer that hasn’t tried gimp and an even harder time finding one that still uses it because of the confusing, unfocused interface. When mastodon stood a real chance of being what Bluesky is becoming, I was jumping up and down flailing my arms trying to get people to work on polishing the user flow and figure out how to communicate what they needed to know concisely. Dismissal dismissal dismissal. “I taught my grandmother how federation works! They just need to read the documentation! Once they start using it they’ll figure it out!” Well, they started using it, didn’t have that gifted grandmother-teaching developer to explain it to them, and they almost all left immediately afterwards.
Just like human factors engineering, UI design is a unique discipline that many in the engineering field think they can intuit their way through. They’re wrong and if you look beyond technical people, it’s completely obvious.
I'm trying to learn acceptance: how not to get so angry at despicable UIs.
Although I admit I'm kinda failing. My minor successes have been by avoiding software: e.g. giving up programming (broken tools and broken targets were a major frustration) and getting rid of Windows.
IMO the fact that code tends to become hard over time in the real world, is even more reason to lower cognitive load. Because cognitive load is related to complexity. Things like inheritance make it far too easy to end up with spaghetti. So if it's not providing significant benefit, god damn don't do it in the first place (like the article mentions).
That depends on who thinks it's going to be a significant benefit - far far too many times I've had non-technical product managers yelling about some patch or feature or whatever with a "just get it done" attitude. Couple that with some junior engineering manager unwilling to push back, with an equally junior dev team and you'll end up with the nasty spaghetti code that only grows.
Sounds like a bunch of excellent excuses why code is not typically well factored. But that all just seems to make it more evident that the ideal format should be more well-factored.
>It's quite easy to imagine a well factored codebase where all things are neatly separated.
If one is always implementing new code bases that they keep well factored, they should count their blessings. I think being informed about cognitive load in code bases is still very important for all the times we aren't so blessed. I've inherited applications that use global scope and it is a nightmare to reason though. Where possible I improve it and reduce global scope, but that is not always an option and is only possible after I have reasoned enough about the global scope to feel I can isolate it. As such, letting others know of the costs is helpful to both reduce it from happening and to convince stakeholders of the importance of fixing it after it has happened and accounting for the extra costs it causes until it is fixed.
>The messy stuff is where the real world concepts need to be transformed into code.
I also agree this can be a messy place, and on a new project, it is messy even when the code is clean because there is effectively a business logic/process code base you are inheriting and turning into an application. I think many of the lessons carry over well as I have seen an issue with global scope in business processes that cause many of the same issues as in code bases. When very different business processes end up converging into one before splitting again, there is often extra cognitive load created in trying to combine them. A single instance really isn't bad, much like how a single global variable isn't bad, but this is an anti-pattern that is used over and over again.
One helpful tool is working ones way up to the point of having enough political power and earned enough respect for their designs to have suggestions of refactoring business processes be taken into serious consideration (one also has to have enough business acumen to know when such a suggestion is reasonable).
>the original author doesn't work here anymore so no one's here to explain the original code's intent.
I fight for comments that tell me why a certain decision is made in the code. The code tells me what it is doing, and domain knowledge will tell most of why it is doing the things expected, but anytime the code deviates from doing what one would normally expect to be done in the domain, telling me why it deviated from expected behavior is very important for when someone is back here reading it 5+ years later when no one is left from the original project. Some will suggest putting it in documentation, but I find that the only documentation with any chance of being maintained or even kept is the documentation built into the code.
The "why" is the hardest part. You are writing to a future version of most probably a different person with a different background. Writing all is as wrong as writing nothing. You have to anticipate the questions of the future. That takes experience and having been in different shoes, "on the receiving side" of such a comment. Typically developers brag what they did, not why, especially the ones who think they are good...
Not necessarily. There are a lot of domains where you're digitizing decades of cobbled together non-computer systems, such as law, administration, or accounting. There's a very good chance that no single human understands those systems either, and that trying to model them will inevitably end up with obscure code that no one will ever understand either. Especially as legislation and accounting practices accrete in the future, with special cases for every single decision.
Plus to everything said. It's an everyday life of "maintainer", picking the next battle to pick the best way to avoid sinking deeper and defending the story that exactly "this" is the next refactoring project. All that while balancing different factors as you mention to actually believe oneself, because there are countless of paths..
We use it to automatically instrument code for tracing. Stuff like this is IMO the only acceptable use to reduce boiler-plate but quickly becomes terrible if you don't pay attention.
Also good for having default activities performed on object or subsystem. For instance, by default, always having an object have security checks to make sure it has permission to perform the tasks it should be (have seen this, and sounds like a good idea at least). And also, to have some basic logging performed to show when you've entered and left function calls.
It's easy to forget to add these to a function, especially with large codebase with lots of developers
This puts things really well. I’ll add into it that between the first white boarding session and the first working MVP there’ll be plenty of stakeholders who change their mind, find new info, or ask for updates that may break the original plan
I am so proud and happy, when I can make a seemingly complicated change quickly, because the architecture was well designed and everthing neatly seperated.
Most of the time though, it is exactly like you described. Or randalls good code comic:
Allmost too painful to be funny, when you know the pain is avoidable in theory.
Still, it should not be an excuse to be lazy and just write bad code by default. Developing the habit of making everything as clean, structured and clear as possible allways pays of. Especially if that code, that was supposed to be a quick and dirty throw away code experiment somehow ended up being used and 2 years later you suddenly need to debug it.
(I just experienced that joy)
I mean really nobody wants an app that is slow, hard to refactor, with confusing business logic etc. Everyone wants good proporties.
So then you get into what you’re good at. Maybe you’re good at modeling business logic (even confusing ones!). Maybe you’re good at writing code that is easy to refactor.
Maybe you’re good at getting stuff right the first time. Maybe you’re good at quickly fixing issues.
You can lean into what you’re good at to get the most bang for your buck. But you probably still have some sort of minimum standards for the whole thing. Just gotta decide what that looks like.
> you also need to write it in a convoluted way because, for various annoying reasons, that's what performs best on the computer.
That's nothing to do with hardware. The various annoying reasons are not set in stone or laws of physics. They are merely the path dependency of decades of prioritizing shipping soon because money.
Assuming you're just referring to repos: not really IMO.
As soon as you split 1 repo into 2 repos you need to start building tooling to support your 2 repos. If your infrastructure is sufficiently robust with 2 repos then you might as well have 3 or 4 or 10. If it's built to _only_ support 2 repos (or 3 or 4) then it's brittle out of the gate.
The value of a monorepo is that you completely eliminate certain classes of problems and take on other classes of problems. Classic trade off. Folks that prefer monorepos take the position that multirepo problems are much harder than monorepo problems most of the time.
> As soon as you split 1 repo into 2 repos you need to start building tooling to support your 2 repos.
No, not really.
If you're talking about projects for modules and components, all you need is a versioning strategy and release consumable packages of your projects.
If you're talking about services, all you need to do is support versioned APIs and preserve your contracts.
No tooling required. For projects you can even make do with git submodules. For services, all you need to do is update clients of your downstream dependencies.
If you aren't using a monorepo, you need some versioning process, as well as procedural systems in place to ensure that everyone's dependencies, stay reasonably up to date. Otherwise, you end up deferring pain in really unwanted ways, and require sudden, unwanted upgrades through api incompatibility due to external pressure.
This also has the downside of allowing api-owning teams to make changes willy-nilly and break backwards compatibility because they can just do it behind SemVer, and then clients of the api need to own the process of migrating to the new version.
A monorepo fixes both of these: you cannot get out of sync, so it is the api-owning team's responsibility to upgrade clients, since they can't break the API otherwise. Similarly, you get a versioning process for free, and clients can never be using out of date or out of support versions of a dependency.
Services work approximately the same either way, since you can't assume synchronous upgrades across service/rpc boundaries anyway.
The classic 0, 1, or N. You can have 0 repos (no-code or outsourced or just some unversioned blob of code). You can have 1 repo (a mono-repo). Or you can have N number of repos (individual micro services). Limiting yourself to a number that's higher than 1 but still finite is just asking for pain. (Of course implementation limits can apply but there shouldn't be logical limits)
Hard disagree. I've found that the best setup is two monorepos - one with feature/app code, and one with operations configuration.
The fundamental mismatch is the feature/app code will have longer testing times, stuff like Puppeteer and creating-infra-per-MR that just fundamentally takes a long time to run. But in ops, you need configuration to roll out quickly, maybe run an autoformatter and a linter or two beforehand and that's it. When you want to rollback, you don't need to wait for all your tests to run.
Yes you need to deal with versioning. You can just use automatic timestamps. You can write easy automation to push new timestamps to the ops/config repository when there are new releases in the app repository. The ops repository has minimal configuration to pull the latest version of the app repository and apply it, where the app repository includes infra and deployment scripts themselves.
The problem you describe with waiting for tests for unrelated code changes (or potentially related but you want to skip it for the reasons you describe) is a problem that seems to deserve a solution even within a single monorepo.
What are the solutions? You would need multiple merge queues.
There isn't really a solution that preserves the vision and advantages of a monorepo. If infra, config, and code are stored in the same repository, then rolling back one necessarily means that the others are supposed to have been rolled back as well. When the version of the code is not a simple identifier pointing to the code but the code itself, there's simply no way to do that safely without recompiling and retesting the code. Same as when a change to code results in a revert to the infrastructure - run plan, run apply, wait for the slow infrastructure APIs, and that must happen before the change to code.
If you try to come up with fancy rules like forbidding changes to infra, config, and code in the same commit, then you're just cramming a polyrepo design into a monorepo, with much more fragility, because commits get rebased and squashed and etc.
Do you not just write code that’s backwards compatible?e.g an api deployment should not break the front end. The api changes should work on the old front end and the updated front end that uses the api change.
As a programmer of over 20 years - this is terrifying.
I'm willing to accept that I just have "get off my lawn" syndrome or something.
But the idea of letting an LLM write/move large swaths of code seems so incredibly irresponsible. Whenever I sit down to write some code, be it a large implementation or a small function, I think about what other people (or future versions of myself) will struggle with when interacting with the code. Is it clear and concise? Is it too clever? Is it too easy to write a subtle bug when making changes? Have I made it totally clear that X is relying on Y dangerous behavior by adding a comment or intentionally making it visible in some other way?
It goes the other way too. If I know someone well (or their style) then it makes evaluating their code easier. The more time I spend in a codebase the better idea I have of what the writer was trying to do. I remember spending a lot of time reading the early Redis codebase and got a pretty good sense of how Salvatore thinks. Or altering my approaches to code reviews depending on which coworker was submitting it. These weren't things I were doing out of desire but because all non-trivial code has so much subtlety; it's just the nature of the beast.
So the thought of opening up a codebase that was cobbled together by an AI is just scary to me. Subtle bugs and errors would be equally distributed across the whole thing instead of where the writer was less competent (as is often the case). The whole thing just sounds like a gargantuan mess.
> The more time I spend in a codebase the better idea I have of what the writer was trying to do.
This whole thing of using LLMs to Code reminds me a bit of when Google Translate came out and became popular, right around the time I started studying Russian.
Yes, copying and pasting a block of Russian text produced a block of english text that you could get a general idea of what was happening. But translating from english to russian rarely worked well enough to fool the professor because of all the idioms, style, etc. Russian has a lot of ways you can write "compactly" with fewer words than english and have a much more precise meaning of the sentence. (I always likened russian to type-safe haskell and english to dynamic python)
If you actually understood Russian and read the text, you could uncover much deeper and subtle meaning and connections that get lost in translation.
If you went to russia today you could get around with google translate and people would understand you. But you aren't going to be having anything other than surface level requests and responses.
Coding with LLMs reminds me a lot of this. Yes, they produce something that the computer understands and runs, but the meaning and intention of what you wanted to communicate gets lost through this translation layer.
Coding is even worse because i feel like the intention of coding should never to be to belt out as many lines as possible. Coding has powerful abstractions that you can use to minimize the lines you write and crystalize meaning and intent.
This presumes that it will be real humans that have to “take care” of the code later.
A lot of the people that are hawking AI, especially in management, are chasing a future where there are no humans, because AI writes the code and maintains the code, no pesky expensive humans needed. And AI won’t object to things like bad code style or low quality code.
I think this is where we lose a lot of developers. My experience has been this is a skill set that isn’t as common as you’d hope for, and requires some experience outside developing software as its own act. In other words, this doesn’t seem to be a skill that is natural to developers who haven’t had (or availed themselves of) the opportunity to do the business analyst and requirements gathering style work that goes into translating business needs to software outcomes. Many developers are isolated (or isolate themselves) from the business side of the business. That makes it very difficult to be able to translate those needs themselves. They may be unaware of and not understand, for example, why you’d want to design an accounting feature in a SaaS application in a certain way to meet a financial accounting need.
On the flip side, non-technical management tends to underestimate and undervalue technical expertise either by ego or by naïveté. One of my grandmothers used to wonder how I could “make any money by playing on the computer all day,” when what she saw as play was actually work. Not video games, mind you. She saw computers as a means to an end, in her case entertainment or socializing. Highly skilled clients of mine, like physicians, while curious, are often bewildered that there are sometimes technical or policy limitations that don’t match their expectations and make their request untenable.
When we talk about something like an LLM, it simply doesn’t possess the ability to reason, which is precisely what is needed for that kind of work.
> 2. Letting the Definition of Done Include Customer Approval
In the olden days we used to model user workflows. Task A requires to do that and that, we do this and this and transition to workflow B. Acceptance testing was integral step of development workflow and while it did include some persuasion and deception, actual user feedback was part of the process.
As much as scrum tries to position itself as delivering value to the customer, the actual practices of modeling the customer, writing imagined user stories for them and pulling acceptance criteria out of llama's ass ensures that actual business and development team interact as little as possible. Yes, this does allow reduce the number of implemented features by quite a bunch, however by turning the tables and fitting a problem inside the solution.
Think about it, it is totally common for website layouts to shift, element focus to jump around as the website is loading or more recently two step login pages that break password managers. No sane user would sign these off, but no actual user has participated in the development process, neither at design phase, nor at testing phase.
You know this future isn't happening anytime soon. Certainly not in the next 100 years. Until then, humans will be taking care of it and no one will want to work at a place working on some Fransketeinian codebase made via an LLM. And even when humans are only working on 5% of the codebase, that will likely be the most critical bit and will have the same problems regarding staff recruitment and retention.
I think this is a bit short sighted, but I’m not sure how short. I suspect in the future, code will be something in between what it is today, and a build artifact. Do you have to maintain bytecode?
> Russian has a lot of ways you can write "compactly" with fewer words than english and have a much more precise meaning of the sentence. (I always likened russian to type-safe haskell and english to dynamic python)
Funny; my experience has been completely the opposite. I've always envied the English language for how compactly and precisely it can express meaning compared to Russian, both because of an immensely rich vocabulary, and because of the very flexible grammar.
I suspect this difference in perception may be due to comparing original texts, especially ones produced by excellent writers or ones that have been polished by generations that use them, to translations, which are almost invariably stylistically inferior to the original: less creative, less playful, less punchy, less succinct. So, if you translate a good Russian writer who is a master of his craft into English, you may feel the inadequacy of the language. Likewise, whenever I try to read translations of English prose into Russian, it reads clumsy and depressingly weak.
Translating is an interpretation of the original text. A translated book can be better than the original. But you often need mastery of the language you translate to.
> A translated book can be better than the original.
Can you give some examples?
> But you often need mastery of the language you translate to.
Professional written translation is virtually always done into your native language, not into a language you've learned later. So that mastery should be taken for granted; it's a prerequisite.
Coding isn't the hard part. The hard part is translating the business needs in code.
You can tell a junior programmer "Make a DB with tables book, author, has Written, customer, stock, hasBought, with the following rules between them. Write a repository, for that DB. Use repository in BooksService and BasketService. Use those services in Books controller and Basket controller." and he will do a fine job.
Ask the junior to write an API for a book store and he will have a harder time.
I write in Clojure and "coding" is perhaps 10% of what I do? There is very little "plumbing", or boilerplate, most of the code directly addresses the business domain. "AI" won't help me with that.
This is a great analogy. I find myself thinking that by abstracting the entire design process when coding something using generative AI tools, you tend to lose track of fine details by only concentrating on the overall function.
Maybe the code works, but does it integrate well with the rest of the codebase? Do the data structures that it created follow the overall design principles for your application? For example, does it make the right tradeoffs between time and space complexity for this application? For certain applications, memory may be an issue and while code the may work, it uses too much memory to be useful in practice.
These are the kind of problems that I think about, and it aligns with your analogy. There is in fact something "lost through this translation layer".
I think translating to russian wasn't worse than translating to English because "russian is more compact".
Probably it was worse, because people in charge in Google speak English. It was embarrassing to watch Google conferences where they proposed Google Translate to translate professional products. It's similarly embarrassing watching people proposing chatGPT lightly, because they lack the ability, or probably just don't care to, analyze the problem thoroughly.
I was helping translate some UI text for a website from English to German, my mother tongue. I found that usually the machine came up with better translations than me.
English and German are EU languages. Russian is not.
The EU maintains a large translation service to translate most EU official texts into all EU languages. So Google Translate is using that to train on. Google gets a free gift from a multinational bureaucracy and gets to look like a smart company in the process.
This is also why English-Mandarin is often poorly translated, in my opinion.
I am a professional translator, and I have been using LLMs to speed up and, yes, improve my translations for a year and a half.
When properly prompted, the LLMs produce reasonably accurate and natural translations, but sometimes there are mistakes (often the result of ambiguities in the source text) or the sentences don’t flow together as smoothly as I would like. So I check and polish the translations sentence by sentence. While I’m doing that, I sometimes encounter a word or phrase that just doesn’t sound right to me but that I can’t think how to fix. In those cases, I give the LLMs the original and draft translation and ask for ten variations of the problematic sentence. Most of the suggestions wouldn’t work well, but there are usually two or three that I like and that are better than what I could come up with on my own.
Lately I have also been using LLMs as editors: I feed one the entire source text and the draft translation, and I ask for suggestions for corrections and improvements to the translation. I adopt the suggestions I like, and then I run the revised translation through another LLM with the same prompt. After five or six iterations, I do a final read-through of the translation to make sure everything is okay.
My guess is that using LLMs like this cuts my total translation time by close to half while raising the quality of the finished product by some significant but difficult-to-quantify amount.
This process became feasible only after ChatGPT, Claude, and Gemini got longer context windows. Each new model release has performed better than the previous one, too. I’ve also tried open-weight models, but they were significantly worse for Japanese to English, the direction I translate.
Although I am not a software developer, I’ve been following the debates on HN about whether or not LLMs are useful as coding assistants with much interest. My guess is that the disagreements are due partly to the different work situations of the people on both sides of the issue. But I also wonder if some of those who reject AI assistance just haven’t been able to find a suitable interactive workflow for using it.
> While I’m doing that, I sometimes encounter a word or phrase that just doesn’t sound right to me but that I can’t think how to fix. In those cases, I give the LLMs the original and draft translation and ask for ten variations of the problematic sentence. Most of the suggestions wouldn’t work well, but there are usually two or three that I like and that are better than what I could come up with on my own.
Yes, coming up with variations that work better (and hit the right connotations) is what I used the machine for, too.
You don't even have to go back in time or use a comparatively rare language pair such as English/Russian.
Google Reviews insists on auto translating reviews to your native language (thankfully, the original review can be recovered by clicking a link). Even for English->German (probably one of the most common language pairs), the translations are usually so poor that you can't always tell what the review is trying to say. To be fair, I think that state of the art machine translation is better than whatever the hell Google is using here (google translate probably), but apparently product people at Google don't care enough to make translations better (or better, allow you to turn off this feature).
> Russian has a lot of ways you can write "compactly" with fewer words than english and have a much more precise meaning of the sentence. (I always likened russian to type-safe haskell and english to dynamic python)
The difference is in fusionality. English does not use inflection and relies heavily on auxiliaries (pre-, post-positions, particles, other modifiers) while Russian (and other Slavic, Baltic languages) rely rather heavily on inflection.
For English speakers, probably the closest is the gerund. A simple suffix transforms a verb into a noun-compatible form, denoting process. In highly fusional languages a root can be combined with multiple modifying pre-, a-, suf-fixes, and inflected on top. This does unlock some subtlety.
> But the idea of letting an LLM write/move large swaths of code seems so incredibly irresponsible
I heard a similar thing from a dude when I said I use it for bash scripts instead of copying and pasting things off StackOverflow.
He was a bit "get off my lawny" about the idea of running any code you didn't write, especially bash scripts in a terminal.
It is obviously the case that I didn't write most of the code in the world by a very large margin, but even not taking it to extremes if I'm working on a team and people are writing code how is it any different? Everyone makes mistakes, I make mistakes.
I think it's a bad idea to run things that you don't at least understand what it's going to do but the speed with which ChatGPT can produce, for example, gcloud shell commands to manage resources is lightning fast (all of which is very readable, just takes a while if you want to look it up and compose the commands yourself).
If your quality control method is "making sure there are no mistakes" then it's already broken regardless of where the code comes from. Me reviewing AI code is no different from me reviewing anyone else's code.
Me testing AI code using unit or integration tests is no different from testing anyone else's code, or my own code for that matter.
> Me reviewing AI code is no different from me reviewing anyone else's code.
I take your point, and on the whole I agree with your post, but this point is fundamentally _not_ correct, in that if I have a question about someone else's code I can ask them about their intention, state-of-mind, and understanding at the time they wrote it, and (subjectively, sure; but I think this is a reasonable claim) can _usually_ detect pretty well if they are bullshitting me when they respond. Asking AI for explanations tends to lead to extremely convincing and confident false justifications rather than an admission of error or doubt.
However:
> Me testing AI code using unit or integration tests is no different from testing anyone else's code, or my own code for that matter.
> Asking AI for explanations tends to lead to extremely convincing and confident false justifications rather than an admission of error or doubt.
Not always true, AI can realise their own mistakes and they can learn. It's a feedback loop system, and / but as it stands this feedback of what is good and bad is provided by end-users and fed back into e.g. Copilot.
That loop is not a short one though. LLMs don't actively incorporate new information into its model while you're chatting with it. That goes into its context window/short term memory. That the inputs and outputs can be used when training the next model, or for fine tuning the current one doesn't change that the distinct steps of training and inference.
No they can't. They can generate text that indicates they hallucinated, you can tell them to stop, and they won't.
They can generate text that appears to admit they are incapable of doing a certain task, and you can ask them to do it again, and they will happily try and fail again.
Sorry but give us some examples of an AI "realizing" its own mistakes, learning, and then not making the mistake again.
Also, if this were even remotely possible (which it is not), then we should be able to just get AIs with all of the mistakes pre-made, so it learned and not do them again, right? So it has already "realized" and "learned" which tasks it's incapable of, so it will actually refuse or find a different way.
Or is there something special about the way that _you_ show the AI its mistakes, that is somehow more capable of making it "learn" from those mistakes than actually training it?
I'm assuming by bullshitting you mean differentiating between LLM hallucinations and a human with low confidence in their code.
I've found that LLMs do sometimes acknowledge hallucinations. But really the check is much easier than a PR/questioning an author - just run the code given by the copilot and check that it works, just as if you typed it yourself.
> just run the code given by the copilot and check that it works
You've misunderstood my point. I'm not discussing the ability to check whether the code works as _I_ believe it should (as you say, that's easy to verify directly, by execution and/or testing); I'm referring to asking about intention or motivation of design choices by an author. Why this data structure rather than that one? Is this unusual or unidiomatic construction necessary in order to work around a quirk of the problem domain, or simply because the author had a brainfart or didn't know about the usual style? Are we introducing a queue here to allow for easy retries, or to decouple scaling of producers and consumers, or...? I can't evaluate the correctness of a choice without either knowing the motivation for it, or by learning the problem domain well enough to identify and make the choice myself - at which point the convenience of the AI solution is abnegated because I may as well have written it myself.
And, yes, you can ask an LLM to clarify or explain its choices, but, like I said, the core problem is that they will confidently and convincingly lie to you. I'm not claiming that humans never lie - but a) I think (I hope!) they do it less often than LLMs do, and b) I believe (subjectively) that it tends to be easier to identify when a human is unsure of themself than when an LLM is.
> I can't evaluate the correctness of a choice without either knowing the motivation for it, or by learning the problem domain well enough to identify and make the choice myself - at which point the convenience of the AI solution is abnegated because I may as well have written it myself.
I think I usually accept code that is in the latter - the convenience is I did not need to spend any real energy implementing the solution or thinking too deeply about it. Sometimes the LLM will produce a more interesting approach that I did not consider initially but is actually nicer than what I wanted to do (afaik). Often it does what I want or something similar enough to what I would've written - just that it can do it instantly instead of me manually typing, doc searching, adding types, and correcting the code. If it does something weird that I don't agree with, I instead modify the prompt to align closer to the solution I had in mind. Much like Google, sometimes the first query does not do the trick and a query reformulation is required.
I wouldn't trust an LLM to write large chunks of code that I wouldn't have been able to write/figure out myself - it's more of a coding accelerant than an autonomous engineer for me (maybe that's where our PoVs diverged initially).
I suspect the similarity with PRs is that when I'm assigned a PR, I generally have enough knowledge about the proposed modification to have an opinion on how it should be done and the benefits/drawbacks of each implementation. The divergence from a PR is that I can ask the LLM for a modification of approach with just a few seconds and continue to ask for changes until I'm satisfied (so it doesn't matter if the LLM chose an approach I don't understand - I can just ask it to align with the approach I believe is optimal).
Multiple times in my s/w development career, I've had supervisors ask me why I am not typing code throughout the work day.
My response each time was along the lines of:
When I write code, it is to reify the part of a solution which
I understand. This includes writing tests to certify same.
There is no reason to do so before then.
> He was a bit "get off my lawny" about the idea of running any code you didn't write, especially bash scripts in a terminal.
I hacked together a CLI tool that provides an LLM a CRUD interface to my local file system for, letting it read, write, and execute, code and tests, and feeds it back the commands outputs.
And it was bootstrapped with me playing the role of CLI tool.
> if I'm working on a team and people are writing code how is it any different? Everyone makes mistakes, I make mistakes.
because your colleagues know how to count
and they're not hallucinating while on the job
and if they try to slip an unrelated and subtle bug past you for the fifth time after asking them to do a very basic task, there are actual consequences instead of "we just need to check this colleague's code better"
AIs are not able to write Redis. That's not their job. AIs should not write complex high performance code that millions of users rely on. If the code does something valuable for a large number of people you can afford humans to write it.
AIs should write low value code that just repeats what's been done before but with some variations. Generic parts of CRUD apps, some fraction of typical frontends, common CI setups.
That's what they're good at because they've seen it a million times already. That category constitutes most code written.
This relieves human developers of ballpark 20% of their workload and that's already worth a lot of money.
Not the parent but this doesn’t seem mind changing, because what you describe is the normal/boring route to slightly better productivity using new tools without the breathless hype. And the 20% increase you mention of course depends a lot on what you’re doing, so for many types of work you’d be much closer to zero.
I’m curious about the claims of “power users” that are talking very excitedly about a brave new world. Are they fooling themselves, or trying to fool others, or working at jobs where 90% of their work is boilerplate drudgery, or what exactly? Inevitably it’s all of the above.. and some small percentage of real power users that could probably teach the rest of us cool stuff about their unique workflows. Not sure how to find the signal in all the noise though.
So personally, if I were to write “change my mind”, what I’d really mean is something like “convince me there are real power users already out there in the wild, using tools that are open to the public today”.
GP mentioned machine assisted translation of a huge code base being almost completely hands-off. If that were true and as easy as advertised then one might expect, for example, that it were trivial to just rewrite media wiki or Wordpress in rails or Django with a few people in a week. This is on the easier side of what I’d confidently label as a game-changingly huge productivity boost btw, and is a soft problem chosen because of the availability of existing code examples, mere translation over original work, etc. Not sure we’re there yet.
> Are they fooling themselves, or trying to fool others, or working at jobs where 90% of their work is boilerplate drudgery, or what exactly?
I wonder about this also. Maybe it's just some of each? Clearly some people fool themselves, the AI companies are doing marketing, some people do have boring jobs...
I have to disagree. If there’s that much boilerplate floating around then the tooling should be improved. Pasting over inefficiency with sloppier inefficiency is just a pure waste.
I spent may entire career trying to eliminate such code as much as I can, so then having copilot write code that I have to fix on almost every step. I frequently have to look for subtle issues and few times they sneaked through, when it produces correct code it frequently is often more verbose than my code.
In a couple of years time I don't see why AI based tooling couldn't write Redis? Would you get a complete Redis produced with a single prompt? Of course not. but if extreme speed is what you want to optimize for, then the tooling needs to be given the right feedback loop to optimize for that.
I think the question to ask is what do I do as a software engineer that couldn't be done by an AI based tool in a few years time? The answer is scary, but exciting.
I can definitely see the value in letting AI generate low stakes code. I'm a daily CoPilot user and, while I don't let it generate implementations, the suggestions it gives for boilerplate-y things is top notch. Love it as a tool.
My major issue with your position is that, at least in my experience, good software is the sum of even the seemingly low risk parts. When I think of real world software that people rely on (the only type I care about in this context) then it's hard to point a finger at some part of it and go "eh, this part doesn't matter". It all matters.
The alternative, I fear, is 90% of the software we use exhibiting subtle goofy behavior and just being overall unpleasant to use.
I guess an analogy for my concern is what it would look like if 60% of every film was AI generated using the models we have today. Some might argue that 60% of all films are low stakes scenes with simple exposition or whatever. And then remaining 40% are the climax or other important moments. But many people believe that 100% of the film matters - even the opening credits.
And even if none of that were an issue: in my experience it's very difficult to assess what part of an application will/won't be low/high stakes. Imagine being a tech startup that needs to pivot your focus toward the low stakes part of the application that the LLM wrote.
I think your concept of ‘what the AI wrote’ is too large. There is zero chance my one line copilot or three line cursor tab completions are going to have an effect on the overall quality of my codebase.
What it is useful for is doing exactly the things I already know need to happen, but don’t want to spend the effort to write out (at least, not having to do it is great).
Since my brain and focus aren’t killed by writing crud, I get to spend that on more useful stuff. If it doesn’t make me more effective, at least it makes my job more enjoyable.
I'm with you. I use Copilot every day in the way you're describing and I love it. The person I was responding to is claiming to code "hands off" and let the AI write the majority of the software.
> But the idea of letting an LLM write/move large swaths of code seems so incredibly irresponsible.
I do think it is kind of crazy based on what I've seen. I'm convinced LLM is a game changer but I couldn't believe how stupid it can be. Take the following example, which is a spelling and grammar checker that I wrote:
If you click on the sentence, you can see that Claude-3.5 and GPT-4o cannot tell that GitHub is spelled correctly most of the time. It was this example that made me realize how dangerous LLM can be. The sentence is short but Claude-3.5 and GPT-4o just can't process it properly.
Having a LLM rewrite large swaths of code is crazy but I believe with proper tooling to verify and challenge changes, we can mitigate the risk.
I'm just speculating, but I believe GitHub has come to the same conclusion that I have, which is, all models can be stupid, but it is unlikely that all will be stupid at the same time.
I think it depends on the stakes of what you're building.
A lot of the concerns you describe make me think you work in a larger company or team and so both the organizational stakes (maintenance, future changes, tech debt, other people taking it over) and the functional stakes (bug free, performant, secure, etc) are high?
If the person you're responding to is cranking out a personal SaaS project or something they won't ever want to maintain much, then they can do different math on risks.
And probably also the language you're using, and the actual code itself.
Porting a multi-thousand line web SaaS product in Typescript that's just CRUD operations and cranking out web views? Sure why not.
Porting a multi-thousand line game codebase that's performance-critical and written in C++? Probably not.
That said, I am super fascinated by the approach of "let the LLM write the code and coach it when it gets it wrong" and I feel like I want to try that.. But probably not on a work project, and maybe just on a personal project.
> Porting a multi-thousand line web SaaS product in Typescript that's just CRUD operations and cranking out web views? Sure why not.
>
> Porting a multi-thousand line game codebase that's performance-critical and written in C++? Probably not.
From my own experience:
I really enjoy CoPilot to support me writing a terraform provider. I think this works well because we have hundreds of existing terraform providers with the same boilerplate and the same REST-handling already. Here, the LLM can crank out oodles and oodles of identical boilerplate that's easy to review and deal with. Huge productivity boost. Maybe we should have better frameworks and languages for this, but alas...
I've also tried using CoPilot on a personal Godot project. I turned it off after a day, because it was so distracting with nonsense. Thinking about it along these lines, I would not be surprised if this occurred because the high-level code of games (think what AAA games do in Lua, and well what Godot does in GDScript) tends to be small-volume and rather erratic within there. Here there is no real pattern to follow.
This could also be a cause for the huge difference in LLM productivity boosts people report. If you need Spring Boot code to put query params into an ORM and turn that into JSON, it can probably do that. If you need embedded C code for an obscure micro controller.. yeah, good luck.
> If you need embedded C code for an obscure micro controller.. yeah, good luck.
... or even information in the embedded world. LLMs need to generate something, o they'll generate code even when the answer is "no dude, your chip doesn't support that".
> they'll generate code even when the answer is "no dude, your chip doesn't support that".
This is precisely the problem. As I point out elsewhere[0], reviewing AI-generated code is _not_ the same thing as reviewing code written by someone else, because you can ask a human author what they were thinking and get a moderately-honest response; whereas an AI will confidently and convincingly lie to you.
I am quite interested in how LLMs would handle game development. Coming to game development from a long career in boutique applications and also enterprise software, game development is a whole different level of "boutique".
I think both because of the coupled, convoluted complexity of much game logic, and because there are fewer open source examples of novel game code available to train on, they may struggle to be as useful.
It is a good example of how we are underestimating the human in the loop.
I know nothing about making a game. I am sure LLMs could help me try to make a game but surely they would help someone who has tried to make a game before more. On the other hand, the expert game developer is probably not helped as much either by the LLM as the person in the middle.
Scale that to basically all subjects. Then we get different opinions on the value of LLMs.
Yeah I think the lack of game code available to train on could be a problem. There's a fair amount of "black art" type problems in games too that a LLM may struggle with just because there's not a lot to go on.
Additionally the problems of custom engines and game specific patterns.
That being said there's parts of games with boilerplate code like any application. In a past game as I was finishing it up some of this AI stuff was first becoming useable and I experimented with generating some boilerplate classes with high level descriptions of what I wanted and it did a pretty decent job.
I think some of the most significant productivity gains for games is going to be less on the code side and more in the technical art space.
> A lot of the concerns you describe make me think you work in a larger company or team and so both the organizational stakes (maintenance, future changes, tech debt, other people taking it over) and the functional stakes (bug free, performant, secure, etc) are high?
The most financially rewarding project I worked on started out as an early stage startup with small ambitions. It ended up growing and succeeding far beyond expectations.
It was a small codebase but the stakes were still very high. We were all pretty experienced going into it so we each had preferences for which footguns to avoid. For example we shied away from ORMs because they're the kind of dependency that could get you stuck in mud. Pick a "bad" ORM, spend months piling code on top of it, and then find out that you're spending more time fighting it than being productive. But now you don't have the time to untangle yourself from that dependency. Worst of all, at least in our experience, it's impossible to really predict how likely you are to get "stuck" this way with a large dependency. So the judgement call was to avoid major dependencies like this unless we absolutely had to.
I attribute the success of our project to literally thousands of minor and major decisions like that one.
To me almost all software is high stakes. Unless it's so trivial that nothing about it matters at all; but that's not what these AI tools are marketing toward, are they?
Something might start out as a small useful library and grow into a dependency that hundreds of thousands of people use.
So that's why it terrifies me. I'm terrified of one day joining a team or wanting to contribute to an OSS project - only to be faced with thousands of lines of nonsensical autogenerated LLM code. If nothing else it takes all the joy out of programming computers (although I think there's a more existential risk here). If it was a team I'd probably just quit on the spot but I have that luxury and probably would have caught it during due diligence. If it's an OSS project I'd nope out and not contribute.
adding here due to some resonance with the point of view.. this exchange lacks crucial axes.. what kind of programming ?
I assume the parent-post is saying "I ported thousands of lines of <some C family executing on a server> to <python on standard cloud environments>. I could be very wrong but that is my guess. Like any data-driven software machinery, there is massive inherent bias and extra resources for <current in-demand thing> in this guess-case it is python that runs on a standard cloud environment with the loaders and credentials parts too perhaps.
Those who learned programming in the theoretic ways know that many, many software systems are possible in various compute contexts. And those working on hardware teams know that there are a lot of kinds of computing hardware. And to add another off-the-cuff idea, so much web interface ala 2004 code to bring to newer, cleaner setups.
I am not <emotional state descriptor> about this sea change in code generation, but actually code generation is not at all new. It is the blatent stealing and LICENSE washing of a generation of OSS that gets me, actually. Those code generation machines are repeating their inputs. No authors agreed and no one asked them, either.
You still use type systems, tests, and code review.
For a lot of use cases it's powerful.
If you ask it to build out a brand new system with a complex algorithm or to perform a more complex refactoring, it'll be more work correcting it than doing it yourself.
But that malformed JSON document with the weird missing quotation marks (so the usual formatters break), and spaces before commas, and the indentation is wild... Give it to an LLM.
Or when you're writing content impls for a game based on a list of text descriptions, copy the text into a block comment. Then impl 1 example. Then just sit back and press tab and watch your profits.
The (mostly useless boilerplate “I’m basically just testing my mocks”) tests are being written by AI too these days.
Which is mildly annoying as a lot of those tests are basically just noise rather than useful tools. Humans have the same problem, but current models are especially prone to it from what I’ve observed
And not enough devs are babysitting the AI to make sure the test cases are useful, even if they’re doing so for the original code it produced
There are very few tutorials on how to do testing and I don't think I have ever seen one that was great. Compared to general coding stuff where there's great tutorials available for all the most common things.
So I think quality testing is just not in the training data at anywhere close to the quantity needed.
Testing well is both an art and a science, and I mean, just look at the dev community on the topic, some are religious about TDD, some say unit tests only, some say the whole range to e2e etc. etc. hard to have good training data when there is no definition of what is "right" in the first place!
> I think about what other people (or future versions of myself) will struggle with when interacting with the code.
This feels like the sign of a good developer!
On the other hand, sometimes you just need executable line noise that gets the job done by Thursday so you can ship it and think about refactoring later.
As far as AI code goes, more often than not, it will read as something very generic, which is not necessarily a bad thing. When opening yet another Java CRUD project, I’d be more happy to see someone copy and pasting working code from tutorials or resources online (while it still works correctly), as opposed to seeing people develop bespoke systems on top of what a framework provides for every project.
> On the other hand, sometimes you just need executable line noise that gets the job done by Thursday so you can ship it and think about refactoring later.
This is a problem too. ChatGPT enables you to write bad code.
It's like Adobe Flash: flash using websites didn't have to be slow, but it was easy to make a slow website with it.
Imagine the LLM is another developer and you're responsible for reviewing their code. Would you think of them the same thing?
While I don't like AI either, I feel a lot of the fear around it is just that - fear and distrust that there will be bugs, some more subtle than others. But that's true for code written by anyone, isn't it?
It's just that you're responsible for the code written by something or someone else, and that can be scary.
But remember that no code should make it to production without human and automated review. Trust the system.
> Whenever I sit down to write some code, be it a large implementation or a small function, I think about what other people (or future versions of myself) will struggle with when interacting with the code. Is it clear and concise? Is it too clever? Is it too easy to write a subtle bug when making changes? Have I made it totally clear that X is relying on Y dangerous behavior by adding a comment or intentionally making it visible in some other way?
Over 20 years of experience, too, but I quit doing that for work. Nobody really really cares, all they care is about time to market and having features they've sold yesterday to customers being done today.
As long as I follow some mental models and some rules, the code is reasonably well written and there is no need to procrastinate and think too much.
When I write code for myself, or I am contributing to a small project with a small number of contributors, then things change. If I can afford and I like it I am not only willing to assure things are carefully thought out, but also I am willing to experiment and test until I am sure that I use the best variant I can come up with. Like going from 99% to 99.9%, even if it wouldn't matter in practice. Just for fun.
As a manager, I wouldn't ask people to write perfect code, nor I would like them to ship buggy code very fast, but ship reasonably good code as fast as they can write reasonable good code.
> Over 20 years of experience, too, but I quit doing that for work. Nobody really really cares, all they care is about time to market and having features they've sold yesterday to customers being done today.
I don't recognise this.
Or at least, I recognise that it can be that way but not always. In places I've worked, I tend to have worked with teams that care deeply about this. But we're not writing CRUD apps or web systems, or inventory management, or whatever. We're writing trading systems. I absolutely want to be working with code that we can understand in a hurry (and I mean, a real hurry) when things go wrong, and that we can change and/or fix in a hurry.
If you write critical systems executing trades, managing traffic lights, landing a rover on the moon, then you should take your time and write the best possible version.
Our code is both easy to read and easy to modify because that allows us to add features fast. It is not the very best possible version of what we can do, because that would cost us much more time.
The code has few bugs, which are mostly caught by the QA teams, is reasonably fast. Maybe not the most elegant, not engineered to take into account future use cases and we push to eliminate from AC some very rare use cases,that will take too much time to implement. Maybe the code it's not the most resource efficient.
But the key aspect is we focus on delivering the most features possible from what customers need in the limited amount of time we have and with the limited manpower we have.
Company is owned by some private equity group and their focus is solely growing the customer base while paying as little as possible. Last year they fired 25% of personnel because their ARR was missing a few millions.
Newertheless, most companies I worked before were in the hurry. With the exception of a very small company where I could work however I see fit.
> But the idea of letting an LLM write/move large swaths of code seems so incredibly irresponsible. Whenever I sit down to write some code, be it a large implementation or a small function, I think about what other people (or future versions of myself) will struggle with when interacting with the code. Is it clear and concise? Is it too clever?
I agree with our conclusion but not your supporting evidence. Not only can you read it to answer all these questions, but you can BETTER answer these questions from reading it. Because you are already looking at it from the perspective you are trying optimize for (future reader).
What is less clear is if it handles all the edge cases correctly. In theory these should all be tested, but many of them cannot even be identified without thinking through the idiosyncrasies of the code which is a natural byproduct of writing code.
It's like worrying about moving bits on a hard drive, or writing nice machine code. Eventually you just won't care. AI / LLMs interacting with code bases in future won't care about structure, clearness, conciseness etc. They'll tokenize it all the same.
Unit, integration, e2e, types and linters would catch most of the things you mention.
Not every software is mission critical, often the most important thing is to go as fast and possible and iterate very quickly. Good enough is better than very good in many cases.
Lots of people. For certain types of software (ISO) they are required.
But I'm in the boat (and also experienced many times first hand) all those tests you write will by definition, never test against that first production bug you get :)
My point was not to question that people would write tests, the point I'm making is that it's tempting to generate both code and tests once you start using an LLM to generate large swaths of code, and then the assurance that tests give you goes out the window.
I'm not convinced that using AI as more than auto-complete is really a viable solution, because you can't shortcut an understanding of the problem domain to be assured of the correctness of code (and at that point the AI has mostly saved you some typing). The theory-crafting process of building software is probably the most important aspect of it. It not only provides assurance of the correctness of what you're building, it provides feedback into product development (restrictions, pushback that suggests alternate ways of doing things, etc.).
> But the idea of letting an LLM write/move large swaths of code seems so incredibly irresponsible.
I think this is where the bimodality comes from. When someone says "I used AI to refactor 3000 loc" some take it to mean they used AI in small steps as an accellerator, and others take it to mean a direct copy/paste, fix compile errors and move on.
Treat AI like a mid level engineer that you are pair programming with, who can type insanely fast. Move in small steps. Read through it's code after each small iteration. Ask it to fix things (or fix them yourself if quick and easy). Brainstorm ideas with it etc etc.
Works well for us nonetheless, also on more complex things. It's not worse than most (including seniors) humans I worked with in the past 40 years, but it is faster and cheaper. On HN it is sometimes forgotten that by far most programmers do not like it; they need money. If you see what comes out of them, you have to puke; yet it's running billion$ businesses and works surprisingly well considering the bad code quality.
It's quite literally incapable of solving many very mid-level things, no matter how much you help it. It's not a reasoning machine, it's basically a different way to search existing answers.
Even in small steps, it fails. I have two cases I test with, nothing special, just some TS generics in one instance and a schema-to-schema mapping tool in another. Both things that Junior devs could do given a couple days, even though they'd need to study and figure out various pieces.
o1 can't get either, no matter how much I break it down, no matter how much prodding. In fact the more you try the worse it gets. And yes I do try starting new conversations and splitting it out. Simply does not help, at all.
It's not to say it isn't really helpful for really simple things. Or even complex things but that are directly in the training set. But the second you go outside that, it's terrible.
Nobody pays for splendid code that isn't in production. They will gladly pay for buggy code that is in production and solves their needs as long as the marketing team does a good job.
I have 10 years professional experience and I've been writing code for 20 years, really with this workflow I just read and review significantly more code and I coach it when it structures or styles something in a way I don't like.
I'm fully in control and nothing gets committed I haven't read its an extension of me at that point.
Edit: I think the issues you've mentioned typically apply to people too and the answer is largely the same. Talk, coach, put hard fixes in like linting and review approvals.
The "Changes to my workflow" part is most relevant, and would be more accurately titled "How Cursor writes code differently to me [a senior developer]".
For example:
1) Cursor/AI more likely to reinvent the wheel and write code from scratch rather than use support libraries. Good to avoid dependencies I suppose, but widely used specialized libraries are likely to be debugged, and mature - able to handle corner cases gracefully, etc. AI "writes code" by regenerating stuff from it's training data - akin to cut and pasting from Stack Overflow, etc. If you're using this for a throwaway prototype or personal project then maybe you don't care as long as it works most of the time, but for corporate production use this is a liability.
2) AI more likely to generate repetitive code rather than write reusable functions (which he spins as avoiding abstractions) means code that is harder to read, debug and maintain. It's like avoiding global symbolic constants and defining them multiple times throughout your code instead. This wouldn't pass typical human code review. When future you, or a co-worker, maybe using a different editor/IDE, fixes a bug, they may not realize that the same bug has been repeated multiple times throughout the code, rather than fixing it once in a function.
We don't have human level AGI yet, far from it, and the code that today's AI generates reflects that. This isn't code that an experienced developer would write - this is LLM generated code, which means it's either regurgitated as-as from some unknown internet source, or worse yet (and probably more typical?) is a mashup of multiple sources, where the LLM may well have introduced it's own bugs in addition to those present in the original sources.
The code is always a burden, a legacy. To support it, one need to look after it, take care of it, bear it's weight.
I have 35+ years of experience on my shoulders. I always celebrate when I can reduce code - and, recently, I do that more and more often, being a support person for a large code base with a 20+ years of history.
PS
You were answering a comment that brags about migrating multithousand LOC code bases from language to the other. Recently I had to review a file that is more than (metric) megabyte (10^6 bytes) and more than 32 thousands LOC in size. I need to find a way to apply a fix for a problem triggered by single statement in several thousands (more that 1M of LOCs) of test cases.
I have a colleague that did it also, moving parts of code and « writing » code quickly with copilot. Because it’s easier to overlook LLM updates he riddled the code with bugs. Subtle things that we undercover later, now that he’s gone.
When you write everything yourself you are more keen to think deeply about changes.
I read today that Google has 25% of their code written by AI. They have an history of trashing huge projects and the quality of their services is getting worse over time. Maybe the industry is going to move to « let’s trash the codebase and ask cGPT 8 to write everything with this new framework »… OP said he’s talking to AI like guiding an other dev. Isn’t he afraid that he will loose the ability to think about solutions for himself ? That’s a trained part of the brain that we can « loose » no ?
If you're building code that's going to go in some medical system, or a space shuttle, then yeah, you probably want to write every small function with great detail.
If you're creating some silly consumer app like a "what will your baby look like in 5 years", then code quality doesn't matter you just need to ship fast. Most startups just need to ship fast to validate some ideas, 99% of your code will be deprecated within a few months
The whole thing just sounds like a gargantuan mess.
Most apps are a gargantuan mess. It's just a mess that mostly works. In a typical large scale web app written in something like Node or PHP, I wouldn't be at all surprised if 95% of the code is brought in from libraries that the dev team don't review. They have no idea about the quality of the code they're running. I don't see why adding AI to the mix makes much of a difference.
> Whenever I sit down to write some code, be it a large implementation or a small function, I think about what other people (or future versions of myself) will struggle with when interacting with the code. Is it clear and concise? Is it too clever? Is it too easy to write a subtle bug when making changes? Have I made it totally clear that X is relying on Y dangerous behavior by adding a comment or intentionally making it visible in some other way?
> It goes the other way too. If I know someone well (or their style) then it makes evaluating their code easier. The more time I spend in a codebase the better idea I have of what the writer was trying to do.
What I believe you are describing is a general definition of "understanding", which I am sure you are aware. And given your 20+ year experience, your summary of:
> So the thought of opening up a codebase that was cobbled together by an AI is just scary to me. Subtle bugs and errors would be equally distributed across the whole thing instead of where the writer was less competent (as is often the case).
Is not only entirely understandable (pardon the pun), but to be expected as algorithms employed lack the crucial bit which you identify - understanding.
> The whole thing just sounds like a gargantuan mess.
As it does to most whom envision having to live with artifacts produced by a statistical predictive text algorithm.
> Change my mind.
One cannot because understanding, as people know it, is intrinsic to each person by definition. It exists as a concept within the person whom possesses it and is defined entirely by said person.
I don't know if you jest but this is likely the next stage, to be released within the next months, that is, AIs doing the first rounds of code reviews. It'll likely be from github / microsoft as they have one of the biggest code review datasets around.
This is already happening - I recently saw a resume which included "Added AI-driven code reviews" as an accomplishment bullet point (the person was working for a large consulting firm).
I’ve tried AI coding assistance tools multiple times. Notably with Ansible and then with some AWS stuff using Amazon Q. I decided I wanted to be curious and see how it went.
With Ansible, it was an unusable mess. Ansible is a bit of an odd bird of a DSL though. You really need to be on top of versions and modules. It’s not well suited for AI because it requires a lot of nuanced understanding. There’s no one resource that will work (effectively) forever, like C or something.
With AWS’s Amazon Q, I started out simple by having it write an IAM policy that I was having a hard time wrapping my head around. It wasn’t very helpful because the policies it provided used conditional keys that weren’t supported in the service that was addressed by the policy.
I’ve found I can typically work with higher quality and less fixing by just writing it myself. I could babysit and teach an AI, but at that point what’s the point?
I’m also unconvinced it’s worth the environmental impact, especially if I need to tutor it or mark up its output anyway.
In any event, it’s easy enough to outsmart/out-clever oneself it a colleague (and vice versa). Adding AI to that just seems like adding a chaotic junior developer to that equation.
> As a programmer of over 20 years - this is terrifying.
>
> I'm willing to accept that I just have "get off my lawn" syndrome or something.
>
> But the idea of letting an LLM write/move large swaths of code seems so incredibly irresponsible.
My first thought was that I disagree (though I don't use or like this in-IDE AI stuff) because version control. But then the way people use (or can't use) SVC 'terrifies' me anyway, so maybe I agree? It would be fine correctly handled, but it won't be, sort of thing.
> But the idea of letting an LLM write/move large swaths of code seems so incredibly irresponsible.
People felt the same about compilers for a long time. And justifiably so, the idea that compilers are reliable is quite a new one, finding compilers bugs used to be pretty common. (Those experimenting with newer languages still get to enjoy the fun of this!)
How about other code generation tools? Presumably you don't take much umbrage with schema generators? Or code generators that take a scheme and output library code (OpenAPI, Protocol buffers, or even COM)? Those can easily take a few dozen lines of input and output many thousands of LoC, and because they are part of an automated pipeline, even if you do want to fix the code up, any fixes you make will be destroyed on the next pipeline run!
But there is also a LOT of boring boilerplate code that can be automated.
For example, the necessary code to create a new server, attach a JSON schema to a POST endpoint, validate a bearer token, and enable a given CORS config is pretty cut and dry.
If I am ramping up on a new backend framework, I can either spend hours learning the above and then copy and paste it forever more into each new project I start up, or I can use an AI to crap the code out for me.
(Actually once I was setting up a new server and I decided to not just copy and paste and to do it myself, I flipped the order of two `use` directives and it cost me at least 4 hours to figure out WTF was wrong....)
> As a programmer of over 20 years
I'm almost up there, and my view is that I have two modes of working:
1. Super low level, where my intimate knowledge of algorithms, the language and framework I'm using, of CPU and memory constraints, all come together to let me write code that is damn near magical.
2. Super high level, where I am architecting a solution using design patterns and the individual pieces of code are functionally very simple, and it is how they are connected together that really matters.
For #1, eh, for some popular problems AI can help (popular optimizations on Stack Overflow).
For #2, AI is the most useful, because I have already broken the problem down into individual bite size testable nuggets. I can have the AI write a lot of the boilerplate, and then integrate the code within the larger, human architected, system.
> So the thought of opening up a codebase that was cobbled together by an AI is just scary to me.
The AI didn't cobble together the system. The AI did stuff like "go through this array and check the ID field of each object and if more than 3 of them are null log an error, increment the ExcessNullsEncountered metric counter, and return an HTTP 400 error to the caller"
Edit: This just happened
I am writing a small Canvas game renderer, and I am having an issue with text above a character's head renders off the canvas. So I had Cursor fix the function up to move text under a character if it would have been rendered above the canvas area.
I was able to write the instructions out to Cursor faster than I could have found a pencil and paper to sketch out what I needed to do.
How do tests account for cases where I'm looking at a 100 line function that could have easily been written in 20 lines with just as much, if not more, clarity?
It reminds me of a time (long ago) when the trend/fad was building applications visually. You would drag and drop UI elements and define logic using GUIs. Behind the scenes the IDE would generate code that linked everything together. One of the selling points was that underneath the hood it's just code so if someone didn't have access to the IDE (or whatever) then they could just open the source and make edits themselves.
It obviously didn't work out. But not because of the scope/scale (something AI code generation solves) but because, it turns out, writing maintainable secure software takes a lot of careful thought.
I'm not talking about asking an AI to vomit out a CRUD UI. For that I'm sure it's well suited and the risk is pretty low. But as soon as you introduce domain specific logic or non-trivial things connected to the real world - it requires thought. Often times you need to spend more time thinking about the problem than writing the code.
I just don't see how "guidance" of an LLM gets anywhere near writing good software outside of trivial stuff.
> How do tests account for cases where I'm looking at a 100 line function that could have easily been written in 20 lines with just as much, if not more, clarity?
That’s not a failure of the AI writing that 100 line monstrosity, it’s a failure of you deciding to actually use the thing.
If you know what 20 lines are necessary and the AI doesn’t output that, why would you use it?
> How do tests account for cases where I'm looking at a 100 line function that could have easily been written in 20 lines with just as much, if not more, clarity?
If the function is fast to evaluate and you have thorough coverage by tests, you couod iterate on an LLMs that aims to compress it down to a simpler / shorter version that behaves identical to the original function. Of course brevity for the sake of brevity can lead to less code that is not always more clear or simpler to understand than the original —LLMs are very good at mimicing code style, so show them a lot of your own code and ask them to mimic it and you may be surprized.
Finally found a comment down here that I like. I'm also with the notion of tests and also iterating until you get to a solution you like. I also don't see anything particularly "terrifying" that many other comments suggest.
At the end of the day, we're engineers that write complex symbols on a 2d canvas, for something that is (ultimately, even if the code being written is machine to machine or something) used for some human purpose.
Now, if those complex symbols are readable, fully covered in tests, and meets requirements / specifications, I don't see why I should care if a human, an AI, or a monkey generated those symbols. If it meets the spec, it meets the spec.
Seems like most people in these threads are making arguments against others who are describing usage of these tools in a grossly incorrect manner from the get go.
I've said it before in other AI threads that I think (at least half?) of the noise and disagreement around AI generated code is like a bunch of people trying to use a hammer when they needed a screwdriver and then complaining that the hammer didnt work like a screwdriver!!! I just don't get it. When you're dealing with complex systems, i.e, reality, these tools (or any tool for that matter) will never work like a magic wand.
I'm sure people who are "terrified" either haven't really tried AI or are so attached to their intellect their egos won't allow them to admit that there's little value now in a lot of the stuff they've memorised over the last few years.
I think this egoic threat is the biggest influence on this kind of thinking tbh.
I'll give you that! Too much of this stuff is sold as the "magic wand" solution... I guess marketing for many products has been like that for a long time...
Sure but you go fast on the simple parts it's good at and slow on the novel/critical part. It's not that hard to understand. You don't drive at top speed everywhere either. You go slower depending on the context.
The real problem with AI coding is not knowing in advance the cases where it's going to spin its wheels and go in a cycle of stupid answers. I lose 20 minutes at a time that way because it seems like it needs just one more prompt, but in the end I have to step in, either with code or telling it specifically where the bug is.
Depending on whether I'm using LLMs from my Emacs or via a tool like Aider, I either review and manually merge offered modifications as diffs (in editor), or review the automatically generated commits (Aider). Either way, I end up reading a lot of diffs and massaging the LLM output on the fly, and nothing that I haven't reviewed gets pushed to upstream.
I mean, people aren't seriously pushing unreviewed LLM-generated code to production? Current models aren't good enough for that.
You could make the same argument for any non-AI driven productivity tool/technique. If we can't trust the user to determine what is and is not time-saving then time-saving isn't a useful thing to discuss outside of an academic setting.
My issue with most AI discussions is they seem to completely change the dimensions we use to evaluate basic things. I believe if we replaced "AI" with "new useful tool" then people would be much more eager to adopt it.
What clicked for me is when I started treating it more like a tool and less like some sort of nebulous pandora's box.
Now to me it's no different than auto completing code, fuzzy finding files, regular expressions, garbage collection, unit testing, UI frameworks, design patterns, etc. It's just a tool. It has weaknesses and it has strengths. Use it for the strengths and account for the weaknesses.
Like any tool it can be destructive in the hands of an inexperienced person or a person who's asking it to do too much. But in the hands of someone who knows what they're doing and knows what they want out of it - it's so freakin' awesome.
Sorry for the digression. All that to say that if someone believes it's a productivity boost for them then I don't think they're being misled.
Employees are not robots. They are human beings. Sometimes human beings have human problems that need the assistance of other humans. This makes humans happier and more productive.
It's depressing to think that there are people who actually believe that optimal use of work resources is even worth calling out as an issue. In 2024.
No amount of discipline was going to make medium-large websites maintainable back then. Today it's actually possible if the creators know what they're doing. Tooling isn't going to prevent people from doing stupid things.
I agree with HTML+CSS+JS websites being not fine. But to be honest, js callbacks are more of a language thing than a framework thing. You don't need to use frameworks to write promises.
I love ambient tunes and listen to them for 1-10 hours a day while doing stuff. Any time I need to focus (even when writing an email) I'll throw on my favorite ambient music.
How is this any different? Why would I get a "premium" account for some random website when there's an entire catalog of ambient music on Spotify/Apple/etc that I could listen to?
The biggest differentiator may be the infinite length of tracks. Once you tune into something that fits it can be left going as long as necessary. With other services there is a need to compose play lists or restart the player when a selection ends.
Such offerings have discontinuities between tracks or beat matched transitions. These generated tracks are constructed to continue with the same themes and sounds as long as wanted. It may or may not be desirable, but it is different.
Oh, this makes sense, I didn't really think of the transition as potentially being distracting or a cue to stop working.
I personally use a "relaxing music playlist/radio", the songs are pretty similar (meditation music), and I don't often notice the song changing, but I do often notice that suddenly I am listening to a completely different melody.
… but there's no continuity with radio … or not a uniform/dependable continuity, at least: the mood is different song-to-song, and there is no control over the "levers" beyond _skip_ and _like_. So, to "train" the radio stations involves interacting with them.
