Even as the field evolves, the phoning home telemetry of closed models creates a centralized intelligence monopoly. If open source atrophies, we lose the public square of architectural and design reasoning, the decision graph that is often just as important as the code. The labs won't just pick up new patterns; they will define them, effectively becoming the high priests of a new closed-loop ecosystem.
However, the risk isn't just a loss of "truth," but model collapse. Without the divergent, creative, and often weird contributions of open-source humans, AI risks stagnating into a linear combination of its own previous outputs. In the long run, killing the commons doesn't just make the labs powerful. It might make the technology itself hit a ceiling because it's no longer being fed novel human problem-solving at scale.
Humans will likely continue to drive consensus building around standards. The governance and reliability benefits of open source should grow in value in an AI-codes-it-first world.
> It might make the technology itself hit a ceiling because it's no longer being fed novel human problem-solving at scale.
My read of the recent discussion is that people assume that the work of far fewer number of elites will define the patterns for the future. For instance, implementation of low-level networking code can be the combination of patterns of zeromq. The underlying assumption is that most people don't know how to write high-performance concurrent code anyway, so why not just ask them to command the AI instead.
> The underlying assumption is that most people don't know how to write high-performance concurrent code anyway, so why not just ask them to command the AI instead.
The data economics reflexivity of LLM input means that when you reduce the future volume of that input to the few experts who "know how to write X anyway", the LLM labs just lost one of the most important inputs. All those non-experts who voted with their judgement and left in the wake of their effort to use the expert-written code, grist for the LLM input weighing mill.
I find it is usually the non-experts that run into the sharp operational edges the experts didn't think of. When you throw the non-experts out of the marketplace of ideas, you're often left with hazardous tooling that would just as soon cut your hand off than help you. It would be a hoot if the LLM's and experts decided to output everything and training in Common Lisp, though.
If handed just Babbage's Difference Engine, or the PDP-11 Unix V7 source code and nothing else, LLM's could speed-run and eventually re-derive the analogs of Zig, ffmpeg, YouTube, and themselves, I'll grant that "just let them cook with the experts" is a valid strategy. The information imparted by the activity around the source code is deeply recursive, and absent that I'm not sure how the labs are going to escape a local minima they're digging themselves into by materially shrinking that activity. If my hypothesis is correct, then LLM labs are industrial-scale stripping away the very topsoil that their products rely upon, and it is a single-turn cheap game that gets enormously more expensive in further iterations to create synthetic topsoil.
There is no shortage of Olympic hopeful elite athletes every four years, despite the incredibly small pool of competitors at each Games.
Same for musicians.
This kind of Winner-Take-All Economics or Superstar Market is what capital wants in their ideal world in markets with near-zero marginal costs of distribution. Even if software creation in the long-term does not fall to this kind of labor market, LLM's can establish a "market can be irrational longer than you can stay solvent" dynamic where capital can run the labor market like this for software for a generation or three before having to face the reflexivity music, like they did for US manufacturing.
And it mostly happens in government funded and/or commercially viable sports, with public schools where kids train for free, scholarships, numerous competitions etc.
To gather those selected elites we take an enormous pool of aspiring athletes and support them from the ground up (usually with our taxes)
Where such support systems don’t exist you have a relatively shallow talent pool, and the best performers are a far cry from what could have been possible otherwise
We're probably close enough to a future with the technology and engineering able to implement it, to justify designing 48-bit perceptual bit depth standards. Optimistically presuming breakthroughs enabling in vivo biological upgrades to our eyes to match those of mantis shrip, we could design 160-bit standards knowing that is a "proven" biological technology capability. That gets within the same galactic supercluster of the known limits of physics limits.
At the currently known limits of physics where Heisenberg Uncertainty Principle, Abbe's Limit, Quantum Shot Noise and such become our sensing barriers, we suspect we need only about 6000 bits per pixel to represent a digital twin of the electromagnetic field of the sensor-covered volume of space. At 60 fps, that is 1.8 Zettabits per second. Scale out data volume accordingly when using using 18.5 sexdecillion fps (Planck time fps).
What surprises me is these "limits of the fabric of reality as we know it" mind experiments fairly concretely point the way on the many roads towards Kardashev Scale implementations, and is not that different from Archimedes' "Sand Reckoner" and Hindu cosmological Kalpa time scales. History doesn't quite repeat, but rhymes yet again.
This I strongly suspect is the crux of the boundaries of their current usefulness. Without accompanying legibility/visibility into the lineage of those decisions, LLM's will be unable to copy the reasoning behind the "why", missing out on a pile of context that I'm guessing is necessary (just like with people) to come up to speed on the decision flow going forward as the mathematical space for the gradient descent to traverse gets both bigger and more complex.
We're already seeing glimmers of this as the frontier labs are reporting that explaining the "why" behind prompts is getting better results in a non-trivial number of cases.
I wonder whether we're barely scratching the surface of just how powerful natural language is.
The challenge not addressed with this line of reasoning is the required sheer scale of output validation on the backend of LLM-generated code. Human hand-developed code was no great shakes at the validation front either, but the scale difference hid this problem.
I’m hopeful what used to be tedious about the software development process (like correctness proving or documentation) becomes tractable enough with LLM’s to make the scale more manageable for us. That’s exciting to contemplate; think of the complexity categories we can feasibly challenge now!
In the enterprise deployments of GitHub Copilot I've seen at my clients that authenticate over SSO (typically OIDC with OAuth 2.0), connecting Copilot to anything outside of what Microsoft has integrated means reverse engineering the closed authentication interface. I've yet to run across someone's enterprise Github Copilot where the management and administrators have enabled the integration (the sites have enabled access to Anthropic models within the Copilot interface, but not authorized the integration to Claude Code, Opencode, or similar LLM coding orchestration tooling with that closed authentication interface).
While this is likely feasible, I imagine it is also an instant fireable offense at these sites if not already explicitly directed by management. Also not sure how Microsoft would react upon finding out (never seen the enterprise licensing agreement paperwork for these setups). Someone's account driving Claude Code via Github Copilot will also become a far outlier of token consumption by an order(s) of magnitude, making them easy to spot, compared to their coworkers who are limited to the conventional chat and code completion interfaces.
If someone has gotten the enterprise Github Copilot integration to work with something like Claude Code though (simply to gain access to the models Copilot makes available under the enterprise agreement, in a blessed golden path by the enterprise), then I'd really like to know how that was done on both the non-technical and technical angles, because when I briefly looked into it all I saw were very thorny, time-consuming issues to untangle.
Outside those environments, there are lots of options to consume Claude Code via Github Copilot like with Visual Studio Code extensions. So much smaller companies and individuals seem to be at the forefront of adoption for now. I'm sure this picture will improve, but the rapid rate of change in the field means those whose work environment is like those enterprise constrained ones I described but also who don't experiment on their own will be quite behind the industry leading edge by the time it is all sorted out in the enterprise context.
Non-technical home users in my circles are fed up with Windows 11's changes from Windows 10 without a suitable transition that eases them into the changes. They are nowhere near good candidates to migrate to any flavor of Linux, though. There are still plenty of sharp edges. So lots of cursing and griping at Windows 11 continues.
More interesting to me however, are the macOS technical friends in my circles. A trickle of them are switching to various Linux desktop distributions. This was inconceivable to me a mere 10 years ago. But I have to admit the quality of the Apple ecosystem has slid an astounding amount, which is driving the more advanced technical users into the arms of Linux. There are still plenty of Apple ecosystem-specific integration points and features that are still not available on Linux, like Apple Notes/iMessage/AirDrop/AirPlay/Handoff between macOS and iOS, system-wide kinetic/momentum scrolling, iCloud sync, system-comprehensive battery management that includes working sleep and suspend, advanced trackpad gestures, uneven Unicode support, uneven human interface guideline adherence, limited laptop LLM inference, etc. So I'm not expecting this trickle to turn into a flood soon, but the solid lock Apple used to have on developer mindshare is not as solid any longer.
> There are still plenty of sharp edges. So lots of cursing and griping at Windows 11 continues.
I wouldn't be so assertive about that. No OS is perfect, and as we see here, windows is no exception. It's mostly a matter of being used to living with those imperfections. At least on Linux, nobody is making those worse for you for "fun" (actually for their own profit at the detriment of yours), and many more nontechnical users sense that just fine (just the way copilot was forced is baffling).
> There are still plenty of Apple ecosystem-specific integration points and features that are still not available on Linux, like Apple Notes/iMessage/AirDrop/AirPlay/Handoff between macOS and iOS
KDE Connect solved that, and much more, many many years ago. I don't know the situation in the Apple walled garden, only that any hurdle there is the result of Apple abusive, user-hostile and anticompetitive practices that should (and will eventually) be illegal outside of the US.
With the Netflix infrastructure, I'm surprised they broadcast it so conventionally. Different channels running at the same time (with the crowd at the bottom, with the crowds as he passed each floor, with his wife watching, with pro climbers talking technical climbing stuff with simultaneous 8K online illustrating graphics, etc.), different audio tracks (with commentators, with crowd at bottom only, etc.*). Alex Honnold was paid only $500K for the event, so maybe there simply wasn't a lot of money allocated to the project to get fancy with the live broadcast.
Inference leans heavily on GPU RAM and RAM bandwidth for the decode phase where an increasingly greater amount of time is being spent as people find better ways to leverage inference. So NVIDIA users are currently arguably going to demand a different product mix when the market shifts away from the current training-friendly products. I suspect there will be more than enough demand for inference that whatever power we release from a relative slackening of training demand will be more than made up and then some by power demand to drive a large inference market.
It isn’t the panacea some make it out to be, but there is obvious utility here to sell. The real argument is shifting towards the pricing.
> They have food and housing, but their life is devoid of meaning.
I find it difficult to relate to such worlds. I make up all kinds of explanations like, "well, it must be because while they have food and housing, they don't have any funds to entertain themselves". Or, "well, it must be because they simply haven't had sufficient education to reach an activation level where the higher tiers of Maslow's come into their line of sight".
And then I read about plenty of counter-examples, like wealthy offspring living the textbook aimless/dissolute/pick-your-adjective life, or the ennui of able-bodied welfare recipients with quite reasonable spending cash from generous Scandinavian welfare regimes when one considers the mind boggling amount of free media, free libraries, free parks, free entertainment in general in the developed world. Perhaps this is just part of their human condition for people suffering from this malaise.
And here I sit, drowning in ideas of what I would be interested to pursue to know our beautiful universe if only I had the time. So much so I write them down into a file just to quiet the cacophony in my head like a dog seeing squirrels everywhere he looks, just so I can get real work done on a timely basis, haha.
When once asked whether I'd ever be bored with eternal youth and boundless resources, I immediately replied an eternity is still too little time to satisfy my curiosity.
I wonder whether we're trending towards a high-sensor variation of "A Young Lady's Illustrated Primer" / Vannevar Bush's Memex that ingests the details of a user's daily life (the smart glasses being a primitive first example products of such) and identifies salient information in their lives can help us perform mass customization of instructions into direct prescriptives, with backing evidentiary data for SME's. Instead of "if X Y and Z then do A, if only X do B", the interaction becomes "do this, anticipate that outcome" to the user, and if an SME (a doctor in your example) asks about it, the system recalls and presents all the factors that went into deciding upon the specific prescriptive.
However, the risk isn't just a loss of "truth," but model collapse. Without the divergent, creative, and often weird contributions of open-source humans, AI risks stagnating into a linear combination of its own previous outputs. In the long run, killing the commons doesn't just make the labs powerful. It might make the technology itself hit a ceiling because it's no longer being fed novel human problem-solving at scale.
Humans will likely continue to drive consensus building around standards. The governance and reliability benefits of open source should grow in value in an AI-codes-it-first world.
reply