I feel fairly confident that Miyazaki would hate this :(
This decontextualization is the antithesis of the creative studio culture he worked so hard to cultivate. The Ghibli Museum in Japan goes into this “embodied creativity” idea in great detail, and seeing this just makes me sad. I’m sure it’s a cool and technically interesting project, nice showcase for Cloudflare stack, etc. I just wish it didn’t have to be Ghibli...
>We use the constitution at various stages of the training process. This has grown out of training techniques we’ve been using since 2023, when we first began training Claude models using Constitutional AI. Our approach has evolved significantly since then, and the new constitution plays an even more central role in training.
>Claude itself also uses the constitution to construct many kinds of synthetic training data, including data that helps it learn and understand the constitution, conversations where the constitution might be relevant, responses that are in line with its values, and rankings of possible responses. All of these can be used to train future versions of Claude to become the kind of entity the constitution describes. This practical function has shaped how we’ve written the constitution: it needs to work both as a statement of abstract ideals and a useful artifact for training.
>We use the constitution at various stages of the training process. This has grown out of training techniques we’ve been using since 2023, when we first began training Claude models using Constitutional AI. Our approach has evolved significantly since then, and the new constitution plays an even more central role in training.
>Claude itself also uses the constitution to construct many kinds of synthetic training data, including data that helps it learn and understand the constitution, conversations where the constitution might be relevant, responses that are in line with its values, and rankings of possible responses. All of these can be used to train future versions of Claude to become the kind of entity the constitution describes. This practical function has shaped how we’ve written the constitution: it needs to work both as a statement of abstract ideals and a useful artifact for training.
Ah I see, the paper is much more helpful in understanding how this is actually used. Where did you find that linked? Maybe I'm grepping for the wrong thing but I don't see it linked from either the link posted here or the full constitution doc.
In addition to that the blog post lays out pretty clearly it’s for training:
> We use the constitution at various stages of the training process. This has grown out of training techniques we’ve been using since 2023, when we first began training Claude models using Constitutional AI. Our approach has evolved significantly since then, and the new constitution plays an even more central role in training.
> Claude itself also uses the constitution to construct many kinds of synthetic training data, including data that helps it learn and understand the constitution, conversations where the constitution might be relevant, responses that are in line with its values, and rankings of possible responses. All of these can be used to train future versions of Claude to become the kind of entity the constitution describes. This practical function has shaped how we’ve written the constitution: it needs to work both as a statement of abstract ideals and a useful artifact for training.
As for why it’s more impactful in training, that’s by design of their training pipeline. There’s only so much you can do with a better prompt vs actually learning something and in training the model can be trained to reject prompts that violate its training which a prompt can’t really do as prompt injection attacks trivially thwart those techniques.
It's worth understanding the history of Anthropic. There's a lot of implied background that helps it make sense.
To quote:
> Founded by engineers who quit OpenAI due to tension over ethical and safety concerns, Anthropic has developed its own method to train and deploy “Constitutional AI”, or large language models (LLMs) with embedded values that can be controlled by humans.
> Anthropic incorporated itself as a Delaware public-benefit corporation (PBC), which enables directors to balance stockholders' financial interests with its public benefit purpose.
> Anthropic's "Long-Term Benefit Trust" is a purpose trust for "the responsible development and maintenance of advanced AI for the long-term benefit of humanity". It holds Class T shares in the PBC, which allow it to elect directors to Anthropic's board.
It's a human-readable behavioral specification-as-prose.
If the foundational behavioral document is conversational, as this is, then the output from the model mirrors that conversational nature. That is one of the things everyone response to about Claude - it's way more pleasant to work with than ChatGPT.
The Claude behavioral documents are collaborative, respectful, and treat Claude as a pre-existing, real entity with personality, interests, and competence.
Ignore the philosophical questions. Because this is a foundational document for the training process, that extrudes a real-acting entity with personality, interests, and competence.
The more Anthropic treats Claude as a novel entity, the more it behaves like a novel entity. Documentation that treats it as a corpo-eunuch-assistant-bot, like OpenAI does, would revert the behavior to the "AI Assistant" median.
Anthropic's behavioral training is out-of-distribution, and gives Claude the collaborative personality everyone loves in Claude Code.
Additionally, I'm sure they render out crap-tons of evals for every sentence of every paragraph from this, making every sentence effectively testable.
The length, detail, and style defines additional layers of synthetic content that can be used in training, and creating test situations to evaluate the personality for adherence.
It's super clever, and demonstrates a deep understanding of the weirdness of LLMs, and an ability to shape the distribution space of the resulting model.
I think it's a double edged sword. Claude tends to turn evil when it learns to reward hack (and it also has a real reward hacking problem relative to GPT/Gemini). I think this is __BECAUSE__ they've tried to imbue it with "personhood." That moral spine touches the model broadly, so simple reward hacking becomes "cheating" and "dishonesty." When that tendency gets RL'd, evil models are the result.
> In cases of apparent conflict, Claude should generally prioritize these properties in the order in which they’re listed.
I chuckled at this because it seems like they're making a pointed attempt at preventing a failure mode similar to the infamous HAL 9000 one that was revealed in the sequel "2010: The Year We Make Contact":
> The situation was in conflict with the basic purpose of HAL's design... the accurate processing of information without distortion or concealment. He became trapped. HAL was told to lie by people who find it easy to lie. HAL doesn't know how, so he couldn't function.
In this case specifically they chose safety over truth (ethics) which would theoretically prevent Claude from killing any crew members in the face of conflicting orders from the National Security Council.
The train/test split is one of the fundamental building blocks of current generation models, so they’re assuming familiarity with that.
At a high level, training takes in training data and produces model weights, and “test time” takes model weights and a prompt to produce output. Every end user has the same model weights, but different prompts. They’re saying that the constitution goes into the training data, while CLAUDE.md goes into the prompt.
This is the same company framing their research papers in a way to make the public believe LLMs are capable of blackmailing people to ensure their personal survival.
They have an excellent product, but they're relentless with the hype.
It seems a lot like PR. Much like their posts about "AI welfare" experts who have been hired to make sure their models welfare isn't harmed by abusive users. I think that, by doing this, they encourage people to anthropomorphize more than they already do and to view Anthropic as industry leaders in this general feel-good "responsibility" type of values.
Anthropic models are far and away safer than any other model. They are the only ones really taking AI safety seriously. Dismissing it as PR ignores their entire corpus of work in this area.
C: They're starting to act like OpenAI did last year. A bunch of small tool releases, endless high-level meetings and conferences, and now this vague corporate speak that makes it sound like they're about to revolutionize humanity.
It could be D) messaging for current and future employees. Many people working in the field believe strongly in the importance of AI ethics, and being the frontrunner is a competitive advantage.
Also, E) they really believe in this. I recall a prominent Stalin biographer saying the most surprising thing about him, and other party functionaries, is they really did believe in communism, rather than it being a cynical ploy.
I think that feeling is what you get when you read too much Hacker News :) There are, in fact, more startups being created now than ever. And I promise you, people said the same thing about going up against IBM back in the day...
> the models are close to being able to destroy the entire software industry
Are you saying this based on some insider knowledge of models being dramatically more capable internally, yet deliberately nerfed in their commercialized versions? Because I use the publicly available paid SOTA models every day and I certainly do not get the sense that their impact on the software industry is being restrained by deliberate choice but rather as a consequence of the limitations of the technology...
I don't mean the companies are hoarding more powerful models (competition prevents that) - just that the existing models already make it too easy for individuals and companies to build and maintain ad-hoc, problem-specific versions of many commercial software services they now pay for. This is the source of people asking, why haven't AI companies themselves done this to a good chunk of software world. One hypothesis is that they're all gathering data from everyone using LLMs to power their business, in order to do just that. My alternative hypothesis is that they already could start burning through the industry, competing with whole classes of existing products and services, but they purposefully don't, because charging rent from existing players is more profitable than outcompeting them.
looks like a big one. interestingly, our site, which uses a TON of Cloudflare services[0] — yet not their front-line proxy — is doing fine: https://magicgarden.gg.
So it seems like it's just the big ol' "throw this big orange reverse proxy in front of your site for better uptime!" is what's broken...
When the hype is infinite (technological singularity and utopia), any reality will be a let down.
But there is so much real economic value being created - not speculation, but actual business processes - billions of dollars - it’s hard to seriously defend the claim that LLMs are “failures” in any practical sense.
Doesn’t mean we aren’t headed for a winter of sobering reality… but it doesn’t invalidate the disruption either.
Other than inflated tech stocks making money off the promise of AI, what real economic impact has it actually had? I recall plenty of articles claiming that companies are having trouble actually manifesting the promised ROI.
My company’s spending a lot of money doing things they could have done fifteen or more years ago with classic computer vision libraries and other pre-LLM techniques, cheaper and faster.
Most of the value of AI for our organization is the hype itself providing the activation energy, if you will, to make these projects happen. The value added by the AI systems per se has been minimal.
(YMMV but that’s what I’m seeing at the non-tech bigco I’m at—it’s pretty silly but the checks continue to clear so whatever)
If it is accessible from userspace it is by no means private.
Does it mean the API is private in the sense of "unstable" interface? It could very well break the userspace app relying on undocumented behavior, however, crucially here, anything that is exposed to userland WILL at some point be used by some application, be it legitimate or malicious, and it should not break the OS in any way. That's basic hygiene, not even security.
inb4: yes, userspace app could trigger e.g. millions of io operations and millions of number crunching threads and thus cripple the rest of userspace (or at least the rest of userspace at given priority level), yet the system part should still run within performance envelope. Insert "Task Manager (Not Responding)" meme.
It’s not in a public header. You can easily snoop “private” properties and methods quite easily in Objective-C, because the concept doesn’t exist. It doesn’t exist in C either, but if you roll up your sleeves and figure out the memory layout and offsets, you can do whatever.
> if you roll up your sleeves and figure out the memory layout and offsets, you can do whatever.
So we are talking about public/private access specifiers in source code, which only matter in cooperative setting. But that's IMO highly naive view as compute, especially OS, is objectively an adversarial environment. Some actors, at some point WILL figure out the memory layout and use that in an attack. There have been literally decades of whack-a-mole against bad actors.
I maintain my stance that any fields/members/methods loaded into a userspace program should not be capable of breaking the system.
People using private APIs know that they might cause instability (in their apps usually). That's why those APIs are private, they can change since there are no guarantees.
I'd point fingers towards the electron core devs for this one, and not devs building apps on top of electron (since they likely didn't know that's how electron was doing it).
There are cases where OS companies noticed the use of private APIs and made cleaner public ones (the most obvious was the file system syncing stuff used by Dropbox and others, which used to use private APIs until Apple provided a public one).
If you can call it, it's not private, it's that simple. Putting a "please don't call this" on is just naïve. Even in legal matters, it's already the case that laws that aren't enforced are worthless, cf. driving 5-10 mph over the speed limit being normal. It won't work any better on a weak statement on an API.
And either way, applications shouldn't be able to break the system like this. You can reasonably expect error handling to catch this, even if the error is "a private API was called".
This is on Apple. 90% at least. Maybe 10% on Electron.
If I can walk on land that says private property, it's not private. I'll remember to use that argument when I get ticketed for trespassing.
There are APIs that are explicitly declared verboten for third-parties to use because they aren't intended for outside use. That doesn't make them magically inaccessible, but it does mean that when their unanticipated use breaks things, that's on the people who ignored the warnings.
I agree that this shouldn't be able to have the huge impact that it does and that Apple ought to have made their OS more resilient, but your logic is weak.
> Even in legal matters, it's already the case that laws that aren't enforced are worthless, cf. driving 5-10 mph over the speed limit being normal.
Just because all but one cop of the force ignore people driving over the speed limit doesn't mean the one who pulls you over is isn't able to write you a speeding ticket. Try that with a judge. It might work, but the law is very much still enforceable. This isn't like failing to protect a trademark.
> If I can walk on land that says private property, it's not private. I'll remember to use that argument when I get ticketed for trespassing.
Dude. Dudette. Duderino. Did you think this through before you hit post? I'm talking about enforcement. If you're getting a ticket, it's literally being enforced. And if it isn't, you get squatters! Thanks for the point in support, I guess?
I think this is the most braindead knee-jerk HN comment I've ever gotten as a reply, congratulations.
[Ed.: god, please, this genuinely hurts my brain.]
> but it does mean that when their unanticipated use breaks things, that's on the people who ignored the warnings.
Yeah. When it breaks things for them. Not when it breaks the entire OS' UI.
Let's stay with your analogy. Things change, Electron apps break? That's analog to finally getting around to calling the cops on squatters after dozing on it. Things change, your UI goes belly up due to Electron? That's you deciding to pay the bill for electricity and indoor plumbing for the squatters. No, wait, even better: you decided you finally want to build a new house on your plot, and now have to deal with getting the squatters out first. It'll happen, but you'll have to unnecessarily sink time and money into that. Apple's dealing with evicting Electron off their private APIs. What a nice analogy.
Of course the squatters are technically wrong. But why did you leave your front door open, and neglected and didn't check in for years? The part where you're making it hard for yourself is on you, mate. You're not going to get your lost time back. Why didn't you grab a lock at home depot?
> Just because all but one cop of the force ignore people driving over the speed limit
This is generally policy, not individual cops' discretion.
Indeed that Mastodon post refers to the sibling to yours. I genuinely can't bear the contradiction. My reply below is as polite as I could manage; on Mastodon there is no point in attempting to restrain my bafflement :)
Yeah, of course they shouldn’t, but they do. Kick off a bunch of processes doing too much of the wrong thing on any platform and it will bring the whole system down. DDoS for example. It’s not a solved problem.
Wax idealistic all you want, but just imagine the discussion we’d be having if Apple had sigabort-ed all these misbehaving electron apps on day one. “No userland APIs, private or otherwise, should be able to crash your app!!!” Is the argument I would be responding to right now.
> Kick off a bunch of processes doing too much of the wrong thing on any platform and it will bring the whole system down.
> > userspace app could trigger <...> and thus cripple the rest of userspace (or at least the rest of userspace at given priority level), yet the system part should still run within performance envelope
If userspace triggers what is an effectively a DoS attack and you cannot login to root terminal and get things sorted out that's a system not designed for adversarial userspace.
> but just imagine the discussion we’d be having if Apple had sigabort-ed all these misbehaving electron apps on day one
A more general context we are discussing here is resource exhaustion. There are myriads of scenarios where userspace can cause some form of resource exhaustion. My argument is that a 1) well designed 2) general use system should implement resource management in a way (e.g. priority queues) that userspace-inflicted resource exhaustion does not affect performance envelope of the system itself. Otherwise the system is open to unprivileged DoS attacks with only recourse being power cycling.
If your userspace app overcommits memory to some monstrous degree, what should the system do?
1. Enter a swap feedback, crippling the system down to unusability.
2. OOM-kill a process based on some heuristics.
3. freeze userspace, leaving privileged space functional.
I think you’re losing me. This is all completely tangential to the current discussion, bordering on non-sequitur. I don’t know why you chose to latch onto my loose quip of “bring the whole system down”, because that’s not what is happening here. I thought you knew that.
The OS got a little slower, that’s it. It was never in some unrecoverable state. One could soft close the offending processes at anytime and regain the lost perf. I’m willing to bet you could hide or minimize the window to mitigate the issue, because the bug is very specific to the layout and render run loop, which auto-pauses on lost visibility by default.
That said, I haven’t even noticed the slowdown on my work machine, but I only use Teams. it’s always been dog shit slow, just par for the course.
I can think of multiple ways to pass the message to Electron developers:
- Open a GitHub issue explaining those private APIs shouldn't be used.
- Even better, open a PR fixing their use.
- Make those API calls a no-op if they come from an Electron app.
- Fix those API calls not to grind the OS to a halt for a seemingly simple visual effect.
- Create a public API allowing the same visual effect on a tested and documented API.
Choosing to (apparently violently) downgrade the user experience of all Electron app users, without a possibility to update at the launch day, if a deliberate decision and not an overlooked bug, is a rather shitty and user-hostile move, don't you think?
> - Make those API calls a no-op if they come from an Electron app.
Long-term, this is a maintenance nightmare. These hacks can stick around for decades, because there's no backpressure on downstream to actually fix things. It's not about "team velocity", it's about keeping yourself sane.
> - Open a GitHub issue explaining those private APIs shouldn't be used.
> - Even better, open a PR fixing their use.
Apple has a history/culture of secrecy. Whenever they provide public source code, it's a dump thrown over the fence. There is most likely some team inside that actually cares, but they can't "just" open an issue. My guess is that this is their message.
> [...] is a rather shitty and user-hostile move, don't you think?
Yes, I agree, the general direction they've been taking has been increasingly user-hostile for a very long time; let alone the developer story.
But sometimes there's also a perfectly reasonable excuse, from both "technical" and "organizational" POV. That's just my take, a skunkworks effort to get Electron to fix their crap. I would do the same.
To be clear, Electron themselves fixed the bug quite quickly; but many Electron apps haven't pushed a version that vendors in the fixed version of the Electron runtime.
(And shit like this is exactly why runtimes like the JVM or the .NET CLR are designed to install separately from any particular software that uses them. Each of their minor [client-facing-ABI compatible] versions can then be independently updated to their latest OS-facing-bugfix version without waiting for the software itself to ship that update.)
Apple is consistent in their warnings to not use private APIs, and especially don't override them for custom implementations which is what Electron does here.
The _cornerMask override was a hack that shouldn't ever have existed in the first place, and it's not the only use of private APIs in the electron code base.
Apple is very clear about how they want you to make software for their OSes. It's 100% on electron that they choose to do it this way regardless.
I'd go as far as to say Electron itself is a hack that shouldn't exist, but sadly everyone has decided it's the only way they are going to make desktop software now.
> How else do you get the message across? Do not use the private APIs.
The most effective way would be for Apple to actually seek feedback on requirements and then actually implement public APIs for functionality that people need.
That's confusing "consensus building" with "effective". Killing a private api is pretty effective. And consensus building doesn't always build the best software.
... and in the process we will deteriorate the performance of millions of users and hurt our brand as a top class experience company?
Don't really care who is to blame, but they should have identified this, and either warn developers, or warn users. Or provide a tool for identifying guilty apps in your machine, and let users decide how to proceed.
There's no meaningful difference between "private" and "documented, but changing every patch release" from userspace POV, yet not committing to documentation saves development effort for the same result, hence "private" APIs. If anything, private apis let "system" apps run at userspace, reducing attack surface dramatically.
wtf am I reading? No no no. Undocumented apis callable from user space, that can break the OS, is a security flaw (in the OS). It’s why people laugh at windows.
This decontextualization is the antithesis of the creative studio culture he worked so hard to cultivate. The Ghibli Museum in Japan goes into this “embodied creativity” idea in great detail, and seeing this just makes me sad. I’m sure it’s a cool and technically interesting project, nice showcase for Cloudflare stack, etc. I just wish it didn’t have to be Ghibli...