Unlike Apple's formal "developer evangelist" and several others I contacted, the guy actually took the time to talk to us, and I was/am grateful for that. He's a cog in a very large corporate machine. Apple is Apple. He's not the CEO. He was doing his job and did me a favor. I am grateful to him.
> Why would someone say "You're a mosquito. Apple will just stomp on you and you will not exist.", it makes zero sense to me given the context laid out here.
I'm telling you what I was told. It's a true story. I was there. It happened to me.
I mean, to be fair, these events happened several years ago. My memory is as faulty as any other human being's, but as far as I remember, this is exactly what happened. These were very memorable events that I remember distinctly. It's possible my memory is distorted, but this is literally what I believe happened to the best of my ability.
Obviously, there was more to the conversation that what I wrote, but these are the actual words that I remember being said.
For more context, the PM at Apple in question was a former colleague of my then girlfriend. I reached out to her to have a friendly catch up. It wasn't positioned as meeting officially with Apple. I was literally just going through LinkedIn trying to figure out who I knew at Apple. So I hit her up on LinkedIn and asked to catch up, then told her about the situation. And this is how she responded. Worth noting: English is not her native language.
Yeah, this is why I was begging investors for a high-level intro to someone at Apple, because I was very afraid that I’d run into some random bureaucrat that would have a similar response.
Our app wasn't running on CPU or GPU –– the actual software we built was running entirely on Apple Neural Engine and it was crazy fast because we designed the architecture explicitly to run that specific chip.
We were just calling the iPhone's built-in face tracking system via the Vision Framework to animate the avatars. That's the thing that was running on GPU.
Okay, though I'm not sure what that has to do with my comment. I understood that from the post: you were concurrently maxing out multiple parts of the SoC and it was overheating as they all contributed to the thermal load. This isn't new or novel -- benchmarks that saturate both the CPU and GPU are legendary for throttling -- though the claim that somehow normal thermal management didn't protect the hardware is novel, albeit entirely unsubstantiated.
That is neither here nor there on CoreML -- which also uses the CPU, GPU, and ANE, and sometimes a combination of all of them -- or the weird thing about MLX.
I don't get what's so weird about MLX. Apple's focus is obviously on MLX / Metal going forward.
The only reason to use CoreML these days is to tap into the Neural Engine. When building for CoreML, if one layer of your model isn't compatible with the Neural Engine, it all falls back to the CPU. Ergot, CoreML is the only way to access the ANE, but it's a buggy all-or-nothing gambit.
Have you ever actually shipped a CoreML model or tried to use the ANE?
>Apple's focus is obviously on MLX / Metal going forward.
This is nonsensical.
MLX and CoreML are orthogonal. MLX is about training models. CoreML is about running models, or ML-related jobs. They solve very different problems, and MLX patches a massive hole that existed in the Apple space.
Anyone saying MLX replaces CoreML, as the submission does, betrays that they are simply clueless.
>The only reason to use CoreML these days is to tap into the Neural Engine.
Every major AI framework on Apple hardware uses CoreML. What are you even talking about? CoreML, by the very purpose of its design, uses any of the available computation subsystems, which on the A19 will be the matmul units on the GPU. Anyone who thinks CoreML exists to use the ANE simply doesn't know what they're talking about. Indeed, the ANE is so limited in scope and purpose that it's remarkably hard to actually get it to use the ANE.
>Have you ever actually shipped a CoreML model or tried to use the ANE?
Literally a significant part of my professional life, which is precisely why this submission triggered every "does this guy know what he's talking about" button.
Yes, MLX is for research, but MLX-Swift is for production and it works quite well for supported models! Unlike CoreML, the developer community is vibrant and growing.
Maybe I am working on a different set of problems than you are. But why would you use CoreML if not to access ANE? There are so many other, better newer options like llama.cpp, MLX-Swift, etc.
What are you seeing here that I am missing? What kind of models do you work with?
I know what MLX is. MLX-swift is just a more accessible facade, but it's still MLX. The entire raison d'être for MLX is training and research. It is not a deployment library. It has zero intention in being a deployment library. Saying MLX replaces CoreML is simply nonsensical.
> But why would you use CoreML if not to access ANE?
The whole point of CoreML is hardware agnostic operations, not to mention higher level operations for most model touchpoints. If you went into this thinking CoreML = ANE, that's just fundamentally wrong at the beginning. ANE is one extremely limited path for CoreML models. The vast majority of CoreML models will end up running on the GPU -- using metal, it should be noted -- aside from some hyper-optimized models for core system functions, but if/when Apple improves the ANE, existing models will just use that as well. Similarly when you run a CoreML model on an A19 equipped unit, it will use the new matmul instructions where appropriate.
That's the point of CoreML.
Saying other options are "better, newer" is just weird and meaningless. Not only is CoreML rapidly evolving and can support just about every modern model feature, in most benchmarks of CoreML vs people's hand-crafted metal, CoreML smokes them. And then you run it on an A19 or the next M# and it leaves them crying for mercy. That's the point of it.
Can someone hand craft some metal and implement their own model runtime? Of course they can, and some have. That is the extreme exception, and no one in here should think that has replaced anything
It sounds like your experience differs from mine. I oversaw teams trying to use CoreML in the 2020 - 2024 era who found it very buggy, as per the screenshots I provided.
More recently, I personally tried to convert Kokoro TTS to run on ANE. After performing surgery on the model to run on ANE using CoreML, I ended up with a recurring Xcode crash and reported the bug to Apple (as reported in the post and copied in part below).
What actually worked for me was using MLX-audio, which has been great as there is a whole enthusiastic developer community around the project, in a way that I haven't seen with CoreML. It also seems to be improving rapidly.
In contrast, I have talked to exactly 1 developer who have ever used CoreML since ChatGPT launched, and all that person did was complain about the experience and explain how it inspired them to abandon on-device AI for the cloud.
___
Crash report:
A Core ML model exported as an `mlprogram` with an LSTM layer consistently causes a hard crash (`EXC_BAD_ACCESS` code=2) inside the BNNS framework when `MLModel.prediction()` is called. The crash occurs on M2 Ultra hardware and appears to be a bug in the underlying BNNS kernel for the LSTM or a related operation, as all input tensors have been validated and match the model's expected shape contract. The crash happens regardless of whether the compute unit is set to CPU-only, GPU, or Neural Engine.
*Steps to Reproduce:*
1. Download the attached Core ML models (`kokoro_duration.mlpackage` and `kokoro_synthesizer_3s.mlpackage`)
2. Create a new macOS App project in Xcode. Add the two `.mlpackage` files to the project's "Copy Bundle Resources" build phase.
3. Replace the contents of `ContentView.swift` with the code from `repro.swift`.
4. Build and run the app on an Apple Silicon Mac (tested on M2 Ultra, macOS 15.6.1).
5. Click the "Run Prediction" button in the app.
*Expected Results:*
The `MLModel.prediction()` call should complete successfully, returning an `MLFeatureProvider` containing the output waveform. No crash should occur.
*Actual Results:*
The application crashes immediately upon calling `model.prediction(from: inputs, options: options)`. The crash is an `EXC_BAD_ACCESS` (code=2) that occurs deep within the Core ML and BNNS frameworks. The backtrace consistently points to `libBNNS.dylib`, indicating a failure in a low-level BNNS kernel during model execution. The crash log is below.
I can't speak to how CoreML worked for you, or how the sharp edges cut. I triply wouldn't comment on ANE, which is an extremely limited bit of hardware mostly targeted at energy efficient running of small, quantized models with a subset of features. For instance extracting text from images.
CoreML is pervasively used throughout iOS and macOS, and this is more extensive than ever in the 25 versions. Zero percent of the system uses MLX for the runtime. The incredibly weird and nonsensical submissions weird contention that because ANE doesn't work for them, therefore Apple is admitting something is just laughable silliness.
And FWIW, people's impressions of the tech world from their own incredibly small bubble is often deeply misleading. I've read so many developers express with utter conviction that no one uses Oracle, no one uses Salesforce, no one uses Windows, no one uses C++, no one uses...
In my conversations with people at Apple, my understanding is that they do not use CoreML. Instead, they have access to lower level libraries that allow more direct programmatic control of the hardware.
CoreML is the crappy middleware they made for 3rd party devs, but never got much love and never took off.
Re: ANE — as stated, ANE is crazy fast when it works. Yes, it’s also more power efficient, but the reason I think it’s actually worth building on is being able to make consumer products where the entire experience depends on speed.
I think you can agree that 5 milliseconds to generate a 512 px resolution image is absolutely insane speed.
mostly because you're asking me to read claude transcripts as though those are actually interesting. i could have simply ignored your article like i do for dozens of others on HN daily, but i loathe the day the internet becomes articles written mostly by ai, summarized by ai, and consumed by ai-addled hobbyist "devs" who don't understand the degree to which they are ruining software
Did you read it? And why do you assume that it can't be interesting?
Honestly, as a writer, I sometimes spend a bunch of time writing an essay that's the result of a conversation, and I realize –– hours later –– that the conversation actually does a better job of exposing the idea and explaining it than the essay I worked so hard to produce.
I agree that Claude convos can be slop, but don't bash it before you read it!
In 2021, I built the first real-time video generation AI app for Apple Silicon. It was revolutionary - until it started melting iPhones. When my demo literally burned Sam Altman's hand, I discovered that Apple's on-device AI hardware was not designed for modern AI.
Look inside any M-series Mac today. Nearly half the silicon - the Neural Engine cores - sits idle, designed for the pre-transformer era of AI. Meanwhile, my ninety-five-year-old father struggles with a seventy-five-inch screen he can barely see, finding brief moments of magic only when speaking to Siri.
The problem isn't just technical - it's ideological. Apple's privacy stance prevents them from gathering the real-world data needed for truly adaptive interfaces. But there's a solution: AppleCare Platinum, a premium human support service that could evolve into a radical new kind of UI - by serving as an Enders Game for AI computer use training data.
I don't see why Apple couldn't just renege their privacy agreement and tell their userbase to pound sand. Not like they'd have any real alternatives anyways.
Or they could afford to lose ~10-50k lifelong customers to accrue the data required to onboard the next million users. I think people vastly overestimate how many Apple customers consider privacy a dealbreaker for their purchase.
Unlike Apple's formal "developer evangelist" and several others I contacted, the guy actually took the time to talk to us, and I was/am grateful for that. He's a cog in a very large corporate machine. Apple is Apple. He's not the CEO. He was doing his job and did me a favor. I am grateful to him.