I guess when it can't be tripped up by simple things like multiplying numbers, counting to 100 sequentially or counting letters in a string without writing a python program, then I might believe it.
Also no matter how many math problems it solves it still gets lost in a codebase
LLMs are bad at arithmetic and counting by design. It's an intentional tradeoff that makes them better at language and reasoning tasks.
If anybody really wanted a model that could multiply and count letters in words, they could just train one with a tokenizer and training data suited to those tasks. And the model would then be able to count letters, but it would be bad at things like translation and programming - the stuff people actually use LLMs for. So, people train with a tokenizer and training data suited to those tasks, hence LLMs are good at language and bad at arithmetic,
Arguments like "but AI cannot reliably multiply numbers" fundamentally misunderstand how AI works. AI cannot do basic math not because AI is stupid, but because basic math is an inherently difficult task for otherwise smart AI. Lots of human adults can do complex abstract thinking but when you ask them to count it's "one... two... three... five... wait I got lost".
Who does fundamentally understand how LLMs work? Many claims flying around these days, all backed by some of the largest investments ever collectively made by humans. Lots of money to be lost because of fundamental misunderstandings.
Personally, I find that AI influencers conveniently brush away any evidence (like inability to perform basic arithmetic) about how LLMs fundamentally work as something that should be ignored in favor of results like TFA.
Do LLMs have utility? Undoubtedly. But it’s a giant red flag for me that their fundamental limitations, of which there are many, are verboten to be spoken about.
You're not doing yourself a favor when you point out "but they can't do arithmetic!" as if anyone says otherwise. Yes, we all know they can't do arithmetic, and that's just how they work.
I feel like I'm saying "this hammer is so cool, it's made driving nails a breeze" and people go "but it can't screw screws in! Why won't anyone talk about that! Hammers really aren't all they're cracked up to be".
Maybe because society has invested $trillions into this hammer and influencers are trying to convince CEOs to fire everyone and buy a bunch of hammers instead.
My comment even said “LLMs have utility”. I gave an inch, and now the mile must be taken.
Saying that the fundamental limitations are things like counting the number of rs in strawberry is boring, though. That's how tokens work and it's trivial to work around.
Talking about how they find it hard to say they aren't sure of something is a much more interesting limitation to talk about, for example.
> Talking about how they find it hard to say they aren't sure of something is a much more interesting limitation to talk about, for example.
Sure, thank you for steelmanning my argument. I didn’t think I needed to actually spell out all of the fundamental limitations of LLMs in this specific thread. They are spoken at length across the web, but are often met with pushback, which was my entire point.
Here’s another one: LLMs do not have a memory property. Shut off the power and turn it back on and you lose all context. Any “memory” feature implemented by companies that sell LLM wrappers are a hack on top of how LLMs work, like seeding a context window before letting the user interact with the LLM.
But that's also like saying "humans don't have a memory property, any 'memory' is in the hippocampus". It's not useful to say that "an LLM you don't bother to keep training has no memory". Of course it doesn't, you removed its ability to form new memories!
So why then do we stop training LLMs and keep them stored at a specific state? Is it perhaps because the results become terrible and LLMs have a delicate optimal state for general use? This sounds like an even worse case for a model of intelligence.
Not entirely a straw man. What is the purpose of storing and retrieving LLMs at a fixed state if not to guarantee a specific performance? Wouldn’t a strong model of intelligence be capable of, to extend your analogy, running without having its hippocampus lobotomized?
Given the precariousness of managing LLM context windows, I don’t think it’s particularly unfair to assume that LLMs that learn without limit become very unstable.
To steelman, if it’s possible, it may be prohibitively expensive. But somehow I doubt it’s possible.
I read this back in 2009, happy to see it's still on the internet.
Obviously with today's electricity prices it would use more than $5 per year but even doubled it is extremely cheap.
My issue with the concept is space and convenience. My upright fridge is about this size but it would take up too much space in my kitchen on its side. Worse again that you can't keep anything on top because that's where the door is.
But more crucially, with a chest freezer you can only easily access the stuff on top. If something is a few levels down you have to move a lot of stuff to access it. I wish they came with shelves that cantilevered out like a toolbox, or a vertical lid on rails that lifted like a drawer
And most fridges do, I have a veg "crisper" and a meat drawer, and two boxes for cheese and other things.
I've read that if you can just minimise the amount of airflow in your fridge, even just by filling it with bottled water, it's more efficient as there's less air to fall out when you open the door. Boxes are essentially this
I'm an atheist, and I don't believe in the antichrist, but it's hard not to see how closely he fits the bill. Only lies come out when he opens his mouth. Even if he says something true, he basically qualifies it with another untruth, and people lap it up. Even the media, even the cynical media, seem to report the things he says at face value. It boggles the mind sometimes.
Last week someone challenged him on his claim that Iran had tomahawks and they bombed their own school. First time I've heard anyone directly challenge him. His response was "I don't know enough about it" classic bs packpedal, like any kid caught in a lie. Next day CNN stories were "trump doesn't know what's happening in the war, others are running it and he's unaware", completely missing the obvious truth, he lied to misdirect people on the school bombing, one person challenged it, he lied again to backpedal.
Some days it's like he has a supernatural ability to get away with lies
I'm an atheist too, but I still see religions as having embedded wisdom - both descriptive of how past societies failed, and prescriptive in that they are parts of the foundations of our present societies.
(Of course they also have a lot of details that are easy to latch onto as mere justifications for doing immoral things. And as moral people move on from traditional religion, then the share of people merely using it as crutch for immorality grows)
The archetype of a leader who engages in abjectly evil behavior while gathering ever more power and followers under a charm spell certainly rings true. But the dynamic is probably more like an individual being particularly adept at releasing the floodgates for our own worst impulses, rather than some supernatural power.
I've yet to meet anyone who wants AI added to anything. If they released a version of windows+office tomorrow that was "guaranteed free of AI" it would be their top seller
But, then all Microsoft's top managers, who apparently have bonuses based on how much AI is shoved down our throats, wouldn't get those bonuses. Nobody's cares whether or not something is a top seller because their incentives are obviously aligned toward cramming AI.
600 is a bargain for a MacBook, but I can't see the public windows users switching en masse. Most people who buy cheap windows laptops do so because
1) they need to replace a broken laptop and want to pay the lowest amount possible
2) they don't want to learn some new thing
600 might seem budget, but it's out of budget for most people. And my guess is PC manufacturers will retaliate against this by cutting prices just a little to drop under that 600 price point for mid range ryzens, with more ram and space.
Any family members I've helped shop for computers only care about how much space it has, how cheap it is, and will it struggle to run things like the last one. As it sits the MacBook is more money for less gigabytes
Also, there are plenty of users such as myself that won't be "switching" but will instead be augmenting my AMD desktop with a laptop. I've not purchased a new Mac since year 2000-ish but I do play to purchase a Neo.
The last time certain family members asked me for a computer recommendation, I gave them a detailed breakdown of which MacBook they could get to meet their lightweight needs for the next decade. They thanked me, agreed, went to Best Buy, and came back with the laptop that the salesperson convinced them was better "because he knows computers". It was an utter piece of crap and they've had nothing but problems with it.[0]
Had this existed when they were shopping, I would've just asked what color they wanted it in, ordered it for them, and been done with it.
[0] OTOH, that got me out of all future tech support duties. "Hey, why can't I connect our new printer to it?" "I'm not sure. Does that Best Buy expert still work there? He might have some suggestions." (Phrased more politely IRL because I'm not a monster, but the intent was there.)
My dad the other month, in need of a computer with webcam and ideally portable, bought some $400-500 HP 17" laptop. He was so proud of it, proud of buying a piece of hardware without asking me, and rather than tell him the truth, I nodded and said "yeah this is neat".
The monitor is awful. Like, the horrible way it changes color and brightness depending on exact viewing angle is sickening; I am shocked California hasn't declared it illegal. It feels cheap, keyboard is cheap, who knows what the battery life is.
If the Apple Neo were available then, and he had asked what to buy, I would have instantly told him to get one.
I broke that circle by having a sibling ultimately follow my recommendation of getting a ThinkPad T at a discount (prev-gen during a sale) and then letting them advertise it to the rest of the family.
If you ask me, for a comparable price range, the ThinkPad still is a much better pick than the MacBook Neo: that thing has no IO and not even enough RAM for nowadays light web browsing.
You're comparing a $1254-minimum laptop[0] with a $599-minimum laptop[1] and asserting that the one that's twice as expensive is nicer.
I'd expect it to be. In fact, I'd demand it.
(I'm ignoring the "old model, found cheaply" bit because that's entirely irrelevant. You can find old Macs on sale around, too, but that doesn't mean you can reasonably compare them to the MSRP of a brand new device.)
And I still stand behind the fact that, for that price, you've got a very competent device that is better specced for light use and friendlier for mom and pop (look, it has a HDMI, you can straight up connect it to the telly! Look, it has USB A ports, so that old camera, hard drive with the family pictures, old weird ergo mouse just works out of the box !).
Again, we’re not comparing a brand new Mac price to an old PC price. Yes, old will be cheaper. Old MacBooks are cheaper than new ones, too.
But for giggles, let’s look at the old PC.
Despite being heavier, wider, taller, thicker, slower, dimmer, lower resolution, hotter, older, and having less battery life, it is, indeed, $20 cheaper.
Put another way, there’s no way on earth I’d pick that over a MacBook Neo to save $20 at the cost of having a worse laptop in almost every way.
That's a valid opinion to hold. I think both machines are Pareto-optimal though. The ThinkPad will likely have a longer useful life because of its heavy build, extra I/O (each port gets less use), and upgradeable parts. The Neo clearly wins on power efficiency, battery life, resolution...
TBH, if I imagined I was the median casual user, I would also take the $20 marginal cost for the Neo. "Worse in almost every way" just depends on how you weight each individual parameter, which for me, is quite atypical.
I don't see why comparing prices between used and new options is unreasonable in this case. If I want a machine to do XYZ (without the stipulation that it be new), then an older model might well be better value. "In $CURRENT_YEAR, how can I get X processing power?"
Of course, old Macs should factor into that too. Also, it's a different story if I do want something brand new.
Here it’s because the old PC they picked is worse in every way than the brand new PC, except for RAM, which the Mac largely mitigates by having ludicrously fast flash hanging off the CPU. Of course an older, worse PC is going to be cheaper than a new Mac. (Except in this case, buying the boat anchor saves you a whopping $20. It’s not even better specs for the same price: it’s worse than the Apple gear that costs the same.)
If we want to compare new vs used, then how much would you have to spend to buy a brand new PC laptop as powerful as last year’s MacBook Pro?
> that thing has no IO and not even enough RAM for nowadays light web browsing.
You can literally open up every app (50+) on it and simultaneously edit 4k video without issues. It handles all of the pro apps really well. So it objectively can handle light web browsing just fine.
LOL I had the exact same experience. Somehow it was a goddamned HP too (oh how I detest HP everything).
And to think I'd explicitly mentioned to him that Apple would probably be releasing the kind of cheap beautiful laptop he was looking for in a month :(
you might be underestimating how much lifting the apple logo on the lid will do for this laptop. If it advertises the whole apple ecosystem thing well, then those people who already have iPhones, AirPods etc they would be very very compelled to go with this versus an Acer or a Lenovo
> 600 might seem budget, but it's out of budget for most people.
Out of budget for my parents but I'll pay the difference myself. It's just painful to see them use their pile of shit $300 laptop that can barely open a text editor, sounds like a jet engine and has about 45 minutes of battery life.
The only haptic feedback they get if the entire fucking thing creaking as soon as you lightly touch it.
They've been through at least 5 of them since I bought my 2015 mbp, which is still working fine in every aspects
The funny thing is that it would do the same for double the price.
You need to spend a ridiculous amount of time on research because the producer itself is selling very different product (very different quality) from a year to another.
I wish a "brand" would be consistant but it's not 99% of the time.
And it's even more painful for me to do the remote tech support for my (80+ years old) parents so paying the difference is a kind of preservation of my mental health...
You need to think about the tech support you do for your parents and decide if it would really be less by moving them to another platform, where "there's no start button" and "where did the top of the window go" and "how do I install this obscure app Ive used for twenty years"
Most of the support questions I field from my folks and in-laws are actually phone things these days. 90% of what I have to deal with is "this thing came up on my phone during the week and I clicked on it, am I hacked? No I don't remember what it said"
Anyone doing accounts and data entry wants a numpad. My dad recently damaged his laptop keyboard. I gave him a spare usb keyboard, and he still went out and bought a new keyboard just for the numpad. There's a reason pc makers keep stuffing those lopsided monstrosities in there
Anyone doing data entry with a numpad will also want a proper one, not a squishy laptop one.
But they're clearly not the majority of the people - the rest of us have to live with a lopsided keyboard because a few people for some reason do data entry on a laptop keyboard.
reply