Hacker Newsnew | past | comments | ask | show | jobs | submit | bconsta's commentslogin

David Graeber in three lines


There is a study that gives a rule of thumb of ~2 bits per param for a model's memorization capacity: https://arxiv.org/abs/2404.05405


Seems they have replicated Gardner's work, without mentioning it, "Maximum Storage Capacity in Neural Networks" (1987), which established that the storage capacity of a neural network is about 2N (2 bits per parameter)


Elizabeth Gardner for those looking.


I had no idea about this. Thanks for sharing


Recent: 3.6 bits per param

https://arxiv.org/abs/2505.24832


You're both right. The classical capacity measure (Gardner's capacity limit) is defined as the maximum number of patterns that can be remembered with zero errors. This remains 2 bits per parameter, proven mathematically.

The capacity definition in this recent paper is completely different - it is defined based on the kolmogorov complexity of predicting a memorized sequence, or in layman's terms: how easy it is to compress known sequences. This allows for some bit "errors", ie some symbols with bad compression ratio, only the total compression ratio of the sequence is measured.

This is somewhat parallel to the classical ECC limits (strict hamming distance constraints) vs modern probabilistic ECC limits.

TLDR when you allow a small number of errors, the capacity increases from 2 bits to 3.6 bits


2 bits out of FP8 would be 25% 2 bits out of FP16 would be 12.5%

I've seen recent work that claimed 70% of the params are used for memorization.


Pixel 9a is probably the closest to what you're describing on the market today.


I love the wide camera bump of the Pixels. It means the phone sits solid on the table without rocking and serves as a nice ledge on the bad for holding it. I wouldn't mind if the phone was thicker for extra battery but the bump is actually a plus in my book.


https://intl.redmagic.gg/products/redmagic-10s-pro is what you want (I have the 9S and absolutely love it)


> 23,000 RPM fan

Phones should not have fans.


I kinda like it. I'm not much of a mobile gamer, so mine only comes on to prolong battery health when the battery is fast-charging. The next model is said to gain water and dust resistance too as well as keeping the active cooling fan: https://www.gsmarena.com/redmagic_11_will_boast_a_world_firs...

I will most likely upgrade to an 11S around this time next year. The other factors that drew me to it were the huge (6500mAh) battery, a real analog headphone jack (used daily with my Etymotic ER4XR), and no dumbass notch cut out of the screen.


Which is such a shame because the Pixel 1 was exactly what they are describing. It was a physically perfect phone. It had no camera bump and the front screen wasn't bulging up past the sides so you could drop it without shattering the screen.


If I'm not mistaken, the idea is use MCP to let a user-facing LLM make tool calls to a VLA model with actions the user prescribes. He mentions using the LeRobot library in another comment.


pareto adequate


Following along these lines, I asked chatgpt to come up with a term for 'haboginking a habogink'. It understood this concept of a 'gorbink' and even 'haboginking a gorbink', but failed to articulate what 'gorbinking a gorbink' could mean. It kept sticking with the concept of 'haboginking a gorbink', even when corrected.


To be fair, many humans would also have problems figuring out what it means to gorbink a gorbink.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: