Let's say we could somehow train an LLM on all written and spoken language from the western Roman civilization (Republic + Western Empire, up until 476 AD/CE, just so I don't muddy the experiment with near-modern timelines). Would it, without novel information from humans, ever be able to spit out a correct predecessor of modern science like atomic theory? What about steam power, would that be feasible since Romans were toying with it? How far back do we have to go on the tech tree for such an LLM be able to "discover" something novel or generate useful new information?
My thought is that the LLM would forever be "stuck" in the knowledge of the era it was trained in. Something in the complexity of human brains working together is what drives new information. We can continue training new LLMs with new information, and LLMs might be able to find new patterns in data that humans can't see and can augment our work, but the LLM's capability for novelty is stuck on a complexity treadmill, rooted in its training data.
I don't view this ability of humans as some magic consciousness, just a system so complex to us right now that we can't fully understand or re-create it. If we're stochastic parrots, we seem to be ones that are magnitudes more powerful and unpredictable than current LLMs, and maybe even constructed in a way that our current technology path can't hope to replicate.
For your thought experiment, I'd assert that the key missing part is experimentation in the real world, as that is what acquires new information, not the complexity of human brains working together.
If you took millions of genius-level immortal humans with all the same Roman data but had them sit in a blank, empty room with their hands tied and simply discuss philosophy for eternity, I'm certain that they would not be able ever spit out a correct predecessor of modern science like atomic theory. Perhaps they could spit out billions of theories including the atomic theory as well, but they would have no data to presume that the atomic theory is more relevant than any other. Extensive information processing can squeeze out every last ounce of knowledge from some data, but anything that isn't in that data can't be acquired by mere thinking about it. On the other hand, if you gave some "LLM++" the ability to toy around with reality and attempt all kinds of experiments to test various hypotheses, then I wouldn't assume that it would not be forever stuck in the knowledge of the era it was trained in.
Yeah, I like that improvement/clarification. Good assertion. Now I wonder if it changes my stance: are the path modern LLMs are on ever going to replicate this environment for acquiring new information that humans currently operate in?
> Let's say we could somehow train an LLM on all written and spoken language from the western Roman civilization (Republic + Western Empire, up until 476 AD/CE, just so I don't muddy the experiment with near-modern timelines). Would it, without novel information from humans, ever be able to spit out a correct predecessor of modern science like atomic theory?
Let's say we could somehow train an LLM on all written and spoken language from the western Roman civilization (Republic + Western Empire, up until 476 AD/CE, just so I don't muddy the experiment with near-modern timelines). Would it, without novel information from humans, ever be able to spit out a correct predecessor of modern science like atomic theory? What about steam power, would that be feasible since Romans were toying with it? How far back do we have to go on the tech tree for such an LLM be able to "discover" something novel or generate useful new information?
My thought is that the LLM would forever be "stuck" in the knowledge of the era it was trained in. Something in the complexity of human brains working together is what drives new information. We can continue training new LLMs with new information, and LLMs might be able to find new patterns in data that humans can't see and can augment our work, but the LLM's capability for novelty is stuck on a complexity treadmill, rooted in its training data.
I don't view this ability of humans as some magic consciousness, just a system so complex to us right now that we can't fully understand or re-create it. If we're stochastic parrots, we seem to be ones that are magnitudes more powerful and unpredictable than current LLMs, and maybe even constructed in a way that our current technology path can't hope to replicate.