For better or worse, humans are very good at anthropomorphizing things. Put Googly Eyes on a trash collector and people call it Mr. Trash Wheel.
People think it’s sentient, because if you talk to it, it responds in sometimes unexpected ways. For humans, this is only possible through sentient thought, and so people think that LLMs are sentient, because they assume it’s like them.
While I don’t think this scenario likely, something that I can’t help but thinking when this sort of statement comes up is, well, how do we know what it’s doing isn’t thinking? Like, I get that it’s ultimately just using a bunch of statistics to predict the next word or token or whatever, but my understanding was that we have fairly limited knowledge of how our own consciousness and thinking works, and so I keep getting the nagging feeling of “what if what our brains are doing is similar somehow, using a physical system with statistical effects to predict stuff about the world, and that’s what thinking ultimately is?”
While I expect that it probably isn’t and that creating proper agi will require something fundamentally more complicated than what we’ve been doing with these language models and such, the fact that I can’t prove that to my own satisfaction makes me very uneasy about them, considering what the ethical ramifications of being wrong about it might be.
Because it’s not. The base architecture of how it works is by probabilistic word suggestion. That isn’t thought.
We have a concept of self. We understand our place. We can interpret and respond to entirely new situations. LLMs routinely fail that. They regularly fall into local minima that keep it on the wrong path, and I’ve personally seen them just… Get lost in the weeds and swing back and forth based on what you tell it.
Give it a protein sequence and tell it to calculate the pI. Then tell it it’s wrong. “Oh my bad yes you’re right it’s {whatever you said it was}.”
Tell it you lied and that the number you said was wrong, and it turns up saying “Yes, you’re correct, the pI is {original value}” - that is objectively false.
That is not the behavior of something that thinks. That’s the behavior of a simple probability model updating priors and weighting things differently by the most recent information you gave it.
LLMs are soulless, brainless, thoughtless word generators. And they have some uses.
Why people are treating LLM as AGI?
Those things do not think. They are trained on our posts on social media and I’m sure there were discussed being banned.
tell a good story and it becomes alive
For better or worse, humans are very good at anthropomorphizing things. Put Googly Eyes on a trash collector and people call it Mr. Trash Wheel.
People think it’s sentient, because if you talk to it, it responds in sometimes unexpected ways. For humans, this is only possible through sentient thought, and so people think that LLMs are sentient, because they assume it’s like them.
Eyes narrow
We better be talking about how we all love Mr. Trash Wheel.
I’m not sure I love Mr Trash Wheel, but we fuck like we’re in love.
Because it uses a natural language interface.
While I don’t think this scenario likely, something that I can’t help but thinking when this sort of statement comes up is, well, how do we know what it’s doing isn’t thinking? Like, I get that it’s ultimately just using a bunch of statistics to predict the next word or token or whatever, but my understanding was that we have fairly limited knowledge of how our own consciousness and thinking works, and so I keep getting the nagging feeling of “what if what our brains are doing is similar somehow, using a physical system with statistical effects to predict stuff about the world, and that’s what thinking ultimately is?”
While I expect that it probably isn’t and that creating proper agi will require something fundamentally more complicated than what we’ve been doing with these language models and such, the fact that I can’t prove that to my own satisfaction makes me very uneasy about them, considering what the ethical ramifications of being wrong about it might be.
Because it’s not. The base architecture of how it works is by probabilistic word suggestion. That isn’t thought.
We have a concept of self. We understand our place. We can interpret and respond to entirely new situations. LLMs routinely fail that. They regularly fall into local minima that keep it on the wrong path, and I’ve personally seen them just… Get lost in the weeds and swing back and forth based on what you tell it.
Give it a protein sequence and tell it to calculate the pI. Then tell it it’s wrong. “Oh my bad yes you’re right it’s {whatever you said it was}.”
Tell it you lied and that the number you said was wrong, and it turns up saying “Yes, you’re correct, the pI is {original value}” - that is objectively false.
That is not the behavior of something that thinks. That’s the behavior of a simple probability model updating priors and weighting things differently by the most recent information you gave it.
LLMs are soulless, brainless, thoughtless word generators. And they have some uses.
They don’t think for now