>Hence, the first question is not whether the AI has an experience of interior subjectivity similar to a mammal’s (as Lemoine seems to hope), but rather what to make of how well it knows how to say exactly what he wants it to say. It is easy to simply conclude that Lemoine is in thrall to the ELIZA effect — projecting personhood onto a pre-scripted chatbot — but this overlooks the important fact that LaMDA is not just reproducing pre-scripted responses like Joseph Weizenbaum’s 1966 ELIZA program. LaMDA is instead constructing new sentences, tendencies, and attitudes on the fly in response to the flow of conversation. Just because a user is projecting doesn’t mean there isn’t a different kind of there there.

Yeah.

That, basically. Been thinking that for a while. In fact I think we've been there for some time now. Just because older, more primitive ones are kind of bad at it doesn't mean they're not actively goal seeking it...