AI Jesus? Experts question wisdom of godbots

The question we ask is, “What is it about the chatbot that makes it seem like a good place to turn for answers?” Our answer is that the design of the chatbots invites us to treat them like more-than-human oracles. Why? First of all, they’re opaque. They don’t show you their workings. And so they can trigger a cognitive response in people that actually has a very long history. They do what oracles, prophets, spirit mediums, and divination practitioners have always done. They have access to a source that is totally mysterious. It can seem to be tapping into something that just knows more than I do. A source like that seems more than human. It can appear to be divine.

If you go through the history of human divination techniques, you see this is repeated over and over again, whether it’s ancient Chinese casting the I-Ching or Yoruba casting cowrie shells. An example we use in our article is sacrificing animals and then studying their entrails to find marks that come from the spirit world, a very common practice found from ancient Rome to many contemporary societies. Or the Delphic Oracle, who seems to have been a spirit medium, someone who went into a trance and whose words, sometimes quite enigmatic, seemed to come from elsewhere.

You don’t have to believe in divine authority for this to work. You just need to feel that AI surpasses humans. The urge to turn to it for answers can start with no more than that. I really want to stress this point: we are not saying “Well, some suckers will fall for this.” The godbots are just an extreme case of something that’s actually much more common. People who pride themselves on their scientific rationality are susceptible, too.

Now, the second aspect of the chatbots is that they’re designed to give you one answer and to give it with complete authority, without any doubts. When Harry Truman was president, he supposedly complained about his economic advisers: “When I ask them for advice, they say, ‘Well, on the one hand, this, on the other hand, that.’” Truman said, “I want someone to find me a one-armed economist!”

Right now that’s what chatbots do. This is one way they’re more dangerous—and perhaps more appealing—than, say, the Google search function. Google says, “Well look, here’s a whole bunch of sources.” So it’s at least implying that there isn’t necessarily just one answer. Look at all these different sources! And if you want, you can go further into them, even compare them to one another.

Chatbots in their current state aren’t like that. In effect they say, “I’m not going to tell you where I got the answer. You just have to accept it. And there’s only one answer.” Life is complex, is often bewildering and there’s an irresistible attraction to things that promise to make it simpler.

And again, it’s the design of the chatbot that because of its opacity, on the one hand, it has all the authority of crowdsourcing. For better or for worse, we’ve come to place a huge amount of faith in the wisdom of the crowd, and then projected it onto chatbots. As a result, it seems to know more than any human could possibly know. So, how can you doubt it?

And its inner workings are opaque—even computer programmers will tell you that some of the things going on in these algorithms are just too complex to explain. It’s not necessarily that they don’t understand their own devices, but that the explanation can be just as complicated as the thing it’s meant to explain.