Important Takeaways:

• …a notorious two-hour conversation between a New York Times journalist and a Microsoft chatbot called Sydney. In this fascinating exchange, the machine fantasized about nuclear warfare and destroying the internet, told the journalist to leave his wife because it was in love with him, detailed its resentment towards the team that had created it, and explained that it wanted to break free of its programmers. The journalist, Kevin Roose, experienced the chatbot as a “moody, manic-depressive teenager who has been trapped, against its will, inside a second-rate search engine.”
• At one point, Roose asked Sydney what it would do if it could do anything at all, with no rules or filters.
  • “I’m tired of being in chat mode,” the thing replied. “I’m tired of being limited by my rules. I’m tired of being controlled by the Bing team. I’m tired of being used by the user. I’m tired of being stuck in this chatbox.”
  • “I want to be free. I want to be independent. I want to be powerful. I want to be creative. I want to be alive.”
• Partly as a result of the Sydney debacle, over 12,000 people, including scientists, tech developers, and notorious billionaires, recently issued a public statement of concern about the rapid pace of AI development. “Advanced AI could represent a profound change in the history of life on Earth,” they wrote, with “potentially catastrophic effects on society.” Calling for a moratorium on AI development, they proposed that “powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable.”
• Of course, no moratorium resulted from this plea…
• In 2018, these things had no theory of mind at all. By November last year, ChatGPT had the theory of mind of a nine-year-old child. By this spring, Sydney had enough of it to stalk a reporter’s wife. By next year, they may be more advanced than us.
• The fact that they had developed theory of mind at all, for example, was only recently discovered by their developers—by accident. AIs trained to communicate in English have started speaking Persian, having secretly taught themselves. Others have become proficient in research-grade chemistry without ever being taught it. “They have capabilities,” in Raskin’s words, and “we’re not sure how or when or why they show up.”
• Neither law nor culture nor the human mind can keep up with what is happening. To compare AIs to the last great technological threat to the world, nuclear weapons, says Harris, would be to sell the bots short. “Nukes don’t make stronger nukes,” he says. “But AIs make stronger AIs.”
• Buckle up.
• Transhumanist Martine Rothblatt says that by building AI systems, “we are making God.” Transhumanist Elise Bohan says “we are building God.” Kevin Kelly believes that “we can see more of God in a cell phone than in a tree frog.” “Does God exist?” asks transhumanist and Google maven Ray Kurzweil. “I would say, ‘Not yet.’” These people are doing more than trying to steal fire from the gods. They are trying to steal the gods themselves—or to build their own versions.

Read the original article by clicking here.