Uncontrollable artificial technology could lead to catastrophe, according to new book

(CN) — Researchers, scientists, companies and governments across the world should stop developing artificial intelligence because it can’t be safely controlled.

And if we don’t — apocalypse awaits, warns Roman V. Yampolskiy, computer engineering and computer science professor at the Speed School of Engineering at the University of Louisville and author of a new book that cautions against the current full-steam ahead approach to AI development.

“We are looking at an almost guaranteed event with the potential to cause an existential catastrophe. This is not a low-risk, high-reward scenario, but a high-risk, negative-reward situation. No wonder many consider this to be the most important problem humanity has ever faced. The outcome could be prosperity or extinction, and the fate of the universe hangs in the balance. A proof of the solvability or non-solvability of the AI control problem would be the most important proof ever,” Yampolskiy writes in his book AI: Unexplainable, Unpredictable, Uncontrollable , published on Monday.

With companies and governments around the world investing billions in the technology to do everything from writing news articles to designing lethal autonomous weapons of war, Yampolskiy contends that AI remains poorly understood and poorly researched, especially when it comes to safety and humanities ability to control it once it reaches “superintelligence.”

After reviewing the current scientific literature on AI, Yampolskiy said that no researcher has proven yet that the technology can be safely controlled.

“We don’t program AI, modern methods rely on machine learning for AI to self-learn from data and we don’t really know what it learns and how it will act in novel situations. For general intelligence, you can’t consider all possible possibilities,” Yampolskiy wrote in an email.

When asked how exactly that means the technology would turn catastrophic, Yampolskiy wrote that it’s not guaranteed, “but very likely as by default most non-human agents would not be value aligned with us.”

“If an AI agent doesn’t have the same values as humans, it will alter states of the environment in ways we will find unfriendly to our existence,” he added.

As AI’s abilities increase, its autonomy will as well, while humanity’s control slips away, Yampolskiy explains. As its capabilities and autonomy grows, its decision-making capabilities and possible failures are infinite, which could cause an infinite number of safety issues.

“Less intelligent agents (people) can’t permanently control more intelligent agents (ASIs). This is not because we may fail to find a safe design for superintelligence in the vast space of all possible designs, it is because no such design is possible, it doesn’t exist. Superintelligence is not rebelling, it is uncontrollable to begin with,” Yampolskiy writes in the book.

But doom is not inevitable. Yampolskiy suggests that AI programs can be made safer by categorizing them as either controllable or uncontrollable, and even implementing moratoriums and bans on different kinds of AI technology.

“It is only for a few years right before AGI is created that a single person has a chance to influence development of superintelligence, and by extension the forever future of the whole world. This is not the case for billions of years from Big Bang until that moment and it is never an option again," Yampolskiy writes in his book. “Given the total lifespan of the universe, the chance that one will exist exactly in this narrow moment of maximum impact is infinitely small, yet here we are. We need to use this opportunity wisely.”