prg.ai meetup: Tomáš Mikolov in a conversation with Rich Sutton

On the verge of the conclusion of the Days of AI 2024, two weeks of AI events for everyone interested in this technology, many AI experts, talents, and enthusiasts gather in the Next Zone event space to listen to the opinions of two distinguished AI scientists, Rich Sutton, and Tomáš Mikolov. The first from the duo starts off the event with an energetic talk on the substance of intelligence.

Intelligence, Cooperation, and Human Flourishing

Sutton begins by pointing out that understanding and creating intelligence have been among the oldest strivings and grand quests in human history. From biology and psychology to the philosophy of mind, people have been inquiring about the capabilities and origins of our mind. Consequently, many inquisitive engineers have been “playing God”, trying to replicate (or even improve upon) human capabilities on inanimate objects. The extent of these experiments is only confirmed by the fact that such efforts have even been imprinted in our mythologies (the legend of Golem comes to mind in the Czech context). So far, improvements in our knowledge about intelligence have always resulted in prosperity and an enriched economy, at least on average (with respect to concrete years and nations). However, can we count on this trend to continue in the future? 

Intelligence, The Ultimate Good?

Rich Sutton takes on a highly (techno-)optimistic view. In his opinion, intelligence is not only the “most powerful phenomenon in the universe” with the potential to “move stars around” in the future. It is also the ultimate good, towards which we all should strive to contribute. Provocatively, he sees possible superintelligent machinery (programs that would overpass humans in most intellectual tasks at once) as our progeny and more noble succession to inefficient humans.

It should be noted that both of these remarks caused many questions among parts of the audience (and the author of this article). I ask the reader to think critically about the following questions: Firstly, is it clear that the sum total of studying and creating greater intelligence will be positive? Or would you require concrete risk-benefit evaluations to support this claim? Secondly, how should we measure nobleness? And to whose benefit should the technology be developed?

Success And Deficits of AI

Sutton then continues by explaining the reasons behind the current success of AI. He points to Moore’s law (which claims that compute power per size of hardware doubles approx. every two years), which is also backed by data showing increasing sizes of state-of-the-art AI models through the years. However, Sutton notes, unlike the improvements in hardware, there have been almost no groundbreaking improvements on the algorithmic side. The currently omnipresent large language models are based on a technology, which is at least seven years old (or older, depending on which part of the algorithm you stress). In his opinion, the current paradigm, which heavily relies on supervised and unsupervised learning, is bound to fail and will soon run into problems with diminishing returns. As an alternative, he recommends reinforcement learning, as learning from feedback should more closely resemble real-life learning of animals and people.

Definition of Intelligence

But when we talk about the right and wrong directions of AI research, shouldn’t we first start with what intelligence is? Throughout history, scientists and engineers have used different working definitions. For example, Turing assumed that intelligent machines should “behave like humans”. Rich, on the other hand, wants the definition of intelligence to be less anthropomorphized. He goes with “[the] ability to achieve goals by adapting behavior”.

Control And Decentralization

But which goals should we program into our AI systems? Sutton imagines we should create a diverse society of interacting agents with many goals. In his libertarian views, adopting a decentralized setting with agents helping each other out of pure mutual benefit is the best way to achieve a safe and beneficial future with AI. On the other side of the barricade, he claims, are calls for centralized (safety) control of AI, which Rich compared to calls for restriction of free speech and other freedoms. I’ll ask the reader to be the judge of the following questions: Will diverse goals more likely lead to cooperation with mutual benefit or competition over scarce resources? Is the regulation of AI another restriction of our rights, or can it have a positive impact on the whole?