The Kenyan Scholar

“We do not know how to control these things,” proclaimed Geoffrey Hinton, recently.

Considering that the above statement was made by the one individual regarded as the ‘godfather of artificial intelligence (AI)’, we ought to pay keener attention to the latest AI advancements, particularly following the release of the various iterations of ChatGPT. Geoffrey Hinton is now scared of the very technology, that he helped pioneer. He has left Google to warn the world about the dangers of AI. Hinton’s decade-long research has shaped the AI products and systems we use today. In 2018, he was a co-winner of the Turing Prize, a sought of Nobel for computer science. Now he regrets his work as he spoke to BBC. The issue to Hinton is, now that we know it works, better than we expected a few years ago, what do we do to mitigate the long-term risks of things more intelligent than us taking control?

If you still are not even the least bit concerned by now, allow me to bring to your awareness the fact Hinton joins a growing chorus of experts worrying that ‘bad AI’ could conceivably even lead to the extinction of the human race. Earlier this month, Samsung banned its stuff from using tools like ChatGPT, citing security concerns. Meanwhile, the IT giant IBM announced that it will pause hiring for roles that AI could potentially fill, which puts nearly 8000 jobs at risk in the next 5 years. So, how do we innovate and protect our future? By guaranteeing the so-called ‘moral alignment’ of this expanding technology.

Before discussing public policy however, we need to understand more about the functioning of AI, to fully appreciate the degree of the existential threat the technology embodies.

I absolutely agree with Hinton’s worry that it is not inconceivable that AI could actually lead to the extinction of the human race. Moreso, it is not only inconceivable, but unfortunately quite likely as maintained by many experts including Conner Leahy, the CEO of AI company, Conjecture. Hinton, the closest person we have to an Einstein in the field of AI, is now taking the risks seriously and going to the public to actually speak about them.

So, what is current danger and the nature of this technology that is so dangerous for us?

Companies that are working on this technology – Google, OpenAI, and other ones – explicitly in their goals, for what they state they are trying to do, is to build God-like intelligence. They are not trying to build just an auto-complete system; this is explicitly their goal, explicitly stated in their founding documents. What is meant by ‘God-like, you may ask. This means something that outstrips humans in every form of capability. It is better than humans at every type of reasoning task, every type of physical task, at some point, every type of skill-based task; more creative in every way. The prevailing belief among critiques is that if a system of any kind that is vastly more intelligent than human race is created, it is not expected to end well. Thus, the question is on what should be done now.

Big AI and tech giants (known brands) signed a letter over a couple of months ago – reaching more than 1000 letters, perhaps 2000 – to call for a pause. The point of the letter was to call for a moratorium, at least for 6 months, on the development of larger and more powerful AI systems that have been built so far.  It is thus vital to first understand how software systems differ from AI systems.

Differences between a software system and an AI system:

A traditional software system requires a programmer to write code which solves a problem -- you have some problem, you want it to do something, and you write code to make it do that. AI is very different. AIs are not really written; they are more like grown. You have a sample of data, of what you want it to accomplish. You don’t know how solve the problem, you just have a description or samples of the problem. Then you use huge supercomputers to crunch these numbers, to kind of like organically almost grow a program that solves these problems. Importantly, humankind still have no idea how these programs work internally. They are complete black boxes; we don’t understand at all how their internals work; this is unsolved scientific problem; and we don’t know how to control these things!

This is the confusing bit, right? Because human beings are the ones making the hardware and other components, so, how do they not know, and therefore how are they not able to control it.

To explain, we can use the example of synthetic evolution in biology. In biology, sometimes you would like a bacterium that produces better milk for example. We don’t really know how all innings in the bacterium work, but we could select for good-milk bacteria. Meaning we can try different bacteria and keep the ones who make really good milk and then breed those and we get some more, and so on. It is quite similar to this, but not exactly. Basically, instead of programmers writing a program, they just try incredible number of programs and search for the ones that are the best programs. But the way these programs are written is not in human language; it is not in code; it is in what is called Neural Weights. These can be sought of imagined as a massive list of numbers, like billions of numbers – billions of knobs on the box and you have the big supercomputer that twiddles all the knobs, billions and billions of times, really first. And then eventually find some setting of the knobs that works, but what those knobs mean? It's unclear.

On the other hand, one can’t help but wonder what is the main positive of AI given all the work devoted to its advancement while the idea of efficiency implies job loss as in the case of IBM. Unfortunately, this always happens when new modern technology and better tools get developed; some people get replaced. Usually new jobs are created, until they are not. You know at some point; the modern civilization will run out of things for humans to do and we are approaching that. For instance, when humankind created the steam engine, it allowed humans to do lots of more cognitive labor. We could think more, we could do more writing and speaking because now the machines can do all the heavy-lifting.

But if the machines also do all the talking and all the thinking, then what is left? We can’t know for sure yet.

Currently, AI systems are still very useful. There are many applications in science and medicine that benefit greatly from AI technology developing better. There are therapeutics to understanding proteins, to also generate art and write code. For instance, many developers use products like Github Copilot, which is an AI system that aids them. It doesn’t replace them, it aids them, it answers questions, it makes writing the code faster, which is quite convenient.

From research, the scary part lies in the fact that the amount resources put into the capability of this AI far outstrips, and the graph is getting wide, the resources put into safety aspect of it, what they call the moral alignment, to make sure it is not bad and destructive. This seems completely unsustainable. Billions of funds and tens of thousands of our brightest engineers and scientist are working day-in-day-out to create ever-more powerful system; while the number of people who work full-time on the alignment problem is probably no more than 200 people according to Leahy’s estimation.

The alignment means making it safe – the moral alignment – controlling a very powerful idea. In the AI alignment field, the question is if we have superhuman/God-like intelligence, how do we make that go well. Most importantly, this is a scientific problem and also an engineering problem we have to understand. It also is a political problem to a larger degree. The challenge here is the number of people working on this and the amount of funding accessible to them is extraordinarily small. Given such a daunting state of reality, the MOST important question is:

Can we put the genie back into the bottle?

The truth is, we don’t know; we can only hope. This capability is going to be necessary to some degree but if the advancement of AI continues at the same pace, it is not going to end well. Critiques belief that the first step should advocate that the public deserves to know what is going on. People have been talking about this for years – heads of these labs have talked about how they think there are extinction risks from these things, some as far back as 2011. These are old discussions that the public is just not informed about.

The governments or parliaments of the major world economies should call upon these labs to testify under oath and actually state what is going on, how risky do you think these things actually are, what can you do about. This is perhaps the first step towards any kind of sensible regulation. We also have to talk about how do we put the genie back into the bottle --how do we progress safely.

A good recommendation is to also get inter-governmental bodies to come together and control AI, and AGI (artificial general intelligence) in particular. There are many smaller applications of AI – ANI (artificial narrow intelligence) – that do not pose significant risk unlike the type of superintelligence research, which is exactly what these large companies currently are doing.

To be very frank:

·         There is currently more regulation on selling a sandwich to the public than there is to building potentially God-like intelligence by private companies.

·         There is no regulatory oversight; there are no audits; there are no licensing processes, there is nothing!

·         Anyone can just grab a billion dollars of venture capitalist (VC) money, big supercomputers, and start doing their cutting-edge work on this and release it on the internet and no one can stop them.

The call for the 6-month pause did amount to anything because a lot of us are just not informed. There is a funny dynamic that happens very often when experts striving for the moral alignment talk with other people in this field, people often reply: oh, we can’t stop this; nothing we can do; people don’t care. But people really do care.