Artificial Intelligence

What is Artificial Intelligence?

Examples

Artificial Intelligence (AI) has set itself the goal of creating “intelligent systems.” The name was first used in 1956, and since then AI has been surrounded by myths and controversy. On one side, some predict paradise on earth: a utopian future without worries, in which intelligent robots provide people with everything. On the other side are people who see Artificial Intelligence as the personification of evil and fear the end of humanity. Over the course of this chapter, it will become clear that both sides are very far removed from what is technically feasible.

But even if AI cannot yet create “intelligent systems,” many useful intermediate products have already been developed [RN10, BA14]:

• Self-driving cars, trains, and subways
• Letters can be dictated to computers
• Texts can be translated into other languages at medium quality
• Robotic vacuum cleaners clean homes
• In 2007, IBM Watson won a game show in the USA
• In 1997, the computer Deep Blue defeated a chess grandmaster

Rational Agents

In AI, the term “agent” is also used. Here, however, an agent is not part of a simulation or a model, but a robot, a computer system, a program, or another machine [RN10]. An agent lives in an environment and perceives information through its sensors. With its actuators, the agent can interact with the environment.

If the agent is a computer program, the environment could be a single computer, a home network, or the internet. As sensors, the agent could, for example, measure the room temperature with thermometers and use its actuators to regulate the heaters accordingly. Today, of course, an agent that only controls heaters would no longer be called intelligent.

An agent could also be a computer program that searches the internet for certain things. As a sensor, it would then have a kind of web browser, and as an actuator it could, for example, send the links to the pages it found by email. The recipient of this email could be another agent. If several agents work together in a system, that system is also called a multi-agent system.

An agent could also be a vacuum cleaning robot or a self-driving car. In this case, the environment would be the physical world.

AI, however, has set itself the goal of developing intelligent agents. But what is an intelligent agent? There are two different possibilities:

• The agent should think and act like a human.
• The agent should think and act rationally, i.e., achieve the optimal result.

Which of the two possibilities is pursued will depend on the agent’s area of application. A personal assistant, such as a virtual secretary, should behave as much as possible like an intelligent human being (but of course without imitating human mistakes, such as stubbornness or a bad mood on Monday morning). A self-driving car, by contrast, should not react as sluggishly as a human being and should “step” on the brake as immediately as possible to avoid an accident.

To test the first of these two possibilities, the so-called Turing test was developed [RN10].

The Turing Test

Via Chat

The Turing test was proposed in 1950 by the British “computer scientist” Alan Turing [RN10]. In this test, a test subject uses a chat app to chat with another person. The test subject does not know, however, whether the other party is a human or an AI, and is supposed to find out. There is no contact other than chat. The Turing test is considered passed if the test subject thinks they have spoken with a human, even though it was an AI.

To accomplish this task, the AI must perform the following steps:

1. Analysis of natural language in the chat (Language Analysis)
2. Extraction and storage of information (Knowledge Representation)
3. Logical processing of information (Semantic Processing)
4. Memorization of information for later use in the dialog (Machine Learning)
5. Determining a logical answer or starting small talk itself (Semantic Processing)
6. Formulation of the answer in natural language (Language Generation)

Steps 1, 2, and 6 deal with natural language. In steps 1 and 2, language is analyzed; in step 6, language is generated. Steps 3, 4, and 5 work with knowledge.

In step 1, a syntactic analysis is performed. Some readers may have learned this in school. The sentence in natural language is transformed into a tree based on the syntactic categories of the individual words. For example, in the sentence “Ich sehe einen Baum” (“I see a tree”), the subject “Ich” (“I”) is recognized as being followed by the predicate “sehe” (“see”), which in turn is followed by the adjective “einen” (“a”) and the object “Baum” (“tree”). The syntactic structure is stored in the computer as a tree.

Such a tree is also called a syntax tree. Teaching a computer the grammar of natural languages is still a relatively easy task. To do this, the computer has to have a database of words and their linguistic information. It also has to know the grammar rules, agreements, and so on. But here the amount of information is limited and relatively static. The grammar of a language usually changes only slowly.

Things become more difficult with the meaning of the words, with semantics. Natural language offers many opportunities for misunderstanding. In a Turing test, one can test whether the AI can correctly resolve pronouns, that is, whether it understands what “him,” “her,” “its,” and so on refer to. If the test subject says the following sentence:

I see a tree and a bottle of orange juice.

Then ignores the AI’s answer and writes:

I'm thinking about whether I should drink it.

How is the AI supposed to know what “it” refers to? Is it the tree or the orange juice? Humans know that one cannot drink a tree, so “it” refers to the juice. The AI would need knowledge about “trees” and about “drinking.” It would need to know something about liquids, that humans can drink liquids, that orange juice is liquid, and that trees cannot be drunk. The AI needs semantic information. A database containing knowledge about the world is also called an ontology. Creating such a database on a large scale, so that an AI has enough knowledge to pass the Turing test, has not yet succeeded [RN10]. Semantic processing and the Semantic Web are still research topics. In the past, only small advances have been made here. So far, such ontologies exist only for small subareas of human life. For example, one can tell a mobile phone, “send a text message to my mother.”

But even if progress is made here, a clever person will be able to trick the AI. They could probe the AI for gaps in its semantic knowledge database. The test subject could, for example, also say:

I wonder what kind of one that is.

So what exact kind of tree, or what kind of orange juice? Humans would reason as follows: since the test subject has already specified the juice more precisely, since it is juice from “oranges,” but not the tree, this sentence probably refers to the tree. The test subject is probably wondering whether it is a birch, an oak, or a chestnut. Maybe they do not know much about biology?

For that, the computer would already need a huge knowledge database and possibly a longer time to “think.” Even small children often still have difficulty understanding exactly what the test subject means. To build up this analytical ability, humans have to learn for many years.

Unfortunately, creating an ontology as a knowledge database is not so easy. It is a very extensive task, and as with computer programming, errors can creep in. This problem will persist until computers can automatically create such ontologies themselves, and how that is supposed to be possible has not yet been discovered.

Via Spoken Language

A simple extension of the Turing test would be to use spoken language instead of chat. The test subject could then talk to the human or the AI through a microphone and speaker. Adding spoken language is no longer difficult. Operating technical devices with natural language is expected to be widespread by 2020. Programs that one can dictate to already exist today. The problem remains semantic processing.

The Visual Turing Test and Humanoid Robots

A further extension of the test would be a conversation by video chat. The test subject sees an image of the human or the AI on the monitor.

There are two possibilities here:

1. The image is simulated, i.e., generated by computer graphics. The AI only exists virtually.
2. The AI is a humanoid robot and is recorded by a camera.

In the first case, the AI would also need image analysis (“computer vision”) in order to interpret the test subject’s mood. As a test of the AI, the test subject could make a face or perform certain gestures with arms and hands. Image analysis is now implemented with neural networks, which are discussed in Section 8.5. Interpreting human gestures and faces is also possible with neural networks.

In addition, the AI would have to have a face that credibly matches what is being said and is judged by the test subject to be “truly human.” This will be possible at some point, but not in the next 10 years.

For the second case, however, a human-looking robot is required, and that is currently still impossible and remains the province of science-fiction films and books.

Rational Agents

If an agent is to think and act rationally and achieve the optimal result, a theory of rationality is required. A rational agent is an agent that does the “right thing.” But what is the “right thing” in a particular situation, given the means available to the agent? How much time does the agent have to “think”? When solving complicated problems, such as playing chess, it is better to have as much time as possible in order to try out as many moves as possible “in one’s mind.” For an agent to appear intelligent, it also has to consider the consequences of its planned actions. A robot that walks into a wall does not appear intelligent. Rational behavior depends on the environment, the quality of the sensors and actuators, the available time, and the “thinking ability.” The “thinking ability” of a technical agent depends on the number of CPUs, the size of the main memory, and the algorithms used. In Section 3.2, we learned about the bounded rationality of humans; a technical agent also has bounded rationality, although with different limits.

Rationality therefore means carrying out the best possible action under the given circumstances. For this, many theories from different sciences are needed:

• Decision Theory
• Behavioral Economics
• Logic
• Statistics and Probability Theory
• Mathematics
• Philosophy
• Knowledge Acquisition and Knowledge Processing
• Complexity Theory
• Economics
• Game Theory
• Computer Science and Formal Systems

The task of creating rational agents is therefore a major interdisciplinary challenge. For this reason, it has so far only been possible to create agents for small application areas, that is, specialists, such as chess-playing programs.

In AI, as with complex systems, there is also a difference between the more mathematical methods that process numbers and the “symbolic” methods that also process texts and symbols. The symbolic methods produce data in a form that humans can understand. Trees, networks, graphs, and logics such as predicate logic are used here as knowledge-representation languages. Another example is the decision trees discussed in Section 7.2. The models of the numerical methods cannot be understood by humans and have to be used by computer programs. An example is the neural networks discussed in the section after next.

Search

Many problems in Artificial Intelligence resemble the search for a needle in a haystack. Since computers can only do one thing after another, the haystack is searched straw by straw until the needle is found. As an example, let us consider the Traveling Salesman Problem (TSP) [CLRS09]. A band wants to perform in four different cities and is planning a round trip in which it visits each city exactly once for a concert. The musicians want to perform in Hamburg, Berlin, Munich, and Cologne, collect information about travel costs, and create the following graph:

Each edge is “weighted” with the cost of the trip. The band wants to start in Hamburg. If they simply set off, for example first to Berlin, then Munich, then Cologne, and back to Hamburg, the trip costs 2+5+3+4 = 14.

But which is the cheapest round trip?

The simplest but most time-consuming way to solve the problem is to calculate all possibilities and then choose the cheapest one. With the four cities above, there are three possibilities from Hamburg, then 2, then 1, for a total of 3*2*1 = 6 possibilities. The costs for these 6 possibilities are listed below.

H B M K H: 2+5+3+4 = 14

H B K M H: 2+1+3+1 = 7

H M B K H: 1+5+1+4 = 11

H M K B H: 1+3+1+2 = 7

H K M B H: 4+3+5+2 = 14

H K B M H: 4+1+5+1 = 11

So it can be cheaper than 14, because the optimal trip costs 7 and goes from Hamburg to Berlin, then to Cologne, then to Munich and back, or in the reverse order. It therefore made sense to search for the optimal solution.

Theoretical computer scientists have studied the Traveling Salesman Problem extensively. They asked whether one really has to try all possibilities to calculate the cheapest route. Or is there a shortcut? Some problems can also be solved well with heuristics, such as always taking the cheapest local connection.

A major practical problem is that the number of possibilities grows very strongly with the number of cities. For many cities, the calculation would take too long even on supercomputers in data centers. Let n be the number of cities. With n cities, the number of possibilities is the so-called factorial of (n-1). That is n-1 * n-2 * n-3 * … * 2. This is a very large number. The Traveling Salesman Problem is a so-called NP-complete problem, because the number of required computation steps grows exponentially with n. The haystack being searched therefore becomes very, very large. We will examine this exponential growth in more detail later in another context, in Section 9.1.

Important: NP-complete problems require an exponential amount of computing time.

This has not yet been proven, but it is very likely¹.

In Artificial Intelligence, there are many other “search problems” [RN10], which luckily are not all NP-complete:

• What are the optimal moves in a game of chess?
• What is the optimal path to clean the floor (vacuum cleaning robot)?
• What is the optimal layout for the transistors on a processor?
• Which stocks should one buy or sell today?
• How can a container port be utilized as fully as possible?
• Can a mathematical formula be proven? (automatic theorem provers)

Search can be compared with System 2 of the human brain from behavioral economics. It is the automatic scanning of possibilities. For NP-complete problems, however, the search for a solution can take a very long time.

Neural Networks

In the course of this book, we have already seen various graphs and networks. The human brain is also a network, although one made of neurons. Mental activity in the brain arises through electrochemical reactions in this network. A neuron can connect with as many as 10 to 100,000 other neurons. These connections are dynamic and change when someone learns something new. Biologically, however, this is far more complicated, because many other elements, such as axons and synapses, are involved.

The following table contains a rough comparison between a PC from 2015 and the human brain [RN10]².

The number of computing and storage units no longer differs greatly between the brain and the PC today. The biggest difference is the clock cycle: brains are much slower than PCs. A neuron switches a million times more slowly than a transistor. Overall, however, the brain is much, much more networked and parallel, and is therefore faster as a whole. In the PC from the table, only 12 cores work in parallel, while in a brain all neurons can work at the same time. The “architecture” is completely different. For this reason, the PC will not automatically become as intelligent as a brain even if the figures in the table converge further in the future. A neuron works quite differently from a transistor. A neuron is biochemical and analog, while a transistor is electronic and digital.

But if the brain is such a good “computer,” then it seems natural at least to imitate it. And that is exactly the idea behind neural networks. As early as 1943, Warren McCulloch and Walter Pitts created a simple model of a neuron. The following figure shows a simplified neuron.

A neuron has any number of inputs; in this example there are three: w0, w1, and w2. The input edges come either from other neurons or from the input of the whole network. Each edge has two pieces of data: an activation level a and a weight w. These are also called “weighted edges”. From the inputs, a weighted sum is calculated in the first component of the neuron. In the second component, the outgoing activation level of the neuron is calculated. In one computation step, the neuron therefore calculates the weighted sum of the activation signals from the inputs and activates its output signal if the sum is greater than a threshold k.

Several such neurons are combined into a neural network. The following figure shows a network with one layer of neurons between input and output.

Such a network can represent various functions. Two inputs are supplied, the neurons calculate their output from the inputs and the weights, and this output can be read at the end. Such a network is already somewhat flexible. If the weights are changed, the result of the network changes. Let us assume that the weights have the following values: w1,1=0.4, w1,2=0.2, w2,1=0.6, and w2,2=0.8, and that input 1 (e1) is 0 and input 2 (e2) is 1. Then neuron 1 calculates the value w1,1*e1 + w2,1*e2 = 0.4*e1+0.6*e2 = 0.6, and neuron 2 calculates the value w1,2*e1 + w2,2*e2 = 0.2*e1 + 0.8*e2 = 0.8.

Such a network can be trained. One specifies both an input and an output and calculates what the weights of the edges would have to look like. This corresponds to supervised learning. A network with only one layer, a single-layer network, however, cannot represent many useful functions, as Marvin Minsky and Seymour Papert proved in their 1969 article “Perceptrons”. As a result, much research funding in the USA was cut and AI research was discontinued. The first “AI winter” arrived, and neural networks were forgotten.

That changed in the mid-1980s, when neural networks were rediscovered. With the “backpropagation” algorithm, it had become possible to train multilayer networks as well. And multilayer neural networks can learn many more functions, even the nonlinear regressions from Section 7.2 [RN10]. Neural networks handle numerical data well; they are “number crunchers.” Often they can also tolerate noisy inputs or inputs with errors. They are fault-tolerant or robust in the sense of Nassim Taleb (see Section 2.7). They are therefore especially well suited for analyzing image data, videos, or speech recordings.

In the following figure, a multi-layer network is shown.

Between the two layers in the middle are the buffers T1 and T2. Real networks of course have many more than just two neurons per layer. Neural networks were used regularly in practice in the 1980s and 1990s, but only for simple tasks. Training often still took too long. Research therefore became quieter again around the end of the 1990s. That changed again in 2010, when GPU computing made it possible to move computation-intensive functions to graphics cards. With a GPU, training a neural network is up to 10 times faster than with a CPU. Starting around 2010, larger networks could therefore be trained with more example data. There was also much more data than ten years earlier. Digital cameras had made great progress. This led to networks with many more layers than before. There were also a number of algorithmic improvements and advances in the theory of machine learning [SB14]. The term “Deep Learning” was introduced for the new techniques. Especially large advances were made in “computer vision.” An image or photo can be looked at from different distances. One can move very close or look at larger parts in context. The trick in deep learning is to introduce a layer for each level of detail. The following figure shows this schematically.

The intermediate layers F1, F2, and F3 store “details” (“features”). In reality, there are networks with more than 10 layers. These “deep” networks can be arranged in a way that makes self-driving cars possible.

This has to be stated very clearly: neural networks learn from example images. “Computer vision” does not mean an actual “understanding” of an image. After appropriate training, a neural network can assign a name to images of already known people with reasonable certainty, or recognize cars on a street. But it is only a mathematical function in a form usable only by computers [GW08]. The knowledge is encoded “numerically.” Neural networks are not themselves “intelligent”; rather, the intelligence lies in the developers who arrange the various layers intelligently, set the weights and thresholds correctly, and train the network with the right data.

As explained at the beginning of this section, neural networks are a simplified imitation of the brain. The biggest difference from a real brain, however, is that a computer is based on silicon transistors with 0/1 logic. A brain is biochemical and analog, so it can represent many different intermediate states. We will return to this later in Section 10.6, in connection with the Singularity.

These neural networks are comparable to System 1 from behavioral economics. They can be used to store heuristics and “fuzzy” logic.

Intelligent Robots

For a human, moving through an apartment is very easy, while playing chess very well is hard. For a robot, it is exactly the other way around. Abstract thinking, which is strenuous for humans, is easier to automate than sensorimotor movements, which are easy for humans. This contradiction is also called Moravec’s paradox, after the robotics researcher Hans Moravec [BA14, RN10].

In terms of Systems 1 and 2 from Section 3.2: System 1 has so far been difficult to automate, System 2 easier. The situation has improved somewhat because of “Deep Learning.” But even today, there is still no robot that can clean an apartment completely without breaking too much. It has been very hard to teach a robot a sense of balance. For this, it needs good sensors that tell it what its environment is like, but also what state its joints are currently in. This has to be calculated in real time and corrected as quickly as possible when there is a risk of imbalance. Recently, however, progress has been made here, and today it looks as if the “balance problem” will be solved in the near future.

Another task for a robot is to find its way in unknown environments. To do this, the robot creates a map of its surroundings. A robot can “understand” its environment, however, only with functioning image recognition. As already mentioned, image recognition was not practical enough before “Deep Learning.” The robot has to enter the objects it has recognized on its “internal” map. At the same time, it has to determine its own position on the map. This is harder than it looks at first glance. A robot moves with motors. It can tell its motors: move me 1 cm forward. But whether it has then really moved 1 cm or only 0.95 cm depends on the resistance of the floor. On parquet flooring, movement is much “smoother” than on a thick carpet. The robot has to check visually where exactly it is afterward. In English, the problem is called “Simultaneous localization and mapping” (SLAM). The right algorithms have since been developed for this. SLAM was still considered a major challenge for robotics in 2008, but it has now been solved satisfactorily for many application areas [BA14].

Today, robots are used in many areas, mainly in industry, for example in automobile production or warehousing. Robots have also entered private households as robotic vacuum cleaners. The city of Tokyo plans to use self-driving taxis at the 2020 Olympic Games. Robots are therefore continuing their advance. The consequences for the labor market are discussed later in Section 11.7.

Status of AI

In the 1960s, many AI researchers predicted the imminent arrival of intelligent computers. AI is a good example of the “horizon problem” in research projects. At first, everything looks very simple; one makes a plan and thinks the work should be finished in five years. During work on the project, however, new questions arise that one had not thought of before and that may previously have been unknown. The “knowledge horizon” shifts during the project. The original plan becomes obsolete. Along the way, however, interesting things have been discovered. It was very important to engage intensively with AI, because otherwise many products would not exist today, such as robotic vacuum cleaners and chess computers. But it must also be said clearly that AI has not reached its ultimate goal of “intelligent agents.” And at present it does not look as if universally intelligent agents will exist in the near future (until 2025). This is because Artificial Intelligence today has the following limitations:

• AIs do not have general universal intelligence; they are specialists and are only used in small, defined areas
• AIs cannot “understand” information
• AIs are not “creative”
• AIs have no introspection, cannot “understand” themselves
• AIs have no consciousness and no self-knowledge

Today’s AIs at most “appear” intelligent, but they are not.

General AI

So far, AIs have been developed only for special application areas and tailored to one task. A person, however, is “intelligent” only if they are flexible and can learn new things. Today, creating a universal AI is not possible in practice.

Semantics

This is partly because computers today cannot perform semantic processing of information. What does the word “Hamburger” mean? It can be food or a person. Humans infer this information from context. These semantic analyses, however, are very complicated. No method has yet been discovered for simply extracting semantic knowledge from the internet. Having experts encode this knowledge manually is also very labor-intensive. But of course progress will be made here over time. A “knowledge database” is conceivable, for example, to which one could pose logical and statistical questions. But progress in this area is slow and will not advance in leaps and bounds. The bottleneck is the creation of large knowledge databases and ontologies. At present, human thinking is still necessary for this.

Consciousness

Can one be intelligent at all without having “consciousness”? Consciousness is not so easy to define; roughly, it means being “aware” of oneself and one’s “existence” [Bla05]. The famous phrase here is René Descartes’ “cogito ergo sum”. Some researchers claim that consciousness is the emergent result of the complex system of the brain [RN10]. But there are also researchers who say that we do not yet know exactly what “consciousness” itself actually is [Bla05]. For example, no special region for consciousness has been found in the brain. And as long as humanity does not know how something works in nature, it cannot rebuild it. The current state of knowledge is therefore not yet sufficient to answer the question of whether computers can have consciousness. Scientists differ in their opinions on this, however [Bro15].

Imitation of Humans

There is a very large difference between AI and humans: AI has no animal heritage. Humans are intelligent mammals descended from apes. In addition to intelligence, humans also possess instincts inherited from animals, such as sexuality. The zoologist and behavioral researcher Frans de Waal found in chimpanzees that they have pronounced social behavior and also help one another with grooming [Waa07]. They already know reciprocity and win-win situations. But when it comes to food and sex, male chimpanzees try to dominate the other males. To do this, they use not only physical violence but also threats, bluffs, and other tricks. According to de Waal, it seems as if chimpanzees intuitively know many passages from Niccolò Machiavelli’s book “The Prince”. Humans also have these “instincts.” As children, humans first have to learn to resolve conflicts peacefully, to reconcile after an argument, and so on. An AI, by contrast, will not have this “animal heritage.” The AI will feel no urge to reproduce or dominate others. An AI is information processing and inference, but not a mammal. For this reason, many predictions about AIs that suddenly develop self-awareness and threaten the world are nonsense, because an AI does not have this “animal” drive for dominance. In the 1982 film “Blade Runner,” very human-looking robots suddenly develop consciousness and want to extend their lives; they therefore have a “survival instinct.” But precisely this “instinct” would first have to be programmed into the AI, because it will not arise by itself [Bro15].

Outlook

Artificial Intelligence can be compared with the development of means of transport. By nature, humans can walk. Some are a little slow at it; others are already faster. Results in 100-meter sprint competitions have become better and better. In means of transport, there has already been substantial progress: bicycles, mopeds, motorcycles, cars, trains, and airplanes. With the help of technology, humans can now move much faster.

The artificial intelligences developed so far correspond more to bicycles. A car would correspond to an AI that could pass the Turing test. AI will be able to “fly” when AIs are as intelligent as humans and have self-awareness. But even reaching the car is still a long way off, and many inventions have to be made first.

Until then, however, Artificial Intelligence will certainly have strong effects on progress and the world of work. It is expected to change many professions, make new professions possible, and also make some professions obsolete. We will discuss this later in Section 11.7. If, however, it should be possible in the long term to create truly “artificial intelligences” that are at least as intelligent as humans, then this could lead to the so-called “Singularity,” which is discussed in Section 10.6.

Interested readers who would like to encounter different positions on the topic of AI are referred to the book “What to Think About Machines That Think”, edited by John Brockman [Bro15].