History of AI - Phase 1

Download as pdf or txt
Download as pdf or txt
You are on page 1of 4

AI: A VERY BRIEF

AND UNFINISHED HISTORY


PART 1
In this two-part video I am going to tell you the “history” possess. The capabilities were: Perception, Learning,
of AI by walking you through 5 key ‘ages‘ or ‘eras.‘ I will Problem-solving/Planning, Reasoning, and Natural Language
characterize each era in terms of five dimensions—(1) Understanding. This school of thought that was trying to
the time period in which it took place, (2) its notion or emulate the human mind became known as “Symbolic AI.”
understanding of intelligence, (3) the paradigm that became
dominant in the era, (4) key AI applications developed Of course, at that time computers possessed none of the
during the era, and (5) the obstacles that ultimately brought capabilities that we ascribe to intelligent human minds, so
the era to an end. Thus, I will tell you the history of AI as a researchers started building these individual capabilities
sequence of shifting paradigms and understandings of what one-by-one. As a consequence, each of these capabilities
artificial intelligence is and the applications for which these was advanced in a separate substream of AI research—some
paradigms and understandings proved useful (or not). of which exist to this day. Fields like machine learning and
automated reasoning started to emerge and the idea of
A couple of things that I would like you to have in mind: First, perceptive systems sparked research on sensors and how to
this is a brief history and thus by design incomplete and interpret sensory information.
highly focused. Second, while I have structured the history
of AI along seemingly self-contained phases, it is important The key paradigm in this ‘Golden Age‘ was a problem-
to note that much of this history is a cumulative, scientific solving technique called ‘Search’. Search systematically
process where each later era builds-on progress made in considers all possible course of action in a ‘search tree‘ and
earlier eras. In other words, you can think of the work done then selects the best (or at least a good) course of action.
in each era as further pushing the “frontier” mentioned in The problem is that these search trees grow unimaginably
an earlier video. Third, we are currently in a boom cycle of fast even for the most simple problems. Take this search
AI. But no one really knows whether and how this boom tree for the very simple game TikTakToe as an example. It
cycle will end and what comes after–so this is necessarily shows the possible courses of action from a certain starting
an unfinished history... (pause)… and we find ourselves in the point—and it demonstrates that from this starting point,
midst of it. So let‘s find out what we can learn for the present you have 5 possible courses of action. That looks like a fairly
and the future of AI by looking into its past. straight-forward problem to solve. But how many courses
of actions would the search tree have if you plotted out the
complete game. A hundred? A thousand? Answer: 255,168!
This phenomenon is called combinatorial explosion and it
The Golden Age implies that any problem that suffers from it is “intractable”,
i.e., it is not possible to find an optimal solution because it is
Our brief and unfinished history of AI begins in the mid impossible to calculate the complete search tree. An example
1950s in an era that is often referred to as “The Golden Age of a game that is intractable even for modern computers, is
of AI”. That the history of AI starts right away with a “golden the game of chess. Its search tree has about 35100 nodes…
age” is no accident. Only a few years earlier Alan Turing had Now, the problem for AI research was that combinatorial
invented the modern computer and intrigued his colleagues explosion is not only an issue for games but also for the
with the possibility that computers might one day be able component capabilities of AI (i.e., problem solving, learning,
to demonstrate the same observable behaviors as intelligent reasoning, etc.)—all of these turned out to be intractable
humans do. problems.

So, a small and tight-knit group of mathematicians and


computer scientists met for a multi-week workshop
pondering the question how such “artificial intelligence”
could be achieved. At that time, these researchers developed
the idea that “artificial general intelligence” requires the
same capabilities that the mind of intelligent humans

THOMAS HUBER - PROFESSOR IN INFORMATION SYSTEMS 4 CORE MODULE I - BASICS OF AI


So, AI research ran against the wall of combinatorial With these two techniques (knowledge representation and
explosion: It was unable to produce applications of practical rules), AI research for the first time in its history, was able
usefulness because in order to keep the search tree small, to produce useful applications that were called “expert
researchers had to use their systems in highly simplified, systems.” Expert systems interacted with the user via a
simulated environments that were nothing like the real dialogue: The system asks questions, and the human expert
world. As a consequence, enthusiasm for AI vanished. This so provides information to the system by giving answers.
called “AI winter” lasted until the late 1970s in which the 2nd Then the system uses this information to make a logical
era of our brief history starts… deduction following pre-specified rules. Here is an example
dialogue from a medical diagnostic tool: (show the following
example on the screen).

The expert system asks: Does the patient cough?


The human user—in this example a doctor--answers: Yes

Expert system: Does the patient have pneumonia?


Human: Yes

Expert system: Has the patient lost taste?


The era of knowledge-based AI Human: Yes

In this era, AI researchers gave up on the goal of developing Based on the answers of the doctor, the expert system then
a general or strong AI and instead focused on developing makes a deduction such as: The patient is most likely to have
systems that were designed to be useful for very narrow Covid-19.
and specific problems. They argued that often we perceive
those humans as intelligent that possess a lot of (expert) However, above and beyond a few useful applications in very
knowledge that they can use in a sensible way to make narrow domains (such as medical diagnosis), the knowledge-
reasonable decisions. Thus, proponents of the new paradigm based approaches to AI failed for two key reasons. First, there
noted that the old paradigm was missing two key ingredients is the knowledge elicitation problem: Human experts find it
to truly emulate the human mind: “knowledge” and routinely hard to articulate their expertise (they do not know
“reasoning.” what rules they follow); and if they do, they often have no
desire to tell someone else. Moreover, the moment you try
To add these two ingredients, AI researchers of that era to make applicability of the system a bit broader, you need
worked on two techniques: First, “knowledge representation” to feed it with very basic common-sense knowledge. Things
to formally capture human knowledge in a machine-readable like: Patients are humans. Humans live in houses. Different
way. Here is an example of such a knowledge representation houses have different ventilation, etc. This is an enormous
technique for a disease you have become all too familiar and very difficult undertaking because it requires us to make
with. This—so called—ontology captures (among others) all things explicit that we ‘just know’ and take for granted. The
the known symptoms of Covid-19 in a way that is accessible second key obstacle faced by knowledge-based AI is the
to machines. fact that the real world is ‘fuzzy’, i.e., full of ambiguity and
contradictions rendering logical deduction often difficult or
even impossible.

Second, to capture the human reasoning process, rules were


expressed as simple if-then expressions as illustrated by the
following example: (show the following IF-THEN statement on This is where part 1 of our brief history ends. The symbolic
the screen) AIs of both ‘The Golden Age‘ and the ‘Era of Knowledge-based
IF patient coughs AI systems‘ seemed to have reached dead ends. In the next
AND has pneumonia video, I will tell you how new paradigms and techniques
AND has lost taste helped AI to back out of these dead ends.
AND has a fever
THEN patient has Covid-19.

THOMAS HUBER - PROFESSOR IN INFORMATION SYSTEMS 5 CORE MODULE I - BASICS OF AI


AI: A VERY BRIEF
AND UNFINISHED HISTORY
PART 2

era of Behavioral AI
By the 1980s, symbolic AI had advanced a variety of
different techniques and approaches—but all of them
seemed to lead to dead ends. Accordingly, the “Era of
Behavioral AI” emerged in strong opposition to earlier AI
research.. It criticized knowledge-based AI for its one-sided
emphasis on higher-level cognitive functions (such as logic
reasoning and expert knowledge) and the AI of the “golden
age” for its reliance on unrealistic simulations.

The new “behavioral paradigm” instead put forward the vision Agent-based AI
that in order for a system to show intelligent behaviors in
real-world settings, we need to build it from bottom-up. As behavioral AI tested out and reached its limits, the
According to this view, complex behaviors emerge from the agent-based paradigm began to emerge in the mid 90s.)
interaction of very simple component behaviors that are
adaptively (re-) combined to react to the environment. This This new paradigm combined behavioral and reasoning
is different from the idea pursued in the golden age, which approaches: From the “golden age”, it took the idea
was all about creating highly complex individual component that proactive behaviors such as planning and problem
behaviors such as “perception” or “problem solving.” By solving are essential. From the behavioral approach, that
contrast, the behavioral paradigm sought to break down reactiveness to the environment is key. This resulted in the
‘hands-on’, real-world tasks such as “vacuum cleaning” into development of “software agents”, i.e., software programs
simple basic behaviors such as “walk randomly”, “avoid that act on behalf of the user. To qualify as ‘intelligent’,
obstacles”, and “vacuum” and then try to make them work software agents do not have to make choices the way the
together as a system that attunes to its environment. human mind does, instead, they have to make choices that
are ‘economically rational.’ Accordingly, software agents try
This approach resulted in a couple of breakthrough robotic to maximize expected utility on behalf of the user building
applications—of which iRobot‘s vacuum cleaners are on a utility function that represents the needs of that user.
arguably the most prominent example. It showed that Software agents take an active role in collaborating with their
for some problems “intelligent” machines can be built users trying to guide them towards utility maximization.
from simple behaviors—without any need for higher-level
cognitive functions. The agent-based paradigm led to a wealth of practically
useful applications. A major breakthrough was the chess
However, this approach only worked when just a few basic program “Deep Blue” that beat the reigning chess world
behaviors had to be combined to show the more complex champion. This was remarkable because chess is an
behavior. For the successful completion of tasks that require intractable problem that suffers from complexity explosion—
a large number of basic behaviors it becomes increasingly the phenomenon that had led to the downfall of symbolic
difficult to understand their interaction and fine-tune the AI. In fact, today software agents are at the core of reusable
overall system. Moreover, while the emulation of higher-level software tools—so called SAT solvers—that can be used to
cognitive functions is not necessary for tasks like vacuum provide good solutions to most intractable problems in a
cleaning, there are other tasks that clearly benefit from reasonable amount of time.
reason and rationality—and for those tasks the behavioral
paradigm proved limited.

THOMAS HUBER - PROFESSOR IN INFORMATION SYSTEMS 6 CORE MODULE I - BASICS OF AI


Agent-based systems worked best for use cases in which Similar to the human brain, artificial neural nets can
the software agents could manage the “real world” and their compute outputs from inputs without being given an
beliefs about it without having to directly interact with it. explicit recipe. They can do this because of their ability to
For example, when the agent could glean data about the real inductively learn the recipe from data. Obviously, this ability
world from existing enterprise systems (such as data about would be tremendously useful for emulating everyday tasks.
the location of delivery trucks, warehouses, and customers), However, for a long time the practical usefulness of neural
the agent could be a powerful tool to optimize resource nets has been severely limited by two constraints: Limited
allocation of a company (e.g., by pushing optimized delivery computing power and scarcity of data. However, after years
routes to the navigation software of delivery trucks). However, of exponential growth, computing power had reached such
agent-based systems turned out to be less powerful for use- high levels that from 2005 onwards, researchers were able
cases that required a more direct interaction with the real to build increasingly deep neural nets, i.e., artificial neural
world—such as a natural-language dialogue with a human structures that are more complex and thus able to perform
being or observing what is going on in the environment. more complex tasks. Moreover, the rise of ‘deep learning’ was
The problem is that for such everyday tasks like talking fueled by the wider digitization of businesses and societies—
and observing, it is almost impossible for us to spell-out a which serves-up increasingly large amounts of data that
clear recipe. We don‘t know how we do it, so we can‘t tell the these machines can use to learn about the world and the
software system either. recipes required to succeed in it. Successful applications of
deep neural nets that have learnt their underlying recipes
And yet, there is a direct line from these early software from data are natural language dialogue, automated
agents to those digital assistants that we now talk to translation, object recognition, and computer vision.
everyday and everywhere: Siri, Alexa, the Google Assistant
to name just a few. The reason is that around 2010, major
breakthroughs in machine learning paired with the ubiquity
of large amounts of data delivered the missing ingredient:
the ability to deal with problems for which we don’t know
how to come up with an appropriate recipe.

So, what can we learn from this history of AI for the present
day and the future? Over the past 70 years, the field of AI
has grown from a niche-y and obscure academic discipline
that was unable to produce anything of practical usefulness
to one of the most vibrant areas of technological progress,
powering sophisticated applications that we now use
every day. However, we also learn that this progress did
Era of Machine Learning not develop linearly. Periods of unbridled optimism and
major leaps forward were punctuated by periods of bitter
The breakthrough ingredient heralding the current “Era of disappointments when AI seemed to have reached another
Machine Learning” —but introduce right column step- dead end. So, we should not be surprised when in a few
wise), was an idea called ‘deep learning.‘ The possibility that years from now, we find ourselves in a situation in which
machines—just like humans—could learn from data and the current excitement around AI has faded away and
experience, had intrigued machine learning researchers for technological progress seemingly comes to a halt. Yet, even
decades. The key technique they had invented to address if this happens, its history suggests that the field of AI has
this problem were called “Artificial Neural Networks” or a remarkable ability to overcome obstacles. So, my guess is
“neural nets.” This technique was not inspired by the human that it will be here to stay. So with this in mind, I invite you
mind but by the human brain. It tried to emulate the to dig deeper into our next series of videos that will introduce
biological neural networks that populate our brains. you to the foundational technologies of modern AI and their
intimate relationship with data.

THOMAS HUBER - PROFESSOR IN INFORMATION SYSTEMS 7 CORE MODULE I - BASICS OF AI

You might also like