Course Code HUM1012

Logic and Language Structure

BL202425040 0921
D21+D22
Asllasbuas
• Unit -1
• Introduction to Natural Language Processing: Human language,
models, ambiguity, processing paradigms, phases in natural language
processing, need for NLP and NLU, relevance of NLP and NLU,
understanding of language (natural language, symbolic language,
computer language), application of natural language in programming,
expert system in AI, recommendation system in AI and cognitive
system in AI.
• What is NLP?
• Natural language processing (NLP) is a subfield of computer science
and artificial intelligence (AI) that uses machine learning to enable
computers to understand and communicate with human language.

• NLP enables computers and digital devices to recognize, understand

and generate text and speech by combining computational
linguistics—the rule-based modeling of human language—together
with statistical modeling, machine learning (ML) and deep learning.
• NLP research has enabled the era of generative AI, from the
communication skills of large language models (LLMs) to the ability of
image generation models to understand requests.
• NLP is already part of everyday life for many, powering search
engines, prompting chatbots for customer service with spoken
commands, voice-operated GPS systems and digital assistants on
smartphones.
NLP also plays a growing role in enterprise solutions that help
streamline and automate business operations, increase employee
productivity and simplify mission-critical business processes.
Benefits of NLP
• A natural language processing system can work rapidly and
efficiently: after NLP models are properly trained, it can take on
administrative tasks, freeing staff for more productive work.
• Benefits can include:
• Faster insight discovery: Organizations can find hidden patterns,
trends and relationships between different pieces of content. Text
data retrieval supports deeper insights and analysis, enabling
better-informed decision-making and surfacing new business ideas.
\
• Greater budget savings: With the massive volume of unstructured
text data available, NLP can be used to automate the gathering,
processing and organization of information with less manual effort.

• Quick access to corporate data: An enterprise can build a knowledge

base of organizational information to be efficiently accessed with AI
search. For sales representatives, NLP can help quickly return relevant
information, to improve customer service and help close sales.
Challenges of NLP
• Biased training: As with any AI function, biased data used in training will skew the
answers. The more diverse the users of an NLP function, the more significant this
risk becomes, such as in government services, healthcare and HR interactions.
Training datasets scraped from the web, for example, are prone to bias.

• Misinterpretation: As in programming, there is a risk of garbage in, garbage out

(GIGO). NLP solutions might become confused if spoken input is in an obscure
dialect, mumbled, too full of slang, homonyms, incorrect grammar, idioms,
fragments, mispronunciations, contractions or recorded with too much
background noise.
• New vocabulary: New words are continually being invented or imported. The
conventions of grammar can evolve or be intentionally broken. In these cases,
NLP can either make a best guess or admit it’s unsure—and either way, this
creates a complication.
• Tone of voice: When people speak, their verbal delivery or even body
language can give an entirely different meaning than the words
alone. Exaggeration for effect, stressing words for importance or
sarcasm can be confused by NLP, making the semantic analysis more
difficult and less reliable.
• Human language is filled with many ambiguities that make it difficult
for programmers to write software that accurately determines the
intended meaning of text or voice data. Human language might take
years for humans to learn—and many never stop learning. But then
programmers must teach natural language-driven applications to
recognize and understand irregularities so their applications can be
accurate and useful.
How NLP works
• NLP combines the power of computational linguistics together with
machine learning algorithms and deep learning.
• Computational linguistics is a discipline of linguistics that uses data
science to analyze language and speech.
• It includes two main types of analysis: syntactical analysis and
semantical analysis.
• Syntactical analysis determines the meaning of a word, phrase or
sentence by parsing the syntax of the words and applying
preprogrammed rules of grammar.
• Semantical analysis uses the syntactic output to draw meaning from
the words and interpret their meaning within the sentence structure.
• The parsing of words can take one of two forms.
• Dependency parsing looks at the relationships between words, such
as identifying nouns and verbs, while constituency parsing then builds
a parse tree (or syntax tree): a rooted and ordered representation of
the syntactic structure of the sentence or string of words.
• The resulting parse trees underly the functions of language
translators and speech recognition.
• Ideally, this analysis makes the output—either text or
speech—understandable to both NLP models and people.
• Self-supervised learning (SSL) in particular is useful for supporting NLP
because NLP requires large amounts of labeled data to train
state-of-the-art artificial intelligence (AI) models.
• Because these labeled datasets require time-consuming
annotation—a process involving manual labeling by
humans—gathering sufficient data can be prohibitively difficult.
• Self-supervised approaches can be more time-effective and
cost-effective, as they replace some or all manually labeled training
data.
Three different approaches to NLP include:
• Rules-based NLP: The earliest NLP applications were simple if-then
decision trees, requiring preprogrammed rules.
• They are only able to provide answers in response to specific
prompts, such as the original version of Moviefone.
• Because there is no machine learning or AI capability in rules-based
NLP, this function is highly limited and not scalable.
• Statistical NLP: Developed later, statistical NLP automatically extracts,
classifies and labels elements of text and voice data, and then assigns a
statistical likelihood to each possible meaning of those elements.

• This relies on machine learning, enabling a sophisticated breakdown of

linguistics such as part-of-speech tagging.

Statistical NLP introduced the essential technique of mapping language

elements—such as words and grammatical rules—to a vector
representation so that language can be modeled by using mathematical
(statistical) methods, including regression or Markov models.
•
• Deep learning NLP: Recently, deep learning models have become the
dominant mode of NLP, by using huge volumes of
raw, unstructured data—both text and voice—to become ever more
accurate.
• `Deep learning can be viewed as a further evolution of statistical NLP,
with the difference that it uses neural network models. There are
several subcategories of models
• Sequence-to-Sequence (seq2seq) models: Based on recurrent neural
networks (RNN), they have mostly been used for machine translation by
converting a phrase from one domain (such as the German language) into
the phrase of another domain (such as English).

• Transformer models: They use tokenization of language (the position of

each token—words or subwords) and self-attention (capturing
dependencies and relationships) to calculate the relation of different
language parts to one another. Transformer models can be efficiently
trained by using self-supervised learning on massive text databases. A
landmark in transformer models was Google’s bidirectional encoder
representations from transformers (BERT), which became and remains the
basis of how Google’s search engine works.
• Autoregressive models: This type of transformer model is trained specifically to
predict the next word in a sequence, which represents a huge leap forward in the
ability to generate text. Examples of autoregressive LLMs include GPT, Llama,
Claude and the open-source Mistral.

• Foundation models: Prebuilt and curated foundation models can speed the
launching of an NLP effort and boost trust in its operation. For example, the IBM
Granite™ foundation models are widely applicable across industries. They support
NLP tasks including content generation and insight extraction. Additionally, they
facilitate retrieval-augmented generation, a framework for improving the quality
of response by linking the model to external sources of knowledge. The models
also perform named entity recognition which involves identifying and extracting
key information in a text.
 History of NLP
• First Phase (Machine Translation Phase) - Late 1940s to late 1960s
• The work done in this phase focused mainly on machine translation (MT). This
phase was a period of enthusiasm and optimism.
• Let us now see all that the first phase had in it −
• The research on NLP started in early 1950s after Booth & Richens’ investigation
and Weaver’s memorandum on machine translation in 1949.
• 1954 was the year when a limited experiment on automatic translation from
Russian to English demonstrated in the Georgetown-IBM experiment.
• In the same year, the publication of the journal MT (Machine Translation)
started.
• The first international conference on Machine Translation (MT) was held in
1952 and second was held in 1956.
• In 1961, the work presented in Teddington International Conference on
Machine Translation of Languages and Applied Language analysis was the high
point of this phase.
• Second Phase (AI Influenced Phase) – Late 1960s to late 1970s
• In this phase, the work done was majorly related to world knowledge and on its
role in the construction and manipulation of meaning representations. That is
why, this phase is also called AI-flavored phase.
• The phase had in it, the following −
• In early 1961, the work began on the problems of addressing and constructing
data or knowledge base. This work was influenced by AI.
• In the same year, a BASEBALL question-answering system was also developed.
The input to this system was restricted and the language processing involved was
a simple one.
• A much advanced system was described in Minsky (1968). This system, when
compared to the BASEBALL question-answering system, was recognized and
provided for the need of inference on the knowledge base in interpreting and
responding to language input.
• Third Phase (Grammatico-logical Phase) – Late 1970s to late 1980s
• This phase can be described as the grammatico-logical phase. Due to the failure of
practical system building in last phase, the researchers moved towards the use of
logic for knowledge representation and reasoning in AI.
• The third phase had the following in it −
• The grammatico-logical approach, towards the end of decade, helped us with
powerful general-purpose sentence processors like SRI’s Core Language Engine
and Discourse Representation Theory, which offered a means of tackling more
extended discourse.
• In this phase we got some practical resources & tools like parsers, e.g. Alvey
Natural Language Tools along with more operational and commercial systems,
e.g. for database query.
• The work on lexicon in 1980s also pointed in the direction of grammatico-logical
approach.
• Fourth Phase (Lexical & Corpus Phase) – The 1990s
• We can describe this as a lexical & corpus phase. The phase had a
lexicalized approach to grammar that appeared in late 1980s and
became an increasing influence. There was a revolution in natural
language processing in this decade with the introduction of machine
learning algorithms for language processing.
•
• study of Human Languages
• Language is a crucial component for human lives and also the most
fundamental aspect of our behavior. We can experience it in mainly
two forms - written and spoken. In the written form, it is a way to
pass our knowledge from one generation to the next. In the spoken
form, it is the primary medium for human beings to coordinate with
each other in their day-to-day behavior. Language is studied in
various academic disciplines. Each discipline comes with its own set
of problems and a set of solution to address those.
• ambiguity and Uncertainty in Language
• Ambiguity, generally used in natural language processing, can be
referred as the ability of being understood in more than one way. In
simple terms, we can say that ambiguity is the capability of being
understood in more than one way. Natural language is very
ambiguous. NLP has the following types of ambiguities −
• Lexical Ambiguity
• The ambiguity of a single word is called lexical ambiguity. For
example, treating the word silver as a noun, an adjective, or a verb.
• Syntactic Ambiguity
• This kind of ambiguity occurs when a sentence is parsed in different
ways. For example, the sentence “The man saw the girl with the
telescope”. It is ambiguous whether the man saw the girl carrying a
telescope or he saw her through his telescope.
• Semantic Ambiguity
• This kind of ambiguity occurs when the meaning of the words
themselves can be misinterpreted. In other words, semantic
ambiguity happens when a sentence contains an ambiguous word or
phrase. For example, the sentence “The car hit the pole while it was
moving” is having semantic ambiguity because the interpretations
can be “The car, while moving, hit the pole” and “The car hit the pole
while the pole was moving”.
• Anaphoric Ambiguity
• This kind of ambiguity arises due to the use of anaphora entities in
discourse. For example, the horse ran up the hill. It was very steep. It
soon got tired. Here, the anaphoric reference of “it” in two situations
cause ambiguity.
• Pragmatic ambiguity
• Such kind of ambiguity refers to the situation where the context of a
phrase gives it multiple interpretations. In simple words, we can say
that pragmatic ambiguity arises when the statement is not specific.
For example, the sentence “I like you too” can have multiple
interpretations like I like you (just like you like me), I like you (just like
someone else dose).
NLP Phases
Following diagram shows the phases or logical
steps in natural language processing −
NLP Phases
• Morphological Processing
• It is the first phase of NLP. The purpose of this phase is to break
chunks of language input into sets of tokens corresponding to
paragraphs, sentences and words. For example, a word
like “uneasy” can be broken into two sub-word tokens as “un-easy”.
• Syntax Analysis
• It is the second phase of NLP. The purpose of this phase is two folds:
to check that a sentence is well formed or not and to break it up into
a structure that shows the syntactic relationships between the
different words. For example, the sentence like “The school goes to
the boy” would be rejected by syntax analyzer or parser.
• Semantic Analysis
• It is the third phase of NLP. The purpose of this phase is to draw exact
meaning, or you can say dictionary meaning from the text. The text is
checked for meaningfulness. For example, semantic analyzer would
reject a sentence like “Hot ice-cream”.
• Pragmatic Analysis
• It is the fourth phase of NLP. Pragmatic analysis simply fits the actual
objects/events, which exist in a given context with object references
obtained during the last phase (semantic analysis). For example, the
sentence “Put the banana in the basket on the shelf” can have two
semantic interpretations and pragmatic analyzer will choose between
these two possibilities.
expert system in AI
• An expert system is a computer program that is designed to solve complex
problems and to provide decision-making ability like a human expert. It
performs this by extracting knowledge from its knowledge base using the
reasoning and inference rules according to the user queries.
• The expert system is a part of AI, and the first ES was developed in the year
1970, which was the first successful approach of artificial intelligence. It
solves the most complex issue as an expert by extracting the knowledge
stored in its knowledge base.

• The system helps in decision making for compsex problems using both
facts and heuristics like a human expert. It is called so because it contains
the expert knowledge of a specific domain and can solve any complex
problem of that particular domain. These systems are designed for a
specific domain, such as medicine, science, etc.
Below are some popular examples of the
Expert System:
• DENDRAL: It was an artificial intelligence project that was made as a chemical
analysis expert system. It was used in organic chemistry to detect unknown
organic molecules with the help of their mass spectra and knowledge base of
chemistry.
• MYCIN: It was one of the earliest backward chaining expert systems that was
designed to find the bacteria causing infections like bacteraemia and meningitis. It
was also used for the recommendation of antibiotics and the diagnosis of blood
clotting diseases.
• PXDES: It is an expert system that is used to determine the type and level of lung
cancer. To determine the disease, it takes a picture from the upper body, which
looks like the shadow. This shadow identifies the type and degree of harm.
• CaDeT: The CaDet expert system is a diagnostic support system that can detect
cancer at early stages.
Characteristics of Expert System
• High Performance: The expert system provides high performance for
solving any type of complex problem of a specific domain with high
efficiency and accuracy.
• Understandable: It responds in a way that can be easily
understandable by the user. It can take input in human language and
provides the output in the same way.
• Reliable: It is much reliable for generating an efficient and accurate
output.
• Highly responsive: ES provides the result for any complex query
within a very short period of time.
• Components of Expert System
• An expert system mainly consists of three components:
• User Interface
• Inference Engine
• Knowledge Base
recommendation system in AI and cognitive
system in AI.
• An AI-powered recommendation system is a machine learning
algorithm that is trained to rank or rate products or users. It is
designed to predict the ratings a user might give to a specific item
and then return those predictions to the user in a ranked list.
• A recommendation engine, also called a recommender, is an artificial
intelligence (AI) system that suggests items to a user.
Recommendation systems rely on big data analytics and machine
learning (ML) algorithms to find patterns in user behavior data and
recommend relevant items based on those patterns.
• Recommendation engines help users discover content, products or
services they might not have found on their own. These systems are
an integral part of generating sales and cultivating engagement for
many online businesses, including e-commerce websites, media
streaming platforms, search engines and social media networks.
• A recommender suggests the next movie or video to watch, a similar
song to listen to, relevant search results or a product that
complements a certain order.
How recommendation engines work
• Recommenders typically operate in 5 phases to predict the most accurate recom
• 1. Data gathering
• Data is the foundation of a recommendation system, so collecting data is a crucial
first step. The 2 key types of data to be gathered include explicit data and implicit
data.
• Explicit data encompasses user actions and activities such as comments, likes,
ratings and reviews. Implicit data comprises user behavior such as browsing
history, shopping cart events, clicks, past purchases and search history.
• Recommenders also use other customer data such as demographics (age or
gender) and psychographics (interests or lifestyle) to find similar users, and
feature data (such as price range or item type) to determine related products or
services.
• mendations:
• 2. Storage
• After data has been gathered, the next step is to store it. The kind of
storage system depends on the types of data collected.
• A data warehouse can aggregate data from different sources to
support data analysis and machine learning, while data lakes can
store both structured and unstructured data.
• A data lakehouse combines the best aspects of data warehouses and
data lakes into a single data management solution.
• 3. Analysis
• The analysis phase employs machine learning algorithms to process
and examine data sets. These algorithms detect patterns, identify
correlations and weigh the strength of those patterns and
correlations. Machine learning models can be trained on large data
sets to make fitting recommendations.
• 4. Filtering
• The final step is to filter the data, showing the most relevant items
from the previous analysis stage. Data filtering involves applying
certain mathematical rules and formulas to the data depending on
the type of recommendation engine used.
• 5. Refining
• An optional refining step might be added to regularly assess the
outputs of a recommendation system and further optimize the model
to continually enhance its accuracy and quality.
Types of recommendation engines
Recommenders differ based on the filtering method that they
use. There are generally 3 types of recommendation engines:
• 1. Collaborative filtering
• A collaborative filtering system filters suggestions based on a particular user’s likeness to
others. Collaborative recommender systems rely on explicit and implicit data and assume
that users with comparable preferences will likely be interested in the same items and
potentially interact with them in similar ways in the future.
• Amazon (link resides outside ibm.com), for instance, uses collaborative filtering for its
product recommendations, just like Spotify (link resides outside ibm.com) does for its
audio content.

• Collaborative filtering recommenders can offer effective suggestions and won’t usually
need detailed item descriptions. However, collaborative filtering is also prone to the cold
start problem, which happens when the system has limited historical data to draw from,
especially for new users.
• There are 2 main kinds of collaborative filtering systems: memory-based and
model-based.
• Memory-based
• Memory-based systems represent users and items as a matrix. They are an
extension of the k-nearest neighbors (KNN) algorithm because they aim to
find their “nearest neighbors,” which can be similar users or similar items.
Memory-based systems are further subdivided into 2 types:
• User-based filtering computes similarities between a particular user and all
other users in the matrix. User-based similarity functions are calculated
between rows in the user-item matrix.
• Item-based filtering computes item similarity through user behavior (how
users interact with items and not item features). Item-based similarity
functions are calculated between columns in the user-item matrix.
• Model-based
• Alternatively, model-based systems create a predictive machine learning model of
the data. The user-item matrix serves as the training data set for the model, which
then yields predictions for missing values, that is, items that a user has not yet
found and will therefore be recommended.
• One of the most commonly used model-based collaborative filtering algorithms is
matrix factorization. This dimensionality reduction method decomposes the often
large user-item matrix into two smaller matrices—one for users and another for
items—having a select few dimensions. The 2 matrices are then multiplied
together to predict the missing values (or the recommendations) in the larger
matrix.
• A more advanced implementation of matrix factorization harnesses deep
learning neural networks. Other model-based systems employ machine learning
algorithms such as Bayes classifiers, clustering and decision trees.
• 2. Content-based filtering
• A content-based filtering system filters recommendations based on an item’s features.
Content-based recommender systems assume that if a user likes a particular item, they
will also like another similar item. Content-based filtering considers item descriptions such
as color, category, price and other metadata assigned by keywords and tags, along with
explicit and implicit data.
• Content-based filtering systems represent items and users as vectors in a vector space.
Proximity is used to determine the similarity between items. The closer 2 vectors are in
space, the more similar they’re considered to be. Vectors similar to previous items
according to their supplied features will be recommended to the user.
• Content-based recommenders apply a user-based classifier or regression model.
Descriptions and features of items a user is interested in act as the model’s training data
set, which then yields predictions for recommended items.
• Content-based recommendation systems can be further improved by using natural
language processing tags. However, this tagging process can be tedious for huge volumes
of data.
• Unlike collaborative filtering, the cold start problem is less of an issue since content-based
filtering is based on metadata characteristics rather than past user interaction. However,
content-based filtering can be limited in exploring new items, as it often suggests those
similar to what users liked previously.
• 3. Hybrid recommendation system
• As its name implies, a hybrid recommendation system merges
collaborative filtering and content-based filtering.
• As a result, a hybrid approach can greatly enhance the performance
of a recommendation engine, but it requires advanced architectures
and intensive computational power.
• Netflix, for instance, uses a hybrid recommendation system (link
resides outside ibm.com) for its movie and TV show
recommendations.
•
cognitive system in AI
• Cognitive AI, also called cognitive artificial intelligence, is software
that tries to think and learn by imitating the way human brains work.
It uses natural language processing (NLP) and machine learning (ML)
to attempt to understand human intention behind queries to deliver
more relevant responses.
• How Would Cognitive Computing Work properly?
• Technologies of cognitive computing combine a variety of
information sources, balancing contextual as well as naturalistic
explanations, to recommend appropriate solutions.
• Cognitive systems use identity techniques that use data mining,
pattern recognition, and natural language processing (NLP) to
comprehend how and why the individual brain processes in order to
accomplish that.
• It takes a lot of structured and unstructured data to address problems
that should be handled by human utilizing technological tools.
Cognitive systems gain the ability to anticipate new issues as well as
simulate alternative solutions as machines continue to improve their
patterns recognition and data processing skills through practice.
• Cognitive computing systems need to possess a few essential
qualities in order to attain such functionalities.
ref
• https://www.ibm.com/think/topics/recommendation-engine