Scribd
Upload
9 views11 pages

Lecture-Feb20&25

The document discusses text classification, which involves assigning labels to texts, including tasks like sentiment analysis and spam detection. It outlines various techniques such as handwritten rules and supervised machine learning classifiers, specifically focusing on generative and discriminative algorithms like Naive Bayes. The document also addresses challenges like data sparsity and the importance of smoothing techniques in classification.

Uploaded by

Janvi Reddy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views11 pages

Lecture-Feb20&25

The document discusses text classification, which involves assigning labels to texts, including tasks like sentiment analysis and spam detection. It outlines various techniques such as handwritten rules and supervised machine learning classifiers, specifically focusing on generative and discriminative algorithms like Naive Bayes. The document also addresses challenges like data sparsity and the importance of smoothing techniques in classification.

Uploaded by

Janvi Reddy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

CSN-528

Lecture-Feb 20&25, 2025


Text Classification

A task of assigning a label or category to an entire text or document.


- Sentiment Analysis (assigning positive/negative tags to a piece of text
(phrase/sentence/document)) is a text classification task.
- Ranging from movie, restaurant, tourism domains to politics.

- Spam Detection (assigning legitimate or illegitimate tag to a mail)


Cues: “online pharmaceutical” or “WITHOUT ANY COST” or “Dear Winner”

- Language Identification in code-mixed language.


- Topic lable (sports/entertainment/education) to a text.
Text Classification

- Even language modeling can also be viewed as classification:


- Each word can be thought of as a class, and so predicting the next word is classifying the
context-so-far into a class for each next word.

- Similarly POS tagging can also be viewed as classification.


Text Classification

Text classification pipeline:

Take a single observation -> Extract (find useful) Features -> Present Data in machine readable
format -> Classify the obeservation into one of the discerete classes using classification algorithm.
Techniques:
1. Handwritten Rules
- Sentiment Analysis using Lexicon and rules.
2. Supervised Machine Learning Based Classifier
- Generative Classification Algorithms
eg., Naive Bayes
- Discriminative Classificaiton Algorithms
eg., Logistic Regression, SVM
Text Classification

Generative classification Algorithm:


We compute P(Observation|Class)
We compute how an observation is generated from a probable class value.

Discriminative classification Algorithm:


We compute P(Class|Observation) directly.
Text Classification

Naive Bayes Algorithm for Classification


- This idea of inference using Bayes Rule has been known since the work of Bayes (1763),
and was first applied to text classification by Mosteller and Wallace (1964).
- Use Bayes rule to transform the desired conditional probability P(Class|document) into some other
expression computable from the corpus.

d – document
c – class
P(d) is constant for all
the Class values
Text Classification

Naive Bayes Assumptions: f is a set of features representing d.

d – document
c – class
P(d) is constant for all the
Class values

1. Order of features doesn’t matter, called bag-of-words assumption.

2. Probabilities P(fi|c) are independent given the class c and hence can be ‘naively’ multiplied.
Text Classification

Generic Form of NB Classifier (A Linear Classifier) for text classification

How to compute prior and posterior?


Word wi in class c

All words in vocab in


class c
Text Classification

Data Sparsity
1. Unknown Words in a class

The zero occurence of fantastic will make the entire P(C|f)


zero, while f = {it’s fun to visit this fantastic place}

Add-1 Smoothing
Text Classification

Data Sparsity
1. Unknown Words in the test data – Ignore the particular word and work with remaining words.

P(sentiment class|love to visit this fantastic place) = ? if ‘visit’ is missing in the training
corpus.
Text Classification

Compute P(Class|predictable with no fun))?

P(+|test) = ((P(predictable|+)*P(with|+)*P(no|+)*P(fun|+)) * P(+))/P(test)


P(-|test) = ((P(predictable|-)*P(with|-)*P(no|-)*P(fun|-)) * P(-))/P(test)

You might also like