“Sentiment Analysis : Detecting polarity in text”

Pragathi R Gowda Vinutha N

Prathiksha M Harshitha M S
Department of computer science and engineering
Gsss institute of engineering & technology for women k.r.s road, Metagalli, Mysuru-570016,
Karnataka

Abstract-The rise of social media in couple of modal based tagger for recognizing part of speech
years has changed the general perspective of and then applied statistics based techniques to
networking, socialization, and personalization. identify sentiments related to subject in speech
Use of data from social networks for different Apoorv Agarwaet al.[2] Using millions of emoji
purposes, such as election prediction, sentimental occurrences to learn any-domain representations
analysis, marketing, communication, business, fordetecting sentiment, emotion and sarcasm.
and education, is increasing day by day. Precise Improve pre-processing techniques of tweets and use
extraction of valuable information from short text baseline machine learning methods
messages posted on social media (Twitter) is a Garin Kilpatrick [3] introduced list of all twitter tools
collaborative task. In this paper, we analyse tweets to collect and analyze Twitter data. He divided all
to classify data and sentiments from Twitter more Twitter tools into 53 categories. These tools provides
precisely. The information from tweets are facility in backup tweets, trend analysis, tweets
extracted using keyword based knowledge translation, voice tweet, and Twitter statistics.
extraction. Moreover, the extracted knowledge is IlknurCelik et al. [4] studied semantic relationship
further enhanced using domain specific seed between entities in Twitter to provide a medium
based enrichment technique. The proposed where users can easily access relevant content, they
methodology facilitates the extraction of are interested in.
keywords, entities, synonyms, and parts of speech MorNaamen et al. [5] studied the users behavior on
from tweets which are then used for tweets Twitter. They applied human coding and qualitative
classification and sentimental analysis. analysis of tweets to understand users activities on
Twitter. They analyzed that majority of users focus
I.INTRODUCTION on self(memoformers) while small portion of users
share information with others(informers).
Data analysis is the process of applying organized Milan et al. [6] extracted tweets topics to map tweet
and systematic statistical techniques to describe, talks to conference topic. They enriched tweets
recap, check and condense data. It is a multistep information by adding Dbpedia topics using
process that involves collecting, cleaning, organizing zamanata an application to extract keywords from
and analysing. Data mining is like applying text and connect them to related topics in Dbpedia.
techniques to mold data to suit our requirement. Data Dept of CSE 6 GSSSIETW, Mysuru
mining is needed because different sources like social Jeonghee Yi et al. [7] presented a model to extract
media, transactions, public data, enterprises data etc. sentiments about particular subject rather than
generates data of increasing volume, and it is extracting sentiment of whole document collectively.
important to handle and analyze such a big data. It This system proceeded by extracting topics, then
won’t be wrong to say that social media is something sentiments, and then mixture model to detect relation
we live by. In the 21st century social media has been of topics with sentiments.
the game changer, be it advertising, politics or
globalization, it has been estimated that data is II.DESIGN
increasing faster than before and by the year 2020;
about 1.7 megabytes of additional data will be System Requirements
generated each instant for each person on the earth. Hardware Requirements
More data has been generated in the past two years • Core i3,5th gen + processor
than ever before in the history of the mankind. It is • 4GB + RAM
clear from the fact that the number of internet users • 80+ Hard disk
are now grown from millions to billions. Software Requirements:
• Programming language: Python
II.LITERATURE SURVEY •Framework:Tweepy (Twitter),Flask(Web)
• IDE : Jupiter Lab, VS Code
Tetsuya Nasukawa et al. [1] used natural language • Operating System : Windows 7+ or Mac ,
processing techniques to identify sentiment related to Linux
particular subject in a document. They used Markov-
System Architecture the use case. The use case diagram describes how a
System architecture is a conceptual model that system interacts with outside actors; each use case
defines the structure, behaviour and more views of a represents a piece of
system.

System flow diagram

Twitter:
This is an application which we are using to read the
data and are giving the output based on the data
provided(here data means the comments). We can
find the results for any of the data which we want that
can be of any domain
Twitter API:
Here in this module when the user creates his/her
personal account they will be able to access to the
data from the twitter. This application is used to
extract the data from the twitter . Here data that is
comments can be extracted by specifying the topic
for which we want to know the results.
Web Application:
Here we code the program in python using different
algorithms so that it works accordingly. We have
used naïve Bayes and multinomial NB algorithms as
these algorithms main purpose is to classify the text.
As we have coded the program in python we are
developing this as an web application (As python
does not support android).
Output:
As shown in the figure our output will be displayed
in the form of graph. Which makes us easy to
understand. Output is shown both in graph and
percentage. IV. IMPLEMENTATION

Use case diagram

Use case diagram is a graph of actors, a set of use Feasibility Study: A feasibility study is a preliminary
cases enclosed by a system boundary, study which investigates the information of
communication associations between the actor and prospective users and determines the resources
requirements, costs, benefits and feasibility of oblivious to the system architecture and does not
proposed system. have access to the source code.
Verification and Validation: The testing process is a
Technical Feasibility: Evaluating the technical part of broader subject referring to verification and
feasibility is the trickiest part of a feasibility study. validation. We have to acknowledge the system
This is because, at the point in time there is no any specifications and try to meet the customer’s
detailed designed of the system, making it difficult to requirements and for this sole purpose, we have to
access issues like performance, costs. verify and validate the product to make sure
everything is in place. Verification and validation are
Operational Feasibility :Proposed project is
two different things. One is performed to ensure that
beneficial only if it can be turned into information
the software correctly implements a specific
systems that will meet the operating requirements.
functionality and other is done to ensure if the
Simply stated, this test of feasibility asks if the
customer requirements are properly met or not by the
system will work when it is developed and installed.
end product.

Naïve Bayes Classifier (NB) : The Naïve Bayes

Final result:
classifier is the simplest and most commonly used
1.Showing results as pie chart with percentage
classifier. Naïve Bayes classification model
computes the posterior probability of a class, based
on the distribution of the words in the document. The
model works with the BOWs feature extraction
which ignores the position of the word in the
document. It uses Bayes Theorem to predict the
probability that a given feature set belongs to a
particular label.
P(label|features) = P(label)∗P(features|label)
P(features) where , P(label) :- is the prior probability
of a label or the likelihood that a random feature set
the label. P(features|label) :- is the prior probability
that a given feature set is being classified as a label.
P(features) :- is the prior probability that a given
feature set is occurred. Given the Naïve assumption
which states that all features are independent, the
equation could be rewritten as follows:
P(label|features) =
P(label)∗P(f1|label)∗………∗P(fn|label) P(features)
Multinomial Naïve Bayes Classifier :Accuracy – 2.Saving each comments in a file defining its polarity
around 75%
Algorithm :
i. Dictionary generation Count occurrence of all
word in our whole data set and make a dictionary of
some most frequent words.
ii. Feature set generation All document is
represented as a feature vector over the space of
dictionary words.
For each document, keep track of dictionary words
along with their number of occurrence in that
document

White Box Testing:White box testing is the detailed

investigation of internal logic and structure of the
Code. To perform white box testing on an
application, the tester needs to possess knowledge of
the internal working of the code .
Black Box Testing: The technique of testing without
having any knowledge of the interior workings of the
application is Black Box testing .The tester is
 V. APPLICATIONS OF SENTIMENT ANALYSIS: (stock market, opinion mining), customer feedback
services, and etc.
• Applications that use Reviews from
Websites
• Applications as a Sub-component REFERENCES
Technology
• Applications in Business Intelligence. [1] “Alchemy api,” (Last visited in March 2012).
• Applications across Domains [Online]. Available: www.alchemyapi.com
• Applications in Smart Homes [2] “American diabetes assosiation,” (Last visited in
october 2012). [Online].Available:
http://www.diabetes.org/diabetes-basics/ common-
terms/
VI. FUTURE ENHANCEMENT
[3] “Archivist,” (Last visited in May 2012). [Online].
Available: http://archivist.visitmix.com/
Due to the lack of multi-lingual lexical dictionary, it [4] “Glossary of diabetes,” (Last visited in october
is current not feasible to develop a multi-language 2012). [Online]. Available:
based sentiment analyser.Further research can be http://en.wikipedia.org/wiki/Glossary\ of\ diabetes
carried out in making the classifiers language [5] “Grabeeter,” (Last visited in November 2012).
independent. The authors have proposed a sentiment [Online]. Available: http://grabeeter.tugraz.at/
analysis system with support vector machines, [6] “Healthcare tweet chats,” (Last visited in October
similar approach can be applied for our system to 2012). [Online]. Available:
make it language independent. The main goal of this http://www.symplur.com/healthcare-hashtags/tweet-
approach is to empirically identify lexical and chats/
pragmatic factors that distinguish sarcastic, positive [7] “Java api for wordnet searching (jaws),” (Last
and negative usage of words. visited in March 2013). [Online]. Available:
http://lyle.smu.edu/∼tspell/jaws/
• Determining neutrality [8] “The twitter hash tag: What is it and how do you
• Potential improvement can be made to our use it?” (Last visited in January 2013). [Online].
data collection and analysis method Available: http://www.techforluddites.com/
• Future research can be done with possible 2009/02/the-twitter-hash-tag-what-is-it-and-how-do-
improvement such as more refined data and you-use-it.html
more accurate algorithm. [9] F. Abel, Q. Gao, G. Houben, and K. Tao,
“Analyzing temporal dynamics in twitter profiles for
personalized recommendations in the social web,” in
VII. CONCLUSION Proceedings of ACM WebSci ’11, 3rd International
Conference on Web Science. ACM, 2011.
The experimental studies performed through the [10] ——, “Analyzing user modeling on twitter for
chapters, successfully show that hybridizing the personalized news recommendations,” User
existing machine learning analysis and lexical Modeling, Adaption and Personalization, pp. 1–12,
analysis techniques for sentiment classification yield 2011.
comparatively outperforming accurate results. For all
the datasets used, we recorded consistent accuracy of
almost 90%.
The first method that we approached for our problem
is Naïve Bayes. It is mainly based on the
independence assumption .Training is very easy and
fast In this approach each attribute in each class is
considered separately. Testing is straightforward,
calculating the conditional probabilities from the data
available. One of the major task is to find the
sentiment polarities which is very important in this
approach to obtain desired output. In this Naïve
Bayes approach we only considered the words that
are available in our dataset and calculated their
conditional probabilities. We have obtained
successful results after applying this approach to our
problem.
Clearly from the success of Naive Bayes, it can
positively be applied over other related sentiment
analysis applications like financial sentiment analysis