IDENTIFICATION OF PUBLIC SENTIMENT OVER COMMENTS THROUGH TWEETS BY DIGITAL INDIA

Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) IDENTIFICATION OF PUBLIC SENTIMENT OVER COMMENTS THROUGH TWEETS BY DIGITAL INDIA Dr. Amrita Chakraborty Assistant Professor, Department of Languages, Literature and Aesthetics (LLA), School of Liberal Studies (SLS), Pandit Deendayal Petroleum University (PDPU), Gandhinagar Official Address: Dr. Amrita Chakraborty, Assistant Professor, Department of Languages, Literature and Aesthetics (LLA), School of Liberal Studies (SLS), Pandit Deendayal Petroleum University (PDPU), Knowledge Corridor, Raisan Village, Gandhinagar – 382007, Gujarat, INDIA Author’s Profile: Dr. Amrita Chakraborty is currently working as an Assistant Professor with the School of Liberal Studies (SLS) under Pandit Deendayal Petroleum University (PDPU) in Gandhinagar. A UGC-NET qualified scholar, she has a total experience of more than 12 years. She has worked with leading corporate brands such as Deloitte, NIIT and Ramoji Film City. She completed her Ph.D. from SLS, PDPU and her area of study was the Digital India initiative by the Government of India. Dr. Amrita Chakraborty, Identification of Public Sentiment over Comments through Tweets by Digital India, -Palarch’s Journal Of Archaeology Of Egypt/Egyptology 17(7),ISSN 1567-214x Abstract: The Digital India initiative by the Government of India is an initiative by Shri Narendra Modi, the Prime Minister of India. Launched in the year 2015, the programme enhanced its scope to various digital services wings. In this study, the researcher attempted to identify public sentiments by studying their comments through Tweets. The researcher considered the Tweets by Digital India for a period of one year from November 21, 2016 to November 20, 2017. With the help of Quasi Systematic 9661 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) Sampling, 572 Tweets were pulled up for analysis. The researcher studied the first two comments of each Tweet. Every comment was categorized based on primary sentiment of the sentence such as ‘Support’, ‘Criticism’, ‘Neutral’, ‘Enquiry’ and ‘Out of Context’. In order to validate the researcher’s conclusion of the sentiment on any comment, the researcher conducted ‘Inter-Rater Reliability Test’ to calculate Kappa Coefficient (k). This study intends to find out the sentiment of the public in majority and the Twitter accounts, which received the higher percentage of supportive, neutral or criticizing comments. The research further tries to identify if the supportive or criticizing comments are higher in English or Hindi. Keywords: Digital India, Sentiment Analysis, Twitter, Public Relations, Twitter Comments Funding: This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. Declaration of Conflicting Interests: The Author declares that there is no conflict of interest. Introduction: What exactly is the Digital India programme? While many scholars use the phrase with relative impunity, it is often misunderstood and misinterpreted. Before delving into the discussion any further, it is important that we understand the concept in its totality. The Government of India launched the Digital India programme so that essential government services are provided to the citizens through electronic means. The principal mechanism to achieve the same was a complete revamping of the online infrastructure and digital empowerment of the people. The official website of Digital India defines the programme thus, “The Digital India programme is a flagship programme of the Government of India with a vision to transform India into a digitally empowered society and knowledge economy” (Digital India, 2019). If we take a closer look at the definition, we shall appreciate that ‘eGovernance’ is the buzzword here. e-Governance by itself is a relatively new concept in India. 9662 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) While the internet was launched in India back during the midnineties of the last century, it was largely an elite concept until the turn of the millennium. The renowned Dot-com Bubble ensured that India also started to gather momentum in terms of the number of internet users. The trend has continued ever since and India today has one of the largest internet subscriber bases globally. According to the Internet and Mobile Association of India (IAMAI), roughly 35 per cent of the Indian populace (close to 481 million people) uses the World Wide Web as of December 2017 (Agarwal, 2018). This has inspired multiple government authorities to adopt e-Governance, which truly has the potential to transform the country into a knowledge economy. If we take a good and hard look at the history of e-Governance in India, we shall find that the concept started taking shape right after the introduction of internet in the country. Of course, the emphasis back then was decidedly on citizen-centric services. Slowly but gradually, a host of states and Union Territories initiated a number of e-Governance schemes. As a case in point, Andhra Pradesh successfully tested a number of e-Governance initiatives. The state’s government in fact clinched three gold awards at the 20th National Conference on e-Governance in 2017 (The Hindu, 2017). It will be interesting to note that the purpose of e-Governance in India has changed over the years. Initially, it revolved around the computerization of records at the government departments. Now, the principal objectives of e-Governance are citizen-centricity and transparency. Notwithstanding all these developments, e-Governance could not make much of headway in India. The villages were still to be covered and the lower strata of the society were yet to be encompassed. To address these gaps, the National e-Governance Plan (NeGP) was launched. The idea was to take a complete view of all the possible e-Governance schemes in the country and bring them 9663 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) under the same roof. The plan was initiated on May 18, 2006. It initially had 27 Mission Mode Projects (MMPs) and eight components. In 2011, four MMPs were added to take the tally to 31. At the end of the day, the intent was to make public services more accessible to the common people, as can be gathered from the Vision Statement of NeGP (Aggarwal, 2018). Despite numerous success stories, e-Governance could not still make the impact that that it was supposed to. Successive governments have felt the necessity of promoting inclusive growth through the wider implementation of e-Governance. The most important areas that were needed to cover were electronic services, products, devices and job opportunities. It was also felt that extra emphasis was needed to be put on electronic manufacturing in the country. To address all these issues and many more, the Government of India kick-started the Digital India programme. The programme is expected to transform the country into a digitally empowered society (Digital India, 2019). The Digital India programme has three key vision areas: 1. Digital Infrastructure as a Core Utility to Every Citizen. 2. Governance and Services on Demand. 3. Digital Empowerment of Citizens (Digital India, 2019). Digital India is an encompassing programme that covers a number of ministries and departments under the Government of India (GoI). Thus, the emphasis areas of the programme also range across many parallel domains. To streamline the functioning of the programme, the authorities have identified nine specific pillars. Each of the pillars falls under multiple ministries and departments. The pillars are: 1. Broadband Highways. 2. Universal Access to Mobile Connectivity. 3. Public Internet Access Programme. 4. e-Governance – Reforming Government through Technology. 9664 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 5. e-Kranti – Electronic Delivery of Services. 6. Information for All. 7. Electronics Manufacturing. 8. IT for Jobs. 9. Early Harvest Programmes (Digital India, 2019). Reasons for Studying Twitter: Twitter calls itself an information network. It stands in stark contrast with platforms such as Facebook, where information is incidental. Although there exist very few statistical studies vis-à-vis the exact status of Twitter as a news aggregator, the fact remains that a majority of the news organizations make it a point to post all their breaking news items on Twitter. Even a cursory glance through the Twitter page of a news organization establishes the primordial nature of Twitter in the world of news. If we follow the different social media handles of Digital India, we shall be able to appreciate that the maximum number of posts concerning the programme are published on the Twitter page itself. In addition, in India, a number of political and social developments are discussed over Twitter. Bureaucrats, politicians and public figures regularly post theirs views and opinions on Twitter thereby making it the dominant social medium. Objectives of the Study: The study focusses on finding objective answers to the following three problem questions: 1. What are the various categories of public comments and what are their respective numbers? 2. What are the top Twitter accounts that received the highest comments falling under Support, Neutral, Criticism and Enquiry categories? 3. Which are the categories of communication (languages/ links) used in the comments by the public and what sentiments do they reflect? Review of Literature 9665 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) This chapter makes a concerted attempt at summarizing the various studies and literature available on Twitter as an effective social media to understand public sentiment. Research on Twitter as a Social Media Platform: There is a largely theoretical paper that makes a distinct attempt at understanding the medium that is Twitter. Titled ‘Understanding Twitter’ and penned in 2012, the paper by Fiona Maclean, Derek Jones, Gail Carin-Levy and Heather Hunter defines and introduces the concept of Twitter within the domain of occupational therapy research and consequent education. According to the paper, Twitter is an accessible, versatile and valuable instrument for the global communication of ideas, thoughts and visions for the future. The paper further says although Twitter poses some distinct challenges, the nature of the medium offers a rather exciting prospect that should and can be embraced. In very simple terms, the paper vouches for the usage of Twitter in multiple ways. Twitter is cross-disciplinary in nature and people from all walks of life can connect and network with each other through the platform. This paper again establishes the importance that Twitter has assumed over the past few years. The current researcher has time and again asserted the reason as to why Twitter has been selected over other social media platforms. This paper confirms and validates that choice. As people are moving ahead, different organizations are adopting Twitter in a big way. As fundamental as it sounds, there is a paper on how Twitter is used by different users. Titled ‘What Do We Do with Twitter?’ and published in 2014, the paper penned by Jimmy Sanderson is a treatise to find out the reason as to why papers on Twitter have multiplied over the past few years. The paper also tries to find out as to why the influence of social media in general and Twitter in particular is increasing when it comes to sports. However, the author also contended that while impact on Twitter is being measured currently, 9666 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) the impact of Instagram or similar other forums will be measured in the near future. This paper establishes the importance of Twitter in sports as well. All in all, Twitter has emerged as one of the most important platforms of communication in practically all the fields. The current researcher maintains that understanding Twitter has become absolutely important in understanding the dynamics of contemporary life. According to this paper, thematic analyses could also be done using Twitter by analyzing the content available on the platform. In very simple terms, what is being communicated through Twitter about any of these fields also make a difference. In an interesting commentary on Twitter, Dhiraj Murthy in 2011 brought out some rather stark facts. Effectively titled, ‘Twitter: Microphone for the Masses?’, the commentary explores Twitter and citizen journalism. As has been noted earlier, Twitter has been responsible for breaking multiple stories at multiple points in time. In fact, the commentary bats for the fact that Twitter has converted normal users into citizen journalists. However, the commentary also tries to find out if the traditional media subsumes the voices on Twitter with its massive presence. The study finds out that subject to the influence of Twitter, news organizations are increasingly Tweeting the headlines of breaking news items, which again underscores the importance of Twitter in journalism. Another issue that comes up in the study is the fact that Twitter remains accessible only to the technical literate thereby taking out the less privileged ones out of the radar. Hoaxes and misinformation on Twitter have the ability to create mess in the otherwise maintained news space. However, there is one thing that comes out of the study – news organizations the world over are increasingly changing the pattern based on which they produce and disseminate news subject to the infusion of Twitter in the scene. 9667 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) Twitter’s Influence on Modifying Public Opinion: There is another study that analyzes the power of Twitter in spreading disinformation and fake news. Titled ‘Disinformation, ‘Fake News’ and Influence Campaigns on Twitter’, this study was conducted in 2018 by Matthew Hindman and Vlad Barash on behalf of the Knight Foundation. With the background being the 2016 American presidential election campaign, the study was one of the largest of its kind that tried finding out the exact nature of fake news on Twitter. The study also made an attempt at finding out how powerful influence campaigns can be on Twitter. In very simple terms, the study tried finding out the strength of Twitter in influencing citizenry. Interestingly, the researchers were of the opinion that the presence of multiple accounts on Twitter that spread disinformation puts serious questions on the efforts made by Twitter to control fake news. In fact, the primary finding of the study was that there are multiple accounts that are often interlinked and that have a definite agenda that maliciously plant fake information on Twitter to misguide people and influence their actions. One can safely deduce that Twitter has become probably the most important platform through which public opinion can be manipulated and tampered with, not to say that it can definitely be influenced. There is another 2015 study that tries to figure out the suitability of Twitter for demographic and social science research. Titled ‘Using Twitter for Demographic and Social Science Research: Tools for Data Collection and Processing’, the researchers Tyler H. McCormick, Hedwig Lee, Nina Cesare, Ali Shojaie and Emma S. Spiro contend that there is a significant potential for Twitter to be used for social science research. This again establishes the importance of Twitter in figuring out public opinion. However, the biggest challenge lies in finding out the demographic details about the users of Twitter. The study designs an appropriate data 9668 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) processing strategy for using Twitter in social science research. The research also tries to find out the reliability of the techniques used. One of the most important deductions of this research is the fact that Twitter remains one of the best sources of information for conducting any kind of social science research. This again underscores the importance of the study in question. Using textual information from Twitter can open up a new window of information. It can be argued that the success or failure of any scheme can be gauged perfectly with the help of an informational study through Twitter. The researcher used Amazon’s Mechanical Truck (AMT) to code a large volume of users profile images efficiently and made a comparison between the evaluations of the truckers with expert coders. The researcher used this method for data collection to conduct a racial comparison. The researcher selected the truckers based on their demographic details to justify the objective of the research. There exists a myth busting study that establishes the exact nature of Twitter research. The study conducted by Andrew Billings in 2014 and titled ‘Power in the Reverberation: Why Twitter Matters, But Not the Way Most Believe’ explored the role of Twitter in setting the agenda of the mainstream sports media. The study contends that Twitter as a social media platform is worth analyzing not because it represents the public in general but because it doesn’t represent the population in general. Again, this study tries to find out the importance of Twitter in another very important area of public attention – sport. As one can understand, the preeminence of Twitter in the overall scheme of public activity is established solidly with this study as well. Interestingly, the followers of players and team are quite high in sports. However, none of the top Twitter influencer is from the arena of sports. The study also points towards the fact that there are some silent followers who are on Twitter without Twitting. This is a catchy area. One can’t really say anything about this section 9669 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) that silently follows Twitter activities. All in all, this study also emphasizes on the importance of Twitter in measuring public opinion. There is another very interesting study that talks about the analysis of Twitter information for evaluating public views on transportation services. In the 2020 research paper titled ‘A framework with efficient extraction and analysis of Twitter data for evaluating public opinions on transportation services’, penned by M.E. Rinker, there is an effort to explore public views on transportation services so that effective policies and decisions can be taken. The paper puts forward a comprehensive framework that lets the extraction and subsequent analysis of opinions using Twitter. The framework is explained using the transportation system of the Miami-Date County. The paper puts forward the view that for any infrastructure project evaluation and eventual planning, the role that public opinion plays is vital. This is exactly where social media can come handy. It can provide an extra stream of information that decision makers and managers can use for comprehending the public perception concerning the transportation services. This paper again proves the importance of Twitter in understanding and mining public perception concerning any activity. Whether it is the government or any other entity carrying out any massive public-facing activity, it is compulsory to look at Twitter and find out how the public perception is shaping up. This research also provides a broad framework under which similar studies can be conducted in the future. Sentiment Analysis on Twitter: In a rather interesting study titled ‘Diabetes on Twitter: A Sentiment Analysis’ in 2018, Elia Gabarron, Enrique Dorronzoro, Octavio Rivera-Romero and Rolf Wynn tried analyzing the sentiments on Diabetes through the platform of Twitter. They believed that contents published on social media in general and Twitter in particular have a strong impact on public 9670 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) sentiments on any issue and diseases are no exception. The study made an explicit attempt at deconstructing the sentiments embedded with the messages on Diabetes on Twitter. In fact, the intent of the study was to find out how public health strategies could be fixed using content on Twitter. The researchers were of the opinion that the public perception about the disease could be found out using the messages that were posted on Twitter. Finding out the perception of common people and people affected by diabetes could significantly help fix the trajectory along which public strategies could be designed to help streamline the situation. In this case, the researchers made a distinction between positive sentiments, negative sentiments and neutral sentiments. One can take the example of the use of emojis in the messages. The use of emojis was considered to be showing a positive sentiment. The study could go a long way in developing a constructive attitude among people towards diabetes. There is another 2018 study that analyzes the sentiments on Twitter rather aptly. Titled ‘A Mixed-Methods Examination of Morality Work through Sentiment Analysis and Qualitative Coding of Twitter Data’ by Tony P. Love, Jenny L. Davis and Joseph M. Calvert, the researchers analyze public sentiments over Twitter. One can assert that public sentiments vary on Twitter depending on multiple parameters. The study shows how public sentiments shift on Twitter depending on the target of the moral judgment. The study details how discourses on Twitter often point towards a dynamic nature of public sentiments. The study outlines a case study and shows the differences in public sentiments. It shows how social media reactions were mostly negative when Ray Rice, an American football player, punched his fiancée. However, when the fiancée married Ray Rice, the sentiments on social media turned less negative although there was a case of abuse earlier. The fickle nature of Twitter discourses could be understood through this episode. It is clear that Twitter still 9671 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) needs some attention to understand the drift of how the members of the public react to specific situations. Another point that was argued by the researchers is that moral boundaries are drawn and redrawn with each debate on Twitter. In this context, one needs to talk about a very important study that was conducted by Yinying Wang and David J. Fikis in 2017. Titled ‘Common Core State Standards on Twitter: Public Sentiment and Opinion Leaders’, the researchers tried examining the public opinion using the Common Core State Standards (CCSS) on Twitter in the United States of America. Sentiment analysis was done as a part of the research and users of Twitter expressed their discontent towards CCSS in all the 50 states of the country. It was found out that most of the opinion leaders were in the given issue were the ones who expressed negative sentiments on the issue. This study is again a very strong indication of how Twitter can be a great platform to understand public sentiments over multiple issues. While this research was centered in the United States of America, similar studies on similar issues in India could throw up rather interesting results. This researcher found out the relevance of all Twitter studies before ascertaining its importance in the study in question. As one can understand, Twitter could be a great mining platform through which the popularity of various governmental and semi- governmental schemes could be enhanced significantly. Research Design and Methodology: The researcher used Content Analysis. Quasi Systematic Sampling (or Systematic Like Sampling) was used for fulfilling the objective. The Sample Size was 572 out of a Population of 2690. The researcher used all the 572 Tweets for study out of which 170 Tweets have no comment. Hence, they were categorized as N/A (Not Applicable). The researcher selected two comments per Tweet. In the process, 881 comments by public were studied. The researcher found 9672 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) out that the order of comments by public in a Tweet changes from time to time. Thus, the top two comments were used for the content analysis. The researcher conducted an Inter-Rater Reliability Test in calculating Cohen’s Kappa (k) to find out the reliability or interobserver agreement between two raters A and B. Inter-Rater Reliability, or precision, happens when one’s data raters (or collectors) give the same score to the same data item (Stephanie, 2014). The researcher selected 30 comments from 881 comments of study through Systematic Sampling by using the formula 881/30 = 29.36. Hence, the researcher considered every 29th comment to study. Considering the population size of 5262 comments by public with a sample size of 881 comments for observation, with 95 per cent confidence level, the margin of error comes to 3.01 per cent. At 99 per cent confidence level, the margin of error is 3.96 per cent. The researcher categorized the comment category with the following codes to study the Kappa Coefficient (k). The researcher coded 1 for rater’s agreement with the researcher and 0 for rater’s disagreement with the researcher. Results and Discussions: It is important to find out the answers to each of the research questions in a systematic and objective manner: 1. What are the various categories of public comments and their numbers? This research question can also be divided into the following survey questions, the answers to which will determine the composite answer. 1.1. Category of Comments and Their Numbers. 9673 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) Table 1 Category of Comments Total Percentage Support 230 26 N/A 170 20 Neutral 135 15 Out of Context 134 15 Criticism 122 14 Enquiry 90 10 Grand Total 881 Figure 1 From the table and the figure above, the following pointers become clear: 1) Supportive comments represent the highest category with a total number of 230 comments or 26 per cent of total comments taken for the study. 2) The next highest category of comments includes neutral statements, which account for 135 comments or just 15 per cent of the total comments. 9674 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 3) 15 per cent of the comments were out of context, which has no relevance with the Tweets of Digital India. 4) There are 122 comments with criticism, which account for 14 per cent of the comments being studied. 5) A number of individuals came up with enquiries for various services of Digital India, which account for 90 comments or 10 per cent. 6) The above pieces of data clearly indicate that the Twitter page of the Digital India campaign significantly turned people’s opinions in its favour. Although 26 per cent cannot be identified as overwhelming public opinion, there are substantial reasons to believe that the situation will only improve with each passing day. The fact that internet pages would always invite irrelevant and out-of-context comments cannot be ignored either. Probably, this is one of the reasons as to why the support percentage is not as high as it should have been. 2. What are the top Twitter accounts that received the highest comments falling under Support, Neutral, Criticism and Enquiry categories? This research question can also be divided into the following survey questions, the answers to which will determine the composite answer. 2.1. Top Six Twitter Accounts with Highest supportive Comments. Table 2 Tweets Posted By Count of Total Percentage Supportive Tweets of Comments Studied Acceptance Digital India 127 541 23.48 Ravi Shankar Prasad 47 128 36.72 Ajay Kumar 7 12 58.33 9675 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) MyGovIndia 5 8 62.50 Ajay Sawhney 4 11 36.36 Grameen Vidyutikaran 4 5 80.00 Figure 2 From the table and the figure above, the following pointers become clear: 1) With 36.8 per cent of supportive comments, it is evident that the Tweets by Ravi Shankar Prasad were more accepted by the people. 2) Tweets by Digital India that were 541 in number were supported only 127 times or with 23.47 per cent of supportive comments. 3) Ravi Shankar Prasad, who was the concerned Union Minister at the time of the study, naturally attracted a lot of public attention and acceptability. Here again, the support percentage, although constituting the largest category, does not cross the halfway mark. The reasons are fairly comprehensible and clear. There would always be a dedicated section of the country’s populace, who would be politically opposed to a certain minister. This section would continue to spew their vitriol irrespective of the ground realities. The same 9676 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) could be applied to the case of Prasad’s Tweets concerning Digital India initiative as well. 2.2. Top Five Twitter Accounts with Highest Neutral Comments. Table 3 Tweets Posted By Count of Total Percentage Neutral Tweets of Comments Studied Acceptance Digital India 83 541 15.34 Ravi Shankar Prasad 27 128 21.09 NITI Aayog 5 27 18.52 NIELIT 4 10 40.00 Ajay Sawhney 3 11 27.27 Figure 3 From the table and the figure above, the following pointers become clear: 1) The table shows that 21.09 per cent of comments through the Tweets of Mr. Ravi Shankar Prasad are neutral. 9677 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 2) Prasad received the higher percentage of neutral comments as compared to the comments received by Digital India account, which is 15.34 per cent. 3) Neutral comments neither express support nor express disapproval. Here as well, the results were not out of expectations. Prasad, who was the concerned minister in charge at the time of the study, naturally is the person to get in touch with. Naturally, therefore, the comments that belong to the neutral category flooded his Tweets. At the same time, the Twitter page of Digital India was not left high and dry in the process. This page also had its share of neutral comments, comments that could become positive if there is a substantial push in the publicity activities of the page. 2.3. Top Five Twitter Accounts with Highest Comments on Criticism. Table 4 Tweets Posted By Count of Total Percentage Criticizing Tweets of Comments Studied Acceptance Digital India 76 541 14.05 Ravi Shankar Prasad 25 128 19.53 NITI Aayog 4 27 14.81 Aadhaar 3 4 75.00 Ajay Kumar 3 12 25.00 9678 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) Figure 4 From the table and the figure above, the following pointers become clear: 1) The above table shows that with a lesser number of Tweets, people criticized the Tweets of Ravi Shankar Prasad more than Digital India Tweets by 5.03 per cent. 2) It needs to be remembered here that the chances of netizens criticizing a person is more than the chances of the same netizens criticizing an official page. Subject to the fact that Prasad has a human face, he was at the receiving end of most of the criticizing and abusing comments. Not all abusive and criticizing comments could be attributed to the performance factor associated with Digital India. Some of the comments are made just for the heck of making comments. The Digital India Tweets by Prasad also invited some unwanted and ill-informed criticizing comments from netizens across the spectrum. 2.4. Top Five Twitter accounts with highest comments on enquiry. 9679 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) Table 5 Count of Comments Total Percentage with Tweets of Tweets Posted By Enquiry Studied Acceptance Digital India 55 541 10.17 Ravi Shankar Prasad 13 128 10.16 GCCS Official 5 44 11.36 NITI Aayog 4 27 14.81 Ajay Sawhney 3 11 27.27 Figure 5 From the table and the figure above, the following pointers become clear: 1) The percentage of enquiries addressed to Digital India and Ravi Shankar Prasad are almost the same. Ajay Sawhney received three enquiries from 11 posts. Those enquiries were mainly on National Digital Payments Mission, the extent of digital platform in the Parliament and its adoption by the Member of Parliament. One of the comments was also for seeking more information on the TCS 9680 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) initiatives to on-board and train merchants on the Aadhaar-enabled platform. 2) The Digital India initiative is an innovative programme launched by the Government of India. No wonder that there are queries vis-àvis its implementation, scopes and plans. Therefore, many netizens wanted to know as to what the scheme is all about. Subject to the fact that the programme has nine different pillars, the enquiries ranged across multiple domains. While some of the enquiries were genuine, some were evidently spurious in nature. However, the nature of the internet is such that there would always be a bunch of people, who would ask the wrong questions and for the heck of asking. 3. Which are the categories of communication (language/ link) used in the comments by public and what sentiments do they reflect? This research question can also be divided into the following survey questions, the answers to which will determine the composite answer. 3.1. Languages of Comments. Table 6 Total No. of Languages or Categories Comments English 608 Hindi 99 Link 5 Image 3 Punctuation 1 None Noted 165 9681 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) Figure 6 From the table and the figure above, the following pointers become clear: 1) Out of 881 comments studied in this research, the researcher found that 204 comments or 69 per cent of the total comments are in the English language. 2) There were 165 Tweets, which received no comments from people. Those Tweets were from various accounts and covered all the eight pillars of the Digital India programme. 3) There were 99 comments, which were posted in the Hindi language. They account for 11 per cent of the total posts. 4) The findings that were arrived above cement some convictions about the state of internet in the country. It remains a fact and is proven beyond doubt by the first finding that English is the dominant language of communication followed by Hindi, which is the most spoken language in the country. If one takes a closer look, it can be seen that English establishes its dominance over Hindi by a distance. There were quite a few Tweets that did not attract any comment and could not be categorized based on languages. It is also interesting that very few Tweets attracted links as comments. 9682 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 3.2. Break-Up of the Languages of Comments. Total Context Out of Neutral Comments No Enquiry Language Criticism Supportive Table 7 English 204 103 84 0 121 96 608 Hindi 23 19 5 0 14 38 99 Link 0 0 0 5 0 0 5 Image 3 0 0 0 0 0 3 Punctuation 0 0 1 0 0 0 1 None Noted 0 0 0 165 0 0 165 Grand Total 230 122 90 172 135 132 881 Figure 7 From the table and the figure above, the following pointers become clear: 1) The highest number of supportive coments were in the English language. 9683 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 2) Out of the 608 comments in English, the percentage of supportive comments account for 33.55 per cent, criticising comments account for 16.94 per cent and neutral comments account for 19.90 per cent. 3) Out of the 99 comments in Hindi, supportive comments account for 23.23 per cent, criticising comments account for 19.19 per cent and neutral comments account for 14.14 per cent. 4) Thus, the percentage of supportive and neutral comments are higher in English language by 10.32 per cent and 5.76 per cent respectively. The percentage of criticising comments are higher in the Hindi language by 2.25 per cent. 5) It is quite interesting to note that the percentage of supportive comments in English is significantly higher than the percentage of supportive comments in Hindi. To an extent, it establishes the citycentric orientation of any digital initiative, with Digital India programme not constituting an exception. The same thing happens when one looks at the neutral comments as well. Here as well, English trumps Hindi in terms of sheer percentages. In very simple words, it could be deduced that the acceptability factor of the Digital India initiative is much higher amongst the English educated class. Inter-Rater Reliability Test to Calculate Cohen’s Kappa (k) for Objective 3: The researcher conducted Inter-Rater reliability test to calculate Cohen’s Kappa (k) to find out the reliability or interobserver agreement between two raters A and B. Interrater reliability, or precision, happens when one’s data raters (or collectors) give the same score to the same data item. The researcher selected 30 comments from 881 comments of study through Systematic Sampling by using the formula 881/30 = 29.36. Hence, the researcher considered every 29th comment to study. The researher categorized the ‘comment category’ with the following codes to study the Kappa Coefficient (k) 9684 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) The researcher further coded 1 for rater’s agreement with the researcher and 0 for rater’s disagreement with the researcher. Table 8 Category Code Support A Neutral B Criticism C Enquiry D Out of Context E N/A F Table 9 Agreement 1 Disagreement 0 Formula for Calculating Cohen’s Kappa: Here, po is the relative observed agreement among raters and pe is the hypothetical probability of chance agreement. The Kappa Coefficient (k) varies from 0 to 1, where: 0 = Agreement Equivalent to Chance. 0.1 – 0.20 = Slight Agreement. 0.21 – 0.40 = Fair Agreement. 0.41 – 0.60 = Moderate Agreement. 0.61 – 0.80 = Substantial Agreement. 0.81 – 0.99 = Near Perfect Agreement 1 = Perfect Agreement. Data Observation: 9685 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 1) 27 conclusions by researcher were accepted by both Rater A and B. 2) 2 conclusions by researcher was rejected by both Rater A and B. 3) Overall Rater A accepted 28 conclusions and rejected 2 conclusions by the researcher. 4) Overall Rater B accepted 27 conclusions and rejected 3 conclusions by the researcher. Calculation of Cohen’s Kappa for This Data Set: 1) Calculation of po, the relative observed agreement among raters:  27 conclusions by the researcher were accepted by both the Raters A and B.  2 conclusions by the researcher was rejected by both the Raters A and B. Therefore, po = Number in agreement/Total = (27 + 2)/30 = 29/30 = 0.96 2) Calculation of pe, the hypothetical probability chance agreement: Probability of both the raters agreeing to researcher:  Rater A accepted 28 conclusions out of 30, i.e. 28/30 = 0.93.  Rater B accepted 27 conclusions out of 30, i.e. 27/30 = 0.9.  Total probability of both the raters accepting the researcher = 0.93 x 0.9 = 0.84. Probability of both the raters rejecting to the researcher:  Rater A rejected 2 conclusions by the researcher out of 30, i.e. 2/30 = 0.06.  Rater B rejected 3 conclusions by the researcher out of 30, i.e. 3/30 = 0.1.  Total probability of both the raters rejecting the researcher is 0.13 x 0.1 = 0.006. Therefore, pe = 0.84 + 0.006 = 0.85. Therefore, applying the values of po and pe in the formula of Cohen’s kappa, 9686 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) = (0.96 - 0.85)/(1 - 0.85) = 0.11/0.15 = 0.73 This final value falls in the range of 0.61 – 0.80. Hence, there is substantial agreement between the two raters. Important Notes: In context of the study done to fulfill Objective 3, it is pertinent that some of the words are explained. 1. Support represents positive comment. In other words, support is categorized as any affirmative comment given in favour of any of the schemes of the Digital India initiative. 2. Criticism represents negative comment. Criticism could be defined as any negative comment given concerning any of the schemes of the Digital India initiative. 3. Neutral statements represent statements related to the post, but not for or against the programme. No content analysis on neutral statements has been done purely because the same cannot be segregated any further. 4. Enquiry means any enquiry on the Digital India initiative. No content analysis on enquiries has been done again because the same cannot be segregated any further. 5. Comments are marked as English or Hindi as per the script used for making the comment. 6. News coverage as comment is studied on the basis of the headline. 7. Full sentence or combination of words is considered for content analysis in order to understand the context of support or criticism. 8. If comments are mixed in Hindi and English script, then the script for the first word has been considered for study. 9. Comments with image have not been considered for study. 9687 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) Consecutive comments by same person is not taken for content analysis for the second time. Consecutive comments by same person is taken for content analysis in cases where there are only two comments. Conclusion: To summarize the analysis, it can be primarily said that the public comments on the Digital India Twitter page could be broadly categorized into supportive, neutral, criticism and enquiry categories. With the numerical calculation involved in the content analysis, it was found out that supportive comments represent the highest category with a total number of 230 comments or 26 per cent of total comments (881) taken for the study. The next highest number of comments fell under the neutral category, which accounted for 135 comments or just 15 per cent of the total comments. There were 122 comments with criticism, which accounted for 14 per cent of the comments being studied. A number of individuals came up with enquiries for various services of Digital India, which accounted for 90 comments or 10 per cent. 15 per cent of the comments were out of context – comments that had no relevance with reference to the Digital India tweets. With 36.8 per cent of supportive comments, it is evident that the Tweets by Ravi Shankar Prasad were more accepted by the people. Tweets by Digital India, with the number being 541, were supported only 127 times translating to 23.47 per cent. 21.09 per cent of comments through the Tweets of Ravi Shankar Prasad were neutral. Prasad got the higher percentage of neutral comments as compared to the comments received by Digital India account, which stood at 15.34 per cent. With a smaller number of tweets, people criticized the Tweets of Ravi Shankar Prasad more than Digital India Tweets by 5.03 per cent. The percentage of enquiries asked to Digital India and Ravi Shankar Prasad were almost the same. The primary reason for 9688 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) Ravi Shankar Prasad having received the maximum number of comments in all categories is his significantly large number of Twitter followers as compared to the Digital India account. Out of the 881 comments studied in this research, the researcher found that 204 comments or 69 per cent of the comments were in the English language. There were 165 Tweets, which received no comments from people. There were 99 comments or 11 per cent of the total comments, which were posted in the Hindi language. Hence, a significantly high number of comments on the Twitter handle of Digital India were in English. The analysis further highlighted that the highest number of supportive comments was in the English language. Out of the 608 comments in English, the percentage of supportive comments stood at 33.55, neutral comments stood at 19.90 per cent and criticizing comments stood at 16.94 per cent. Out of the 99 comments in Hindi, supportive comments stood at 23.23 per cent, neutral comments stood at 14.14 per cent and criticizing comments stood at 19.19 per cent. Thus, the percentages of supportive and neutral comments were higher in English language by 10.32 per cent and 5.76 per cent respectively. The percentage of criticizing comments was higher in the Hindi language by 2.25 per cent. Hence, public comments that were posted in English had a higher inclination towards supporting the Tweets by Digital India. Limitations: Although the researcher tried to make the study as error-free as possible, there were some limitations that crept into the study. One can look at some of the bigger limitations first. Here they are: 1. The total number of Tweets posted on the Digital India Twitter page was 2,690 for a period of one year ranging from 21/11/2016 to 9689 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 20/11/2017. Due to limitation on time available for the study, the researcher could not consider all the Tweets for study and hence obtained the data with Quasi-systematic Sampling (or Systematiclike Sampling) by taking a representation of two days per week, maintaining a gap of seven days between each set. 2. The total number of comments for 572 Tweets was 5,262. Due to limitation of time, the researcher studied only the first and second comments of each Tweet, which translates to 881 comments. 3. Only the first tweet for each of the days was considered for the study. 4. The total number of replies in 572 posts was 5262. Therefore, the average number of replies per Tweet is 9.199, which is equivalent to 10 posts per day. Hence, the researcher decided to consider the first comment of each day due to the limitation of time. 5. The final comment is ranked as either Support, Neutral, Enquiry, Criticism or Out of Context. 6. Some comments were given only with YouTube links. Those links that do not play were not considered for the study. 7. The categorization of comments written in both English and Hindi was done based on the script with which the comments start. To sum it up, it could be safely deduced that the Twitter page of the Digital India initiative has been significantly successful in making people aware about the initiative turning their opinion in favour of the programme. While there is no denying that the programme itself has the potential to transform the fortunes of the country, the public relations activities of its official Twitter page cannot be discounted either. At a time when branding something properly has become the cornerstone of success for any initiative, the Twitter page of Digital India can be considered a good example of how such a page should function. References: 9690 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 1. Aggarwal, S. (2018, June 20). National e-Governance Plan. Retrieved from National e-Governance Plan Web Site: https://nisg.org/files/documents/A05140001.pdf 2. Digital India, M. o. (2019, March 13). About Digital India. Retrieved from Digital India Website: http://www.digitalindia.gov.in/content/about-programme 3. Digital India, M. o. (2019, March 13). Digital India Power to Empower. Retrieved from Digital India Web Site: https://www.digitalindia.gov.in/ 4. Digital India, M. o. (2019, March 13). E-GOVERNANCE POLICY INITIATIVES UNDER DIGITAL INDIA. Retrieved from Digital India Web site: http://digitalindia.gov.in/content/e-governance- policy-initiatives-under-digital-india 5. Digital India, M. o. (2019, March 13). ELECTRONICS MANUFACTURING. Retrieved from Digital India Web site: http://digitalindia.gov.in/content/electronics-manufacturing 6. Digital India, M. o. (2019, March 13). How Digital India will be realized: Pillars of Digital India. Retrieved from Digital India Website: http://digitalindia.gov.in/content/programme-pillars 7. Digital India, M. o. (2019, June 19). Vision of Digital India. Retrieved from Digital India Website: https://www.digitalindia.gov.in/content/vision-and-vision-areas 8. Reuters (2017, October 2). Smartphones made in India? Manufacturing ambition hits hurdles. Retrieved from ET tech: https://tech.economictimes.indiatimes.com/news/mobile/smartphones -made-in-india-manufacturing-ambition-hits-hurdles/60911110 9. Government of India. (2018, June 19). Digital Payment Methods. Retrieved from Cashless India Web site: http://cashlessindia.gov.in/digital_payment_methods.html 9691 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 10. PMINDIA. (2018, June 19). Make In India. Retrieved from PM India Web site: http://www.pmindia.gov.in/en/major_initiatives/make-inindia/ 11. Press Information Bureau (2015, March 25). Approach and Key Components of e-Kranti: National e-Governance Plan 2.0. Retrieved from Press Information Bureau Website: http://pib.nic.in/newsite/PrintRelease.aspx?relid=117690 12. Press Information Bureau (2017, March 3). India to Host 10TH International Conference on Theory and Practice of Electronic Governance: ICEGOV 2017. Retrieved from Government of India Web site: http://pib.nic.in/newsite/PrintRelease.aspx?relid=158823 13. Press Information Bureau (2017, July 21). India to Host Global Conference on Cyber Space 2017 – World’s Largest Conference on Cyber Space. Retrieved from Government of India Web site: http://pib.nic.in/newsite/PrintRelease.aspx?relid=168850 14. Statista (2019, April 27). Penetration of leading social networks in India as of 3rd quarter 2017. Retrieved from The Statistical Portal Web site: https://www.statista.com/statistics/284436/india-social- network-penetration/ 15. Agarwal S. (2018, February 20). Internet users in India expected to reach 500 million by June: IAMAI . Retrieved from The Economic Times Website: https://economictimes.indiatimes.com/tech/internet/internet-users-inindia-expected-to-reach-500-million-by-juneiamai/articleshow/63000198.cms 16. FE Online (2018, May 31). BHIM app receiving this big update: Know what it is and how to use it. Retrieved from Financial Express Web site: https://www.financialexpress.com/industry/technology/bhim-appreceiving-this-big-update-know-what-it-is-and-how-to-useit/1188686/ 9692 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 17. Fiona Maclean, D. J.-L. (2013). Understanding Twitter. British Journal of Occupational Therapy, 295-298. 18. Sanderson, J. (2014). What Do We Do With Twitter? Communication & Sport, 127-131. 19. Murthy, D. (2011). Twitter: Microphone for the masses? Media, Culture & Society, 779-789. 20. Matthew Hindman, V. B. (2018). Disinformation, ‘Fake News’ and Influence Campaigns on Twitter. Miami: Knight Foundation. 21. Tyler H. McCormick, H. L. (2017). Using Twitter for Demographic and Social Science Research: Tools for Data Collection and Processing. Sociological Methods & Research, 390-421. 22. DWIVEDI, VEDVYAS JAYPRAKASH, and Yogesh C. Joshi. "PRODUCTIVITY IN 21 st CENTURY INDIAN HIGHER EDUCATION INSTITUTIONS." International Journal of Human Resource Management and Research 9.4 (2019): 61-80. 23. Billings, A. (2014). Power in the Reverberation: Why Twitter Matters, But Not the Way Most Believe. Communication & Sport, 107-112. 24. Rinker, M. (2020). A framework with efficient extraction and analysis of Twitter data for evaluating public opinions on transportation services. Travel Behaviour and Society, 10-23. 25. Radhika, R., and Ramesh Kumar Satuluri. "A study on life insurance penetration in India." International Journal of Human Resource Management and Research (IJHRMR) 9.1 (2019). 26. Elia Gabarron, E. D.-R. (2018). Diabetes on Twitter: A Sentiment Analysis. Journal of Diabetes Science and Technology, 1-6. 27. Tony P. Love, J. L. (2018). A Mixed-Methods Examination of Morality Work Through Sentiment Analysis and Qualitative Coding of Twitter Data. SAGE Research Methods Cases, 1-12. 28. JINDAL, NEENA, KRITIKA THAKUR, and TANIA SHARMA. "DIGITAL INDIA: CHALLENGES, SOLUTIONS AND ITS IMPACT ON SOCIETY." International Journal of Environment, Ecology, Family and Urban Studies (IJEEFUS)9. 2, Apr 2019, 83-90 9693 Identification of Public Sentiment over Comments through Tweets by Digital India PJAEE, 17 (7) (2020) 29. Yinying Wang, D. J. (2017). Common Core State Standards on Twitter: Public Sentiment and Opinion Leaders. Educational Policy, 1-34. 30. Vanan, C. KURINCHI, and R. Subramani. "Digital divide: rural and urban college students ‘attitude towards technology acceptance." International Journal of Communication and Media Studies (IJCMS) 5.4 (2015): 1-8. 9694