International Journal of Engineering Applied Sciences and Technology, 2022

Vol. 7, Issue 12, ISSN No. 2455-2143, Pages 111-115

Published Online April 2023 in IJEAST (http://www.ijeast.com)

CHAT GPT: EXPLORING THE CAPABILITIES

AND LIMITATIONS OF A LARGE LANGUAGE
MODEL
Shubham Singh, Srishti Tiwari, Prabhat Kumar Yadav, Dr. Atul Kumar, and Dr. Vibha Srivastava
Department of Information Technology,
Shri Ramswaroop Memorial College of Engineering and Management,
Lucknow, India

Abstract: ChatGPT is a popular computer program that II. CREATION OF CHATGPT

simulates human communication. It can comprehend Leading artificial intelligence research facility OpenAI
normal language and respond in human-like ways. It developed the GPT (Generative Pre-trained Transformer)
uses complicated algorithms that have been trained on a series of language models. The transformer architecture was
large quantity of text data to provide logical responses. used to train the first GPT model, GPT-1, which was made
This paper explains the design and training process of public in 2018. It was trained on a vast amount of text data.
ChatGPT, as well as its capacity to respond to a range of GPT-1 excelled at natural language processing tasks like
inquiries. This research also investigates ChatGPT's question answering and language translation, but later
disadvantages, such as its proclivity to create biased models in the series outperformed it. GPT-2, which was
replies, as well as the potential ethical implications of its launched in 2019, demonstrated state-of-the-art performance
employment. in a range of language tasks, including text production. It
was trained on an even larger dataset.
Keywords:ChatGPT, NLP, Chatbot, Conversational AI,
Language Translation, Text Generation.

I. INTRODUCTION
ChatGPT was first released in June 2020 by OpenAI, an
artificial intelligence research organization based in San
Francisco, California. The term ―ChatGPT‖ is a
combination of two parts: ―Chat‖, which refers to the
program’s ability to simulate conversation with humans, and
―GPT,‖ which stands for ―Generative Pre-trained
Transformer‖, the name of the deep learning architecture Figure 1: Evolution from Transformer architecture to
that ChatGPT is based on. The researchers who created the ChatGPT. Source:
initial GPT model, which was made available by OpenAI in https://pwr.edu.pl/fcp/aGBUKOQtTKlQhbx08SlkTUhZeUT
2018, were the first to use the phrase "Generative Pre- gtCgg9ACFDC0RFTm9PFRYqCl5tDXdVGnoV/1/public/n
trained Transformer." With the aid of a sizable corpus of ews_team/w4_dr_sobon_laser/evolution_1.jpg
text data and a deep learning architecture referred to as a
transformer, the GPT model was created to produce text in a 2020 saw the introduction of GPT-3, an even bigger model
manner comparable to that of humans. In order to reflect the that was trained on a massive dataset of more than 45
ChatGPT model's emphasis on mimicking discussion with terabytes of text data. It displayed the ability to carry out a
humans, OpenAI merged the terms "GPT" and "Chat" when variety of natural language processing tasks, including
they published it in 2020. Since then, the program's natural language translation, question answering, and
language processing capabilities have grown to be known as summarization, and was able to produce prose that was
"ChatGPT." In the area of artificial intelligence, ChatGPT astonishingly human-like.
has already shown that it has a lot of potential, and it is
anticipated to have a big impact in the years to come. One of the many GPT-3 applications created expressly for
However, something to keep in mind are the ethical conversational AI is ChatGPT. Figure 1 above shows the
implications of ChatGPT, including matters like bias, evolution from transformer architecture to ChatGPT. It can
privacy, and safety. have talks that sound natural because it was trained on a big
dataset of conversational data. Its growth is a continuation

111
 International Journal of Engineering Applied Sciences and Technology, 2022
Vol. 7, Issue 12, ISSN No. 2455-2143, Pages 111-115
Published Online April 2023 in IJEAST (http://www.ijeast.com)

of language model evolution, and future advancements are presenting it with a huge corpus of text data throughout the
to be anticipated. training process. Language modeling is the term for this
procedure.
III. CHATGPT AND ITS UNDERLYING The model can capture long-term dependencies and
TECHNOLOGIES contextual linkages between words in a sequence because of
A. Deep Learning the numerous layers of self-attention processes that make up
Deep learning heavily influences ChatGPT's architecture ChatGPT's transformer design. The model may assign
and functionality. A deep learning architecture called the varying weights to various components of the input
transformer model is particularly well suited for natural sequence thanks to the self-attention mechanism, allowing it
language processing (NLP) applications such as language to concentrate on the components that are crucial for
modeling and text production. Due to its various layers of producing the next word in the sequence.
self-attention mechanisms, the model can capture long-term The GPT (Generative Pre-training Transformer) architecture
dependencies and contextual interactions between words. is the name of the transformer architecture that ChatGPT
The transformer concept is applied in ChatGPT to produce employs. It can produce high-quality content in a variety of
replies to text-based inputs. The model is trained to predict circumstances because it has been pre-trained on a vast
the following word in a sequence given the preceding words amount of text data. Additionally, the GPT architecture can
by presenting it with a huge corpus of text data throughout be customized for certain NLP tasks like text
the training process. The ability of ChatGPT to provide summarization, sentiment analysis, or language translation.
coherent and contextually relevant responses is based on a
process known as language modeling. Using large corpora D. Pre-training
of data, it can be fine-tuned for specific tasks like language Before fine-tuning the model for a particular job, such as
translation or conversation production. The model is trained language translation or dialogue production, pre-training
to provide responses that are appropriate for the job at hand entails training the model on a sizable corpus of text data.
using task-specific data that are submitted to it during fine- In the instance of ChatGPT, the model is pre-trained using a
tuning. language modeling objective on a vast amount of text data.
A wide variety of publications, including books, papers, and
B. Natural Language Processing (NLP) online pages, make up the pre-training data. The model is
The area of artificial intelligence known as NLP is trained to anticipate the following word in a sequence given
concerned with processing and interpreting natural language the words that came before it during pre-training. The model
text and speech. To comprehend the input text and produce can learn the fundamental relationships and patterns in
appropriate responses, ChatGPT employs several NLP natural language text thanks to this method.
approaches. These methods consist of: ChatGPT can gain a profound knowledge of the structure
 Tokenization: ChatGPT separates the input text into and nuanced aspects of natural language by pre-training the
discrete tokens, such as words or subwords, and then model on a vast amount of text data. As a result, it may
feeds these tokens into the model to be processed. produce excellent responses to a variety of conversational
inputs. The model can be fine-tuned for certain tasks like
 Part-of-speech tagging: ChatGPT uses part-of-speech
dialogue generation or language translation after it has been
tagging to determine each word's grammatical function
pre-trained.
in the input text.
 Named entity recognition: ChatGPT employs named
IV. IMPACT OF CHATGPT ON DIFFERENT
entity recognition to locate entities mentioned in the
SECTORS
input text, such as persons, businesses, and locations.
 Dependency parsing: ChatGPT utilizes dependency A. Healthcare
parsing to examine the connections between words in a The healthcare sector has been significantly impacted by
phrase. ChatGPT. It has been used to facilitate patient
 Sentiment analysis: ChatGPT can determine the tone or communication, speed up the diagnosing process, and
sentiment of the input text via sentiment analysis. advance medical research thanks to its language
understanding capabilities and capacity to produce replies
C. Transformers that resemble those of humans. ChatGPT can be used to
The field of Natural Language Processing (NLP) has examine symptoms and offer possible diagnoses, enhancing
undergone a profound change due to the Deep Learning the precision and promptness of patient care. To speed up
Architecture called "Transformers". medical research and the creation of new treatments and
Transformers are used in ChatGPT to produce answers to therapies, it can also be used to extract important data from
text-based inputs. The model is trained to predict the electronic health records and evaluate huge amounts of
following word in a sequence given the preceding words by medical data. Furthermore, ChatGPT can be utilized to offer
patients individualized mental health assistance and

112
 International Journal of Engineering Applied Sciences and Technology, 2022
Vol. 7, Issue 12, ISSN No. 2455-2143, Pages 111-115
Published Online April 2023 in IJEAST (http://www.ijeast.com)

guidance, making mental health interventions more Based on a customer's financial background and objectives,
available and practical. it can also be used to offer tailored financial advice, such as
investment suggestions or debt management techniques. To
protect both clients and financial institutions, ChatGPT can
also be used to analyze financial data and spot potential
fraud or suspicious behaviour. Financial companies may
boost productivity, lower the risk of financial fraud, and
improve customer service by utilizing ChatGPT.

V. ETHICAL CONSIDERATIONS OF CHATGPT

Concerns regarding ChatGPT use were stated in 58/60
Figure 2: Summary of benefits/applications of ChatGPT in (96.7%) records including ethical, copyright, transparency,
health care education. and legal issues, the risk of bias, plagiarism, lack of
originality, inaccurate content with risk of hallucination,
limited knowledge, incorrect citations, cyber security issues,
B. Education
and risk of infodemics[2].
ChatGPT can be utilized to give students individualized
learning experiences, offer online coaching, and support, A. Bias
and help with grading and assessment. By offering language ChatGPT may be biased depending on the training set of
data. The effectiveness and potential biases of ChatGPT can
translation services and assisting in the removal of language
be significantly influenced by the training data's quality and
barriers, ChatGPT can also be utilized to increase access to
collection methods.
education. Additionally, it can be used to create interactive
learning materials like quizzes or simulations and analyze The model may produce biased text if the training set used
student performance data to pinpoint areas that require more to create ChatGPT is biased or lacking. The model might
help. Teachers may deliver more individualized and not work well on text from different demographic groups or
areas, resulting in biased content, for instance, if the training
accessible learning experiences while also easing
data primarily consists of text from a certain demographic
administrative activities like grading and assessment by
group or region.
utilizing ChatGPT's language comprehension capabilities.

C. E-commerce
Much progress has been made by ChatGPT in influencing
the e-commerce industry. Its language comprehension skills
and capacity to produce responses that sound human can be
used to enhance customer satisfaction, speed up the
purchasing process, and boost sales. Based on a customer's
browsing and purchasing history, ChatGPT can make
tailored recommendations. It can also be utilized to offer
customers virtual support, responding quickly and
effectively to problems and common questions. ChatGPT
Figure 3: Gender bias in performance feedback written by
can also be used to provide customized product descriptions,
ChatGPT. Source: https://textio.com/blog/chatgpt-writes-
evaluations, and other marketing content, all of which can
performance-feedback/99766000464
enhance the overall buying experience. E-commerce
businesses can offer more individualized and successful
It is critical to remember that ChatGPT is a language model
marketing strategies and ultimately boost their sales by
and is impartial by nature. Figure 2 above shows the gender
utilizing ChatGPT's language comprehension skills.
bias in performance feedback written by ChatGPT.
To make sure that the training data is diverse,
D. Financial Services
representative, and devoid of any prejudice or stereotypes, it
The financial services industry has been significantly
is essential to address any potential biases in the data. Using
impacted by ChatGPT. It can be used to automate customer
inclusive and varied training data is one way to make sure
care, enhance fraud detection, and give customers
the model is exposed to a variety of texts and viewpoints.
individualized financial advise by utilizing its language
Using bias mitigation approaches throughout the training
understanding capabilities and capacity to produce human-
and testing phases is another strategy for minimising any
like responses. Customers can receive virtual support
potential biases in the model's output.
through ChatGPT, which can respond to inquiries regarding
their accounts, transactions, and other financial issues.

113
 International Journal of Engineering Applied Sciences and Technology, 2022
Vol. 7, Issue 12, ISSN No. 2455-2143, Pages 111-115
Published Online April 2023 in IJEAST (http://www.ijeast.com)

VI. CONCLUSION
B. Traffic ChatGPT is a large language model that could completely
The computational resources available to ChatGPT restrict alter how we communicate with both machines and one
its capacity to react to requests. There is a limit to how another. This study report examined ChatGPT's strengths
many queries it can handle at once, despite its ability to and weaknesses, concentrating on its capacity to produce
process many them in simultaneously. The volume of traffic coherent text, its biases, ethical issues, and privacy
that ChatGPT is handling can have an impact on how well it concerns. Although ChatGPT has shown outstanding ability
responds. The model might need to dedicate resources to in producing text that is coherent and appropriate for the
handle all the queries when many come in at once, which environment, it is crucial to address ethical issues including
could influence its capacity to produce high-quality bias, fairness, and openness during the development and
responses. This can cause replies to be given more slowly or implementation process. To further secure user privacy and
with less accuracy. uphold faith in the technology, privacy issues relating to
data gathering, data sharing, and data breaches must be
C. Limited Knowledge addressed. . Despite many limitations and weaknesses, these
Although ChatGPT has been trained on a staggering amount results suggest that very large language models may be an
of text material and can produce answers to a wide range of important ingredient in the development of adaptable,
queries and prompts, there are still some subjects or fields of general language systems [10].Ultimately, ChatGPT is an
knowledge in which it might lack the knowledge or innovative technology with the potential to completely
comprehension to give accurate or helpful answers. The data change how we communicate with both machines and one
that ChatGPT was trained on, which has a cutoff date of another. So,it is crucial to approach this technology with
2021, limits the knowledge that it has access to. Even prudence and to keep investigating its potential and
though this covers a lot of ground, it does not consider any constraints in a moral and responsible way.
recent changes or occurrences. It's also important to note
that ChatGPT is not able to actively seek out new REFERENCES
knowledge from experience. It only uses the data that it was
trained on, and while it may adapt to new inputs to some [1]. Lund, B. D., & Wang, T. (2021). Chatting about
extent, it is only as smart as the data that it was taught on. ChatGPT: How may AI and GPT impact academia
and libraries?. Journal of Academic Librarianship,
D. Privacy 47(5), 102519.
A natural language processing model called ChatGPT can https://doi.org/10.1016/j.acalib.2021.102519
produce text that is human-like, which creates various [2]. Sallam, M. (2021). ChatGPT Utility in Health Care
privacy issues. The gathering of data is one of the main Education, Research, and Practice: Systematic
privacy issues with ChatGPT. The user's input and any Review on the Promising Perspectives and Valid
personal information contained in the text that is created are Concerns. Journal of Medical Internet Research,
both examples of data that the model may gather and retain 23(8), e30218. https://doi.org/10.2196/30218
about the user. Sensitive information including private [3]. Choudhury, A., & Shamszare, H. (2021).
preferences, opinions, and beliefs may be included in this Investigating the Impact of User Trust on Adoption
data. It is crucial to guarantee that the data acquired by and Use of ChatGPT: A Survey Analysis.
ChatGPT is safe and used only for legal purposes in order to Information, 12(6), 244.
preserve the user's privacy. Data sharing is a further privacy https://doi.org/10.3390/info12060244
issue with ChatGPT. The model's creators or other [4]. Ruby, M. (2021, September 22). How ChatGPT
organisations, for example, may receive access to the text Works: The Model Behind The Bot. Forbes.
that ChatGPT generates. Appropriate security measures https://www.forbes.com/sites/mollyruby/2021/09/2
must be in place to guarantee that the user's data is not 2/how-chatgpt-works-the-model-behind-the-
disclosed without their consent in order to protect their bot/?sh=47bf4d9d45a4
privacy. [5]. Brown, T. B., Mann, B., Ryder, N., Subbiah, M.,
Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020).
Language models are few-shot learners. Advances
in neural information processing systems, 33, 1-17.
[6]. Korteling, J.E.; van de Boer-Visschedijk, G.C.;
Blankendaal, R.A.M.; Boonekamp, R.C.;
Eikelboom, A.R. Human- versus Artificial
Intelligence. Front. Artif. Intell. 2021, 4, 622364.
[GoogleScholar] [CrossRef] [PubMed]
Figure 4: Summary of risks/concerns of ChatGPT

114
 International Journal of Engineering Applied Sciences and Technology, 2022
Vol. 7, Issue 12, ISSN No. 2455-2143, Pages 111-115
Published Online April 2023 in IJEAST (http://www.ijeast.com)

[7]. Sallam, M. (2023). ChatGPT Utility in Healthcare

Education, Research, and Practice: Systematic
Review on the Promising Perspectives and Valid
Concerns. Healthcare, 11(6), 887.
https://doi.org/10.3390/healthcare11060887
[8]. Chatterjee, J.; Dethlefs, N. This new conversational
AI model can be your friend, philosopher, and
guide … and even your worst enemy. Patterns
2023, 4, 100676. [Google Scholar] [CrossRef]
[PubMed]
[9]. Stokel-Walker, C. AI bot ChatGPT writes smart
essays—Should professors worry? Nature, 9
December 2022. [Google Scholar] [CrossRef]
[10]. Chen, T.J. ChatGPT and Other Artificial
Intelligence Applications Speed up Scientific
Writing. Available online:
https://journals.lww.com/jcma/Citation/9900/Chat
GPT_and_other_artificial_intelligence.174.aspx
(accessed on 16 February 2023).
[11]. Nature editorial. Tools such as ChatGPT threaten
transparent science; here are our ground rules for
their use. Nature 2023, 613, 612. [Google Scholar]
[CrossRef]
[12]. Gordijn, B.; Have, H.t. ChatGPT: Evolution or
revolution? Med. Health Care Philos. 2023, 26, 1–
2. [Google Scholar] [CrossRef]

115