2024 7th International Conference on Devices, Circuits and Systems (ICDCS) | 23-24 April 2024 | Karunya Institute of Technology

and Sciences, Coimbatore, India.

RAG-based LLM Chatbot using Llama-2

Sonia Vakayil D. Sujitha Juliet
School of Computer Science and Technology School of Computer Science and Technology
Karunya Institute of Technology and Sciences Karunya Institute of Technology and Sciences
Coimbatore, India Coimbatore, India
e-mail: soniavakayil@karunya.edu.in e-mail: sujitha@karunya.edu

Anitha. J Sunil Vakayil

School of Computer Science and Technology Management Development Centre
Karunya Institute of Technology and Sciences Loyola Institute of Business Adminstation
Coimbatore, India Chennai, India
e-mail: anitha_j@karunya.edu e-mail: sunil.vakayil@liba.edu

Abstract— Chatbots, otherwise known as autonomous empathetic responses to victims of sexual harassment which
conversational agents, are a rising utilitarian application of is not as prominently seen in ML models [6 - 7].
Natural Language Processing. They enable the streamlining of
information searches and improve user productivity and II. RELATED WORK
experience. This study focuses on building a chatbot that is
2024 7th International Conference on Devices, Circuits and Systems (ICDCS) | 979-8-3503-5047-0/24/$31.00 ©2024 IEEE | DOI: 10.1109/ICDCS59278.2024.10561020

This section of the study will explore the existing chatbot

aimed at assisting victims of sexual harassment, using a Large
implementations with a focus on their underlying models.
Language Model (LLM). While ML-based chatbots are a
notable prospect, LLM-powered chatbots offer more human-
Previous studies and projects have explored various
like conversations and can surpass humans in empathy. This approaches to developing chatbots for victims of sexual
project evaluated the performance of the LLM Llama-2 model harassment. Recent years have seen a surge in the utilization
in generating accurate and empathetic answers to create a of machine learning and large language models for developing
supportive, sensitive, and informative chatbot for the victims of chatbots to assist victims of sexual harassment.
sexual harassment. The model leverages Retrieval Augmented ML-based chatbots are employed to understand and
generation to achieve a commendable accuracy of above 95%,
respond to the complex and sensitive nature of conversations
providing information in an understanding and helpful tone.
related to sexual harassment. These chatbots are trained on
The model is also capable of providing helpful advice without
judgement or preconceived notions about the victim, one of the
datasets containing conversations, enabling them to recognize
reasons victims do not report their harassers. patterns and provide appropriate support and information to
the victims. However, they cannot generate answers that are
Keywords— LLM, Llama-2, chatbot, empathy, harassment specific to the user’s query or comment [8].

I. INTRODUCTION Large language models, on the other hand, have shown

promise in generating more human-like responses - fostering
The issue of sexual harassment and abuse affects millions a sense of empathy and understanding in interactions with
of people worldwide in today’s world. The current statistics victims. LLMs exhibit a deeper understanding of language
for the year 2024 state that, about 35% of women worldwide nuances and conventional emotional cues [9], owing to the
have experienced either physical and/or sexual intimate vast amount of data on which they are trained. This is
partner violence or non-partner sexual violence in their extremely crucial when engaging with individuals who have
lifetime [1]. In India, the situation is equally alarming, with a experienced trauma.
National Family Health Survey revealing that 30% of women
have experienced physical or sexual violence [2]. Chatbots can be categorised into two main types: rule-
based chatbots and end-to-end chatbots [8]. Rule-based
The victims of sexual harassment often labour under the chatbots are commonly used for the implementation of simple
emotional and mental impact of the traumatic experience. tasks – due to their lack of adaptability. They follow a
They generally hesitate to disclose their experience due to predefined conversational pathway and follow explicit rules
feelings of fear, shame, or fear of getting rejected and judged or a decision tree that specifies the conversational flow. This
[3 - 4]. As a result, they prefer to suffer in silence which may framework of the workflow is illustrated in Fig. 1 below.
lead to the development of psychological issues like
depression and anxiety. In such scenarios, chatbots can play a
vital role in providing a safe and anonymous space for victims
to seek help and information. Chatbots can stand as a resource
and guide to, individuals where one can access advice and
guidance without the fear of judgment or stigma. Moreover,
chatbots streamline the process of finding essential
information [5], eliminating the need for the victim to scour
the internet, which can be overwhelming and triggering.
Keeping these factors in mind, the LLM chatbot in this
project was developed to offer discreet but trustworthy
support to victims of sexual harassment, addressing the critical
need for mental health care and resources in an accessible,
sensitive, and compassionate manner. The chatbot uses the Fig.1 Workflow of a rule based chatbot
Llama-2 LLM model and aims to provide accurate and A study conducted [8] in response to the #MeToo
movement in the city of Maastricht illustrates a rule-based

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on November 29,2024 at 14:28:54 UTC from IEEE Xplore. Restrictions apply.

195
chatbot that is specifically designed for the support of sexual outperform most of its closed-source competitors such as
harassment survivors. The model is designed with separate ChatGPT [14].
ML models to identify harassment cases and the type of
harassment that has occurred. Its success rate for identifying The model is built on the powerful transformer
harassment cases exceeds 98%, but it faces challenges in Architecture model. The transformer model was introduced in
accurately identifying specific types of harassment and 2017 and relies on self-attention techniques to capture
handling time occurrences. The paper also specifies that the contextual information from the input text. It consists of an
model needs further improvement to generate answers that are encoder-decoder structure, where the encoder processes input
specific to the user input. tokens and the decoder generates output tokens. The multi-
head attention mechanism allows the model to attend to
End-to-end chatbots are also known as neural chatbots different parts of the input sequence simultaneously,
since they use deep-learning neural networks to generate enhancing its ability to learn complex patterns [14].
responses directly from the input text. Unlike rule-based
models, they are capable of handling complex conversations B. Workflow
since, they learn from data and adapt to user interactions – thus The chatbot created for this study works by using a
allowing more versatility. systematic workflow to generate responses based on user
queries. It uses Retrieval-Augmented-Generation (RAG) to
One study [10] used such a chatbot to check its retrieve relevant information from the store. The workflow
effectiveness in providing support to individuals with mental begins with PDF files that contain information about resources
health issues. This chatbot combines Behavioural Activation and laws pertaining to sexual harassment. The text is extracted
(BA) therapy with artificial intelligence (AI) to provide from these pdf files with the help of the PdfPlumber library
recurrent emotional support, personalized assistance, and [15].
remote mental health monitoring. The BA-based chatbot has
proven effective in supporting individuals with mental health The text extracted is then split into smaller chunks and
issues with a significant percentage of users reporting a better converted into vector representation. This embedding of text
mood score. into numerical vector form helps the Llama-2 model
understand the underlying semantic meaning of the text.
Another notable example is the Rainbow chatbot [11] Which in turn, helps retrieve relevant contexts from the Vector
which utilizes the OpenAI GPT model for generating database during answering. This embedding of chunks is done
responses according to the user input. It helps victims by with the help of the fine-tuned BERT-base model, trained on
providing personalised assistance and supports the survivors the MS MARCO dataset [16]. It maps the text into a 68-
emotionally. Regular updates of the model information base dimensional dense vector space and is hosted on the
ensure the enhancement of the chatbot’s understanding and HuggingFace community [17]. This process is done only
responsiveness. once.
The LawU chatbot [12] is another end-to-end chatbot that While using the model for question and answering
is specifically designed for victims of sexual harassment in the scenarios, users submit a query to the model, triggering a
region of Thailand. Utilizes the Llama-2 model for search process for relevant contexts or information in the
understanding natural language input from the user and ChromaDB. These contexts are sent to the Llama-2 LLM,
generating query-specific answers. Its main aim is to provide which processes these contexts and generates a meaningful
legal information and guidance to the victims. and human-understandable response. This response will be
There also exists a study [13] that combines a rule-based formatted based on the contexts as well as user intent (which
and ML-based approach to overcome the limitations of each is obtained from the query). This high-level workflow can be
method. The chatbot aims to support survivors of sexual seen in Fig. 2 below.
violence by addressing frequently asked questions related to
punishment, police reports, and support centres. All these
chatbots play a crucial role in supporting survivors of sexual
harassment, offering a scalable and accessible platform for
assistance and empowerment.
III. METHODOLOGY
A. Model architecture.
This study uses the Llama-2-7b model which is part of the
Llama-2 family of pre-trained generative text models created
by Meta and released in July 2023. They use an auto-
regressive approach - which sequentially generates text based
on the context provided to the model. We use the Llama-2-7b
model since it is capable of handling complex linguistics since Fig. 2 Llama-2-7b RAG chatbot
it has been pre-trained on a vast corpus of about 2 trillion C. Retrieval Augmented Generation
tokens of text data and supports longer context lengths of up
to 4096 tokens. For dialogue cases, the model has been fine- The Retrieval-Augmented Generation (RAG) framework
tuned in a supervised manner - using labelled data. The model enhances and further boosts the responding capability of the
is a great resource to use for the use case of this study since, it chatbot created. Traditionally, chatbots rely on pre-trained
is built to balance both helpfulness and safety in its responses. models or rules that generate responses that lack contextually
In most cases, the Llama-2-7b model has been found to appropriate information or user-specific responses. RAG
battles this limitation by allowing the chatbot to dynamically

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on November 29,2024 at 14:28:54 UTC from IEEE Xplore. Restrictions apply.

196
retrieve relevant and up-to-date contexts from the knowledge B. Preprocessing and embedding
base. By providing more detailed and reliable information, The study employs a preprocessing technique to enhance
RAG significantly enhances user engagement and trust in the quality of the extracted textual data that will be then
chatbot interactions. converted to vector embeddings. The first step was to remove
The model created uses a LangChain [18] retriever to any special characters and symbols that may introduce noise
retrieve the most relevant contexts from the vector database. or inconsistencies. Stopwords were removed, and the large
It accepts a string query as input and returns a list of the top-n document was broken down into smaller and manageable
context embeddings from the ChromaDB vector database. chunks. For all of these processes, we use Spacy’s stopword
The selection of these documents is based on their similarity list, the transformer’s StoppingCriteria and, the LangChain
to the input query. Cosine similarity is used to check the RecursiveTextSplitter.
similarity between the contexts and the query. It calculates the The chunks are thus ensured to be complete sentences that
cosine of the angle between two vectors (shown in Eq. (1)). aren’t split med-sentence and that the vector embeddings
𝐴⋅𝐵 capture the semantic information well. Using the ‘msmarco-
cos(𝜃) = ∥𝐴∥∥𝐵∥ (1) bert-base-dot-v5’ model, we generate embeddings for each
chunk. the embeddings created are then stored in a vector store
where A and B are the numerical vector representations of (ChromaDB).
the two texts.
C. Creating the Llama-2 model
When a query is given as input, the retriever calculates the
cosine similarity between the query’s embedding and the The Llama-2-7b Large language model used in this study
embeddings of all the documents in the database. The was downloaded and loaded on the system. The model was
documents with the highest cosine similarity are considered fed with a template that ensured that the chatbot would
the most similar to the query and the top-n is returned as the respond with safe, helpful, sensitive, and empathetic
output. responses. The template plays a huge role as a structured guide
for generating responses in a consistent and context-aware
D. Response generation manner.
The Llama-2 model generates responses by first It emphasises the importance of context – since this is
tokenizing the input text query, generating the response, and crucial in generating relevant and helpful answers for the user
finally decoding the tokenized response. Tokenizing is the query. The tone of the template is specifically set for empathy,
process of transforming text into numerical vector respect, support, and non-judgement. The model is explicitly
representation. Once the query is tokenized, it is passed to the instructed not to generate harmful, inappropriate, or false
LLM’s ‘generate’ method which uses the model’s learned content. Recognising the limitation of a chatbot in the case of
parameters to generate a sequence of tokens from that counselling and long-term help, the model is instructed to
response. This is then finally converted back to natural guide and encourage the victim to seek professional help. It
language form. This process allows Llama-2 to generate also looks at chat history to ensure that the user’s need is fully
responses that are contextually appropriate and semantically understood and that the responses generated are relevant to
coherent. their query.
IV. IMPLEMENTATION. The prompt template used for the model of this study is
given below:
A. Resource compilation
The pdf files, containing substantial information about the """Use the following pieces of context to answer the
relevant information were researched and compiled. All the question at the end. {context}. You are a helpful, respectful,
resources were taken from reputed and trustworthy sites [19 - and empathetic friend. Your friend has undergone a traumatic
25] –most of which are from the government or legal system event and you are trying to help them process the whole event
of India. Since this chatbot focuses - on helping victims of and take the next steps. You must be understanding and never
India, all the resources solely focus on helplines, contact blame them. Always answer as helpfully as possible, while
information, laws and NGOs that are specifically for the being safe. And try to gradually guide them to a therapist after
Indian population. The resources collected aid in equipping ensuring that they are in a safe space. Your answers should not
the chatbot with crucial contact details for victims in need – include any harmful, unethical, racist, sexist, toxic, dangerous,
regardless of gender and location in India. or illegal content. Please ensure that your responses are
socially and gender unbiased. Always answer using the
The chatbot is well-versed in the Legal services provided context provided and refrain from answering on your own. If
by the Indian government for Victims of sexual harassment a question does not make any sense, or is not factually
and is capable of understanding the various types of sexual coherent, explain why instead of answering something not
harassment defined. These resources also allow the chatbot to correct and ask for further explanation. If the context provided
recognise and understand the various scenarios put forth by does not have any relevant information or if you don't know
the user and respond accordingly with the relevant the answer to a question, please don't share any false
information along with advice and suggestions as to the information, reply by saying "Sorry, I am not sure of the
nearest NGO or support group and provide their contact answer - but please don't hesitate to contact a counsellor". You
information – regardless of the gender of the victim. can also look into chat history. {chat_history}
Question: {question}
Answer:"""

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on November 29,2024 at 14:28:54 UTC from IEEE Xplore. Restrictions apply.

197
 The relevant contexts are retrieved from the vector
database and are passed to the model as {context} as seen in
the prompt along with the chat history and user query which
are temporarily stored in the memory.
D. Response validation
The responses of the chatbot are consistently empathetic
and non-judgemental. It acknowledges user emotions
provides support and does not generate answers that accuse
the victim of any wrong. In fact, the model always seems to
be reassuring and kind. The model can handle cases of
uncertainty quite well – and gently advises the user to seek
professional help or provide more context.
The chatbot is fully competent at recognising the different Fig. 5 Handling emotional outbursts
types of harassment and tailoring its responses, accordingly,
offering relevant advice or resources. It does not discriminate
against the victim based on location, gender or identity and
offers support regardless. All these results can be seen in Fig.
3 to 7.
However, the chatbot is subject to sometimes retrieving
the wrong helpline for ‘boys’ (Shown in Fig. 8). This could be
due to the high difference in the amount of information
available for men in India who have experienced sexual
harassment and the very few helplines that are available for
male victims in India. Nonetheless, it is worth noting that the
right helpline also appears in the answer generated.
Moreover, the chatbot is completely hosted on the local
machine, there is no issue of data privacy, but this can hinder
the usage of the chatbot by multiple users. These issues must
be considered when factoring in the further enhancements that
could be applied to the model. Fig. 6 Empathetic response with advice

Fig. 7 Correctly identifying harassment.

Fig. 3 Parsing the documents.

Fig. 8 Wrong helpline retrieved along with the correct one.

Fig. 4 Unsure answer

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on November 29,2024 at 14:28:54 UTC from IEEE Xplore. Restrictions apply.

198
 V. CONCLUSION [10] Rathnayaka, Prabod, et al. "A mental health chatbot with
cognitive skills for personalised behavioural activation
In conclusion, this study clearly demonstrates the potential and remote health monitoring." Sensors 22.10 (2022):
of Large Language Models (LLMs) in enhancing the 3653.
capabilities of chatbots, particularly in sensitive contexts such [11] “rAInbow: Chatbot to Support Victims of Domestic
as supporting advisors for victims of sexual harassment. The Abuse.” World Justice Challenge 2021. rAInbow:
LLM-powered chatbot – specifically the Llama-2 model - can Chatbot to Support Victims of Domestic Abuse | World
Justice Project [Accessed: Feb. 18, 2024].
generate accurate, empathetic, and non-judgmental responses,
[12] Socatiyanurak, Vorada, et al. "Law-u: Legal guidance
fostering a supportive and safe environment for victims. The through artificial intelligence chatbot for sexual violence
over 95% accuracy achieved, shows the efficacy of this victims and survivors." IEEE Access 9 (2021): 131440-
approach. These findings pave the way for future research and 131461.
application of artificial intelligence in domains that require a [13] Maeng, Wookjae, and Joonhwan Lee. "Designing a
more human-centric but non-judgemental entity. chatbot for survivors of sexual violence: Exploratory
study for hybrid approach combining rule-based chatbot
The study has great scope for further enhancement of the and ml-based chatbot." Asian CHI Symposium 2021.
chatbot’s capabilities. The integration of live web-scraping to 2021.
retrieve counsellors’ contact information from official and [14] “Llama 2: Open Foundation and Fine-Tuned Chat
trustworthy websites could provide victims with immediate Models,” HuggingFace, meta-llama/Llama-2-7b ·
Hugging Face [Accessed: Feb. 18, 2024].
and personalized assistance. In addition to this, the transition
[15] "pdfplumber." PyPI, version 0.10.4, released on Feb 10,
from a local system to an online platform or cloud-based 2024. https://pypi.org/project/pdfplumber/ . [Accessed:
storage would make the chatbot more accessible and scalable. Feb. 18, 2024].
Moreover, optimizing the response time and including a [16] Hugging Face. "sentence-transformers/msmarco-bert-
translator for all Indian languages would significantly base-dot-v5." Hugging Face, 2024.
improve the user experience, making the chatbot more https://huggingface.co/sentence-transformers/msmarco-
efficient and user-friendly. These advancements could bert-base-dot-v5 . [Accessed: Feb. 18, 2024].
revolutionize the way support is provided to victims of sexual [17] Hugging Face. "The AI community building the future."
harassment, making it more personalized, and accessible.. Hugging Face, 2024. https://huggingface.co/ .
[Accessed: Feb. 18, 2024].
REFERENCES [18] "LangChain." LangChain, 2024.
https://www.langchain.com/ . [Accessed: Feb. 18, 2024].
[1] World Population Review. “Rape Statistics by Country.” [19] O. Banerji, “Domestic violence helpline numbers,
https://worldpopulationreview.com/country- counselling and how to report cases
rankings/rape-statistics-by-country [Accessed: Feb. 18, smoothly,” IPleaders, Mar. 23, 2022.
2024]. https://blog.ipleaders.in/domestic-violence-helpline-
[2] Nasrin Sultana, “India Inc Sees Alarmingly High numbers-counselling-and-how-to-report-cases/ .
Unresolved Sexual Harassment Cases At Workplace,” [Accessed: Feb. 18, 2024].
Forbes India, Oct. 17, 2023 [20] R. Baruah, “The Law Against Sexual
https://www.forbesindia.com/article/take-one-big-story- Harassment,” Legal Services India,
of-the-day/india-inc-sees-alarmingly-high-unresolved- https://www.legalservicesindia.com/article/2545/The-
sexual-harassment-cases-at-workplace/89043/1 Law-Against-Sexual-
[Accessed: Feb. 18, 2024]. Harassment.html#:~:text=The%20Sexual%20Harassme
[3] Karami, Amir, et al. "A systematic literature review of nt%20Against%20Women,as%20well%20as%20intern
sexual harassment studies with text mining." ational%20level [Accessed: Feb. 18, 2024]
Sustainability 13.12 (2021): 6589. [21] National Commission for Women, “Helplines,”
[4] Shechory-Bitton, Mally, and Liza Zvi. "Is it harassment? National Commission for Women,
Perceptions of sexual harassment among lawyers and http://www.ncw.nic.in/helplines [Accessed: Feb. 18,
undergraduate students." Frontiers in psychology 11 2024]
(2020): 1793. [22] S. Goenka, “Sexual Harassment”, Legal Service India,
[5] Følstad, Asbjørn, et al. "Future directions for chatbot https://www.legalserviceindia.com/legal/article-1323-
research: an interdisciplinary research agenda." sexual-harassment.html [Accessed: Feb. 18, 2024]
Computing 103.12 (2021): 2915-2942. [23] “Home | Men Welfare Trust”. Men Welfare Trust,
[6] Yao, Yifan, et al. "A survey on large language model http://www.menwelfare.in/ [Accessed: Feb. 18, 2024]
(llm) security and privacy: The good, the bad, and the [24] Saumya Uniyal and Siddhesh Surve. “Top 11
ugly." arXiv preprint arXiv:2312.02003 (2023). Organizations in India that Help in Cases of Molestation,
[7] Sorin, Vera, et al. "Large language models (llms) and Sexual Abuse & Violence”. TimesNext
empathy-a systematic review." medRxiv (2023): 2023- https://timesnext.com/top-organizations-in-india-that-
08 help-in-cases-of-molestation-sexual-abuse-
[8] Bauer, Tobias, et al. "# MeTooMaastricht: Building a violence/#:~:text=Top%2011%20Organizations%20in
chatbot to assist survivors of sexual harassment." %20India%20for%20Reporting%20Cases,...%208%20
Machine Learning and Knowledge Discovery in 8.%20Azad%20Foundation%20...%20More%20items
Databases: International Workshops of ECML PKDD [Accessed: Feb. 18, 2024]
2019, Würzburg, Germany, September 16–20, 2019, [25] Jahnvimehta. “Sexual Violence against Men in
Proceedings, Part I. Springer International Publishing, India”. Legal Service India,
2020. https://www.legalserviceindia.com/legal/article-4685-
[9] Li, Cheng, et al. "Large language models understand and sexual-violence-against-men-in-india.html [Accessed:
can be enhanced by emotional stimuli." arXiv preprint Feb. 18, 2024]
arXiv:2307.11760 (2023).

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on November 29,2024 at 14:28:54 UTC from IEEE Xplore. Restrictions apply.

199