CASE STUDY 1:

Problem Addressed:

The authors (Kanclerz et al., 2020) are tackling the challenge of sentiment analysis,
specifically focusing on the decreasing availability of resources for languages with fewer
users, such as Polish. Due to copyright restrictions and a diminishing user base, the resources
for sentiment analysis in less common languages are becoming scarce. This scarcity hinders
the development and evaluation of effective solutions for sentiment analysis in these
languages. The authors highlight the need for robust datasets to evaluate natural language
processing solutions, emphasizing the challenges posed by the legal restrictions on content
use.

Solution: To address this problem, the authors utilize cross-lingual sentiment analysis
techniques, specifically employing the LASER (Language-Agnostic SEntence
Representations) toolkit. LASER provides a solution by enabling the creation of a universal
semantic space where sentences from different languages are embedded. This allows for the
development of models that perform sentiment analysis across multiple languages, even for
languages with limited available resources.

Detailed Solution Steps:

 LASER Toolkit: The authors leverage the LASER toolkit developed by Facebook
Artificial Intelligence Research (FAIR).
 Language-Agnostic Embeddings: LASER generates language-agnostic sentence
representations by embedding sentences from various languages into a common
semantic space.
 Universal Representation: LASER's approach involves creating a common semantic
space that is independent of the language in which sentences are written. This is
achieved by embedding sentences from different languages into this shared space.
 Bidirectional Long Short-Term Memory (BiLSTM): LASER uses a five-layer
BiLSTM for encoding sentences, producing a 1024-dimensional vector representation
for each sentence.
 Lossless Transformation: LASER's methodology allows for a lossless
transformation of models used for Natural Language Processing tasks in one language
 into models that achieve similar results in the analysis of texts written in different
languages.
 Cross-Lingual Comparison: LASER facilitates the comparison of sentences written
in different languages by assigning them to points in the shared semantic space. The
proximity of points indicates similarity in meaning.
 Efficiency and Universality: LASER's language-agnostic embeddings are efficient
and can be used for various NLP tasks, providing a universal representation of
sentence meanings.

By adopting LASER and its language-agnostic embeddings, the authors overcome the
challenges associated with sentiment analysis in languages with limited resources. This
approach not only allows for the development of effective sentiment analysis models for less
common languages like Polish but also enables the extension of models to analyze texts in
more than 90 languages. The authors demonstrate the effectiveness of this cross-lingual
approach in their evaluation on the PolEmo 1.0 Sentiment Corpus and the MultiEmo-Test 1.0
datasets.

Aspect Details
Datasets Used - Polish Sentiment Corpora (PolEval2017, HateSpeech, PCSN)
- PolEmo 1.0 Sentiment Corpus (Customer reviews from Hotels,
Medicine, School, Products)
Models and Methods - Neural Network Architectures: CNN, BiLSTM
- LASER Toolkit: Language-Agnostic SEntence Representations
Evaluation Metrics - F1-scores for sentiment labels (SP, WP, 0, WN, SN, AMB)
- Micro-AUC, Macro-AUC
Results - LASER+BiLSTM outperformed other models in single-domain
and multi-domain datasets
- Cross-lingual approach (LASER) evaluated using MultiEmo-Test
1.0
- LASER+BiLSTM performed better than LASER+CNN in most
cases
Challenges - Variations in dataset sizes
- Presence of less frequent sentiment labels impacting model
performance
Applications - Cross-lingual sentiment analysis using LASER for effective
 classification in multiple languages

CASE STUDY 2:

Sentence Classification with Dynamic Convolutional Neural Networks (DCNN)

Problem: (Kalchbrenner et al., 2014)

The case study addresses the challenge of sentence classification, particularly focusing on
sentiment analysis and question type classification. The problem involves understanding the
order and relationships of words in a sentence to accurately classify its sentiment or
determine its question type.

Solutions:

The proposed solution is a Dynamic Convolutional Neural Network (DCNN) with a unique
architecture. The DCNN utilizes dynamic k-max pooling, wide convolutions, and folding
layers to capture intricate dependencies and features within sentences. The model is designed
to be sensitive to word order, enabling it to recognize specific n-grams and maintain their
order and relative positions.

Solution Steps:

 Wide Convolution: Apply wide convolutions to capture features at different orders.

 Dynamic k-max Pooling: Use dynamic k-max pooling to subsample and maintain
important features.
 Folding: Implement folding layers to simplify feature dependencies between different
rows.
 Feature Graph: Create an internal feature graph over the input sentences induced by
convolution and pooling layers.
 Training: Train the network using backpropagation, Adagrad optimization, and GPU
parallelism.

Significance of Findings:

The DCNN exhibits high performance in various sentence classification tasks, outperforming
other neural and non-neural models. It demonstrates sensitivity to word order, captures long-
range dependencies, and is applicable to hard-to-parse sentences. The feature graph induced
by the DCNN provides a unique perspective on the relationships between words in a
sentence.

How They Solve:

 Sentiment Analysis: Achieve superior accuracy in sentiment prediction on movie

reviews datasets compared to other models, including Naive Bayes, SVM, and
recursive neural networks.
 Question Type Classification: Attain competitive results in classifying question
types in the TREC questions dataset, even without extensive hand-crafted features.
 Twitter Sentiment Prediction: Showcase the DCNN's effectiveness in sentiment
prediction on Twitter data using distant supervision with emoticon-based labels.

The case study demonstrates that the DCNN, through its innovative architecture, can
effectively capture complex sentence structures and dependencies, making it a powerful tool
for various natural language processing tasks.

CASE STUDY 3:
Problem:

The challenge addressed is the automatic generation of text abstracts, which involves
summarizing lengthy documents or articles to provide concise and coherent summaries.

Solutions:

The literature identifies two main approaches to automatic text abstracts:

I. Extraction-based Approach:

Utilizing Deep Learning Structures: This approach involves employing deep learning
architectures specifically designed for extraction purposes. These structures are trained to
identify and extract key information or sentences from the original text, forming the basis of
the generated abstract.

II. Generation-based Approach:

Generating Sentences through Deep Learning Method Training: In this approach, deep
learning methods are utilized to train models for the generation of coherent and contextually
relevant sentences. The model learns to generate new sentences that capture the essence of
the original text, forming a summary.

Significance of Findings:

 Efficiency and Automation: Automatic text abstract generation is crucial for

efficiently summarizing large volumes of information without manual intervention.
 Adaptability to Deep Learning: The identified approaches leverage the capabilities
of deep learning, showcasing the adaptability and effectiveness of these techniques in
handling complex natural language processing tasks.

How They Solve:

Extraction-based Approach:

 Implement deep learning structures designed for extraction tasks.

 Train models to recognize and extract key sentences or information from the input
text.
 Use the extracted content as the basis for the generated abstract.
Generation-based Approach:

 Utilize deep learning methods for training models in sentence generation.

 Train models to understand contextual relationships and generate coherent sentences
that convey the main ideas of the text.
 Generate text abstracts by allowing the model to create summaries based on the
learned patterns.

These approaches address the need for automated text abstract generation, providing
solutions that leverage the capabilities of deep learning to enhance the efficiency and
accuracy of the summarization process.