2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC)

Automatic Detection of NoSQL Injection Using

Supervised Learning
Md Rafid Ul Islam∗ , Md. Saiful Islam† , Zakaria Ahmed‡ , Anindya Iqbal§ , and Rifat Shahriyar¶
Department of Computer Science and Engineering
Bangladesh University of Engineering and Technology
Dhaka, Bangladesh
Email:∗ rfd.009@gmail.com, † saifulislam@cse.buet.ac.bd, ‡ zakaria.buet13@gmail.com,
§ anindya@cse.buet.ac.bd, ¶ rifat@cse.buet.ac.bd,

Abstract—With the advancement in big data, NoSQL SQL injection vulnerability in their servers in 2012. Seals [3]
databases are enjoying ever-growing popularity. The increasing shows that SQL injection attack was used to steal the personal
use of this technology in large applications also brings security details of 156,959 customers from British Telecommunications
concerns to the fore. Historically, SQL injection has been one of
the major security threats over the years. Recent studies reveal company TalkTalk’s servers in 2015. Hacker0 et al. [4] show
that NoSQL databases also have become vulnerable to injections. that it is possible to steal bitcoin with SQLi. Hence, this is
However, NoSQL security is yet to receive the attention it deserves still a major security threat.
from the industry or academia. In this work, we develop a tool Researchers found that NoSQL databases also face the risk
for detecting NoSQL injections using supervised learning. To the of being affected by injection attacks [5]. NoSQL injection
best of our knowledge, our developed training dataset on NoSQL
injection is the first of its kind. We manually design important vulnerability is reported by Diaspora [6] in its social commu-
features and apply various supervised learning algorithms. Our nity framework in 2010. Sullivan [7] demonstrates JavaScript
tool has achieved 0.93 F2 -score as established by 10-fold cross- query based code injection attacks for MongoDB. As NoSQL
validation. We also apply our tool to a NoSQL injection gen- databases are getting more and more popular, vulnerability
erating tool, NoSQLMap and find that our tool outperforms issue is also becoming a major concern. An article published
Sqreen, the only available NoSQL injection detection tool, by
36.25% in terms of detection rate. The proposed technique is in 2015 shows that about 40,000 web apps that use MongoDB
also shown to be database-agnostic achieving similar performance databases are vulnerable to injection attacks2 . According to
with injection on MongoDB and CouchDB databases. OWASP top 10 security ranking of 2017, injection is the
Index Terms—NoSQL, Injection, Database Security, Mon- topmost security threat for applications and NoSQL injections
goDB, CouchDB, Automatic Detection, Machine Learning, Su- are among them3 4 .
pervised Learning
Since NoSQL injection is a relatively new type of threat,
I. I NTRODUCTION there has not been adequate work addressing this problem.
Existing works mostly discuss the types of attacks that are
NoSQL (Not only SQL) is an alternative to traditional SQL.
applicable to NoSQL. For example, Ron et al. [8] and Hou et
NoSQL databases give us the ability to work with large sets of
al. [9] show some ways to generate injection queries. These
distributed data with greater efficiency. Applications requiring
works only discuss the types and severe effects of NoSQL
high performance and scalability can be effectively developed
injection and present some mitigation techniques that can be
using NoSQL databases. NoSQL is able to process a very
applied in the development phase of a system. Based on the
large amount of data and distribute them across computing
literature, these injections can be classified into four types -
clusters faster than SQL databases [1]. Along with providing
PHP array injection, NoSQL OR injection, Javascript based
high scalability and high performance, NoSQL is designed
injection, piggybacked queries.
to deal with large volumes of rapidly changing structured or
Basic protection against injection would be input sanitiza-
unstructured data, a flexible data model for big data, object-
tion. However, it does not save applications from all types of
oriented programming, and the like. Because of these benefits,
injections. Since the injection completes the query string by
these databases are getting popular for large-scale cloud and
balancing the start and end of each string in the query, this type
web applications. Google, Facebook, Adobe, eBay, Cisco, etc.
of injection works even if PHP string sanitization is applied
are using NoSQL databases for their web applications1 .
on the query. For example, OR-injection also works after
SQL injection attack is one of the oldest and most fatal
sanitization since it also balances out the string quotations.
security threats. Even today, many large organizations are
So, to protect applications from the risk of NoSQL injection,
frequently falling prey to SQL injection into their traditional
SQL databases. Keizer [2] mentions that hackers stole more
than 450,000 login credentials from Yahoo by exploiting an 2 https://www.securityweek.com/thousands-mongodb-databases-found-
exposed-internet
3 https://www.owasp.org/index.php/Top 10-2017 Top 10
1 https://www.mongodb.com/who-uses-mongodb 4 https://www.owasp.org/index.php/Top 10-2017 A1-Injection

978-1-7281-2607-4/19/$31.00 ©2019 IEEE 760

DOI 10.1109/COMPSAC.2019.00113
it is important that an automated NoSQL injection detection support vector machine (SVM) [20], k nearest neighbor (IBk)
tool is developed. [21], and XGBoost [22]. We evaluate their performance using
Eassa et al. [10] attempt automatic detection of NoSQL 10-fold cross-validation. Based on the experimental results on
injection using a syntactic parser. Joseph et al. [11] design a MongoDB, we have found that a model trained with a neural
tool using non-deterministic finite automata. They test their network provides the highest mean recall (92.94%) along with
tools against a very small number of injections. Joseph et 91.87% mean accuracy, 93.55% mean precision, and 0.9343
al. only mention two examples in their study which cover mean Fβ (β = 2) score.
only trivial javascript injections. Eassa et al. can only detect For CouchDB dataset, despite dataset size is smaller by
PHP array injections. Diglossia [12] detects injection based 65.12%, Fβ score degrades by 3.85% only on average; which
on processing the part of the query containing user input. This indicates that our approach is database-agnostic and can easily
approach involves converting user inputs to shadow characters be extended to other NoSQL databases as well.
and then detecting injections applying a dual parser which Note that in the context of threat detection, recall measures
is based on the shadow characters. These studies are not the percentage of the vulnerable components correctly pre-
well described and we failed to implement these models. The dicted as such and is widely considered an effective criterion
tools and datasets are also not available. So, the evidence for recommending a model. Fβ (β = 2) score combines the
and outcomes of these studies are not enough to convince us recall and precision measures and imposes lower weight on
that the necessity for developing a reliable automatic NoSQL precision.
injection detection tool is addressed. We also generate a separate test dataset containing injec-
Sqreen [13] provides several publicly available tools for tions only using a NoSQL injection generation tool named
security monitoring and protection monitoring including sup- NoSQLMap5 . The set of queries are independent of our
port for NoSQL injection detection. However, Sqreen severely training dataset and direct output of NoSQLMap without any
fails to detect Javascript based injection and piggybacked kind of manual processing. We find that our tool can detect
queries. Another weakness of this tool is it takes almost 10- 36.25% more injections than Sqreen.
20 seconds to detect an injection. Incorporating this tool may In summary, the specific contributions of our work are:
lead to slowing down the server extremely. So, an acceptable • We generate a dataset of 1354 NoSQL queries (including
automatic detection tool is yet to be designed which is very around 75% benign and 25% injection) and validate the
important for dealing with attacks once a system is already dataset by practically testing on a local server. To the best
deployed. of our knowledge, this is the first labeled dataset of this
Recent success of supervised learning in automatic fraud kind.
and malware detection motivated us to explore this direction. • We design 19 features for classifying benign and injection
Guruswamy [14] explains in his article on Forbes why machine queries. These features seem to be highly effective as our
learning models are better than rule-based systems. While tool performs quite good despite being trained on a small
rule based detection approaches suffer from possible attacks number of samples.
beyond the coverage of the rules, machine learning based • We demonstrate that popular supervised learning models
approaches are likely to train themselves with properties of can effectively solve the NoSQL detection problem.
injections that are not visible while formulating the rules. • We provide a publicly available tool and describe how to
Hence, they can detect new types of injection when attacked. integrate our tool with an existing mechanism. To the
Since no benchmark dataset is available for NoSQL injection best of our knowledge, our tool performs better than
queries, we first generate a dataset of benign and malicious any publicly available tool of this kind. To encourage
MongoDB queries with extensive study of available relevant reproduction, we release the dataset and tool6 .
resources. The literature is far from being enriched. So, we are The remainder of the paper is organized as follows. Section II
able to find a very small dataset to train our model which is introduces NoSQL injection, its threat, and different types. We
not sufficient. We manually generate a large number of benign discuss studies in relevant fields in Section III. In Section IV,
and injection queries. Then we manually augment our dataset we describe our approach towards the design of an automated
by applying cross-overs (combining parts of two queries) and tool for detecting injections. In Section V, we evaluate the
mutations (tweaking one element of a query) over the existing performance of our tool. In Section VI, we present how our
dataset. We validate the generated queries by developing a tool can be integrated with a web application. We show the
simple, vulnerable website which works on top of a MongoDB comparison of our tool with existing ones in Section VII.
database. While replicating the experiments for CouchDB Finally, Section VIII concludes the paper.
database, we follow the same procedure. Finally, Our dataset
contains 1004 MongoDB (including 203 injections) and 350 II. P RELIMINARIES
CouchDB (including 50 injections) queries. We model the In this section, we demonstrate the risks of NoSQL
detection problem as a binary classification (benign query and databases and how NoSQL injection attacks are executed.
injection query) problem and use this dataset to train popular
supervised learning methods i.e., decision tree (ID3) [15], 5 https://github.com/codingo/NoSQLMap

random forest [16], AdaBoost [17], neural network [18], [19], 6 https://github.com/anonymous1363101/nosql-injection-detection

761
Later, we introduce different types of NoSQL injections we TABLE I
PHP A RRAY I NJECTION E XAMPLE
intend to detect.
Database Type Query Injection
A. NoSQL Injection MongoDB db.logins.find({ username: { $ne: 1 }, { $ne: 1 }
NoSQL database is a schema-free database that supports password:{ $ne: 1 } })
CouchDB POST /users/ find HTTP/1.1 Accept: { “$ne”: null }
easy replication, simple API, and high consistency. This type application/json Content-Type: applica-
of database provides higher performance and speed and con- tion/json Host: localhost:5984 { “selec-
tor”: { “username”: { “$ne”: null } }
sumes fewer resources. The most common data models in }
NoSQL databases are column-based, document-based, key-
value mapping-based, graph-based, and multi-model. NoSQL
databases, such as MongoDB, CouchDB, etc, are yet to be “password” => array(“$ne” => null)
robust against security attacks. Malicious users can exploit ));
these security vulnerabilities to execute privilege escalation
attacks to get access to other user accounts of same or higher This query eventually exposes all the entries where username
privilege levels. When NoSQL is first introduced, it is thought and password are not null. Thus an attacker is able to get
to be free of injections, unlike the traditional SQL databases. unauthorized information from MongoDB.
But later the works by Hou et al. [23], Okman et al. [24], and An attacker may also append an additional query with
Ron et al. [8] show that NoSQL databases are also vulnerable the original one by manipulating input. For example, when
to some injections similar to SQL injections. username is G. R. R. Martin the query is,
In 2015, three students of University of Saarland, Germany
showed that about 40,000 MongoDB databases on the internet db.doc.find({ username: ‘G. R. R. Martin’ })
are vulnerable7 . They claimed to be able to get read and write
access to thousands of databases containing sensitive customer Now, if an attacker put G. R. R. Mar-
data from web shops without any special hacking tools. They tin’});db.dropDatabase(); db.insert( {username: ‘dummy’,
reported the existence of many MongoDB web servers that password: ‘dummy as username, the following query will
remain vulnerable to injection attacks. be executed:
OWASP8 and an IBM study [8] have also shown that
NoSQL databases are vulnerable to injection attacks, although db.doc.find({ username: ‘G. R. R. Martin’});
they do not use traditional SQL syntax. db.dropDatabase(); db.insert({username: ‘dummy’,
Consider the following script for a login form where user password: ‘dummy’})
inputs username and password.
MongoDB treats this query as three independent queries
$collection->find(array( instead of one and runs all of them. Here, the second query
“username” => $ GET[‘username’], deletes the database completely which is disastrous.
“password” => $ GET[‘password’] Sqreen [13] shows that it is very easy to attack a MongoDB
)); database using injection and change the content of the database
if no security measure is taken by the developer9 . A Node.js
When a user provides the username and password, it application with JSON data format is also vulnerable if no
sends an http request. For example, if username is admin security mechanism is applied.
and password is 12345678, the corresponding http request is
B. Types of NoSQL Injections
login.php?username=admin&password=12345678 Here, we introduce 4 types of injections applicable to
NoSQL. Although all types of injections are possible for Mon-
The script matches the username and password and returns goDB, we find that only 2 types are applicable to CouchDB.
true if both are correct. Now, an attacker can alter the query 1) PHP Array Injection: The Table I shows a scenario
by passing an array as input like this, where user input in a login form is exploited to execute an
injection attack. PHP array injections inject PHP codes into
login.php?username[$ne]=null&password[$ne]=null an application so that the query conditions are modified. When
the server executes this modified query, the attacker gains
This creates the following MongoDB query, information that is not supposed to be retrieved by the original
query.
$collection->find(array( 2) NoSQL OR Injection: Unlike SQL queries, JSON struc-
“username” => array(“$ne” => null), ture makes ‘OR injections’ hard in MongoDB and CouchDB,
7 https://www.securityweek.com/thousands-mongodb-databases-found-
9 https://blog.sqreen.io/mongodb-will-not-prevent-nosql-injections-in-
exposed-internet
8 https://www.owasp.org/index.php/Testing for NoSQL injection your-node-js-app/

762
 TABLE II TABLE IV
OR I NJECTION E XAMPLE P IGGY- BACKED Q UERY

Database Type Query Injection Query Injection

MongoDB db.doc.find({ username: ‘tolkien’, ’, $or:[ {}, { ‘a’:
$or:[ {}, { ‘a’: ‘a’,password: ‘a’,password: ‘’} ], db.doc.find({ username: ‘G. R. ’}); db.dropDatabase(); db.insert(
‘’} ], $comment: ‘successful $comment: ‘successful R. Martin’}); db.dropDatabase(); {username: ‘dummy ’, password:
MongoDBinjection’}) MongoDBinjection db.insert({username: ‘dummy ’, ‘dummy
CouchDB POST /users/ find HTTP/1.1 Accept: ”, “$or”: [ { password: ‘dummy ’})
application/json Content-Type: applica- ” }, { “password”: { “$ne”:
tion/json Host: localhost:5984 { “selec- “null” } } ]
tor”: { “username”: “vchaulk0”, “$or”:
[ { “password”: “12345” }, { “pass- TABLE V
word”: { “$ne”: “null” } } ] } } SQL INJECTION DAMAGE STATISTICS

Organization Damage Year

TABLE III
Yahoo 450,000 plain text passwords stolen 2012
JAVA S CRIPT I NJECTION
LinkedIn 6.5 million hashed passwords hacked 2012
Query Injection Bitcoin Hacker shows bitcoin can be stolen with SQLi 2016
db.stores.mapReduce (function() { for a); } },function(kv) { Arizona voter database Data of 200,000 voters stolen 2016
(var i = 0; i ¡ this.items.length; i++) return 1; }, { out: ‘x’ }); Qatar National Bank Sensitive financial information leaked 2016
{ emit(this.name, this.items[i].a); } db.injection.insert ({success:1});
Hetzner South Africa Over 40,000 customer details including bank 2017
},function(kv) { return 1; }, { out: return 1;db.stores.mapReduce
accounts leaked
‘x’ }); db.injection.insert ({success:1}); (function() { { emit(1,1
return 1;db.stores.mapReduce (function() {
{ emit(1,1); } }, function(name, sum) { return
Array.sum(sum); }, { out: ‘totals’ });”
in SQL statements and compare them with safe user inputs
to detect injection, this method parses SQL statements and
but still, it is possible to bypass security procedures by creates parse trees, which are fed to a clustering algorithm.
injecting an always true condition (for example, an empty Then the tree edit distance is used to measure the distance
string) using ‘OR’ keyword. among the parse trees. The challenge of this approach is to
In Table II, we find that an empty expression is attached learn to characterize benign SQL statements so that it can
to the input using an OR condition. An empty expression is accurately identify one from an injection attack statement.
always true and consequently, it makes the password check Most of the automatic detection studies use parsing based
ineffective. analysis [26] [27]. In recent years, supervised learning has
3) JavaScript Based Injection: An attacker can forcefully becoming popular for automatic fraud or malware detection
return true or put a condition inside $where operator that with high accuracy. Impression fraud detection by Haider et
always results as true. For example, if there is a condition like al. [28], Android malware detection by Amos et al. [29],
‘’ == ‘’, it will always give true value. As MongoDB allows JavaScript malware detection by Wang et al. [30], malicious
execution of JavaScript codes in order to perform complex web content detection by Hou et al. [31], web application
queries, it is also possible to inject malicious commands by vulnerability prediction by Shar et al. [32], predicting cross-
manipulating JavaScript functions. This type of injection is not site scripting (XSS) security vulnerabilities by Gupta et al.
possible in CouchDB. An example is shown in Table III. [33], and predicting vulnerable software components via text
4) Piggybacked Queries: In MongoDB, attackers can ex- mining by Scandariato et al. [34] are some of the examples.
ploit assumptions in the interpretation of escape sequences and In spite of the growing popularity of NoSQL databases,
special characters (such as termination characters like carriage there are few recent works pointing out its security issues.
return [CR], line feed [LF], closing braces, and semicolons) Leavitt et al. [35] discusses issues such as limitations, advan-
to end a query and insert additional harmful queries like tages, concerns, and doubts regarding NoSQL databases. Ron
db.dropDatabase() to be executed by the database, which et al. [8] lists some possible types of code injection in NoSQL
can lead to disastrous effects like deleting all users from the databases. Swathy Joseph and Jevitha Kp [11] discuss an
database. CouchDB does not have this type of injections. Table automata-based approach to prevent NoSQL injections. Eassa
IV shows an injection of this type. et al. [10] proposes a tool called ‘NoSQL Racket’ which can
detect only ‘PHP array injections’ by comparing code static
III. R ELATED W ORKS code analysis and runtime code analysis. Hong et al. [36]
Database and web security are among the most threatening proposes a parse tree based injection detection mechanism for
areas in information security. Albeit many works have been NoSQL databases. Okman et al. [24] reviews two of the most
performed on SQL injection, it is still one of the major vul- popular NoSQL databases (Cassandra and MongoDB) and out-
nerabilities of a database. Some of the notable SQL injection lines their main security features and problems. Some existing
attack incidents are presented in Table V. works only address injection detection in queries from trusted
SOFIA [25] is a programming-language and source-code client applications. For example, Diglossia [12] converts user
independent tool which also can be used with various attack input to shadow characters and employs a methodology to
generation tools. While most approaches detect user inputs separate user input from the query. Then, based on user input

763
 TABLE VI TABLE VII
S OURCES U SED FOR C OLLECTING N O SQL B ENIGN Q UERIES S OURCES U SED FOR C OLLECTING N O SQL I NJECTION Q UERIES

https://docs.mongodb.com/manual/, https://www.idontplaydarts.com/2010/07/mongodb-is-vulnerable-to-
https://www.tutorialspoint.com/mongodb/, sql-injection-in-php-at-least/,
https://www.journaldev.com/6221/mongodb-findandmodify-example, https://zanon.io/posts/nosql-injection-in-mongodb,
http://php.net/manual/en/mongocollection.findandmodify.php , http://blogs.adobe.com/asset/files/2011/04/NoSQL-
https://specify.io/how-tos/find-documents-in-mongodb-using-the-mongo-shell, But-Even-Less-Security.pdf,
http://no-fucking-idea.com/blog/2012/04/01/using-map-reduce-with-mongodb/ , http://www.syhunt.com/?n=Articles.NoSQLInjection,
http://thejackalofjavascript.com/mapreduce-in-mongodb/ , http://docs.mongodb.org/manual/faq/developers/#how-does-mongodb-address-
http://www.querymongo.com/ , sql-or-query-injection,
https://stackoverflow.com/questions/30435073/mysql-to-mongodb-query- http://php.net/manual/en/mongocollection.find.php,
conversion-issue , http://blog.websecurify.com/2014/08/hacking-nodejs-and-mongodb.html,
https://stackoverflow.com/questions/27915598/how-to-convert-group-by- http://blog.websecurify.com/2014/08/attacks-nodejs-and-mongodb-part-to.html,
having-query-from-mysql-to-mongodb-in-phalcon , https://security.stackexchange.com/questions/83231/mongodb-nosql-
https://stackoverflow.com/questions/42692413/sql-query-convert-to-mongodb , injection-in-python-code,
http://docs.couchdb.org/en/2.1.1/ https://www.infoq.com/articles/nosql-injections-analysis,
https://www.owasp.org/index.php/Testing for NoSQL injection

analysis, they detect injections. Hou et al. [23] also proposes

a strategy to prevent injections, which also depends on the We have generated a labeled dataset for both MongoDB and
proper control of client application. CouchDB. MongoDB dataset contains more samples because
it is being used by many web applications and thus its benign
IV. D ESIGN OF D ETECTION M ODEL
and injection examples more available. Table IX contains the
In this study, we have used feature based supervised learning summary of our dataset.
classifiers to detect injections. To train the classifiers, we
had to generate our own dataset since there is no labeled B. Feature Design
dataset on NoSQL injection available. The methodology for Selecting appropriate features are very important for any
the development of the proposed tool is shown in Figure 1. feature-based supervised learning classifier. We have designed
19 features to start with. Then we have selected the 10 highest
A. Training Dataset Generation ranked features based on information gain and correlation. It
Since the literature is far from being rich, we first generate is to be noted that any individual feature will not dictate the
a dataset of benign and malicious MongoDB queries with decision of detection, rather in combination with other features
extensive study of available relevant resources. So, we are able they yield a weighted outcome of the classifier that determines
to find a very small dataset to train our model which is not our prediction. The designed features and the intuitions behind
sufficient. The benign queries are collected from MongoDB choosing these are briefly explained below.
and CouchDB official sites and some other links (Table VI). • Contains Empty String: A lot of NoSQL injections use
And, the injection data is collected from popular security an empty string to create a condition that evaluates to
sites, blogs, and studying the state of the art works (Table true for most database entries.
VII). Then, we manually generate a large number of benign
and injection queries by augmenting the dataset by applying • Contains Injection Payload: A payload file contains some
cross-overs (combining parts of two queries) and mutations common substrings or signatures of NoSQL injections.
(tweaking one element of a query) over the existing dataset. We have taken the original NoSQL injection payload
We validate the generated queries by developing a simple, vul- file from cr0hn’s GitHub repository10 . Then we have
nerable website which works on top of a MongoDB database. added a few more of injection payloads found on the web.
While replicating the experiments for CouchDB database,
we follow the same procedure. Finally, Our dataset contains • Contains Not Equal: $ne keyword is present in most of
1004 MongoDB (including 203 injections) and 350 CouchDB the PHP array injections.
(including 50 injections) queries.
In our dataset, we have included all 4 types mentioned in • Contains Comparison: find(), find.sort(), $eq, $gt, $gte,
Section II-B, i.e. PHP array injection, NoSQL OR injection, $lt, $lte, $ne, $in, and $nin are used to select the relevant
JavaScript-based injection, and piggybacked queries. We have entries that a user needs. These keywords are found in
tried to create a balanced dataset, but the number of ‘OR most NoSQL injections, as well as benign queries.
injection’ is less than the other injections in our dataset. The
reason is the absence of many variations in ‘OR injection’ like • Contains Logical Operator: $or, $and, $not, and $nor
we have found in other types of injections. In Table VIII, we keywords are found to be used to create always true
present some examples and the distribution of different types
of injections in our training dataset. The details of these four 10 https://github.com/cr0hn/nosqlinjection wordlists/blob/master/
types of injections have already been discussed in Section II. mongodb nosqli.txt

764
 Fig. 1. Conceptual model of our solution strategy

TABLE VIII
M ONGO DB AND C OUCH DB I NJECTION Q UERIES S UMMARY

Injection Type No of Samples Example

PHP array injection 134 db.logins.find({ username: { $ne: 1 }, password: { $ne: 1
} })
OR Injection 20 db.users.find({ username: “dummy”, $or: [ {}, { pass-
word: “” } ]})
JavaScript based Injection 43 db.users.find({username: ‘admin’, $where:
function(){return 1}})//’, password: ‘abcd’})
Piggybacked queries 56 db.users.find({username: ‘’});db.users.drop();
db.users.insert({username:‘rafid’, password: ‘rafid’})

TABLE IX command. But most JavaScript benign queries contain

M ONGO DB AND C OUCH DB DATASET S TATISTICS
return, too. We have found three variants that are most
Database Query Type No of Samples in Dataset of the times considered malicious. These are - ;return,
Benign 801 return true, and return 1.
MongoDB
Injection 203
Benign 300 • New Query: If a new query starts after another where
CouchDB should have been only one query, it is a type of injection
Injection 50
called ‘Piggy Backing’.

• Always True Expression: /./, /.*/ etc indicate ‘any’ in

statements. regular expression. When they are used as injection the
query becomes true for every entry in the database.
• Contains Evaluation Query Operation: Most JavaScript Thus, privacy breach may happen.
and some PHP injections contain $where, $mod, $regex,
and $text command. • Contains Element Query Operations: MongoDB element
query operations e.g., $exisits and $type are strategically
• Presence of ;return, return true, and return 1: Diglossia used in injection attacks.
[12] shows that some injections are possible using return

765
 • Null comparison: Most NoSQL injections contain null. TABLE X
F EATURE R ANKING BY I NFORMATION G AIN AND C ORRELATION
If null is present inside the query, then it may be
comparing something with null which always yields true Rank By Information Gain By Correlation
in NoSQL (e.g. MongoDB) syntax.
1 Contains Comparison Contains Comparison
2 New Query New Query
• Targets Table: createTable() and showTable() commands
can respectively create a new table or show the current 3 Contains Empty String Contains Empty String
table. Both of which can be used as malicious commands 4 Contains Not Equal Contains Not Equal
to create an access point or to get confidential data. 5 Contains Payload Contains Payload
6 Presence of Return Always True Expression
• Alters Collection: createCollection() and drop() 7 Always True Expression Presence of Return
commands affect the database directly. These are 8 Evaluation Query Operation Evaluation Query Function
usually not allowed to be performed through user input
9 Contains Logical Operator Element Query Operation
or input from a Rest API.
10 Element Query Operation Contains Logical Operator
• Drop Database: dropDatabase() command deletes the
entire database and its entries.
A. Evaluation Methodology
• Update query: $update and $save commands can change We design the detection problem as a binary classification
the data entries. (where the two classes are Benign and Injection) using 10
selected features mentioned in Table X. We use supervised
• Remove query: An attacker can use the $remove learning classifiers such as - decision tree based ID3 algo-
command to remove important data from a database. rithm [15], artificial neural network [18], [19] with back-
propagation, random forest [16], AdaBoost [17], k nearest
• Limit keyword: Limit keyword is used to restrict access neighbor (IBk) [21], support vector machines (SVM) [20],
to all data entries. But attackers can exploit it to get and XGBoost [22]. We investigate the performance of the
access to more data than they have access to. classifiers using 10-fold cross-validation. In 10-fold cross-
validation, the dataset is randomly partitioned into 10 equal
• Infinite Loop: while(true) will send the server to execute folds. Then one of the folds is selected as the validation set
an infinite loop which may be used to commit a denial and the remaining 9 folds are selected to train the classifier. We
of service attack. repeat it 10 times to use each of the folds as the validation
set exactly once. The final estimation is the average of the
• Contains ;}//: Diglossia [10] showed that ;}// can be 10 results from the folds. We also test our model with a
used tactically for stronger Javascript attacks. Hence, if separate test dataset where injections are generated using
a query contains ;}//, it has a higher possibility of being the NoSQL injection generation tool named NoSQLMap11
an injection. (both MongoDB and CouchDB). This tool is not used while
generating our original dataset.
Our training dataset is imbalanced. For MongoDB dataset
C. Feature Selection of 1004 queries, the ratio of benign to malignant queries is
3.95 : 1. And, for CouchDB dataset of 350 queries this ratio of
We use WEKA’s ClassifierSubsetEval [37] with
benign to malignant is 6 : 1. Hence, we use oversampling with
J48(decision tree) [38], IBK(k nearest neighbor) [21]
SMOTE (synthetic minority oversampling technique) [39] to
classifiers, and greedy step-wise search with backward
improve the ratio of benign and malignant queries to 1.13 : 1
elimination to select and rank 10 out of initially designed 19
(for MongoDB) and 1.1 : 1 (for CouchDB). We tune SMOTE
features based on information gain and correlation (Table X)
parameters such as SMOTE percentage to 250% and Number
separately. This is done by combining both our dataset of
of Neighbors to 2 for the MongoDB dataset and SMOTE
MongoDB and CouchDB.
percentage to 450% for CouchDB dataset. Table XII shows
We select these 10 features to improve the performance of the performance measures after applying oversampling.
our classifiers and find that reducing feature dimension sig- We experiment with the 7 classifiers tuning their hyper-
nificantly improves our model in terms of accuracy, precision, parameters to obtain better trained models. Based on the
recall, and Fβ (β = 2) score. consistency of performance metrics on training and validation
sets, we can claim that our model is not overfitted. The
V. E VALUATION A ND R ESULTS parameter values given in Table XI are found to be optimal
In this section, we discuss the evaluation methods and for each classifier.
present the performance measures of different classifiers.
11 https://github.com/codingo/NoSQLMap

766
 TABLE XI TABLE XII
PARAMETERS U SED FOR S EVEN C LASSIFIERS P ERFORMANCE M EASURES OF 10- FOLD C ROSS -VALIDATION OF
D IFFERENT C LASSIFIERS
Classifier Parameter Name Value
Dataset Classifier Accuracy Precision Recall F2 Score
Decision Tree(ID3) No Parameters Null
Decision Tree 91.6642% 93.4370% 92.6929% 0.932872
Size Per Bag 100 (ID3)
Random Forest Number of Iterations 200 Random Forest 91.8772% 93.5465% 92.9375% 0.932460
MongoDB AdaBoost (boost- 91.7880% 93.4722% 92.8735% 0.933518
Number of Trees 200
ing with J48)
Classifier Used J48
Neural Network 91.8772% 93.5537% 92.9392% 0.934302
Number of Iterations 1000 SVM 89.4552% 91.0479% 91.5189% 0.91
AdaBoost
Use Resampling True k Nearest Neigh- 91.6196% 93.3030% 92.7668% 0.931952
Percentage of Weight Mass 100 bor
to base XGBoost 89.5101% 90.79104% 87.9179% 0.884429
Learning Rate 0.05 Decision Tree 88.3333% 90.7801% 85.3333% 0.896358
(ID3)
Maximum Epochs 2000 Random Forest 88.5666% 90.8256% 85.8% 0.897641
Neural Network CouchDB
Number of Hidden Layers 4 AdaBoost (boost- 88.6333% 90.8386% 85.9333% 0.898132
Number of Nodes in Hidden 10, 10, 6, 10 ing with J48)
Layer Neural Network 88.6667% 90.8451% 86.00% 0.898328
Type of SVM C-SVM, C = 1 SVM 85.2% 84.6685% 85.9667% 0.849250
2 k Nearest Neigh- 88.6667% 90.8451% 86.0% .898328
SVM Kernel Function e−γ|u−v|
bor
Class Weights {1, 1} XGBoost 85.36% 85.00% 84.06% .842463
k Nearest Neighbor Number of Neighbors 5
Maximum Depth 2 TABLE XIII
XGBoost
Objective Function binary logistic C ONFIDENCE I NTERVAL OF C LASSIFICATION E RRORS OF T HE
C LASSIFIERS

Dataset Classifier Confidence Interval of

We ensure a wide coverage of different types of classifiers Classification Error
that are commonly used for similar problems.
MongoDB Neural Network [7.9974%, 8.0501%]
B. Results CouchDB Neural Network [11.2297%, 11.4368%]
The results of our selected feature set (Table X) with respect
to evaluation measures such as accuracy, precision, recall, and
Fβ score for 7 classifiers are given in Table XII. These results In (1), error implies the classification error, const is the
are obtained using the oversampled dataset. Before applying constant value that defines the likelihood and N is the number
oversampling, the precision has been 89.06% (for MongoDB), of observations used to evaluate the model. The constant value
94.68% (for CouchDB) and recall has been 76.65% (for we have used is 1.96 for 95% likelihood.
MongoDB) and 64.4% (for CouchDB), respectively. After Table XIII shows the result of the confidence interval of
oversampling, we see that recall has increased for both datasets classification error of neural network for 95% likelihood. For
with only a slight decrease in precision in CouchDB. neural network classifier on MongoDB dataset, the value of
We observe from Table XII that even though precision confidence interval implies that there is a 95% likelihood that
is higher in other classifiers, recall and F2 score is higher the true classification error of the model is in the interval
in neural network. As even one injection query execution [7.9974%, 8.0501%] for unseen data.
can compromise the whole system, we think recall is the VI. D EPLOYMENT S TRATEGY
most important measure in this case. That is why we choose
Fβ (β = 2) score with less weight on precision to be our Our proposed tool will work as a server plugin. It works
deciding performance measure. With the highest F2 score, in two steps. First, a listener listens to the port where the
neural network is selected as our representative model for server communicates with the NoSQL database and forwards
detecting NoSQL injections for both MongoDB and CouchDB. it to the port our tool (classifier) listens on. For example, if
We also calculate the confidence interval of classification the server code sends a query to MongoDB on port 100 and
error for neural network from our 10-fold cross-validation our tool listens on port 101, our listener which intercepts the
results. We use the Wilson Score Interval [40] to calculate data coming through port 100 will forward the query to port
the confidence interval denoted by I to assess the reliability 101. Next, our tool filters and sends only the benign query
of the result. back to MongoDB’s port 100. Hence, every query would be
filtered through our tool before running in a NoSQL database

error × (1 − error) management system. The conceptual model of the proposed
I = error ± const × (1) system is presented in Figure 2.
N

767
 our tool is platform independent. Sqreen supports only one
NoSQL database, i.e., MongoDB. On the contrary, our scheme
can be extended to work with other NoSQL databases with
minor adjustments as we have already demonstrated it with
CouchDB.
From a careful study, we have found that Sqreen can
detect only a few types of injections and fails against some
important ones. It can detect some PHP array injections and
OR injections, however, is helpless against JavaScript-based
injections(Table - III) and piggy-backed injections(Table - IV).
Our scheme can detect all of these with moderate accuracy,
as the designed supervised learning approach does not rely on
any particular syntax structure.
We also have found some studies on detecting NoSQL
Fig. 2. Conceptual model of our proposed system injection attacks. The works of Eassa et al. [10] and Joseph
et al. [11] are the most relevant ones. However, they have
not released their tools and hence we could not compare with
20 those.
Number of injections detected(Out of 20)

20
18 18 VIII. C ONCLUSION AND F UTURE W ORKS
16 Despite having significant security risks, prevention of
15 NoSQL injection is not getting the attention it deserves. In
this work, we propose an automated system to detect any
13
12
type of query which may lead to NoSQL injection attack and
demonstrate the performance of the system for MongoDB and
10 CouchDB databases. Our major contribution is the generation
10
of the labeled dataset containing around 1350 NoSQL queries.
8 We explore multiple machine learning methods with careful
tuning of hyper-parameters for classification and present the
performance of these methods in terms of accuracy, precision,
account populatedb userdata orderdata recall, and Fβ score. We can claim that our system can detect
most of the injections based on extensive experiments. As
Our tool Sqreen
detecting an injection is our major concern, we recommend
neural network as the most effective method as it provides the
Fig. 3. Performance comparison between Sqreen and our tool
highest recall and F2 score. We also compare our study with
the only available tool, Sqreen and observe that our system
VII. C OMPARATIVE S TUDY significantly outperforms it.
We have used an independent third-party tool, To the best of our knowledge, this is the first work to
NoSQLMap12 to generate injections for testing our tool propose a methodology based on supervised learning which
against injection attacks outside our original dataset. can detect NoSQL injection attacks. Our tool provides high
NoSQLMap has three sample databases such as shops, accuracy and enables server injection vulnerability testing by
customers, and appUserData. This tool also provides four professionals without disclosing the confidential application
vulnerable web applications such as account, populatedb, code of an enterprise. However, the automatic design of
userdata, orderdata. We have generated 20 injections through features from relevant literature can be an interesting research
NoSQLMap for each of these four web applications and direction, but it is quite impossible as the relevant corpus is
tested the generated injections for Sqreen [13] and our tool. very small. A larger dataset is also more likely to improve the
Sqreen has detected 13, 10, 12, and 8 injections for each performance. We leave these issues as possible future works.
test set and our tool has detected 20, 18, 18, and 16 injections,
ACKNOWLEDGMENT
respectively. The performance comparison between the two
tools is shown in Figure 3. The detection rate of our method We want to express our gratitude to the authors of ‘SOFIA:
is 36.25% higher on average than Sqreen. an automated security oracle for black-box testing of SQL
Sqreen’s NoSQL injection detection support is only avail- injection vulnerabilities’ for providing their dataset on benign
able for Ruby and Node.js based servers. On the contrary, SQL queries. We also want to thank Vladimir de Turckheim
and other members of Sqreen team for providing us valuable
12 https://github.com/codingo/NoSQLMap instructions on using their tool and other information. And

768
last but not least, we thank Samsung for supporting us by [25] D. A. L. C. B. Mariano Ceccato, Cu D. Nguyen, “Sofia: an automated
security oracle for black-box testing of sql-injection vulnerabilities,” in
their research grant ”Code Review Usability Measurement”. ASE 2016 Proceedings of the 31st IEEE/ACM International Conference
on Automated Software Engineering. ACM New York, NY, USA, 2016,
pp. 167–177.
R EFERENCES [26] C. Cowan, C. Pu, D. Maier, J. Walpole, P. Bakke, S. Beattie, A. Grier,
P. Wagle, Q. Zhang, and H. Hinton, “Stackguard: automatic adaptive
[1] MongoDB, “Nosql databases explained,” detection and prevention of buffer-overflow attacks.” in USENIX Security
https://www.mongodb.com/nosql-explained. Symposium, vol. 98. San Antonio, TX, 1998, pp. 63–78.
[2] G. Keizer, “Yahoo fixes password-pilfering bug, explains who’s at risk,” [27] J. Newsome and D. Song, “Dynamic taint analysis for automatic
2012. detection, analysis, and signature generation of exploits on commodity
[3] T. Seals, “Sql injection possible vector for talktalk breach,” Infosecurity software,” 2005.
Magazine, 10 2015, accessed: December 02, 2017. [28] C. M. R. Haider, A. Iqbal, A. H. Rahman, and M. S. Rahman, “An
[4] e. . hacker0 (25), oumar (57), “How i hacked hundreds of bitcoins! ama,” ensemble learning based approach for impression fraud detection in
Steemit, 8 2016, accessed: December 02, 2017. mobile advertising,” Journal of Network and Computer Applications,
[5] “Infoq,” https://www.infoq.com/articles/nosql-injections-analysis, 1 2018.
2017. [29] B. Amos, H. Turner, and J. White, “Applying machine learning classi-
[6] “Security lessons learned from the diaspora launch,” fiers to dynamic android malware detection at scale,” in 2013 9th Inter-
http://www.kalzumeus.com/2010/09/22/security-lessons-learned-from- national Wireless Communications and Mobile Computing Conference
the-diaspora-launch. (IWCMC), 2013, pp. 1666–1671.
[7] B. Sullivan, “Server-side javascript injection,” Senior Security Re- [30] J. Wang, Y. Xue, Y. Liu, and T. H. Tan, “Jsdc: A hybrid approach for
searcher, Adobe Secure Software Engineering Team, 6 2011. javascript malware detection and classification,” in Proceedings of the
[8] A. Ron, A. Shulman-Peleg, and E. Bronshtein, “No sql, no injection? 10th ACM Symposium on Information, Computer and Communications
examining nosql security,” arXiv preprint arXiv:1506.04082, 2015. Security. ACM, 2015, pp. 109–120.
[31] Y.-T. Hou, Y. Chang, T. Chen, C.-S. Laih, and C.-M. Chen, “Malicious
[9] B. Hou, Y. Shi, K. Qian, and L. Tao, “Towards analyzing mongodb nosql
web content detection by machine learning,” Expert Systems with
security and designing injection defense solution,” in Big Data Security
Applications, vol. 37, no. 1, pp. 55 – 60, 2010.
on Cloud (BigDataSecurity), IEEE International Conference on High
[32] L. K. Shar, L. C. Briand, and H. B. K. Tan, “Web application vulnera-
Performance and Smart Computing (HPSC), and IEEE International
bility prediction using hybrid program analysis and machine learning,”
Conference on Intelligent Data and Security (IDS), 2017 IEEE 3rd
IEEE Transactions on Dependable and Secure Computing, vol. 12, no. 6,
International Conference on. IEEE, 2017, pp. 90–95.
pp. 688–707, Nov 2015.
[10] A. M. Eassa, O. H. Al-Tarawneh, H. M. El-Bakry, and A. S. Salama,
[33] M. K. Gupta, M. C. Govil, and G. Singh, “Predicting cross-site scripting
“Nosql racket: A testing tool for detecting nosql injection attacks
(xss) security vulnerabilities in web applications,” in 2015 12th Interna-
in web applications,” International Journal of Advanced Computer
tional Joint Conference on Computer Science and Software Engineering
Science and Applications, vol. 8, no. 11, 2017. [Online]. Available:
(JCSSE), July 2015, pp. 162–167.
http://dx.doi.org/10.14569/IJACSA.2017.081178
[34] R. Scandariato, J. Walden, A. Hovsepyan, and W. Joosen, “Predicting
[11] S. Joseph and K. Jevitha, “An automata based approach for the pre- vulnerable software components via text mining,” IEEE Transactions on
vention of nosql injections,” in International Symposium on Security in Software Engineering, vol. 40, no. 10, pp. 993–1006, Oct 2014.
Computing and Communication. Springer, 2015, pp. 538–546. [35] N. Leavitt, “Will nosql databases live up to their promise?” Computer,
[12] S. Son, K. S. McKinley, and V. Shmatikov, “Diglossia: detecting vol. 43, no. 2, 2010.
code injection attacks with precision and efficiency,” in Proceedings [36] H. Ma, T.-Y. Wu, M. Chen, R. Yang, and J.-S. Pan, “A parse tree-based
of the 2013 ACM SIGSAC conference on Computer & communications nosql injection attacks detection mechanism,” 2017.
security. ACM, 2013, pp. 1181–1192. [37] A. W. Moore and M. S. Lee, “Efficient algorithms for minimizing
[13] Sqreen, “Web application and user protection,” https://www.sqreen.io. cross validation error,” in Eleventh International Conference on Machine
[14] K. Guruswamy, “Data science: Machine learning vs. rules based sys- Learning. Morgan Kaufmann, 1994, pp. 190–198.
tems,” Forbes, Dec 2015. [38] J. Quinlan, C4.5: Programs for Machine Learning. Elsevier Science,
[15] C. Jin, L. De-Lin, and M. Fen-Xiang, “An improved id3 decision tree 2014.
algorithm,” in Computer Science & Education, 2009. ICCSE’09. 4th [39] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “Smote:
International Conference on. IEEE, 2009, pp. 127–130. synthetic minority over-sampling technique,” Journal of artificial intel-
[16] T. K. Ho, “Random decision forests,” in Proceedings of the Third ligence research, vol. 16, pp. 321–357, 2002.
International Conference on. IEEE, 1995. [40] E. B. Wilson, “Probable inference, the law of succession, and statistical
[17] S. R. Freund Y., “A desicion-theoretic generalization of on-line learning inference,” Journal of the American Statistical Association, vol. 22, no.
and an application to boosting,” in Lecture Notes in Computer Science. 158, pp. 209–212, 1927.
Springer, Berlin, Heidelberg, 2005, pp. 23–37.
[18] J. J. Hopfield, “Artificial neural networks,” IEEE Circuits and Devices
Magazine, vol. 4, no. 5, pp. 3–10, 1988.
[19] Y. LeCun, “A theoretical framework for back-propagation,” in Artificial
Neural Networks: concepts and theory, P. Mehra and B. Wah, Eds. Los
Alamitos, CA: IEEE Computer Society Press, 1992.
[20] N. Cristianini and J. Shawe-Taylor, “An introduction to support vector
machines,” 2000.
[21] D. Aha and D. Kibler, “Instance-based learning algorithms,” Machine
Learning, vol. 6, pp. 37–66, 1991.
[22] T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting
system,” CoRR, vol. abs/1603.02754, 2016. [Online]. Available:
http://arxiv.org/abs/1603.02754
[23] B. Hou, K. Qian, L. Li, Y. Shi, L. Tao, and J. Liu, “Mongodb
nosql injection analysis and detection,” in Cyber Security and Cloud
Computing (CSCloud), 2016 IEEE 3rd International Conference on.
IEEE, 2016, pp. 75–78.
[24] L. Okman, N. Gal-Oz, Y. Gonen, E. Gudes, and J. Abramov, “Security
issues in nosql databases,” in Trust, Security and Privacy in Computing
and Communications (TrustCom), 2011 IEEE 10th International Con-
ference on. IEEE, 2011, pp. 541–547.

769