Pentest Application Web
Pentest Application Web
Pentest Application Web
WEB APPLICATION
PENETRATION
TESTING METHODOLOGY
https://cyberpublicschool.com/
Table of Contents
Introduction: ................................................................................................................................ 1
Security in SDLC:.......................................................................................................................... 2
Penetration Testing Methodology: Case Study ..................................................................... 3
Information Gathering .........................................................................................................................3
Vulnerability Testing............................................................................................................................7
OWASP TOP 10 Vulnerabilities .......................................................................................................... 8
Injection Flaws .................................................................................................................................... 8
Broken Authentication and Session Management ..........................................................................11
Cross site scripting ............................................................................................................................14
Insecure direct object references .....................................................................................................17
Security Misconfiguration ................................................................................................................18
Sensitive data exposure ....................................................................................................................20
Missing Function Level Access Control ...........................................................................................22
Cross--‐Site Request Forgery (CSRF) ..................................................................................................... 23
Using Components with Known Vulnerabilities .............................................................................26
Unvalidated Redirects and Forwards ..............................................................................................27
Risk Ratings & Assessment .............................................................................................................. 31
Reporting ............................................................................................................................................. 33
1
Introduction:
The dramatic rise of web applications in the last decade has revolutionized every
aspect of our lives. Today, web applications drive many businesses across the
globe. It is needless to mention about their presence in social networking,
banking, entertainment, insurance, health, automobiles and many other domains.
We all enter a lot of sensitive personal information into these applications every
day. From the user’s perspective, they take it granted that these applications are
secure. For instance, when a user has his private conversations with his friends in
Facebook, he never wonders if those messages can be seen by a third person.
Users take it granted that Facebook maintains them securely. Similarly, from the
corporate perspective, web applications are crucial since they drive their core
businesses. Any damage to them is going to affect their business directly. In order
to win the user’s trust and to make sure that business is not hit, it is imperative
that these web applications are maintained in a secure manner.
In the late 90’s, the attacks on companies were mostly network based. Therefore,
a hacker had to gain access first to a system in a network and from there had to
grab control of a server within that network, perform his attack and then finally
clear the footprints. But, as network security improved and robust control came
into picture over the years, the attacks slowly moved towards the web application
end. In addition, with the advent of dynamic pages and several other new
technologies, the client increasingly started interacting with server. This opened
the gates of opportunity for hackers who no longer had to take the trouble of
getting access to network, erasing the footprints etc. In web application hacking,
with a well-crafted payload, an attacker can download all the details present in a
database. The recent hacks on Sony, Adobe, JP Morgan Chase, eBay and several
other companies ring alarm bells to prepare for the worst. Hence securing web
applications is the need of the hour and Penetration testing of web applications is
one way of achieving it. In the remaining part of this course, we are going to see
what penetration testing is all about, and the methodology that needs to be in
place to secure web applications effectively.
1
Security in SDLC:
One of the main problems with security of web applications today is the lack of a
coordinated and well-planned program that lasts throughout the SDLC. Security
needs to be incorporated in every phase of SDLC (what is now being called as
‘secure SDLC’) to make sure that the final product is robust and secure. The
section below lists important tasks that need to be done in each phase to ensure
security in all phases.
2
Penetration Testing Methodology: Case Study
Penetration testing is a process that tries to identify the security loopholes
present in the application by actually performing the attack. The pen tester
attacks the application assuming the role of a hacker, identifies vulnerabilities
present in the application, and then reports them to the application owner. There
is no definitive procedure for performing penetration testing on web applications
due to wide range of technologies that are now being used and also due to the
fact that each application is very much different from the other. While
penetration testing is definitely one of the primary testing programs to secure
web applications, it is necessary but not sufficient.
Consider a fictional bank BestBank who has their services available online through
a web application. BestBank now wants to assess the security posture of their
web application and outsources this to a security firm to conduct penetration
testing. We will now analyze various phases involved in conducting penetration
testing on the BestBank application. Penetration testing methodology explained
here lists out below phases:
▪ Information Gathering
▪ Vulnerability Testing
▪ Risk Assessment
▪ Reporting
Information Gathering
Prior to the start of pen test; the tester needs to gather information such as URL,
valid credentials, roles, valid test data from the BestBank company. Information
gathering phase is concerned about gathering as much information about the
application as possible. This includes understanding the server & technology,
application entry points, technologies used, understanding application structure
and so on. Below are some of the steps that can be followed to gather as much
information as possible about the web application:
3
▪ Understand application functionality & structure:
Understanding the functionalities present within the application is
crucial to identify the vulnerabilities associated with them. Make a list of
crucial functionalities that an attacker might be interested in. For
example account balance of a user; transferring funds to third party,
updating account details are all highly sensitive. Identifying these is very
important. Its only when you identify the functionalities you will be able
to figure out applicable threats.
GET /shopping/Product.asp?CustID=100&ITEM=mobile&PRICE=112
Host: example
4
Cookie: SESSIONID=dsfafdjnj324n324234q242qn432j4n23j4nj34jnjn
...
In the above case, note down all the parameters of the request such as
CUSTOMERID, ITEM, PRICE and the Cookie. These are considered entry
points to the application that can be tampered (explained in later
sections).
5
Example:
Example: For instance, if you want to look for those URL’s in a web site
(www.bestbank.com) which contains the word ‘admin’ just type
“inurl:admin site:www.bestbank.com” in Google to see respective pages.
6
paths, underlying web server and the technology, entry and exit points etc. With
this information in hand, we can move to next section, which is Vulnerability
Testing.
Vulnerability Testing
Vulnerability testing is the phase, which involves analysis of the application for
security weaknesses, technical flaws, or vulnerabilities. This would require
penetration testing skill set. The security issues identified in this phase will have
to be drafted and presented to BestBank along with the remediation solutions.
Depending on the application type, there are different areas that need to be
tested. Below is a comprehensive list of areas that are to be tested for a given
web application.
• Authentication Testing
• Authorization Testing
• Cryptography
• Error Handling
There is no defined method to test the application for the above issues in any
particular order. It usually depends on the tester’s approach. While it is beyond
the scope of this document to dig deep into each of these areas, we will
nevertheless look at some of the most common vulnerabilities that are present in
the web applications today. OWASP, an open source project, releases the list of
7
TOP 10 critical web application security flaws, which helps us, understand the
kind of vulnerabilities that are identified in this phase. Below is the list of OWASP
top 10 vulnerabilities.
OWASP TOP 10 Vulnerabilities:
▪ Injection
▪ Broken Authentication and Session Management
▪ Cross-Site Scripting (XSS)
▪ Insecure Direct Object References
▪ Security Misconfiguration
▪ Sensitive Data Exposure
▪ Missing Function Level Access Control
▪ Cross-Site Request Forgery (CSRF)
▪ Using Components with Known Vulnerabilities
▪ Unvalidated Redirects and Forwards
We will go through each of them in detail in the below sections:
Injection Flaws
Injection flaws occur when an application takes in some untrusted data and sends
it to the interpreter. Injection flaws are one of the most common and dangerous
issues found in many web applications. Depending on the underlying interpreter,
there are different injection flaws such as SQL injection, Xpath Injection, LDAP
injection, OS command injection and so on. We will examine more about SQL
injection from here on, as it is the most common vulnerability associated with
web applications.
Example:
1. Consider a BestBank user who tries to view his account balance. When the
user enters the following URL:
http://www.bestbank.com/users/AccountBalance.asp?uid=123
8
2. Therefore, if an attacker tampers the value the interpreter is going to parse
whatever value is supplied by the attacker.
3. Consider the case where an attacker injects something malicious like the
below:
http://www.bestbank.com/users/AccountBalance.asp?uid=123; DROP TABLES-
-
4. The interpreter would execute the above command and drop the tables
because it has no way to know that it is a malicious command. Thus, lack of
separation of “data” from the “code” results in injection vulnerabilities.
How to identify?
The basic issue is to check whether all use of interpreters clearly separates
untrusted data from the command or query. During pen testing this can be done
by sending input to the application and seeing how it responds. Look for the areas
where there is a scope to hit the database. For example to identify presence of
SQL injection vulnerability, inserting a single quote might result in some error due
to syntax issues as shown in the following screenshot:
However, this is not always true because there could be SQL injection even in
cases where the application does not respond with an error message and instead
shows some custom page. This case is called “Blind SQL Injection” which needs to
be identified with true or false conditions.
9
http://www.bestbank.com/users/AccountBalance.asp?uid=123 AND 1=1
Depending on the type of exploitation, the SQL injection flaws can also be
categorized as Union based and error based.
Injection vulnerabilities can have serious impact as they allow an attacker to:
Remediation Suggestion:
● The preferred option is to use a safe API, which avoids the use of the
interpreter entirely or provides a parameterized interface. Be careful with
APIs, such as stored procedures that are parameterized, but can still
introduce injection under the hood.
● If a parameterized API is not available, you should carefully escape special
characters using the specific escape syntax for that interpreter. OWASP’s
ESAPI provides many of these escaping routines.
● Positive or “white list” input validation is also recommended, but is not a
complete defense, as many applications require special characters in their
input. OWASP’s ESAPI has an extensible library of white list input validation
routines.
10
Broken Authentication and Session Management
Example:
2. Now he enters some other user’s ID (say admin) and clicks submit.
3. After this, the attacker just requests some internal page such as
viewprofile.jsp and he logs in as admin.
11
This works because the application wrongly sets the session attribute when forgot
password process is initiated. The attacker takes advantage of this and exploits it
by requesting it in a sequence. This is known as Session Puzzle attack.
How to identify?
Session management flaws are spread across the application. Below are a few
cases that can help you in identifying some of the serious issues:
12
Remediation Suggestion:
13
Cross site scripting
Cross-site scripting (XSS) is one of the most frequently encountered issues in web
application security testing. XSS flaws occur when the application takes in user
input and sends it back to the user without proper validation. In other words, an
attacker will be able to execute scripts in the end user’s browser by sending them
as input to the application. The following example will help you to understand the
issue better.
Example:
Consider a search page on the BestBank site that takes in some user input and
displays it back to the user.
1. For example enter the text “xss” and observe that the text reflects back in
the same page as shown in the below screenshot.
2. Now this value sits in the response between some html tags as shown
below:
14
3. Thus inserting certain scripts will result in their execution as the browser
parses the html normally unaware that it is sent by an attacker.
How to identify?
Before we talk about identifying XSS issues, it’s important to note that there are
three varieties of XSS vulnerabilities:
1. Reflected: The value inserted is reflected in the same page (Ex: Search
page).
2. Stored: The value inserted is stored in the database and is reflected later on
when the user visits the page. (Ex: comments posted by users in forums).
3. DOM: In this case, the value inserted does not hit the server i.e., the source
of the data is in the DOM, the sink is also in the DOM, and the data flow
never leaves the browser.
15
During pen testing look for the locations where user supplied values are reflected
back in the response. This can be done by capturing the traffic using a proxy tool
and observing the response to find the inserted value. The insertion points can be
query parameters, POST parameters etc. while the reflection points can be
anything in the response (including hidden fields). Once the reflection scenario is
identified check if the application validates the input. To verify whether the
application validates input, try sending special characters such as <, > etc. and see
if the application responds in a different way. If not, based on the response
location, frame a payload to test for XSS.
XSS vulnerabilities exploit the trust a user has in a web site. The victim clicks on
the link sent by the attacker because he trusts the domain. By exploiting XSS, an
attacker can
● Input Validation: Validate the input and allow only those characters that are
applicable to the context. While validating do not go with black listing as
hackers always find ways to bypass the filters. Instead, the white listing
approach, which allows only limited known values, is recommended.
● Output Encoding: Output encoding the data will help browser understand
what needs to be treated as html code and what needs to be displayed.
Applications need to ensure that all variable output in a page are encoded
before being returned to the end user. Encoding variable output substitutes
HTML markup with alternate representations called entities. The browser
16
displays the entities but does not run them. For example <script> gets
converted to <script>.
Applications often refer to their resources with actual names when generating the
web pages. Developers need to include authorization checks in the application
before granting access to any of the internal resources. If authorization checks are
not implemented, an attacker can just tamper the parameter values to get access
to unauthorized web pages that may contain sensitive data.
Example:
Consider a case where a user (say Tom) logs into his BestBank account and clicks
on ‘View Account Balance’ link:
http://www.bestbank.com/ ViewBalance.jsp?user=tom
If the application had not implemented proper authorization checks, Tom can
tamper the value of ‘user’ parameter and see the balance of some other user (say
John) by requesting the below URL:
http://www.bestbank.com/ ViewBalance.jsp?user=John
How to identify?
17
Impact of such vulnerabilities can be moderate to critical. It depends on what kind
of data the object reference is pointing to. Exposure of any sensitive personal
information leakage through these vulnerabilities can be severe.
Remediation Suggestion:
● Mapping the values: Instead of directly using the object references, the
application can use identifiers (such as numbers from 1 to 6) per user which
again map to actual database key on the server. In this case, attackers
cannot directly target the unauthorized resources.
● Authorization checks: Ensure that the application has proper access control
or authorization checks in place before grating access to the internal
resources of the application.
Security Misconfiguration
This is related to misconfiguring settings that may belong to platform, web server,
database, application server etc. This also includes missing security patches, use
of default accounts and so on.
Example:
For example, consider a case where directory listing is not disabled on the
BestBank’s server. If an attacker discovers this vulnerability, he can simply list
directories to find any file that is present on the BestBank Server. Attacker can
then download sensitive documents or code file present on the server and then
proceed to find a serious access control flaw in your application. Similarly, an
application using Heart bleed vulnerable version of OpenSSL is an example for this
kind of vulnerability.
How to identify?
18
Pen tester needs to analyze the security hardening posture of the application by
asking below questions:
● Default installations: Does the application allow use of default accounts and
their passwords? For example if the application uses SAP, check for default
usernames and passwords of SAP accounts.
19
Impact can be critical as some of the vulnerabilities can even result in complete
system compromise without the knowledge of the user. For example, heart bleed
vulnerability can result in an attacker getting sensitive information present on the
server’s memory.
Remediation Suggestion:
This is one of the most common vulnerabilities found in most of the websites. This
vulnerability refers to the scenario where sensitive data is not encrypted and is
exposed. Even when the data is encrypted, there could be a chance that the
algorithm is broken or key generation and management is not proper.
Example:
How to identify?
20
● Is sensitive data transmitted in clear text internally or externally? This
would refer to lack of SSL mechanism for the application. In addition, the
end-to-end transmission between various servers should also be secured.
Of course the pen tester may not have the opportunity to verify all these
cases but if there is a chance look for these areas too.
● Is the sensitive data stored on the server in clear text? This would allow
database admins and attackers to grab the information. This may not be
straightforward to identify unless you exploit some vulnerability such as
SQL injection to get details from the database.
● Does the application not use security directives or headers that address
security issues at browser level? For example, lack of proper caching
headers in the response results in the pages being cached in the browser.
This can be verified by looking at the response header captured by the
proxy tool. Any sensitive information present in these pages can later be
accessed by an attacker.
What is the impact?
Since the issue is about sensitive data, the impact is always critical. Nevertheless,
depending on the application and business type it may vary greatly.
Remediation Suggestion:
● Identify sensitive data & secure it: First step is to identify which areas deal
with sensitive data. It could include SSN’s, credit card numbers, and
21
passwords and so on. Next thing is to ensure that this data is encrypted
during transmission and when at rest.
● Use of strong algorithms & keys: Broken algorithms such as MD5 are not
supposed to be used with sensitive information. Only strong algorithms are
to be used when encrypting values related to sensitive data. The key sizes
should also be long enough to ensure that an attacker cannot perform
brute force related attacks.
This is a weakness in the application where access control checks are not properly
implemented. Many a times, function level protection is managed via
configuration, and the system may not be correctly configured. If the developers
do not enforce proper code checks, it could result in missing function level access
control flaw.
Example:
http://bestbank.com/app/employeesInfo.aspx
22
If the BestBank application does not properly implement authorization checks, an
attacker simply browses to target URLs can view the data. Not only for
unauthenticated users, but this also applies to all the users who are not of
manager role. For instance if a normal employee logs in and then accesses this
page successfully it a flaw.
How to identify?
The best way to find out if an application is vulnerable to function level access is
to first try to login with a privileged account and identify sensitive functions, copy
the URLs and then login with a less privileged account and request those URL’s. If
you can access those pages, it means that the access control checks are not
properly implemented.
In short using a proxy, browse your application with a privileged role. Then try to
access these pages using a less privileged role. If the server returns valid
response, it means that the application doesn’t have any access control
mechanism.
Remediation Suggestion:
Ensure that the application has authorization checks implemented at all levels
before granting access to them. These authorization checks can be implemented
as a separate module and invoked every time a request is handled. Please note
that the problem is not solved by removing the link from UI. An attacker has
several ways to know the internal URL’s of privileged accounts.
23
CSRF allows an attacker to perform unwanted actions (such as updating or
deleting data) on a currently logged in user’s account without his knowledge.
CSRF takes advantage of the fact that most web apps allow attackers to predict all
the details of a particular action. Browsers send cookies automatically for
respective domains when a request is passed to them. Attackers take advantage
of this and create malicious web pages that generate forged requests that are
similar to the actual requests.
Example:
Consider a scenario where a BestBank user logs into his bank account to transfer
money to his friend account.
1. As soon as the user (Tom) clicks on ‘Send Money’ button that allows him to
send money (say 1000$) to his friend (Mark), the below request would be
triggered:
http://www.bestbank.com/SendMoney.aspx?From=Tom&To=Mark&Amt=
1000
2. Now an attacker who has account in the BestBank studies this pattern and
now crafts a request that would get triggered if Tom was to send money to
attacker:
http://www.bestbank.com/SendMoney.aspx?From=Tom&To=Attackr&Amt
=20000000
3. The attacker now sends this link to Tom and lures him to click on it using
social engineering techniques.
24
4. If Tom clicks on the above URL when he is signed in, the request would be
processed because cookies automatically go along with the request. Hence,
money would be transferred from Tom’s account to attacker’s account.
How to identify?
Attackers can trick victims into performing any state changing operation the
victim is authorized to perform, e.g., updating account details, making purchases
and so on. Hence, depending on the underlying case the impact needs to be
analyzed.
Remediation Suggestion:
The main reason why CSRF is possible is the fact that attacker is able to construct
the request because he knows its pattern and the values. However, if there is
some unique token whose value is not guessable and which is validated on the
server for every request, then that solves the problem. Therefore, the best
solution is to include the unique token in a hidden field. This causes the value to
be sent in the body of the HTTP request, avoiding its inclusion in the URL, which is
more prone to exposure. Having a CAPTCHA also solves the issue but it might not
be practical to have CAPTCHA in every page of the application.
25
Using Components with Known Vulnerabilities
Example:
There are several examples that fall under this category. For instance, check if
BestBank is vulnerable to Heart bleed attack on OpenSSL cryptography library, a
widely used component in the implementation of the Transport Layer Security
(TLS) protocol. This flaw allows a remote attacker to expose sensitive data,
possibly including user authentication credentials and secret keys, through
incorrect memory handling in the TLS heartbeat extension. This can be checked
using Heartbleed scripts in nmap tool. Alternately there are several online sites
available (such as https://filippo.io/Heartbleed/) which give the results instantly.
Similarly, Apache Struts remote code execution, Apache Cookie disclosure
vulnerability etc. are other examples.
How to identify?
Although it may sound easy to figure out if you are using some vulnerable
component, it is not so in reality. This is because there are no such tools that
point out all the vulnerable libraries or components used in a standard and
searchable way. This is because not all libraries use a proper versioning system. In
addition, there is no central database where we can search for vulnerable
versions of all the components (CVE & NVD are now slowly filling up this gap).
Apart from this, vulnerabilities are reported everyday on several components. So
a pen tester has to keep in touch with these updates and look for them while pen
testing the application.
26
What is the impact?
The impact could vary from low to critical and usually depends on what the
vulnerability is about and how it can be exploited.
Remediation Suggestion:
The best solution is to have a process in place to address the issue of updating to
newer versions. Many a time’s development teams have no idea about using old
and obsolete software and its danger effects. Below are other things that need to
be considered:
● Establish security policies & standards: Have security standards for the
component use, such as requiring clearances from an authorized team for
third party component usage.
Applications frequently have the necessity to redirect the users to different sites.
For this, a target page is specified in a parameter that is generally appended to
the URL. The unvalidated redirect issue arises if this parameter is not validated
allowing the attackers to define the destination site.
Example:
Consider BestBank link that has a page called 'redirect.jsp' which takes a single
parameter named 'url'. As soon as the user clicks on a link the below link would
be triggered:
http://www.bestbank.com/redirect.jsp?url=desination.com
27
If the application does not validate the value coming from the 'url' parameter, an
attacker can craft a malicious URL that redirects users to a malicious site that
performs phishing and installs malware.
http://www.bestbank.com/redirect.jsp?url=evil.com
How to identify?
Try to identify on what basis the application is redirecting the user to another
page. If the destination URL value is sent from the client end, try to tamper the
value and see how the application responds.
Remediation Suggestion:
The destination parameters can be a mapping value, rather than the actual
URL or portion of the URL, and that server side code translate this mapping
to the target URL.
Automated Scanning
28
Automated scanning cannot replace the manual process described above but it
can certainly aid penetration tester to some extent. Below are some of the
advantages and disadvantages of automated scanning process:
Advantages:
● Tools may not identify all the vulnerabilities but they can help you with the
low hanging fruits or the obvious ones. This could in turn help the pen
tester to focus on critical ones while the tool looks for easy and
straightforward issues.
● Tools run fast. In other words, they cover lot of ground in a fraction of time.
While a manual penetration testing might take somewhere between 10-15
days depending on the size of the application, a scan for the same
application takes only a few hours to complete. Hence, if you want to test
lot of application in a limited period of time, automated scanning can help
you.
Disadvantages:
● A tool cannot replace human mind. A tool will not identify logical flaws
present in the application, as it cannot understand the functionality.
● You cannot be sure that the tool has scanned all the pages and for all the
vulnerabilities.
29
● Tools may not work with new technologies and may not identify issues such
as information leakage etc.
With these points in mind, one needs to have mix of both manual and automated
scanning during the security assessments. There are several open source tools as
well as commercial tools available in the market as listed below.
Commercial Tools:
Arachni, Grabber, Zed Attack Proxy, Powerfuzzer, W3AF, Wapiti, Watcher, Zero
Day Scan.
All of these scanners require you to login to the web application by configuring
the tool. The tool then parses through all the pages and constructs a hierarchical
structure. After this the tool sends payloads to test for various vulnerabilities.
Following is a screenshot of IBM Appscan tool reporting various vulnerabilities:
30
Pen tester has to evaluate these by manually verifying them so as to remove the
false positives. Once the false positives are removed, report about remaining
findings can be generated using these tools which we will see under reporting
section.
31
described in the earlier sections. Risk is considered to be product of
likelihood and impact. Hence the next step is to identify both likelihood and
impact of the issue.
32
III. Would this result in non compliance to any state laws?
IV. Would this affect CIA (Confidentiality, integrity, authenticity)?
● Determine severity of the risk: In this step the likelihood estimate and the
impact estimate are put together to calculate an overall severity for this
risk. This can be done by figuring out whether the likelihood is low,
medium, or high and then do the same for impact. OWASP proposes a 0-9
scale rating and overall risk rating based on the below table.
It may not be possible for an external pen tester to adequately assess the actual
business impact caused by successful exploitation of a vulnerability.
Reporting
Penetration testing includes not only technical assessment but also proper and
detailed documentation of the identified findings. Remember that the client may
not be aware of any of the terminologies such as cross site scripting or cross site
request forgery. Hence a detailed report highlighting important issues needs to be
presented at the end of the test. A typical report contains the below sections:
Summary:
This gives an overview of the security posture of the application and sums up the
overall findings of the assessment and provides a high level view of the issues
present in the application. This is intended for people at executive level who may
not be very much aware of the technical issues. Hence the summary needs to be
33
in simple words explaining how the issues present in the application can impact it
and what needs to be in place to improve the overall security.
Test Details:
● Objective: This section outlines the project objectives and the expected
outcome of the assessment.
● Scope: This section outlines the scope of the test.
● Schedule: This section outlines start and end dates of the test.
● Targets: This section lists the number of applications or targeted systems
(BestBank’s application).
● Limitations: This section outlines every limitation which was faced
throughout the assessment. For example, limitations of project-focused
tests, limitation in the security testing methods, performance or technical
issues that the tester come across during the course of assessment, etc.
● Findings Summary: This section outlines the vulnerabilities that were
discovered during testing.
● Remediation Summary: This section outlines the action plan for fixing the
vulnerabilities that were discovered during testing.
Vulnerability details:
34