Cypher Tronics Inc.  

​www.cyphertronics.com Info@cyphertronics.com

Vendor Security Risk Management (​VSRM​) System 

Test Plan

Module 7 - Assignment 3 

April 17, 2020 

By: Julian Bennett, John Hasinsky, Chirag Shah, Kindra Smith, Garrett Williams

1
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Approvals
Approved By: Signature Date
Michelle Moore April 17, 2020
Chirag Shah, Kindra Smith, April 15, 2020
John Hasinsky, Julian
Bennett, Garrett Williams

Document Control
Name VSRM Software Test Plan
Doc. Ref. No. VSRM-TP-001
Document Status Issued
Date of Issue April 19, 2020

Change History
Doc. Author Date Description / Change
Version
1.0 Chirag Shah, April 15, 2020 This document is the test plan
Kindra Smith, for VSRM System Test Plan
John Hasinsky,
Julian Bennett,
Garrett Williams

Distribution List
Name Role
Michelle Moore Sponsor and Approver
Chirag Shah Contributor, Reviewer & Approver
Kindra Smith Contributor, Reviewer & Approver
Julian Bennett Contributor, Reviewer & Approver
John Hasinsky Contributor, Reviewer & Approver
Garrett Williams Contributor, Reviewer & Approver

2
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

1.0 INTRODUCTION 5

2.0 OBJECTIVES 6
2.1 Purpose 6
2.2 Overview 6
2.3 Summary 7
2.4 Tasks 8

3.0 SCOPE 9
3.1 General 9
3.2 Tactics 10

4.0 TESTING STRATEGY 11

4.1 Mission statement 11
4.2 Test Levels 12
4.3 Testing Life Cycle & Methodology 13
4.4 Bug Defect Life Cycle 14
4.5 Unit Testing 15
4.6 System and Integration Testing 16
4.7 Performance and Stress Testing 17
4.8 User Acceptance Testing (UAT) 18
4.9 Batch Testing 21
4.10 Automated Regression Testing 23
4.11 Beta Testing 28
4.12 Data and Database Integrity Testing 30
4.13 Function Testing 31
4.14 Business Cycle Testing 32
4.14.1 User Interface Testing 32
4.15 Load Testing 33
4.15.1 Performance and Stress Testing 34
4.16 Volume Testing 35
4.16.1 Security and Access Control Testing 36
4.17 Configuration Testing 38
4.18 Test Plan & Reporting Template 40

5.0 REQUIREMENTS FOR TEST 40

5.1 Security Assessment Questionnaire 40
5.2 Prioritize Vendors based on Risk and Tier levels 4​1
5.3 Show approved List of suppliers 41

3
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

5.4 Vendor Assessment Risk Reporting 42

5.5 Software Integration 42
5.6 Effective Vendor Security Assessment Request 42

6.0 HARDWARE REQUIREMENTS 43

7.0 ENVIRONMENT REQUIREMENTS 43

7.1 Physical & Logical Environments 43
7.2 Workstation 44

8.0 TEST SCHEDULE 45

9.0 CONTROL PROCEDURES 45

9.1 Problem Reporting 45
9.2 Change Requests 46

10.0 FEATURES TO BE TESTED 46

10.1 Data and Database Integrity 46
10.2 Functions 46
10.3 User Interface 47
10.4 Performance, Load & Stress 47
10.7 Volume 47
10.8 Security and Access Control 48
10.9 Configuration & Integration 48
10.10 Installation 48

11.0 FEATURES NOT TO BE TESTED 48

12.0 RESOURCES/ROLES & RESPONSIBILITIES 49

13.0 SCHEDULES 51

14.0 SIGNIFICANTLY IMPACTED DEPARTMENTS 52

15.0 SUSPENSION CRITERIA 52

16.0 DEPENDENCIES 53

17.0 RISKS/ASSUMPTIONS 53

18.0 TOOLS 53

19.0 APPROVALS 55

4
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

1.0 INTRODUCTION
Cypher Tronics has designed a very unique approach to help support small, medium and
large organizations with Security Risk Management. Cypher Tronics’ Vendor Security
Risk Management (VSRM) System is uniquely developed with many customization and
feature rich approaches to handle security risk assessments for any size of organization.
VSRM has the ability to comply with many different security and compliance requirements
including but not limited to PCI-DSS, ISO 27001, SOC2, COBIT, FedRAMP and other
financial institution’s requirements. VSRM can be customized as per organization’s needs
and can be installed as on-premise solution or Cloud based solution.

Third-party vendors are extremely important to today’s businesses. They let host
organizations systematize certain business processes they can’t do themselves or are too
costly to do on their own. For instance, third-party vendors can provide payroll services,
HR support, technological services, and do sales for you.

While the third-party vendors can help save time and money and boost efficiency, there
are also risks associated with using third-party vendors. One of the major risks posed by
third-party vendors pertains to cybersecurity. Unfortunately, many businesses
underestimate the cybersecurity risks resulting from third parties.

The security risks resulting from third-party vendors have witnessed a steep rise over the
last couple of years. The rate of data breaches has increased to an unprecedented level.
Almost all industries are now targeted.

Cypher Tronics has developed a very unique and feature rich customizable Vendor
Security Risk Management system that will assess vendors who supply technology related
products or services. VSRM is one of a kind vendor risk management system that can
handle any size of the company with state of the art security, performance and regulatory
compliant data protection methodology.

5
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

2.0 OBJECTIVES

2.1 Purpose
This document describes the plan for testing the architectural prototype of the Vendor
Security Risk Management (VSRM) System. This test plan supports the following
objectives:

● Listed VSRM System requirements by product manager

● List the recommended test requirements (high level).
● Recommend and describe the testing strategies to be employed.
● Identify the required resources and provide an estimate of the test efforts.
● List the deliverable elements of the test activities.

2.2 Overview

A recent survey conducted by Ponemon Institute reveals that 56% of companies have
experienced a 3rd-party breach in 2017, which is an increase of 7% compared to previous
year. Another survey conducted by Deloitte in 2016 was more depressive, reporting that
87% of organizations
have experienced a
disruptive incident with
third-parties in the last
2-3 years. Another
research in 2016,
sourced by Soha
Systems, reports that
63% of all breaches
were related to third
parties. The findings in
these studies confirm
that third-party cyber risk
assessment is a must.
As per Cyber Security
research organization ISC2, 50% organizations have developed Vendor Security
Assessment process but it is all based on manual assessment via Microsoft excel
worksheets. Such processes are not effective and are prone to data leakage risk with

6
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
third party (vendors) products and services providers. The remaining 50% of
organizations are either not performing vendor security assessment due to cost, overhead,
or not finding effective solutions to handle the massive vendor risk management task on
hand. Some organizations who run Vendor Security Assessment solutions complain
about its effectiveness, performance, lack of customization and assessment
questionnaires are not being updated over time. Government agencies are in a situation
where they handle more than 1,000 vendors, at any given time and cost, overhead of
solution management and resource utilization is a nightmare for them.

Cypher Tronics has designed a very unique approach to help support small, medium and
large organizations with Security Risk Management. Cypher Tronics’ Vendor Security
Risk Management (VSRM) System is uniquely developed with many customization and
feature rich approaches to handle security risk assessments for any size of organization.
VSRM has the ability to comply with many different security and compliance requirements
including but not limited to PCI-DSS, ISO 27001, SOC2, COBIT, FedRAMP and other
financial institution’s requirements. VSRM can be customized as per organization’s needs
and can be installed as on-premise solution or Cloud based solution.

Third-party vendors are extremely important to today’s businesses. They let host
organizations systematize certain business processes they can’t do themselves or are too
costly to do on their own. For instance, third-party vendors can provide payroll services,
HR support, technological services, and do sales for you.

2.3 Summary
While the third-party vendors can help save time and money and boost efficiency, there
are also risks associated with using third-party vendors. One of the major risks posed by
third-party vendors pertains to cybersecurity. Unfortunately, many businesses
underestimate the cybersecurity risks resulting from third parties.

The security risks resulting from third-party vendors have witnessed a steep rise over the
last couple of years. The rate of data breaches has increased to an unprecedented level.
Almost all industries are now targeted.

Cypher Tronics has developed a very unique and feature rich customizable Vendor
Security Risk Management system that will assess vendors who supply technology related
products or services. VSRM is one of a kind vendor risk management system that can
handle any size of the company with state of the art security, performance and regulatory
compliant data protection methodology.

7
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

2.4 Tasks
This plan lists all the required tasks that will help make this tool the most stable and
effective. The tool should be able to serve multiple customers 100+ and 10,000+ vendors
at a time which will put a load on the system. The pre-release testing is very important for
this tool and it should be considered as the final outcome before the tool gets deployed
into the production environment. This test plan lists all the activities in the staging/QA
environment in an AWS container which is segregated from the production environment.
After pre-release testing, it will be important for the product management team to continue
to collect feedback from customers where this tool will get deployed and also QA/Testing
team so feedback can be tested post-testing and post product release as well. The
problem reporting will be a very important task which will be covered as feedback gets
collected from customers and the product management team.
No. Task Title Task Description

1 Plan Test ● Identify Requirements for test

● Assess Risk
● Develop Test Strategy
● Identify Test Resources
● Create Schedule
● Generate Test Plan

2 Design Test ● Workload Analysis

● Develop Test Suite
● Identify and Describe Test Cases
● Identify and Structure Test Scripts
● Review and Access Test Coverage

3 Implement ● Setup test environment

Test ● Record or Program Test Scripts
● Develop test stubs and Drivers
● Identify test-specific functionality in the design and
implementation model
● Establish external data sets

4 Execute ● Execute Test scripts

Test ● Evaluate execution of test
● Recover from halted test

8
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

● Verify the results

● Investigate unexpected results
● Log defects

5 Evaluate ● Evaluate test case coverage

Test ● Evaluate code coverage
● Analyze Defects
● Determine if test completion criteria and success criteria have
been achieved
● Create test evaluation report
● Signoff on successful test completion

3.0 SCOPE

3.1 General
An organization of any size can utilize Cyber Tronics’ VSRM system. The VSRM system
can handle 20,000+ users at a time and is a highly scalable and customizable solution.
The typical users of the VSRM system are IT or Security professionals who are tasked to
report Enterprise Risk Management as part of the Governance, Risk and Compliance
function of the security team of an organization. The scope of the VSRM is to provide out
of the box assessment questionnaires that are easily customizable for organizations who
are in business to procure technology hardware systems, services, software subscriptions
and cloud solution providers.

The Goal:

9
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

3.2 Tactics
This Test Plan describes the integration and system tests that will be conducted on the
architectural prototype following integration of the subsystems and components identified
in the Integration Build Plan for the Prototype of Vendor Security Risk Management
System.

It is assumed that unit testing already provided thorough black box testing, extensive
coverage of source code, and testing of all module interfaces.

The purpose of assembling the architectural prototype was to test feasibility and
performance of the selected architecture. It is critical that all system and subsystem
interfaces be tested as well as system performance at this early stage. Testing of system
functionality and features will not be conducted on the prototype.

The interfaces between the following subsystems will be tested:

● Requester submits Vendor Security Assessment request

● Risk Registration System (Database repository)
● Catalog of Vendors and Risk Levels

The external interfaces to the following devices will be tested:

● Local PCs
● Remote PCs
● Vendor Response Time

10
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
● Mobile/Tablet Devices

The most critical performance measures to test are:

● Response time for remote

login (Vendor) to the
Vendor Security
Assessment system.
● Response time to access
the Vendor Data
Repository (Risk
Registration System)
● Response time to access
the Vendors and Risk
level Catalog Subsystem.
● Vendor response time
when system loaded with
10,000+ logged in
Vendors simultaneously.
● Department Requester (Requester who wants to work with vendor) response time
when 100+ simultaneous accesses to the Vendor Security Risk Management
Request submission engine
● Security Staff who assesses vendor’s response and data/reports they have
provided

4.0 TESTING STRATEGY

Cypher Tronics has a unique software development testing strategy. In an agile
environment, where we work in short sprints or iterations, each sprint focuses on only a few
requirements or user stories, so it is natural that documentation may not be as extensive, in
terms of both number and content.

The purpose of the test strategy is to list best practices and some form of structure that the
software development and testing teams can follow.

11
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

4.1 Mission statement

Cypher Tronics has implemented a testing strategy which aligns with our goal to
Constantly Deliver secure and stable Software product that meets Customer’s
Requirements by means of providing fast feedback and defect prevention, rather than
defect detection.

Supported by

● No code may be written for a story until we first define its acceptance criteria/tests
● A story may not be considered complete until all its acceptance tests pass

We follow a unique hybrid testing methodology, our testing methodology has elements
of Agile testing. In this Agile Test Strategy, we would also include a reminder to
everyone about Quality Assurance.

● Cypher Tronics VSRM system development defines QA as a set of activities

intended to ensure that products satisfy customer requirements in a systematic,
reliable fashion.
● In SCRUM (agile) QA is the responsibility of everyone, not only the testers. QA is all
the activities we do to ensure correct quality during the development of new
products.

4.2 Test Levels

12
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

● Testing is continuous: Our Agile development team tests continuously because it is

the only way to ensure continuous progress of the product.
● Continuous feedback: Agile testing provides feedback on an ongoing basis and this
is how your product meets the business needs.
● Tests performed by the whole team: In a traditional software development life cycle,
only the test team is responsible for testing but in agile testing, the developers and
the business analysts also test the application.
● Decrease time of feedback response: The business team is involved in each
iteration in agile testing & continuous feedback shortens the time of feedback
response.
● Simplified & clean code: All the defects which are raised by the agile team are fixed
within the same iteration and it helps in keeping the code clean and simplified.
● Less documentation: Our Agile development teams use a reusable checklist, the
team focuses on the test instead of the incidental details.
● Test Driven: In agile methods, testing is performed at the time of implementation
whereas, in the traditional process, the testing is performed after implementation.

4.3 Testing Life Cycle & Methodology

The Cypher Tronics agile testing life cycle includes the following 5 phases:

13
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
1. Impact assessment
2. Agile Testing Planning
3. Release Readiness
4. Daily Scrums
5. Test Agility Review

Cypher Tronics follows Acceptance Testing Driven Development (ATDD). ATDD

methodology focuses on involving team members with different perspectives such as the
customer, developer, and tester (Three Amigos). Three Amigos meetings are held to
formulate acceptance tests incorporating perspectives of the customer, development, and
testing. The customer is focused on the problem that is to be solved, the development is
focused on how the problem will be solved whereas the testing is focused on what could
go wrong. The acceptance tests are a representation of the user’s point of view and it
describes how the system will function. It also helps to verify that the system functions as it
is supposed to. In some instances acceptance tests are automated.

14
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

4.4 Bug Defect Life Cycle

1. Tester finds the defect

2. Status assigned to defect- New
3. A defect is forwarded to Project Manager for analyze
4. Project Manager decides whether a defect is valid
5. Here the defect is not valid- a status is given "Rejected."
6. So, the project manager assigns a status rejected. If the defect is not rejected then
the next step is to check whether it is in scope. Suppose we have another function-
email functionality for the same application, and you find a problem with that. But it
is not a part of the current release when such defects are assigned as a postponed
or deferred status.
7. Next, the manager verifies whether a similar defect was raised earlier. If yes defect
is assigned a status duplicate.
8. If not, the defect is assigned to the developer who starts fixing the code. During this
stage, the defect is assigned a status in- progress.
9. Once the code is fixed. A defect is assigned a status fixed
10. Next, the tester will re-test the code. In case, the​​Test​​Case​​passes the defect is
closed. If the test cases fail again, the defect is reopened and assigned to the
developer.
11. Consider a situation where during the 1st release of the Vendor Security request a
defect was found in request registration that was fixed and assigned a status

15
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
closed. During the second upgrade release the same defect again re-surfaced. In
such cases, a closed defect will be reopened.

4.5 Unit Testing

Cypher Tronics uses a very unique way of testing software. It gives us advantages over
normal testing methods and capabilities.
The advantages are shown as below:

● Reduces Defects in the Newly

developed features or reduces
bugs when changing the existing
functionality.
● Reduces Cost of Testing as defects
are captured in the very early
phase.
● Improves design and allows better
refactoring of code.
● Unit Tests, when integrated with
build gives the quality of the build
as well.

We use all Three different types of Unit

Testing shown as below:

● Black Box Testing - Using which the user interface, input and output are tested.
● White Box Testing - used to test each one of those functions behaviour is tested.
● Gray Box Testing - Used to execute tests, risks and assessment methods.

❏ In this testing we will test individual functionality of modules.

❏ We will test functions and its output.
❏ The main aim is to check the functioning of individual modules.
❏ Isolate each unit of the system to identify, analyze and fix the defects.
❏ Defects are captured in the very early phase.
❏ Build gives the quality of the build as well.

❏ Build means whatever version is coming for testing, a part of the

application.

16
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Example: ​if a developer is developing a loop for searching functionality of an application

which is a very small unit of the whole code of that application then to verify that the
particular loop is working properly or not is known as unit testing.

Participants:

QA Manager

QA Analyst

Test Manager

Developer

Product Manager

Project Manager

4.6 System and Integration Testing

● System shall install and connect with database without any errors both in
cloud and on-premise
● Configurations and integration points shall work and shall be tested.
○ Integration points with ticketing systems

17
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
○ Integration points with IT-GRC Systems
○ Integration points with Reporting tools like Tableau...

4.7 Performance and Stress Testing

Cypher Tronics performance testing measures response times, transaction rates, and
other time sensitive requirements. The goal of Performance testing of VSRM modules is
to verify and validate the performance requirements have been achieved. This
Performance testing will be executed several times, each using a different "background
load" on the system. The initial test will be performed with a "nominal" load. A second
performance test will run using a peak load.

Additionally, this Performance test will be used to profile and tune a system’s performance
as a function of conditions such as workload or hardware configurations.

NOTE: Transactions below refer to "logical business transactions." These transactions are
defined as specific functions that an end user of the system is expected to perform using
the application, such as add or modify a given contract.

Type Description

Definition Objective Validate System Response time for designated

transactions or business functions under a the following
two conditions:

● normal anticipated volume

● anticipated worse case volume

Participants ● QA Analyst
● QA Manager

Methodolog Technique ● Use Test Scripts developed for Business Model

y Testing (System Testing).
● Modify data files (to increase the number of
transactions) or modify scripts to increase the
number of iterations each transaction occurs.
● Scripts should be run on one machine (best case to
benchmark single user, single transaction) and be
repeated with multiple clients (virtual or actual, see
special considerations below).

18
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Completion ● Single Transaction / single user: Successful

criteria completion of the test scripts without any failures
and within the expected / required time allocation
(per transaction)
● Multiple transactions / multiple users: Successful
completion of the test scripts without any failures
and within acceptable time allocation.

Special ● Comprehensive performance testing includes

Consideratio having a "background" load on the server. There
n are several methods that can be used to perform
this, including:
○ "Drive transactions" directly to the server,
usually in the form of Oracle DBMS calls.
○ Create "virtual" user load to simulate many
(usually several hundred) clients. Remote
Terminal Emulation tools are used to
accomplish this load. This technique can
also be used to load the network with
"traffic."
○ Use multiple physical clients, each running
test scripts to place a load on the system.
● Performance testing should be performed on a
dedicated machine or at a dedicated time. This
permits full control and accurate measurement.
● The databases used for Performance testing should
be either actual size, or scaled equally.

4.8 User Acceptance Testing (UAT)

Type Description

Definition Objective Cypher Tronics UAT test has the primary objective to
make sure that​the client's needs are met with the
developed solution. This means that the developed
software (e.g. web app or mobile app) must be largely
feature-complete.

19
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Following are few points to be considered for making sure

UAT is successful.

Scope:

● Department should be able to submit and view

requests via URL and the department requester
should be able to configure vendor contact emails
and automated email settings.
● Vendor Contact should be able to receive
automated emails and should be able to login to the
VSRM portal to answer questions, save answers
and be able to revisit and eventually submit VSRM
requests all together.
● Vendors should be able to see their score through
the reporting UI.
● Department requesters should be able to see
overall vendor score through reporting UI.
● Security Engineer should be able to get automated
email from the tool once the vendor's contact
completes the VSRM request.
● Security Engineers should be able to see risks with
vendors in graphical and also per control/question.

Participants ● Security Engineer

● QA Analyst
● QA Manager
● Product Manager
● Project Manager
● VSRM Developer

20
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Methodolog Technique ● Requirements based test cases

y
Test cases must cover the business requirements, each
test case should be linked to specific requirements based
on an ID number stated in the project document. Test
cases could be written shortly after the requirement
specification is defined and it is called requirement driven
test cases. The disadvantage of this approach is that if the
requirements contain mistakes then test cases would also
go wrong.

● Business process based test cases

Business process based test cases are written to make

sure that the system that is delivered will work specifically
in supporting the business processes. The test cases must
be able to show that the requirements have been met in a
way that reflects how the organization is going to use the
system.

● User Interface driven test cases

User Interface driven test cases are structured around

forms or screens that need to be completed. Test cases
are based on data entry, interactions via the screen, and
reporting. User Interface driven test cases can be
embedded within business process based test cases
where the business process involves data entry,
interaction or reporting.

Entry criteria ● Business Requirements must be available.

● Application Code should be fully developed
● Unit Testing, Integration Testing & System Testing
should be completed
● No Showstoppers, High, Medium defects in System
Integration Test Phase -
● Only Cosmetic error is acceptable before UAT
● Regression Testing should be completed with no
major defects
● All the reported defects should be fixed and tested
before UAT

21
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

● Traceability matrix for all testing should be

completed
● UAT Environment must be ready
● Sign off mail or communication from System
Testing Team that the system is ready for UAT
execution

Completion Before moving into production, following needs to be

criteria considered:

● Application Code should be fully developed. Unit

Testing, Integration Testing & System Testing
should be completed. No Showstoppers, High,
Medium defects in System Integration Test Phase -
Only Cosmetic error is acceptable before UAT.
● No critical defects open
● Business process works satisfactorily
● UAT Sign off meeting with all stakeholders

Special Following points needs to be considered to make UAT

Consideratio Success:
n
● Prepare UAT plan early in the project life cycle
● Prepare Checklist before the UAT starts
● Conduct Pre-UAT session during System Testing
phase itself
● Set the expectation and define the scope of UAT
clearly
● Test End to End business flow and avoid system
tests
● Test the system or application with real-world
scenarios and data
● Think as an Unknown user to the system
● Perform Usability Testing
● Conduct Feedback session and meeting before
moving to production

22
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

4.9 Batch Testing

Cypher Tronic QA team has developed a batch testing framework that validates an
actively trained version of VSRM tool to measure its prediction accuracy. This batch test
helps us view the accuracy of each intent and entity in our active version, displaying
results with a chart. Goal is to Review the batch test results to take appropriate action to
improve accuracy, such as adding more example utterances to an intent if VSRM app
frequently fails to identify the correct intent or labeling entities within the utterance.
Type Description

Definition Objective QA team will run Batch Testing in Automation which will
be running the whole test set by selecting the Run Test set
from the Execution Grid. In this process the Scripts get
executed one by one by keeping all the remaining scripts
in Waiting mode.

● Normal anticipated volume

● Anticipated worse case volume
● Performance of the system
● Execute sets of scripts or batches.
● Group related tests or dependencies into a single
execution.
● Automate batch distribution to multiple destination
environments.
● Validate requirements with requirement use case
batches.
● Confirm key functionality across each supported
environment configuration.
● Determine product readiness using comprehensive
batch reporting.
● Exercise targeted functionality with feature-specific
batches.
● Regression test more in less time, unmanned, and
against multiple environments.

Participants ● QA Analyst
● QA Manager
● Project Manager

23
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Methodolog Technique Execute sets of scripts, or batches, across multiple

y destination environments. With each targeted batch,
confirm a specific set of functionality, across one or more
environment configurations. Create and execute a library
of batch scripts, targeting portions of the application under
test (AUT), or the entire AUT. Quickly validate batch
results from the comprehensive report, summarizing the
testing success as well as identifying and detailing any
regressions.

Completion ● Single Transaction / single user: Successful

criteria completion of the test scripts without any failures
and within the expected / required time allocation
(per transaction)
● Multiple transactions / multiple users: Successful
completion of the test scripts without any failures
and within acceptable time allocation.

Special ● Run all scripts or groups of scripts..QA should run

Consideratio multiple scripts with Automation Tool. ​​Utilize QTP
n and Winrunner both support batch tests.
● In a sprint each time a new build is given and based
on the build functionality implemented or fixed, our
regression tasks are various. To overcome this use
Test Batch Runner (TBR), thus some builds may
require 10 test scripts to run while others require
only 5 out of these 10 in a sequence manner.

4.10 Automated Regression Testing

Automated Regression testing involves testing the unchanged parts of an app again and
again. It ensures that the previous functionality of an application is working as intended
and the new feature that has been added doesn’t introduce any new bug or error. This is
basically a process of verification.

Regression testing is started as soon as the developer adds a new functionality to VSRM
application or fixes any bug. This is because of the dependency between the newly added
and previous functionality. This testing/verification holds immense importance, especially
when there are continuous changes or improvement in the application.

24
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Type Description

Definition Objective Cypher Tronics Regression Testing is a full selection of

already executed test cases which are re-executed to
ensure existing functionalities work fine.

Objective for this testing is done to make sure that new

code changes should not have side effects on the existing
functionalities. It ensures that the old code still works once
the latest code changes are done.

● Test cases which have frequent defects

● Functionalities which are more visible to the users
● Test cases which verify core features of the product
● Test cases of Functionalities which has undergone
more and recent changes
● All Integration Test Cases
● All Complex Test Cases
● Boundary value test cases
● A sample of Successful test cases
● A sample of Failure test cases

Participants ● QA Analyst
● QA Manager
● Project Manager

25
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Methodolog Technique Our approach is to have test automation that needs the
y least possible maintenance and that builds confidence in
stakeholders that the release is of good quality.
So, we have designed a test strategy for critical business
use cases, that would ensure there is no regression issue
introduced, and, of course, that we could implement fast.
This is our setup process:

● The production database backup is restored twice

● Two parallel test systems are set up
○ One with production-released code
○ One with a current version of the application
under test

This provides two identical setups with code differences in

only one version:
Keeping the two setups the same is critical, as this
ensures that any issue is only from the new changes being
pushed.
Test cases are split, so from the standard process of
“Perform action and verify reaction,” the actions are
performed from one milestone to another for the workflow,
and then reports are compared. This is the key in
identifying unexpected changes.
When a QA Analyst is focused on a feature or change, the
test typically is for the change and ensuring the change is
in place. Regression testing is different in the sense that it
has to verify that ​nothing else​ has changed. This
difference in mindset is reflected in automation scripts. It
makes the feature test scripts unsuitable for finding
regression issues, so we need a different approach to the
problem.
Additionally, the methodology will apply on:

New functionality:​ This is the most common

reason for us to run regression testing. The old and
new code must be fully compatible. When

26
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

developers introduce new code, they don’t fully

concentrate on its compatibility with the existing
code. It is up to regression testing to find possible
issues.

Functionality revision:​ In some cases,

developers revise the existing functionality and
discard or edit some features. In such situations,
regression testing checks whether the feature in
question was removed/edited with no damage to
the rest of the functionality.
Integration. In this case, regression testing will
assure that the software product performs
flawlessly after integration with another product.

Bug fixes:​ Surprisingly, developers’ efforts to

patch the found bugs may generate even more
bugs. Bug fixing requires changing the source
code, which in turn calls for re-testing and
regression testing.

Completion ● Application Code should be fully developed.

criteria ● Full regression testing should be completed.
● Only Cosmetic error is acceptable before UAT.
● No critical defects open
● Business process works satisfactorily
● QA Manager Sign off meeting with all stakeholders

Special Consider:
Consideratio
n ● An appropriate structure to manage the regression
tests, including test suites and test cases
● An appropriate tool and medium to store and
maintain these structured regression tests
● An appropriate UI for QA analyst
● Generic script that drives the required Test Suite
and runs the tests

27
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

● Ability to define a migration path through the

application lifecycle.

AND

● Formulate a policy on a regular basis for regression

testing
● Perform desired action and check expected
response for correctness
● Regression test must not be outdated and must be
correct
● Analyze defect escaping detection during the
process
● Should create a logical batch of test cases instead
of having one large regression test
● Test suites must be designed on the basis of 80/20
principle of management
● Perform regression testing after every successful
compile for smaller projects
● Design regression testing based on the risk factors
across the business
● Identify application area with high risk of failure
● Link regression testing with functional testing
● Re-run successful functional test cases
● Regression testing must be considered as an
integral part of extreme programming method

28
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

4.11 Beta Testing

At Cypher Tronics we believe that running a Beta Test provides a complete view of the
true customer experience of your product. It gives our QA team a perspective that cannot
be achieved through any other testing methodology, and is critical to ensuring the success
of VSRM.

Type Description

Definition Objective Beta Testing is one of the Acceptance Testing types,

which adds value to the product as the end-user (intended
real user) validates the product for functionality, usability,
reliability, and compatibility.

Inputs provided by the end-users helps in enhancing the

quality of the product further and leads to its success. This
also helps in decision making to invest further in the future
products or the same product for improvisation.

Objective is to:

● Complete overview of the true experience gained

by the end users while experiencing product
● Test all the known issues
● Uncover the hidden bugs and gaps in the final
product
● Combination of real platforms by testing on a wide
range of devices, OS, browsers and more.

Participants ● QA Analyst
● QA Manager

29
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Methodolog Technique ● All the components of the Product are ready to start
y this testing.
● Documentation that has to reach the end users
should be kept ready – Setup, Installation, Usage,
Uninstallation should be detailed out and reviewed
for correctness.
● The Product Management team should review if
each and every key functionality is in good working
condition.
● Procedure to collect Bugs, feedback etc should be
identified and reviewed to publish.

Completion ● The product is expected to be at least 90% – 95%

criteria completed (stable enough on any of the platforms,
all features either almost or fully complete).
● No Showstopper bugs in any of the platforms.
● All Major bugs discovered in the Beta Test phase
should be fixed.
● Beta Summary Report.
● Beta Testing Sign Off.

Readiness Checklist before launching it.

Few of them are:

● All the components of the Product are ready

to start this testing.
● Documentation that has to reach the end
users should be kept ready – Setup,
Installation, Usage, Uninstallation should be
detailed out and reviewed for correctness.
● The Product Management team should
review if each and every key functionality is in
good working condition.
● Procedure to collect Bugs, feedback etc
should be identified and reviewed to publish.

30
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Special ● All major and minor issues are closed

Consideratio ● Feedback report should be prepared from public
n ● Delivery of Beta test summary report

4.12 Data and Database Integrity Testing

The databases and the database processes should be tested as separate systems. These
systems should be tested without the applications (as the interface to the data). Additional
research into the DBMS needs to be performed to identify the tools / techniques that may
exist to support the testing identified below.

Type Description

Definition Objective Ensure Database access methods and processes

function properly and without data corruption

Participants ● Database Engineer

● QA Engineer
● QA Analyst
● Project Manager

Methodology Technique ● Invoke each database access method and

process, seeding each with valid and invalid
data (or requests for data).
● Inspect the database to ensure the data has
been populated as intended, all database
events occurred properly, or review the
returned data to ensure that the correct data
was retrieved for correct reasons.

Completion All database access methods and processes function

criteria as designed and without any data corruption.

Special ● Testing may require a Oracle DBMS

Consideration development environment or drivers to enter
or modify data directly in the databases.
● Processes should be invoked manually.

31
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

● Small or minimally sized databases (limited

number of records) should be used to increase
the visibility of any non-acceptable events.

4.13 Function Testing

Function Testing of the application should focus on any target requirements that can be
traced directly to use cases (or business functions), and business rules. The goals of
these tests are to verify proper data acceptance, processing, and retrieval, and the
appropriate implementation of the business rules. This type of testing is based upon black
box techniques, that is, verifying the application (and its internal processes) by interacting
with the application via the GUI and analyzing the output (results). Identified below is an
outline of the testing recommended for each application:

Type Description

Definition Objective Ensure proper application navigation, data entry,

processing, and retrieval.

Participants Testing Team ● QA team will work with Product Management

and Product Delivery team to conduct
functions testing
● UI Designer
● Product Manager
● Project Manager

Methodology Technique ● Execute each use case, use case flow, or

function, using valid and invalid data, to verify
the following:
● The expected results occur when valid data is
used.
● The appropriate error / warning messages
are displayed when invalid data is used.
● Each business rule is properly applied.

Completion ● All planned tests have been executed.

criteria ● All identified defects have been addressed.

32
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Special ● Access to the Linux Server (on-premise or

Consideration cloud) and the existing vendors in the
database System is required to run some of
the identified System Tests on the Prototype.

4.14 Business Cycle Testing

4.14.1 User Interface Testing

User Interface testing verifies a user’s interaction with the software. The goal of UI Testing
is to ensure that the User Interface provides the user with the appropriate access and
navigation through the functions of the applications. In addition, UI Testing ensures that
the objects within the UI function as expected and conform to corporate or industry
standards.

Type Description

Definition Objective Verify the following:

● Navigation through the application properly

reflects business functions and
requirements, including window to window,
field to field, and use of access methods
(tab keys, mouse movements, accelerator
keys)
● Window objects and characteristics, such as
menus, size, position, state, and focus
conform to standards.

Participants ● UI Designer
● Product Manager
● Project Manager
● QA Engineer
● Database Engineer

Methodology Technique ● Create / modify tests for each window to

verify proper navigation and object states for
each application window and objects.

33
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Completion ● Each window successfully verified to remain

criteria consistent with benchmark version or within
acceptable standard

Special ● Not all properties for custom and third party

Consideration objects can be accessed.

4.15 Load Testing

Load testing subjects the system-under-test to varying workloads to evaluate the system’s
ability to continue to function properly under these different workloads. The goal of load
testing is to determine and ensure that the system functions properly beyond the expected
maximum workload. Additionally, load testing evaluates the performance characteristics
(response times, transaction rates, and other time sensitive issues).

NOTE: Transactions below refer to "logical business transactions." These transactions are
defined as specific functions that an end user of the system is expected to perform using
the application, such as add or modify a given contract.

Type Description

Definition Objective ● Verify System Response time for

designated transactions or business cases
under varying workload conditions.

Participants ● QA Engineer
● QA Manager
● Product Manager
● Project Manager
● Database Engineer
● IT Engineer

Methodology Technique ● Use tests developed for Business Cycle

Testing.
● Modify data files (to increase the number
of transactions) or the tests to increase the
number of times each transaction occurs.

34
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Completion criteria ● Multiple transactions / multiple users:

Successful completion of the tests without
any failures and within acceptable time
allocation.

Special ● Load testing should be performed on a

Consideration dedicated machine or at a dedicated time.
This permits full control and accurate
measurement.
● The databases used for load testing
should be either actual size, or scaled
equally.

QA engineers will test load and performance of 10,000+ users from Vendor side
(simultaneous login) and 100+ users from internal company department/requesters at a
time.

4.15.1 Performance and Stress Testing

This is a type of performance test implemented and executed to find errors due to low
resources or competition for resources. Low memory or disk space may reveal defects in
the target-of-test that aren't apparent under normal conditions. Other defects might result
from competition for shared resources like database locks or network bandwidth. Stress
testing can also be used to identify the peak workload the target-of-test can handle.

Type Description

Definition Objective Verify that the target-of-test functions properly

and without error under the following stress
conditions:
● little or no memory available on the server
(RAM and DASD)
● maximum actual or physically capable
number of clients connected or simulated
● multiple users performing the same
transactions against the same data or
accounts

35
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

● worst case transaction volume or mix (see

Performance Testing above).

Participants ● QA Engineer
● QA Manager
● Product Manager
● Project Manager
● Database Engineer
● IT Engineer

Methodology Technique ● Use tests developed for Performance

Profiling or Load Testing.
● To test limited resources, tests should be
run on a single machine, and RAM and
DASD on the server should be reduced or
limited.
● For remaining stress tests, multiple clients
should be used, either running the same
tests or complementary tests to produce
the worst-case transaction volume or mix.

Completion criteria ● All planned tests are executed and

specified system limits are reached or
exceeded without the software failing or
conditions under which system failure
occurs is outside of the specified
conditions.

Special ● Stressing the network may require network

Consideration tools to load the network with messages or
packets.
● The DASD used for the system should
temporarily be reduced to restrict the
available space for the database to grow.
● Synchronization of the simultaneous
clients accessing the same records or data
accounts.

4.16 Volume Testing

36
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Type Description

Definition Objective Verify that the target-of-test successfully

functions under the following high volume
scenarios:
● Maximum (actual or physically- capable)
number of clients connected, or simulated,
all performing the same, worst case
(performance) business function for an
extended period.
● Maximum database size has been
reached (actual or scaled) and multiple
queries or report transactions are
executed simultaneously.

Participants ● QA Engineer
● QA Manager
● Product Manager
● Project Manager
● Database Engineer
● IT Engineer

Methodology Technique ● Use tests developed for Performance

Profiling or Load Testing.
● Multiple clients should be used, either
running the same tests or complementary
tests to produce the worst-case
transaction volume or mix (see Stress
Testing above) for an extended period.
● Maximum database size is created
(actual, scaled, or filled with representative
data) and multiple clients used to run
queries and report transactions
simultaneously for extended periods.

Completion criteria ● All planned tests have been executed and

specified system limits are reached or
exceeded without the software or software
failing.

Special ● 1.3 seconds for loading data should be

Consideration considered an acceptable time for high
volume conditions.

37
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

4.16.1 Security and Access Control Testing

Security and Access Control Testing focus on two key areas of security:

● Application security, including access to the Data or Business Functions, and

● System Security, including logging into / remote access to the system.

Application security ensures that, based upon the desired security, users are restricted to
specific functions or are limited in the data that is available to them. For example,
everyone may be permitted to enter data and create new accounts, but only managers can
delete them. If there is security at the data level, testing ensures that user "type" one can
see all customer information, including financial data, however, user two only sees the
demographic data for the same client.

System security ensures that only those users granted access to the system are capable
of accessing the applications and only through the appropriate gateways.

Type Description

Definition Objective ● Function / Data Security: Verify that

users can access only those
functions / data for which their user
type is provided permissions.
● System Security: Verify that only
those users with access to the
system and application(s) are
permitted to access them.

Participants ● QA Engineer
● QA Manager
● Security Engineer
● Project Manager
● Database Engineer
● IT Engineer

Methodology Technique ● Function / Data Security: Identify

and list each user type and the

38
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

functions / data each type has

permissions for.
● Create tests for each user type and
verify permission by creating
transactions specific to each user
type.
● Modify user type and re-run tests
for the same users. In each case
verify those additional functions /
data are correctly available or
denied.
● System Access (see special
considerations below)

Completion ● For each known user type the

criteria appropriate function / data are
available and all transactions
function as expected and run in
prior Application Function tests
● Validate SAML2.0/SSO/MFA
access via tools like Okta, Duo,
OneLogin and more.

Special ● Access to the system must be

Consideratio reviewed / discussed with the
n appropriate network or systems
administrator. This testing may not
be required as it may be a function
of network or systems
administration.

4.17 Configuration Testing

Configuration testing verified operation of the software on different software and hardware
configurations. In our production environment, the particular hardware specifications for
the client workstations, network connections and database servers vary. Client

39
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
workstations may have different software loaded (e.g. applications, drivers, etc.) and at
any one time many different combinations may be active and using different resources.

Type Description

Definition Objective ● Validate and verify that the client

Applications function properly on
the prescribed client workstations
and mobile devices

Participants ● QA Engineer
● Project Manager
● Database Engineer

Methodology Technique ● Use Integration and System Test

scripts
● Open / close various PC
applications, either as part of the
test or prior to the start of the test.
● Execute selected transactions to
simulate user activities into and
out of various PC applications.
● Repeat the above process,
minimizing the available
conventional memory on the
client.
● Repeat all of the above for mobile
and tablet devices

Completion ● For each combination of the

criteria Prototype and PC application,
transactions are successfully
completed without failure.

Special ● What PC Applications are

Consideration available, accessible on the
clients?

40
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

● What applications are typically

used?
● What data are the applications
running (i.e. large spreadsheet
opened in Excel, 100 page
document in Word).
● The entire systems, network
servers, databases, etc. should
also be documented as part of
this test.

4.18 Test Plan & Reporting Template

5.0 REQUIREMENTS FOR TEST

5.1 Security Assessment Questionnaire

Populate latest Information Security Assessment Questionnaire on basis of different
vendors who provide solution, systems or services to US Government

41
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
● Cloud Service Provider
● Hardware Manufacturers and Resellers
● Software Solution providers
○ Off the shelf software
○ Custom software Solution

5.2 Prioritize Vendors based on Risk and Tier levels

Color Tier Risk Risk Description

Descripti Level Severity
on

4 Critical Vendors who are critical to operation, and whose failure or

inability to deliver contracted services could result in US
Government’s systems or services failure but Vendor has
systems and services that are vulnerable to the open
critical risks and vendors who don’t meet US Government
imposed minimum security and compliance requirements.

3 High Vendors (1) who have access to US Government and its

customer’s data and have a high risk of information loss;
and / or (2) upon whom US Government is highly
dependent operationally.

2 Medium Vendors (1) whose access to US Government and its

customer’s information is limited; and / or (2) whose loss
of services would be disruptive to organization.

1 Low Vendors who do not have access to Government and its

customer’s data, systems and environment and whose
loss of services would not be disruptive to Government.

5.3 Show approved List of suppliers

1. Risk basis
● Lowest Risk Vendors

42
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
● Medium Risk Vendors
● High Risk Vendors
● Critical Risk Vendors
2​. Technology basis
● Cloud Service Provider
● SaaS provider
● Off-the shelf software
● Custom Software
3. Security & Compliance basis
● FedRAMP Certified
● SOC2/SSAE16/18 Certified
● PCI-DSS Certified
● ISO 27001/27002 Certified
● COBIT Certified
● CSA aligned
● GDPR/CCPA Compliant
● Security Clearance achieved

4. Show Suppliers who are due for renewals

5. Show when was last security assessment conducted

● What has changed from last assessment

5.4 Vendor Assessment Risk Reporting

● Executive Reporting
● Operational Reporting

5.5 Software Integration

● Most ticketing systems including
Ex: JIRA, ServiceDesk, ServiceNow and more..

● Risk Management Systems such as IT-GRC tools

Ex: Archer, RiskVision, ZenGRC, LockPath, Qualys, SymantecGRC

43
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

5.6 Effective Vendor Security Assessment Request

Requesting an assessment from a vendor should be a simple process, and should not
include a back and forth email conversation. Software should be able to send a request
right within the platform, and the vendor will get a friendly email inviting them to complete
the assessment.

Highlights:

● Automated emails and reporting functionality with Vendors and US

Government internal Information Security, Compliance & Risk Management
staff
● Customized and complete vendor reporting.
● Evaluate vendor performance.
● Archives contracts that outline the rights and responsibilities of all parties.
● Interactive vendor portal – Have your personnel enter vendor data or allow
vendors to login and upload the information themselves.
● Provides evidence of due diligence in selecting a vendor.
● Organizes and rates each vendor objectively.
● Distributes and monitors compliance surveys to vendors.
● Automatically manages tasks like periodic contract reviews and compliance
evaluations.

6.0 HARDWARE REQUIREMENTS

No. Resource Description

1 Server Need a Oracle Database server and install MySQL server Web
server which install Apache Server

2 Test Tool Develop a Test tool which can auto generate the test result to the
predefined form and automated test execution or use existing off
the shelf test tools.

44
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

3 Network Setup a LAN Gigabit and 1 internet line with the speed at least 5
Mb/s or Wi-Fi with Wireless Access Points.

4 Computer At least 4 computer run Windows 10, Linux, Unix and Macbook
Pro, Ram 4GB, CPU 3.4GHZ, HDD 16 GB

5 Cloud AWS EC2 Instance with Oracle DBMS, Linux Server and 32 GB
Environment RAM with 200 GB HDD, Key Management, Encryption and
Firewall/IDS subscribed.

7.0 ENVIRONMENT REQUIREMENTS

7.1 Physical & Logical Environments

No. Resource Need Description

1 Testing Lab Desired Testing lab is required to make sure that test is
conducted in isolated environment and appropriate
hardware and communication channels are available and
pre-installed/active prior to test

2 IT & Desired Different Hardware and OS images should be available

Networks along with Wireless Access Points, Network connectivity
room to and from the test lab should be implemented and
tested prior to test.

3 Security Availabl Each access point should be managed through a point to

e point VPN tunnel along with MFA and an encrypted
channel. Environment should be segregated physically
and logically. Only key personnel who are responsible
for testing should be allowed via key card. CCTV

45
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

cameras should record all activities going in and out of

the testing lab.

4 Software Availabl All required testing software should be made available for
e testers and should be deployed in AWS EC2 staging
instances. Microsoft Project, Office 2020 and Visio
should be pre-installed on endpoint machines so QA,
Product Management and Project Management teams
can perform their duties.

5 Cloud Availabl AWS EC2 Instance with Oracle DBMS, Linux Server and
Environmen e 32 GB RAM with 200 GB HDD, Key Management,
t Encryption and Firewall/IDS subscribed.

7.2 Workstation

Resource Need Description

Workstation Desired Microsoft Windows DELL

with Intel Chipset, Microsoft
Windows HP/Acer with
AMD and Intel Chipset,
variations of RAM (4, 6, 8,
12, 16, 18, 20, 22, 24, 32
GB), variations of HDD
(SSD and normal with 6
different HDD space),
Apple iPad, iPhone,
Android Phone, Apple

46
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

McBook with variations of

RAM and HDD space.

8.0 TEST SCHEDULE

Testing of the VSRM system and Architectural prototype incorporates test activities for
each of the test efforts identified in the previous sections.

Task Effort Start Date End Date

Test Planning 3 days April 7th 2020 April 11th 2020

Test Design 3 days April 11th 2020 April 14th 2020

Test Development 4 days April 14th 2020 April 18th 2020

Test Execution 4 days April 18th 2020 April 22nd 2020

Test Evaluation & 1 day April 22nd 2020 April 23rd 2020
validation

9.0 CONTROL PROCEDURES

9.1 Problem Reporting

All alerts and incidents should be reported and documented in detail via JIRA ticket
system. Reporting template can be utilized (section 4.9) to report the test results. Any bug
should be documented in the JIRA ticket as well. It is important for all testers to utilize
standard operating procedures supplied by the IT team to learn about JIRA ticketing and
documentation.

9.2 Change Requests

Any modification to the software should be documented and approved by product

development, project management and product management team. Each ticket should be
logged in JIRA with appropriate documentation of why the modification is needed, what

47
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
the timelines are, subject matter experts and owners of the development environment.
Document the process of modifications to the software. If the changes will affect the
existing programs, then these modules need to be identified, documented and approved
by JIRA change management ticketing system.

10.0FEATURES TO BE TESTED

The listing below identifies those items (use cases, functional requirements, non-functional
requirements) that have been identified as targets for testing. This list represents ​what​ will
be tested.

10.1 Data and Database Integrity

● Verify access to VSRM Core Database.
● Verify simultaneous record read accesses.
● Verify lockout during VSRM CVSS, Risk and Assessment Questionnaire
updates.
● Verify correct retrieval of update of database data.

10.2 Functions

● The system shall interface with the existing Vendors in Database System.
● The system shall interface with the existing Assessment categories and
shall support the integration with ticketing systems such as JIRA,
ServiceNow, ServiceDesk and Risk Management systems such as
Archer-GRC tool, Zen GRC, Symantec GRC, Qualys GRC, AllGress and
more.
● The server component of the system shall operate in the cloud - AWS and
GCP and shall run under the LINUX Operating System
● If requested, the server component of the system shall operate on the
Co-location or on-premise location as per requirements listed in SoW .
● The client component of the system shall operate on any computer (laptop or
desktop)

48
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

10.3 User Interface

● The desktop user-interface shall be platform/OS agnostic.
● The desktop user-interface shall be compliant with any OS.
● The user interface of the VSRM System shall be designed for ease-of-use
and shall be appropriate for a computer-literate user community with no
additional training on the System.
● The user-interface shall work with any Mobile or Tablet device.

10.4 Performance, Load & Stress

● Verify response time to access external Vendors.
● Verify response time to access external Sales/Marketing subsystems.
● Verify response time for remote login.
● Verify response time for remote submittal of request by department.
● The system shall provide access to the legacy browsers and Database with
no more than a 10 second latency.
● Verify system response when loaded with 100+ logged on department
requesters
● Verify system response when 10,000+ simultaneous end-users/vendors
access VSRM Portal to submit answers.
● Verify system capabilities and handling with lower memory space
● Verify system capabilities and handling with lower HDD space
● Verify system capabilities and handling with lower CPU specs
● Verify system capabilities with shared resources such as database locks and
lower network bandwidth availability.

10.7 Volume
● 10,000 + end users at a time
● 100+ departments submitting tickets/requests at a time

10.8 Security and Access Control

● Verify Login from a local PC.
● Verify Login from a remote PC.

49
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM
● Verify Login from Mobile & Tablet devices.
● Verify Login security through username and password mechanisms.
● Verify SAML2.0/SSO authentication

10.9 Configuration & Integration

● The client component of the system shall run on any endpoint operating
system (Windows, Mac, Linux, Unix) and Mobile/Tablet devices.
● The web-based interface for the VSRM System shall run in all available
browsers (Google Chrome, IE, Firefox, Microsoft Edge, Apple Safari,
Opera…)
● The web-based interface shall be compatible with the latest Java VM runtime
environment.

10.10 Installation
● System shall install and connect with database without any errors both in
cloud and on-premise
● Configurations and integration points shall work and shall be tested.

11.0FEATURES NOT TO BE TESTED

Failover / Recovery Testing

○ No testing required.
○ Reason: ​ ​Failover testing will check the system's ability to provide extra
resources and the ability to move to back-up systems during the system's
failure due to one or the other reasons. This testing will be conducted by the
security & compliance team in partnership with the cloud operations team to
make sure failover happens in time with limited resources and system
availability is successfully completed. Quality Assurance team will work with
Security & Compliance as well as Product Management team to make sure
this gets completed and results are noted as part of testing execution and
delivery of the test report.

50
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

12.0 RESOURCES/ROLES & RESPONSIBILITIES

Role Job Description Responsibilit Minimum

y Requireme
nt

QA Manager Identifies, prioritizes, and implements test Bobby Singh 1

cases

Responsibilities:

● Generate test plan

● Generate Test Suite
● Evaluate effectiveness of test effort

QA Analyst Ensures test environment and assets are James 3

managed and maintained. Mitchell,
Jason
Responsibilities: Blessing,
Karina
● Administer test management system Stewartson
● Install / manage worker access to test
systems

QA Engineer Executes the tests Suzie Aarons, 2

Yuri Yaport
Responsibilities:

● Execute tests
● Log results
● Recover from errors
● Document defects

Product Work side by side with Sales Engineer to Tim Suzockie, 1

Manager understand feedback from prospects and
customers, include that in product design and
testing criterias.

Sales Represent customer and prospect feedback to Catherine 2

Engineer the Product Management team. Zeta Jones,
Angelina Jolie

51
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Installation Verify and validate install and working James 2

Team conditions of the application on multiple OS Kuzock
and devices Simon Says

Developers Work with Product Management and QA team Steve Jobs, 7

to understand defects and performance and Bruce Willis,
continue to enhance the product Arnod Aaron,
Tom Cruise,
Michael
Douglas,
Ranveer
Singh,
Jitendra
Bakshi

Technical Work with the Product Manager and Product Bill Gates 1
writer Development team to design FAQs, Help Files
and Knowledgebase including install and
implementation guidelines.

Configuration Validate configuration and integration Martha 2

Manager requirements, work with Technical writers to Stewart, Jerry
have it documented and perform tests to make Mcguire
sure they are working as stated in original
requirements.

Test Manager Provides management oversight Simon Jones 1

Responsibilities:

● Provide technical direction

● Acquire appropriate resources
● Management reporting

Database Provides technical oversight and support help James Jones 1

Engineer
Responsibilities:

● Provide technical direction

● Troubleshoot and support

52
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

UI Designer Provide creative UI related direction and Samantha 1

support Igor

Responsibilities:

● Provide creative direction

● Troubleshoot and support

Security Provides guidance over security requirements: Jimmy Chu 2

Engineer ● Secure code training and Rocky
● Security Awareness Balboa
● OWASP Top 10 Guidelines
● Application Security
● Mobile Development Security
● Vulnerability Scanning (System, App -
SAST and DAST)
● Penetration Testing

13.0SCHEDULES

Below is outline of the criteria that denote a successful

completion of a test phase

● Run rate is mandatory to be 100% unless a

clear reason is given.
● Pass rate is 80%, achieving the pass rate is
mandatory

Deliverables Members Estimate Timeline

effort

Test Plan QA Manager/Project 25 man hour March 23rd 2020

Manager

Test Cases QA Analyst 30 man hour March 27th 2020

53
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Test Incident Report Tester/QA Analyst 25 man hour April 1st 2020

Test Delivery Report QA Manager 30 man hour April 3rd 2020

Create the test specification Test Designer 170 man hours April 5th 2020

Perform Test Execution Tester, Test Administrator 120 man hours April 10th 2020

Total Total Test Complete 400 man hour April15th 2020

Test Levels & Deliverables:

● Integration Testing (Individual software modules are combined and tested as a

group)
● System Testing: Conducted on a complete, integrated system to evaluate the
system's compliance with its specified requirements
● API testing: Test all the APIs create for the software under tested

14.0SIGNIFICANTLY IMPACTED DEPARTMENTS

Below are departments/teams/groups who are responsible to manage the product and
outcomes throughout the testing life cycle.

Department/Team Name

Quality Assurance

Product Development

Sales & Marketing

Security & Compliance

Product Management

Project Management

IT & Networks

54
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

15.0SUSPENSION CRITERIA

● If the team members report that there are 40% of test cases failed, suspend testing
until the development team fixes all the failed cases.

16.0DEPENDENCIES
● Resource availability and budget allocation
● QA requested tools procurement

17.0RISKS/ASSUMPTIONS
● High risk assumption is budget allocation and not enough testing resources
available due to other testing projects running in parallel.

18.0TOOLS

Tool Name Test Type

Xray Test Management

55
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

Headspin Automated testing

froglogic Automated testing

Lambdatest Cross browsing test

Browsera Cross browsing test

WebLoad Load Testing

Loadrunner Load Testing

Jira Defect Tracking and Ticketing

Espresso Mobile Testing

API Testing SoapUI

NetSparker Security Testing

Acunetix Security Testing/Vulnerability Scanner

Qualys Security Testing/Vulnerability Scanner

W3C CSS Validator Tool

Other Software:

Tool Name

Microsoft Office

Microsoft Project

Microsoft Visio

Adobe Acrobat

No. Resource Description

1 Server ● Need a Oracle Database server and MySQL server.

● Web server with Apache Server

56
Assignment 3: Test Plan ​Cypher Tronics​ - ​VSRM

2 Test Tool Develop a Test tool which can auto generate the test result to the
predefined form and automated test execution or use existing off
the shelf test tools.

3 Network Setup a LAN Gigabit and 1 internet line with the speed at least 5
Mb/s or Wi-Fi with Wireless Access Points

4 Computer At least 4 computer run Windows 10, Linux, Unix and Macbook
Pro, Ram 4GB, CPU 3.4GHZ, HDD 16 GB

5 Cloud AWS EC2 Instance with Oracle DBMS, Linux Server and 32 GB
Environment RAM with 200 GB HDD, Key Management, Encryption and
Firewall/IDS subscribed.

19.0APPROVALS

Team Name Date

PRODUCT TIM SUZOCKIE April 9th 2020

MANAGEMENT

PRODUCT STEVE JOBS, BRUCE 15th April 2020

DEVELOPMENT WILLIS, ARNOLD AARON,
TOM CRUISE, MICHAEL
DOUGLAS, RANVEER
SINGH, JITENDRA BAKSHI

SECURITY & JIMMY CHU, ROCKY 10th April 2020

COMPLIANCE BALBOA

QUALITY ASSURANCE BOBBY SINGH 8th April 2020

CUSTOMER IGOR MITTAL 15th April 2020

ADVOCATE

57