2015 IEEE International Conference on Computational Intelligence & Communication Technology

Testing Framework for 4G Wireless Communication Systems

Sunil Parida Ajay Sharma

Digital Networking Group. Digital Networking Group.
Freescale Semiconductors PVT LTD Freescale Semiconductors PVT LTD
Noida, India Noida, India

Abstract— With the increasing requirement of higher reproduce the issue required to define mitigation
data rates to support various multimedia applications, a steps
demand for an enhanced wireless communication ƒ perform minimal test condition suggested by
To
system has become a necessity. During the development- standard document for a commercial grade
stage of any such wireless communication system it is products
necessary to perform continuous functional and ƒ
Regular track of codebase if any new issue
performance tests for a reliable and efficient end-
product. It is also necessary to develop a cost effective introduced.
and easy to use framework. In addition, to meet the B. Testing Requirments and Understanding
reduced time to market requirement, the infrastructure
required for testing and validation needs to be A 4G LTE/LTE-A product selling point is not
automated in order to speed up the product only based on solutions, feature sets and use cases
development cycle and prevent errors due to manual but also on the different test aspects which are
intervention. In this paper we aim to introduce an covered to label its capabilities. When it comes to
effective and low cost test and validation framework testing it is invariantly true that an end to end system
that can be utilized for a 4G wireless system such as an testing is a crucial aspect in all possible real time
LTE/LTE-A. Considering the performance scenarios. However for solution providers who are
requirements specified in the 3GPP specifications, the primarily interested to test their LTE solution,
framework is enhanced to perform various levels of investing on very expensive equipment’s is defiantly
testing in a fully automated environment and a not always viable solution, also bearing in mind that
provision is included to generate different detailed level that such equipment’s will become obsolete in near
of reports based on the outcomes of the tests and future due to new requirement coming from
validations. This framework also keeps the track of the technology evolution.. To design an efficient testing
previous results and hence product health progress can methodology, it is required to gather and understand
be monitored quickly. the requirement specifications and performance
expectations.
Keywords-LTE, system test design, 3GPP, test
automation
In LTE/LTE-A systems a wide range of testing and
I. INTRODUCTION validation requirements are derived using
specifications provided by 3GPP standard document
A. Importance of Testing
[6-11].LTE/LTE-A systems are built using several
Over the span of a decade, demands for an logical and numerical network elements. The
advanced testing methodology and infrastructure for functional split between the EPC (evolved packet
emerging new telecommunication technologies have core) and E-UTRAN (Evolved universal terrestrial
become a major requirement. Following are the radio access) is shown in Fig 1. In testing and
driving factors for a low cost, easy to use and an validation, the framework focus is restricted to the
automated testing and validation framework PHY (Physical layer) functionality and performance
requirements: requirements. All layer elements such as RLC,
PDCP, RRC, S-GW etc. have very little scope and
ƒ Continues growth in communication system in are considered obsolete.
terms of new feature addition to meet the high
data rate requirements. From Fig1 it can be seen that the PHY is agnostic
ƒ Reliable, efficient and error-free products. to the rest of the layers except for the MAC layer.
ƒ An early detection of real time field issues by The PHY interacts with the MAC layer to obtain the
creating and testing such scenarios prior to the following details:
product release to the end users
ƒ During support and maintenance, there is a need ƒ
Sub-frame configuration parameters for UP-
of a testing and validation framework to quickly LINK.

978-1-4799-6023-1/15 $31.00 © 2015 IEEE 454

DOI 10.1109/CICT.2015.143
 Fig 1: Functional split between E-UTRAN and EPC.

space of the tested system requires infinite testing

ƒ Sub-frame configuration parameters for DOWN- time/ resource hence is practically infeasible [1].
LINK.
ƒ One time configuration parameters.
ƒ DOWN-LINK PDU Data. II. LTE/LTE-A TESTING FRAMEWORK
Utilizing the standard testing method described in
In the case of PHY testing since the requirements Fig 2, an improved and simplified framework is
are limited between PHY and MAC messaging, a designed. Details of the framework are given in
testing framework can be developed using a locally section A. Section B describes the use cases and
generated configuration files mimicking the actual advantages based on the framework design
MAC layer functionality. The idea behind the
framework design is to provide a real time effect of The 4G LTE/LTE-A testing framework described
the system and to be able to test the PHY layer as if it here, is aimed not only to be a simple design, but to
were working in a complete end to end system. also be capable enough to perform various automated
tests. Fig 3. Shows the
C. Standard Testing Framework
ƒ Unit Testing.
A standard testing methodology is shown in Fig ƒ System Testing.
2. As shown in the diagram, design of an effective ƒ Integration Testing.
testing system starts from defining the test space i.e. ƒ Acceptance Testing.
identifying the target and boundary of the measure of
tested system. Covering the entire system operational The framework design by us incorporates all the
listed testing methods mentioned above and also
performs some advance level testing’s which is
described in the upcoming sections.

Automated Test Framework (ATF): The ATF

encapsulates six major blocks as listed below. The
ATF runs on a PC system which performs
automation tests at regular intervals. The user has the
flexibility to select the different kinds of test suits to
be executed during the automation, also to select the
environments in which the PHY testing is to be
performed.

b Configuration Setup: The configuration setup

is a software application which gives the flexibility to
Fig 2: Standard Testing Method. schedule the automation framework, to select or de-
select test suites and to select or de-select the test

455
environments to run. When configured, the system
will read the configuration file which lists the options
selected and performs the automation.

Fig3: Improved Testing Framework Block Diagram

c Test Suite: Test Suites are a set of use cases e LTE/LTE-A Test Model: The LTE/LTE-A Test
which will validate specific feature, functions or Model is a tested and verified module, either
stability of a system under test. All the use cases are hardware or software based. In our case this module
written in a single or multiple xls. For example is a MATLAB module. The test module contains a
consider a Feature Based Test Suite which can wrapper code written in TCL used to read the
contains n number of features. In this case n different configuration file fromc. Based on the configuration
test suites can be defined containing x number of user settings specified in the file the wrapper will
cases. Hence a total of n * x use cases would ensure configure the test model to read the specified test
complete test of a suite. List of different types of suites to be executed. Once the use cases are listed in
suites identified are given below. the test model system the model is executed. The test
model will then provide the results of the test
ƒ Sanity Test Suite performed f along with test vectors g which will be
ƒ Regression Test Suite used by the different environment systems to perform
ƒ Feature Based Test Suite the test on the PHY development protocol under test.
ƒ Conformance Test Suite Certain output reference data are also provided as
ƒ System Test Suites reference data check points to perform extensive
testing e.g. unit level testing. Reference data can
Depending on the test suite being selected certain range from basic CRC checks to extensive level like
level of testing may or may not be possible for a test channel estimation data, equalized data, SINR,
environment. Manual intervention would be required measurement etc. f Result Logger: The result logger
at this stage to manage the test suites and understand is aimed to log all results obtained from the LTE Test
validity of the test suite for a test environment. Model. This data logs allow the user to log
However selection/de-selection of test suite is more information from PHY to MAC and MAC to PHY
or less a check box click via the configuration setup. message exchange, various computation such as
SINR, Measurements etc. These data act as a bench

456
mark to validate the PHY system when testing outputs with that of the intermediate vector outputs
commences with the different environments. Various generated via the Test Model. The results of the test
graphs can be projected on the system performance. will be stored captured and store in the Result Logger
More effective mechanism can be used to automate f The UTE can also perform different levels of
different types of xls which can provide extensive testing:
amount of statically data.
ƒ Configuration Test: Testing different L2
g Test Vector Storage: Through the LTE/LTE-A configurations and parameter sweeps.
Test Model various test vectors can be generated ƒ Channel & Algorithm Test: Testing different
depending on the test suite run. These test vectors can channel conditions e.g. EVA5, EPA5, and
further be utilized to drive the automation test system ETU70 etc.
to perform different levels of tests with different test
environments. Some test environments that can be INTEGRATION TEST ENVIRONMENT (ITE):
utilized in the automation framework are listed The ITE is a test environment that can be utilized to
below: perform an end to end PHY testing for a complete
UPLINK data/control path scenario or DOWNLINK
ƒ Black box Testing. data/control path scenario. ITE can be considered as
ƒ Unit Level Testing. one-step higher than that of UTE. Some different
ƒ Integration Testing. types of testing that can be performed using the ITE
are the following:
The test vector storage stores all the vectors under
the suite names which have been configured viab. ƒ Single sub-frame test.
The test environment may or may not use the test ƒ Multi sub-frame test.
vectors generated via the test model. For the test ƒ HARQ test.
environments that do use the vectors a wrapper code ƒ Complete UP-LINK data and control path test.
is written in TCL to read the input test vectors into ƒ Complete DOWN-LINK data and control path
the specific environment. test.

The ITE similar to the UTE reads the test vectors

h Environment System: The Automation Setup which are stored in Test Vector Storage g via TCL
can utilize different environment systems to perform script. The test case will be executed and the results
various levels of tests. These different environment will be stored captured similar to how the UTE is
systems can provide very comprehensive details. For performed.
example: Unit level testing breaks down the complete
PHY into small units such as channel estimation, BLACKBOX TEST ENVIRONMENT (BTE): The
SINR computation, equalization, CQI decoding BTE can be used to perform testing utilizing a MAC
which are all considered as a separate single unit Layer simulator. This can allow multi sub-frame
entity. Each unit entity can undergo its own level of testing up to ten sub frames or more. The purpose of
testing. It is also possible to have a group of entities using a BTE is be able to simulate a environment
to undergo testing. For example channel estimation, system which is almost like a real system having
equalization even though being different unit entities parameters arriving from higher layers in a similar
can be paired as a single entity to undergo testing. A fashion end to end system works. BTE works
detailed explanation of different kinds of testing different to UTE and ITE. The BTE reads the
that’s can be achieved from the automation setup is configuration for the use case via script mechanism
given in detailed below: and configures the system and generates random
data. The test model generated test vectors are not
UNIT TEST ENVIRONMENT (UTE): The PHY utilized in BTE. The
system can be broken down up as UP-LINK and
DOWN-LINK and further broken into several small
unit entity. The idea behind breaking down the PHY III. RESULTS AND ADVANTAGES
system into separate unit entities is to be able to
perform a detailed testing considering each entity The Framework described in this paper has be
independent. This allows the mitigation of any issues utilized and has provided great number of
such as bugs, performance of certain algorithms etc. advantages. Some of the advantages are listed below:
The UTE utilizes the MATLAB generated input
vectors generated via Test Model e and stored in ƒ Sanity/Regression can be done periodically every
Test Vector Storage g read directly via a TCL script day/week in order to keep a constant health
and executes the test case. At the end of the test, the check of the software code. When multiple teams
TCL script will compare the intermediate buffer work on a single code base it mandatory to keep

457
 a constant watch on the code base to assure that REFERNECES
the basic functionality is not broken. Sanity level [1] Catherine Tsao, Automation Design for 4G Mobile Internet
testing can be performed at different levels as Communication Systems Testing, 2004 lEEE Autotestcon.
listed below: [2] Ray Dolan, The Mobile Broadband future, international
Engineering Consortium, Annual Review of Communications,
- Unit level sanity testing Volume 54. 2001
[3] Genichi Taguchi, System of Experimental Design,
- Integration level sanity testing. 1988, Kraus International Publications, New York
- System level sanity testing. [4] Catherine Tsao, Advanced Measurement System
- Algorithm Performance level sanity testing. [5] Design for 4G Mobile Internet Systems Structural and Behavior
Testing, IMTC 2004.
ƒ Defining a correct use case for a test suite [6] TS 36.211 Evolved Universal Terrestrial Radio Access (E-UTRA);
Physical channels and modulation.
requires a considerable amount of effort. By [7] TS 36.212 Evolved Universal Terrestrial Radio Access (E-UTRA);
using the automation framework one can Multiplexing and channel coding.
validating the correctness of the use case and [8] TS 36.213 Evolved Universal Terrestrial Radio Access (E-UTRA);
save considerable amount of time. Physical layer procedures.
[9] TS 36.214 Evolved Universal Terrestrial Radio Access (E-UTRA);
Physical layer; Measurements.
ƒ The automation frame work can be utilized to [10] 3GPP TS36.141 Evolved Universal Terrestrial Radio Access (E-
check the performance of the overall system and UTRA); Base Station (BS) conformance testing.
can give a deeper insight on requirements such [11] 3GPP TS36.104 Evolved Universal Terrestrial Radio Access (E-
UTRA); Base Station (BS) radio transmission and reception.
has optimization scope, algorithm performance
etc. The automation framework can easily be
enhanced with very low cost for testing advance
features.
ƒ
IV. CONCULSIONS
In this paper an improved testing framework has
been proposed. The testing framework described here
has allowed a relative amount of flexibility to test a
system efficiently in different environment system at
a bare minimum cost. The effectiveness of this
framework design has been utilized in a complete life
cycle of commercialized PHY releases in the market
as of today.

458