OFTLARE

ESTING
ETHODoLOGIES
AssigNMEN T

B.ViKAs
J8 l8BH1AOSIG
CSE A
A
OWhat ts the pupose of tesltng ¢ Tplatn
about the model
fo lestipq
Testinq
Testing
Os
thepsocess Of
us cxc>cutstn9

evluattng System o &s

oa

Qulomate d
Coponents by manctal os

oeans to vertytng that tt Satisftes

Pectfted sequtrements.
Tbe
CPucose qf Testinq
Tes tng Consumeso at least haf of the

ttme andwork equtsed to p soces8

Produce afunctional psogpa.

MYTH: Gwd pseqsammes wst te cocle

otthout buqo.
*Hsto Says that even well wsttten
Pso qa am su bave -3bugs per hundsedl
Stotements.

Mocdel ox Testnqs
Tbeoosld h e Model hlonld
Tbe envtsonment >tnusonment Dexpected
Model
the 0soq sam
P5ogsam
Model Tests utcome
Noture &
Bug
Pycboleg8 Moclel
Amode gor testog
Above Jgure a model of desttng (P*oces
ts

Tt tncludes theve models. A model of

the
eputonment, a model of the Proqsom ano
a model of the
epected bug
* tnutronsment
the havduave
93am3 enuaonment e6
ond 8oftuware xequtsed to make tt sun.for
Ooltne
ystems the enutsonment may tnelude
O,tesminals
CommuDICott op koes, othes dyséemo,
and peratord

The enutsonment also tneludes all proqams

that tptesact wrtb and are used to cxeate
thepogsas under tes6 - 8uch as 08, tnkage

ed tor, loader. Coplier, utl1ty. routines

Because the hardware and ftivase ar

Stable, tt t8 Dot 8mart to blasme the

epvtropmeot for bugs.

#(Ps0qsam
too compltcatedl to
Most psoqam
undesstand in detail.
to be
sam t
Tbe corcop£ of the pxoq

tn osder to tcst
tt.
implftc dDesn't

s t m p l e mod el of the progsam we

w e ma Y
may
þehaviour,
the unepectcd;
eplatn model to tnelude
the proqram
have to modity that fails
o r e a c b s and debali. And 1
the rOqranm.
ue may ave to modidy
 Bugo
>
aTe more tnsidous than eves we

ePect them o be.

A n unexpectedd test vesult may Lead
us to
char
charge ever natiorn of what a
bu
s and our model of bugo.
ome optimstic notions than even many
P9ammess o3 testers bave albout buqs ase

Lsuaily
usually
unable
unableto test ejfectvely and
tojustfy the dr»ty test most
Pams need.
Optimtstie nations about bu93

Bengtn Buq +Hypotbest6 The bele that

bugs ase nice,fame and logteal,

Tbe beltef that a

Sug bocalty typotbesiö -
bug dscovesed wtth er a
compopent affect
only tbat comyponents behaviouT
The that
Coaol byq TDominadce belkef
thooe essoYs tn the Control 8tuctuicg Of
dootnent tbe bug.
9 ams
Code d ata Seporakon The belef thet
xespeet the spaxation Of code and
bugs data.
stngua Saluatare Est The beltef that
 -

tbe
lanqugge
,U 8yntax and 6enantie
eltmtnates most bug.
Cossecttons -brde: be mtstaker beluf
that a corected bug eroain Cossectrd,

that
Srlver bullets Tbe mtstaken beltef
a Sadtstic &tseak low cunoung and tntuilion

ae Sudftcrent to etmtnate most bugs. Tougb

bug beed melhodology anod Lechntques.
Aegelie The bel:ef that testers
Testers
belter at test design tharn P9aamtD
3 at Cocde destgn.
Tests
must
Tests axefosmal poceduse, Inputs
be pspaed, oülcomed ahouldpredtct, test

should be documented, commands necd t6

obsesue d
be executed, and aesult 8 ase to be
8ubjectedl to
exsov.

all tho8 e co0ss ase

We do thvee d:stinet knds testinq on

Hptcal softwave system

Tbey axe
A untt s the
testinqs
O r t Component 80ftuoaxe
that can

S mallest
testable piece of ete
assembled,
toked, loaded
complted,
be wosk and one
untt ts usually the Scveredl hundsel
and consiats of
tcscing u
a
p rm
0 9 sm ee
amm dd
Code. Ontt

oxeoer
ltnes of
 do to show that the unit
the
testing we

oesr t Satisfy utsunclional pecificaton

Os tha tmplementatton Structuse
doesn't oatch the tntended desro
dtsuotuse. A component t8 an tnte9 atedl

99*9te, 0f one os move untt8. Compo neof

Te6 tingt8
Tes tng the testng we do to Shou
that tbe compopent does'ot 8at:sty *tts

Junctioal Spectftecdzor ox that uts

Pplementattorn stsucture doesn't match the

tntended destgn 8tsuctuve doesn't matcb5
the tntendeddesign stauctuse.
the
Integvcttor Testing Tnteqsatton ts
poces by whrch components ave
agggated to cseate lasges Coponents,
Tnteqsation tesling t tes tung done to show
that even thsough. the Components were

tmtuclually satisfactoxy. checks the

Combtnatton of ComponenE6 dye incusrent or

tnconsstent.

Syetem Tes ttngA systerm ts a btg9

component. Oystem testtog atmed at ts
Teveallng bugs. that cannot be attstbuted
to Components.
T t includes testing fov
accountahr!:E
pefomanre, securt ty
 ond seCCe
Conftqusatton Senst(tuity, 3tostup
oole f Madeb:
Tbe aal f tesling consts of Cseatirn,
our
and bevstnq modeds.
Selecting, c*plasir
abltty to go thvough thus process dpends
OD tbe numbes of d:ffesent models, e houe

at hand and thet» ab:kly to cxpress a

eqams behoutour.

p l a t n about Taxonomy of bugo

There t no untver sally Cossect way

catgqostze bugs. The taxonomy

ts not si9tt.
o
Agtven bug Can be put.tnto one

another categovy dependto9 on tts bistov

and the proqsamroes's &tate of rotrd.

T h e major coteqovies ase s O Gequtvement s,

featuvcs
deatuvcs andfuncttonalty Bugs Stucturel
Oata Bugs Codog bugs Itface
Bugs and test
Inteqsatron and system Bugs Test
Desg bugs.
and Functional1ty
Goegutrement s, featuses
Catergories
tn cqutoements,
equtoements,
Vastous.
Bugd
teatures bugs tnclude.

and dpectficatto ns Bags

equtoendents

specifteations developed
developPed
and
Requt ements
uts
 rom them can be tncomplete ambtqudus
O self contsadtc tovy They can be
idunde 8tood or tmposible to undesatand ,
Tbe &Pectftcatton that don't have
bave
Flaws tn tbem may change cobrle the
tbe

destgn Dpoqses 8. The featured

ase

added mod:fted and deletedl.

cquirements, espectally, as epressed
p Pecificattons ase a major Souace o f

epensive bug3
T h e ange 3foor> a few pecentoges to
mose than 50 dependtng on -the RPlteattop-
and enuttornmentt.
What busts moot about the bugs ts that
they
tbey ase the easliest to tnvade the system
ard the Last to leaue.

eatu Bugs
S p ecificatton psoblems uoually Coeate
1
Cosseonding fectusePsoblems.
A featuse
featuse can be wrong. mts0iox

8upesfluous. ot wi 58iD featuse Ov case

easie to defect and covrect. A ovong
feature oO Case s casieto defect andX
feature
COse ct. A
emoUing theeatures mqbt complccute
Tesousce s, and fo
mose ta
Consume

the 804twase,
Rose bugs.
 eadure Interactton Bugs
aod
Pourdrox
testable
coorect, clear, tmplemerntable
feature Spectftcations ts Dot
and

enougb.
Feotures usually Come tn ,qsouPS or
each
eselatedfeatures.
(ated The feaduves of
gsouP and thbe tnteraction offeatures
cottbinthe soup are usually well tcsted.
The Psoblem ts unpsedectable, tntesac tions
betuoeen eature gsoups os even between
todlturdual features. fos cxample, jou
telepboDe ts prourded orth cal holding

and call foswasdip9.

t v e y 9Pplicatton ha3 untts (Peculiar set of
eatures aDd a much bggev set of features
nteactionpottntials and tbere payo
ntesaction potertials and theve fo re desutt 1
eatuse totesaction byg3.

What ts Contaol flouw 9oph: tplain about

path predtcates and acbieveable(paths g
Cootaól flou a*ph
A contsol flow g*oph ts the gophical
contsol flow os Computatior
sepreseotation 0f
8 P r e s e o t a t t o n

ustnq the exceeution of p9ams o

applicatiomd controflo g*phs ase
mostl
coell as
used t Statte analyats a8
Comptes Rplcotion s as tbey can
accusat ely epsedent the flow tnöide of a

Pograss untt. Tbe cootsol flow gph

was o;gtnally deueloped ytsance s EA=

Patb pvedteakes
Apredicale asso ctated otth a
poth is
called a path (Pved1cate. tor eple
tS qscater than zeso" Y9090" ' w
etther Degatrue ov equal to to t8tsue" ts
asequence of predicotes obose tutb
volues wtll cause
Cause the out1ne to take a

Pecepatb.
Multt Branches
Tbe
(path takeD thsough a multplay
bsanch such as a
Computedl -G0 TOLS,
case
&taterment O Jump tables canno
be dvectly expsessed tn Taue/False
tes 3.
Athou9b ut tó po36ible to descsibe
such atesnatives by udi mult: valued
Logtc, an ePedient ts to ePess8

multplay branches ao ap equualant set

of the-else statement

exemple a three oay

fox Cade
statemert can be osttte as Tf ca8e =)
I F Case =2 ELIE
DoAI ELOE DoAz
Do A3 ENO I 1) END1f
Inputss
> In testipq the wosd topt to not

yeststcted to dtse ct tnputs Such as

vastables tn a subaoutioe call but tndudes
all data
objecls vefesenced by the
soutine
soutine whose values ave,ftxed Pstor to
eptesing tt
o example, t p u t s t a callnDg

dala stsuctuse, valurs

Sequcpce object8 tn a

Let to eqtstesd os any Combnation of

object yr.
T b e inputsfor a pasttculax test us
maRp d as a one dtmenatopal asray
called as an Iput Vector.

Wutte a shot note oo(path Senstttztpg

euteo acbievabte and unachtevable Paths
We want to delect and test enough (paths
to acbieve a Satiufactory notton of test .

3uch Cas Ct C
Completenes8
Extsact the psoqsams Contoolflouo

-and deleet a set of tentative Covesing

5ph
Paths the
TOY
any path tn tbat set, to teapset
Predtcekes alorg the path o3 Deeded as
to
of the put vectors
eypse85 them t
epsea8 tesmo
 sa cc the path thaoqb, oultipytn
the todturdual compouod os
pedicotes to achieve a booleah epeS5iC

Sucb as

Mufy out the expses5ton to achieve

SuT)
O Pod ucts fovm
ADFG-+1IJkL t AEFGHIJKL t BcOfGiHiJklt

BcDEFGtl
E a c b product tesm denotes a set of

tnequaltte s solued o l l yteldd an

thoct u
tput vectos that col dive the routine
aulong tbe dessqoatedl (path.
Solve ay one f tbe tnequaltty setsjfor
the chosen pcotd and you havefourd a set f

tput vedues fos tbepath.

you can ftnd a 8olution then the
Patb isacbtevable.
Hou Can't ftncl a solution to any
the Seds of iequal1tees tbe path s
unacbtevablc.

The act o Jtndinq a 3et of Solutions

to the (potb prrdicate expTes3ion 18 catled
poth Sensitizotion.
oth
 Heuststc pvocedusr for Snottizin9patbs:
înstead
Thrs ts a workable aRproach,
o Seleciipq thepaths wrtbot Comstdeioq
to choode a
bouo to Sentize, attempt
to
set thtt tt t8 easy
covertn9 path
Covestng
Sensrttze and prck bard. to Senottire(paths
to achteve coverage
ony you must
as

vastables thot affect tbe

Identify all
olectotonp.
dependeot
Classty the predtcates
as

Os tndPendent predtcates.
been achteved usin9
I Covexaqe bas not

tndeperdent uncorelated. precdiette,

8et ustng Garelate d
extend thepath
Ptdi cates
Coverage bas not been achreve

the cases that tnuolue

extend to thoJe
dependert prdicafes
covelated, dependertPredicaf
last, use

a Tsansactton flow trplarn

What s

about
tsansaction flow testinq techn:9ues

Tsaso ctrom flou gepb intro du ced

as a
ase
sanaaction flows
a ysems (psoces819
presantatton of
 The methods tbat ase Uplied to
then used
Contsobl
floosoph3 ae

Jo functional eotipg.
xanoaction -flowo and tsansaction

Flow testng ave to the tndependent

ath
System tester what Contsolflouwd a
Lesing ase to the
proqammer .

T h e tzans action flow grophics to Ceat

'that
a behavio6al model of the P q am

leads t o funetiondl testtnq.

The Laansaction flow q3oph ts a model
o5 the sisucture of the 8yotemo bebauiou
>An example o a transaction flow ts os
Carcel
foltow.

O Reoder
User Request hlatt ccept (Proccs
0cler From
Begt e Ovdler jom
o m CPO
CPU

wait ccept sansmit

Tsanemtt C Anymos D
B Page to Freld
answess
felds to cPU
Tesminal rput
 Wait fo a
Accept Valkd Move User
Ett
D cants
CPU Confm CPU
Conftim
Set
lsansmtt
dtgnostte Seview

to teymnol

Tsansactioo tlouo
testtng Techntque 6
G t tbe tapsaction8 tlood:
lot
Complteated oystems tbeut poce3 s a

Toansaction s
diffeven-t, compltcat ecl
.

of
Should have cptctt ePresentatt
ons
of the
0 the equivalent.
tsansacttons flouwS,
Tansactton ftows ave uke Control flow

and conseq uently we sbould expect

gpbs
3ehs detal.
levels of
have them tn tnczeastng
to
documentatton should
T h e ey3tems destqn
that details
Conta:n an overUtew8ectior
the
the mato transactionflou. a Tmandatos
Oetailed
tsansaction flow 8 axe

the dational des1q fa

eseqursite to
ystem'sfunetonal test.

Revtews and kllk

throughs
Inipectons, natural agenda fo
Q9endafoOr
Daturat
are
Toansactor lous
o v inpe
ction8.
em devtous
Syst
y alkthroughss, Hou
the walkenough
ye sheuld
Conducting
In
 Dricus3 paths Lbough tronsactibp
to
type
account fo 18-997/. of the
Caanda cdion the System us
cpecded op*ce
Dcus3palh6 throughflow infuockian
atbe than tecbntcal terma.
Ask the dearqneaa -to velate evey lou,
Lo he
Pecftcatt on and to show houw tha
laanaacdion; diaertly o todisecly, folou by
aom tbe dequtsements
Make transaclton flows ttstng the coney
Stone Ox
of Systemfunctional tsttn 9,just a
ptb Lestinq t8he conea &tone of untt

>Sclect additional flouw testtng the

Cosoe 3Lone of Systenfunctiona domain

boundasted.

>desiqD move tst coses to valtdate all

btths and deatbs.
Publish and d:stbdte the Selecte
test paths througb n the Tsaniactton
Jloua
flous a cay asposstble that tbey wtl
exest the maxtmu beneftctal effect on the
dPject.
Path Selectton
Select a8et o-f Covert
9 9 Path s (G1C3
using the andlogous
cattevia you usecfor
stsuctunalpatb testng.
>Selectng a Coveving et paths based
on
functionally Senstble tvanjactions aYou
woud for Cotsol
flow gphs.
Ty
y to nd the moot tos tuous, longt,

staanqes t path fom the entay to the eit

O the tsansaction flo
Path Sensttrzation:
Most o the normal paths ase vey

easy t o 8ensrtrze -
80 - I57 transaction

flow covexage (C14) ta

usually teasy to
a chteve
T h e emaintng Gmall Pescentage t ften
vey dfcalt.
act f dynng the
Sensitization ts the

tsansact ron, tf there a e dendrtization

then bet on
Poblem8 on the casy paths,
tsansaction flow s or a des9n
eithes a
bu tn

b
Poth tnotrumentation

a &9geyole tn
umentation (plyd
Inslr than tn
must be flo edtinq
tansaction

un:tpath edting.
 The tnfomation Of the patb takrp
jf q
tsanjaclion must be kpt wrth that
gtue
bandion and can be vecoveled by o
Centsal sancction dypatched or by the the
tndividual pvocessinq nod ul as,
I nD Some
Some SystemS, 6uch tao ce s 0se
cvicded by the qpesaling stem 6 os a
oovrdecd

sunning g
What ts data flou
a
testirg Klhat ase the
Stsateqtes tn data flow testing
Data flou Testing*
Dauta flow testing u s the Dame
to
gtvep
afamtly o tesé 8t3ateqies basec on
Selecting paths haough the progsam's Contwl
flo tn osder to explove sequences of events

elatedto the Status of data objets,

tor example; pick enough paths to asiue
that evey data object bas been tottialized
cveyy
P to use o that all desincd objrets hae

een used for Somethin

Motivation It s our belief that just

as One would notfeel Contd ent about a
aro wrthout executing every statement
tn it as par of Some ttst, one should
not feel Confidept a beut a (p*cqsam otthoctb
 the ualue
having Seen the edfect cf using
poduce d by each and evey Cooputation.

Strateqtes tn data flow ts ting:

The stzuetural test staateg1es discussed
below, are based on the p9qam' Control

extent to
So *oph. They dijfer tn the

obtchpredtcatcs udes and/ or computational

u6es af vasiables ase toduded tn the test

8et vasioud tupes of data flouw teoting

their
Drder of their
8tsate9tes in ddecsea$tn,q
efecttveness are;
AN-du (Pathe (ADUP: The all- du -paths
(ADU
t8 the stsongst
dataflo testin 9 co

stsat
Stsod
eg8
Stsategy
dtscussed here
thak
.
It equires
egy
u (Pathfrom evey deftnition
of very
evey
every ome test.
vastable to evey
Vasiables x andY are

Vasiable x and Y:
tor
tor
test that stasts-
link () any
used
onl
only on

Satifies
thts ottesion,
Ctteoion.

a tt
a the entry
vasiable
The
8ttuation for
vasiable z: vasiable
tos beCaese
the
compltcated
t5
more
o r the
(places.
edefned t many exerci3e
execse
ts
(,3) e must
must
on
l:nk a n ,s,s
deftntéron
ine(udes suhpath s 3 , 4
that
paths
aths

on link G. 5) s
olejtnrlion
The
 COvesed palb thad tocludas (5.c)
by any
osuhpatb (.3, ,5.c.). The (s.c ) defn
equnes paths hat tnelude subpoth
5,,3.4) and (5, 6,1,3).
D Vasiable Vavtable V:s
V dejtned only
Once on ltok (1,3) Becauwe v has a

Pecdicable use clt node 2 and the

dubseqeunt path to the end must be
oveed for both dveotions at entoy /erit
paths
aths anc at least one
patb that tnclude
to
loo(P
loop cawed
by,4).
Note that must testPath that
we
tnelude
both
suhpath s(3.4,5) and &5) cven
thougb netthes of th«se ha[ v defotti on.Th
must be tnclude d becaude theY rovi de
attenate du path to the V lm
use on
k(s.c
Athoy9h 4 ts not used tn {he test
Set
that coves the
,
pedicate uses of
asay
vosiable vc and U.
The all- du -paths Stsateqy as a
stson
Coitevion, but tt doesn'4 one take as
many
ests as tt nigbt 8eem at fhst because any
one test Simu-laneously 8atisfie s the Cojtesto
Sor Sevegalo defrnt tions and use6 of
Severoal diffevent vasinhleA
AU uses taategya): "he all uses
stratq ts that ad least on defiotdiorn
clear path fom evesy defnt ton evey.
vaviable to cvesy ose af hat d:ftotti on be
cve
Execised unde
Some test.
Just as we Ted uce d Ow ambrttons by
otegping dounftm al path (P) to branch
Covesa 9 Ca , say, we Can seduce the number
Ctvesa
test Cases by askinq that the test set
Shouldtnclude at least one (Palh Sgmeryt
cm eves oletimtton to evesy use that can be
seache by that defioition
Tos V: NDUP T«quves thet
Vaiable w
tnclude Subpaths G.4,5) and , ) * some
test because Subsequent usts of v, Such a
tnk be rcacbed by eithes
5.c)
o can
altesnattve. In AU ether (3.4,5) os (3.5) Con
be but we don't have
used to 6tartPaths,
to se both stmilasly, we can skp the(8.10)
tnk tf we've ncluded the (29,10) 8uubpath

AuP- uoes some C-uses

3taate9y (PU iC)
that
vaviable and evesu diftnttion of
4osevey
forevesy
toclude at Leat one deftnttion
vastable,
o t h e dfinttito to evesy
fee pah here are
dfinttions oJ the

.use: u
pedicate
that hot
Coveye d by the above
by
use
v a s i a b l e s

adld computatlonal
ption, then
vescipt
est Cadrs as sequted to Cover eve
evea
dedmttion.
o vavtable Z
Tov POC we can select poths Lhat

oll take the P e s t n k (12,13) and

theve fove we do not covey the c-use of

b u t tht's okay accordling to the
Stsadeg's definttion because evesy definitio
t8 Coveve d.

tnk s .3) GG5) (5,.6 ) and a,8) must be

tncluded because they contain deftnttions
variab le zunks ) 3.5) (8.9)(3, 0]
(0) and G.10) must be tp cluded because
they contain odjtnttionsfor variable z,
ftnd a covesing3et of ttst Cased andey
APU t0fo ll vaxtattons tn thtg example,
tonly ake two tests.
o s Vasiable V APUt¢ t8 acheteved for v by

,3, 5,6,4, 8,t0, 1, t3

S, G, 4, 8, 1O, u, IR(tPPer)
13,2 and 1, 3, 5,G,A,3, t0, 1, 12 (ower),

13,2note that the C- use a

.10) need
b e tncluded undes the APO +e
colterton
 A C- uscs Some p- uses stad egy cU1P):
The ull
C usesSome P- uses 3tratqg
Aco +P) ts toftrst enoure coverage by
Computaktondl use casés and 1f any
defniéton ts not Coveved by theprevtous

seleetedpotha/add such predtcote u e c a s e

a8are needed to assuY e that every dlefnttiors
tncluded tn Some test

os
tos Vaviable zi AcP +P Coverage us acheveoe
Jorzz
Tor by patb 3,4,5,6, 7, 3, 10, u,12,13
Several
lowea), a) but the pedtcat e uses
of
defuotttons ave not coveed. &pect ftcally, the
3 ) deftntto ts not covee d Jor the , 5 )
not Coverro
(Puse, the,3 ) definition ts

Sor the ,1) G.c) and O.lo) pases

for

that APO+ e
The above examples tmply
than brap ch Coverage but AcF
t& 8taonger
roay be cweaker than, or tnamparable to,

bsanch covedaq e.

Fos Vaxiable 2 Path l 3 ¢, 5, G,,

6atote S to vas table V

e uses APU), AU1 computotiono-

COAl predical
uses Aco) 8toateqtes. The
all (precdtcot e uses

stsategy s dextveod from -pu4e stsate9y

TS by dpendto the vequrement thaf

se
tnclude. Aco1f
the eQutremen ts
6txotegg doppn
thatte tnelude
Jo the vastable td there a
a
p-use
use
no c-us eSor
the variable .
It ts tntui tiue ly obvious that ACU
should be weaker than
AcOP and tha
APU Should be oeaker
hat PO tC.