BUDT758K: Computer Simulation

for Business Applications

Spring 2018
Prof. Sujin Kim
Lecture 24
 Announcements

▪ Submit your course evaluation!

CourseEvalUM http://CourseEvalUM.umd.edu
▪ Download: Model 12-02
▪ HW5, Question 3
“ … Customer line switching occurs whenever the difference
between the queue lengths is 3 customers or more …”
- condition for scanning: Use NQ (xxx.Queue) variable
▪ Lecture 25: Case Study with in-class quiz
Section 12.4
Variance Reduction

OUTPUT ANALYSIS IV

3
 Variance Reduction

▪ Random input  random output (RIRO)

▪ In other words, output has variance
• Higher output variance means less precise results
• Would like to eliminate or reduce output variance

4
 Variance Reduction

▪ Easy (brute-force) variance reduction: just simulate some

more
• Terminating: additional replications
• Steady-state: additional replications or a longer run
▪ But sometimes can reduce variance without more runs —
free lunch (?)
▪ Key: unlike physical experiments, can control randomness in
computer-simulation experiments via manipulating RNG
• Re-use same “random” numbers

5
 Variance Reduction

▪ Several different variance-reduction techniques

• For comparisons — common random numbers
• For single system — antithetic variates, control variates,
importance sampling, stratified sampling, …
• Usually requires thorough understanding of the mathematical
structure of the model and the code – challenging!
▪ We will discuss only common random numbers

6
 Common Random Numbers (CRN)

▪ Applies when objective is to compare two (or more)

alternative configurations or models
• Interest is in difference(s) of performance measure(s) across
alternatives
• Model 7-2 (small mfg. system)
− Total WIP = EntitiesWIP(Part 1) + EntitiesWIP(Part 2) +
EntitiesWIP(Part 3)
− Total avg. WIP output — two alternatives
A. Base case (as is)
B. 3.5% increase in business (interarrival-time mean falls from 13
to 12.56 minutes)
7
 Common Random Numbers (CRN)

• Same run conditions, but change model into Model 12-1:

− Remove Output File Total WIP History.dat
− Add entry to Statistic module to compute and save to a .dat file
total avg. WIP on each replication
➢ DAVG ( Dstat ID )—Average value. DAVG records the average
of the dstat (time-persistent) expression throughout the
replication.

8
 “Natural” Comparison
▪ Run case A, make change to get to case B and run it, then Compare
Means via Output Analyzer:

▪ Difference is not statistically significant

• Were runs of A and B statistically independent?
• Did we use the same random numbers running A and B?
9
• Did we use the same random numbers intelligently?
 CRN Intuition

▪ Get sharper comparison if you subject all alternatives to same

“conditions”
• Then observed differences are due to model differences rather
than random differences in “conditions”
• Small mfg. system: For both A and B
− “Same” parts arrive at same times
− Be assigned same attributes (job type)
− Have same process times at each step
▪ Then observed differences will be attributable to system
differences, not random bounce

10
 Synchronization of Random Numbers in
CRN
▪ Generally, get CRN by using same RNG, seed, stream(s) for all
alternatives
• Already are using same stream, default = stream 10
• But its usage generally gets mixed up across alternatives
▪ Must use the same random numbers for the same purposes
across alternative scenarios — synchronization of random-
number use
• Usually requires some work, understanding of model
• Usually use different streams in RNG
• Usually different ways to do this in a given model
• Sometimes can’t synchronize completely for complex models —
settle for partial synchronization 11
 Synchronization of Random Numbers in
CRN
▪ Synchronization by source of randomness
• Sources of randomness: interarrival time, processing times, …
• Assign stream to each point of variate generation
• Separate random-number “faucets”
▪ Synchronization by entity
• Whenever a new entity arrives assign all random attributes

12
 Synchronization of Random Numbers in
CRN: Model 12-2
▪ Synchronize by source of randomness
− 15 sources of randomness, separate stream for each, modify into
Model 12-2
✓ Define a Variable for each stream number: ex) S P1C1
✓ Use the variable name in the variate-generation expression:
» Ex) TRIAL(6,8,10, S P1 C1)
✓ Skip over stream 10: Arena’s default stream, used for built-in
nonstationary-Poisson process generator and Chance-type
Decide module
− 10 replications to each alternative: Arena random number
generator will automatically advance to the next substream within
each stream for each replication

13
 Synchronization of Random Numbers in
CRN : Model 12-2
Stream # Usage Variable Name Location of Change
1 Interarrival Times S Interarrivals Create Module
2 Part Indices S Part Index Assign Module
3 Part Type 1 at Cell 1 S P1 C1 Expression Module
4 Part Type 1 at Cell 2 S P1 C2 Sequence Module
⋮
15 Part Type 3 at Cell 3 S P3 C3 Sequence Module

▪ Fairly simple but might not ensure complete synchronization

• Maintains synchronization if alternatives disagree on number of
random numbers used per stream per replication
• Synchronization of streams for built-in nonstationary-Poisson
process generator and Chance-type Decide module may not be
ensured
14
 Effect of CRN

“Natural”
Comparison

Synchronized
CRN

▪ CRN was no more computational effort

▪ Effect here is fairly dramatic, but it will not always be this strong
15
• Depends on “how similar” A and B are
 CRN Statistical Issues

▪ In Output Analyzer, Analyze > Compare Means option, have

choices of Paired-t or Two-Sample-t
• Paired-t subtracts results replication by replication — must use
this if doing CRN
• Two-Sample-t treats samples independently — can use this if
doing independent sampling, often better than Paired-t

16
Paired-t vs. Two-sample-t

17
 Synchronization of Random Numbers in
CRN: Model 12-2 Modified
▪ Synchronization by entity
• When a new part arrives, pre-generate every possible random
variate that an entity might need, assign to attributes, used
downstream
• Better synchronization insurance but uses more memory

18
Synchronization of Random Numbers in
CRN by Entity

19
 Mathematical justification for CRN

▪ Let X = output r.v. from A, Y = output from alternative B

▪ Compare 𝐸(𝑋) and 𝐸(𝑌) ↔ 𝐸(𝑋 − 𝑌)
▪ Pared t-test: Estimator = 𝑋ത − 𝑌ത
▪ 𝑉𝑎𝑟 𝑋ത − 𝑌ത = 𝑉𝑎𝑟(𝑋 − 𝑌)/𝑛
▪ If 𝑋 and 𝑌 are independent (without CRN),
𝑉𝑎𝑟(𝑋 – 𝑌) = 𝑉𝑎𝑟(𝑋) + 𝑉𝑎𝑟(𝑌)
▪ The variance of the CRN estimator is
𝑉𝑎𝑟(𝑋 – 𝑌) = 𝑉𝑎𝑟(𝑋) + 𝑉𝑎𝑟(𝑌) – 𝟐 𝑪𝒐𝒗(𝑿, 𝒀)
≤ 𝑉𝑎𝑟(𝑋) + 𝑉𝑎𝑟(𝑌)
≥ 0 with CRN
in most cases
20
 Variance Reduction

▪ Standard deviation: 0.994 vs. 10.4

▪ In terms of variance: approximately 1 vs. 100

21
 Synchronization of Random Numbers in
CRN
▪ There is no additional computational overhead in implementing
CRN
▪ The efficiency improvement is guaranteed as long as positive
correlation is induced between the competing system designs.
▪ However, the random-number stream synchronization must be
carefully conducted with an understanding of how your model
works.

22
FINAL EXAM

23
 Final Exam Info

▪ 11:00-12:20 pm/ 2:00-3:20 pm, Thursday, May 10

▪ Coverage: Lecture 13-16, 19-24 (skip L17&18 Entity Transfer),
HW4 and HW5, Quiz 10-13
▪ Can bring one page of hand-written formula sheet (download
the form from the course website)
▪ Office hours for next week
• Tuesday, May 8 (regular office hours): 12:20 pm – 1:20 pm
• Wednesday, May 9: 10:30 am –12:00 pm, 1:00pm- 3:00 pm

24
 L13-L15: Ch5

▪ Model 5-1: Call Center

• Basic Process: Variable Data Module
• Advanced Process
− Seize, Delay, and Release Modules
− Expression Data Module
• Block: Delay Block
• Entity priority, sharing queue

25
 L13-L15: Ch5

▪ Model 5-2: Enhanced call center

• Nonstationary Poisson arrival process
• Schedule module: Arrival, Capacity
• Sets – Resource, Counter
• Statistic data module
• Record module
▪ Review Model 5-3

26
Schedule Module for Arrival

27
 L19-L24

▪ L19, 20: Ch9 Balking & Reneging, Holding & Batching

▪ L21: Section 6.4-6.5 Output Analyzer, Process Analyzer
▪ L22: Point estimator, Interval Estimator (Confidence Interval),
Hypothesis Test – Goal, Concept, and how to compute
Output Analysis for Terminating Simulation
▪ L23: Output Analysis for Steady-State Simulation
▪ L24: CRN

28
 Final Exam

▪ False Questions
▪ Multiple choice questions
▪ Conceptual problem
▪ Arena Model Question

29
 Arena Model Example

Create Arrivals Hold for Signal Batch Entities Delay for Process
0
0

0 True
W ait Q ueue Less than
2

0 False

Create Scan Delay for 8

Scan for Condition Record Startups
Entity minutes
0

Assign Batch Signal for Batch Record Batch

Size Dispose 1
Release Size
0

30
31
32