See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/220610457

HyperVerse: simulation and testbed reconciled

Article in International Journal of Advanced Media and Communication · March 2010

DOI: 10.1504/IJAMC.2010.032142 · Source: DBLP

CITATIONS READS
8 1,475

5 authors, including:

Jean Botev Markus Esch

University of Luxembourg Hochschule für Technik und Wirtschaft des Saarlandes
68 PUBLICATIONS 375 CITATIONS 41 PUBLICATIONS 379 CITATIONS

SEE PROFILE SEE PROFILE

Ingo Scholtes Peter Sturm

University of Zurich Universität Trier
110 PUBLICATIONS 1,670 CITATIONS 61 PUBLICATIONS 535 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Surveys on self-organization View project

Privacy-preserving data-mining View project

All content following this page was uploaded by Ingo Scholtes on 16 May 2014.

The user has requested enhancement of the downloaded file.

 HyperVerse: Simulation and Testbed Reconciled
Jean Botev† , Markus Esch† , Hermann Schloss∗ , Ingo Scholtes∗ and Peter Sturm∗
∗ University of Trier
System Software and Distributed Systems
D-54286 Trier, Germany
Email: {schloss,scholtes,sturm}@syssoft.uni-trier.de
† University of Luxembourg
Faculty of Sciences, Technology and Communication
1359 Luxembourg, Luxembourg
Email: {jean.botev, markus.esch}@uni.lu

Abstract—When dealing with dynamic large-scale topologies to deterministically perform tests in a synthetic environment
such as those underlying peer-to-peer (P2P) distributed virtual in which every aspect can be controlled by the experimenter.
environments (DVE), one inescapably reaches the point where Algorithms, parameters and topologies can be compared under
either a) simulations lack human behavior and assessment or
where b) practical experiments on a small scale do not yield different conditions e.g. using different user mobility models,
significant results. The restrictions resulting from the separation churn dynamics, simulated network latencies and so forth
of simulation and testbed environments hinder a comprehensive while leaving other simulation parameters unchanged.
assessment and efficient development of adaptive algorithms and Within the HyperVerse project [1], both simulations and
techniques for DVEs as they are investigated in our HyperVerse practical tests have been performed in the past to evaluate its
research project. In this article, we present a hybrid evaluation
system designed to combine the advantages of simulations and underlying concepts. Simulations have been performed using
testbeds. The proposed infrastructure exhibits great flexibility the TopGen environment [2], which is being developed within
particularly alluring in view of the multitude of potential research our group since 2006. Practical tests on small scales have so far
in the context of DVEs. been conducted using an actual DirectX-based implementation
of the algorithms and topologies in question. During these
I. M OTIVATION tests, implementation and deployment has proven to be time-
It is beyond controversy that the design of scalable DVEs consuming, especially dealing with external testers and firewall
is a challenging task. Their commissioning and maintenance traversal for peer-to-peer topologies.
being both laborious and costly, all concepts, topologies and Acknowledging the fact that both approaches, simulation
algorithms utilized need to be rigidly tested and evaluated well and testbeds, offer unique opportunities, a combination of
in advance. For this purpose DVE designers and researchers both seems alluring. Within our project the need for such a
usually rely on two separate strategies: Computer simula- hybrid evaluation system has been additionally fueled by a
tions are used to quickly obtain reproducible results, while special scenario: For a couple of years, our group teaches basic
small-scale testbed deployments are made to obtain a more programming principles to secondary level pupils at the age
realistic view under real-world conditions. Both approaches 10 and above. For this we embrace both well-known concepts
have unique advantages and disadvantages when evaluating like simplified programming environments as well as novel
massive-scale distributed virtual environments. approaches like learning and interacting in distributed virtual
The foremost advantage of testbed deployments is the fact environments. The latter naturally raises the question whether
that they provide a realistic view of a system’s performance both projects can synergetically benefit from each other by (a)
under real-world conditions such as realistic network latency, enlisting the group of pupils as “testing personnel” and (b)
congestion, user behavior and so forth. Unfortunately the using our DVE research prototype as software platform for
reproducibility and tracing of results is hampered by the real teaching. In this article we discuss a hybrid evaluation system
world’s intrinsic non-determinism and the scale is limited by that has been developed for this purpose. It allows to use a
the availability of physical devices. The fact that real user generic user client implementation to be used for the evaluation
behavior is essential for realistic tests furthermore complicates of different algorithms and topologies without having to deploy
the usage of research testbeds like e.g. PlanetLab1 . The us- any code changes. These actual clients are connected to a
ability of practical tests in research is usually restricted by the simulation environment which simulates actual communication
comparably small number of available physical machines and and data distribution while possibly additionally simulating
staff. While this could theoretically be alleviated by including hundreds of more clients.
external testers, arousing the interest of a massive number The remainder of this article is organized as follows: Having
of people has proven to be difficult. Furthermore, quickly laid out the general architecture of the implemented system in
obtaining results is prevented by the necessary deployment of section II, in the following section III we demonstrate how it
software, and the development of test prototypes is complicated has so far been used to evaluate concepts and topologies in our
by the heterogeneity introduced by external resources. At this HyperVerse research project. After having discussed related
point, simulations provide adequate means to evaluate DVE work in section IV, we conclude with our contributions and
concepts using a moderate amount of resources. Here the propose the usage of our evaluation system within the DVE
number of simulated entities can be varied within a wide range, research community.
being limited only by available processing capacities and time.
Further advantages include that simulations offer the possibility II. A H YBRID E VALUATION I NFRASTRUCTURE
In this section, we seek to introduce the architecture of
1 http://www.planet-lab.org our hybrid evaluation infrastructure. Herein lightweight client
applications can be attached to simulations running in a central per-vertex on all available processor cores in parallel. The
simulation server in real-time. Actual users can interact with main advantage of using this module is, that the implementor
simulated entities and simulated entities can be programmed benefits from the future availability of additional processing
to react to user interaction. The central component of this cores without the need to explicitly deal with multi-threaded
system is TopGen, an open source simulation framework for programming and synchronization issues. Switching between
graph and network mesoscopics that is being developed in our single-threaded or parallelized code is merely a matter of
group since 2006. TopGen has so far proven to be capable changing the header of a loop iterator. An important aspect
of simulating systems as diverse as router topologies [2], when using multi-threaded simulations is to retain deter-
self-organized monitoring schemes for complex networks [3], minism. For this purpose, transaction-based graph and state
DVE topologies and algorithms [1], [4] as well as bio-inspired abstractions are used. Changes performed by a certain thread
self-synchronization phenomena [5]. The TopGen simulation are not made visible to other threads until all parallel tasks
framework runs on top of the Microsoft .NET or MONO2 started in a specific simulation step have been completed.
runtime environment, the latter being an open source im- The experimentation component provides a framework that
plementation of the ECMA/ISO standard for the Common can be used to implement custom experimentation modules.
Language Infrastructure (CLI). MONO is freely available for It provides distinct simulation modes for modules that (a)
a multitude of operating systems and processor platforms. compute graph and network metrics (e.g. diameter, degree
The basic architecture of TopGen is shown in Figure 1. In distribution, etc.), (b) hook into deterministic, discrete time-
order to maximize platform independence and provide for a step simulations by subscribing to certain events (e.g. vertex
scripted batch operation, the graphical user interface (GUI) of added, vertex removed, simulation step, etc.) or (c) intend to
TopGen has been decoupled from the actual framework. By perform self-paced (non-deterministic) simulations in a self-
this means, a memory-efficient command line version can be managed thread. The experimentation component also provides
started without the GUI portions of the software. Furthermore a framework that can be used to enrich modules with meta-
it rids the core framework as well as the command line information like e.g. which graph types a module should
interface from any particular GUI library dependencies and be allowed to operate on or how result data sets shall be
thus furthers platform independence. interpreted. Based on the CLI’s powerful reflection mechanism,
The main advantage of TopGen is that it offers a generic yet user-implemented experimentation modules can be attached
intuitive approach to network simulations based on so-called or detached to simulations dynamically at run-time without
Experimentation Modules. These modules may contain topol- interrupting running simulations.
ogy generation schemes, metrics computations or simulations Apart from its core components, the TopGen framework
of complex network algorithms and can be freely combined contains a simple model that allows to extend the framework
in order to facilitate a “building blocks”-like definition of by custom graph-based abstractions. These can e.g. be used
complex simulation experiments. Integrated data logging and in own experimentation modules while harnessing the power
plotting facilities as well as a real-time visualization compo- of TopGen’s existing components. So far, extension modules
nent complement the environment’s functionality. implementing abstractions for cellular automata, large Internet
router networks and geometric intersection graphs have been
TopGen GUI implemented. Using a circle as geometric intersection primi-
tive, the latter can e.g. be used to simulate mutual avatar visibil-
Real-Time
Visualization Real-Time
ity in DVEs. Finally the framework is complemented by two
User components managing the composability of experimentation
Data
Interface
Video Monitoring modules as well as the loading and saving of graphs.
Recording
Although all experimentation modules combined to a certain
TopGen Framework
simulation run share a common graph topology, situations
with local and possibly inconsistent per-vertex graphs (as
Experiment e.g. commonly found in P2P networks) can be simulated in
Graph Management
Management different ways. Since TopGen supports directed graphs, it is
simple to represent vertices with differing/inconsistent neigh-
Extensions Experimentation bor lists. Alternatively (e.g. if multiple views with inconsistent
Graph
Cellular Geometric undirected edges are required) modules are free to additionally
Facilities Automata Graph
(Data Logging,
Multicore Support )
maintain and manipulate local per-vertex graphs.
Experimentation Modules From the perspective of a user willing to implement a
Module Module Module
...
custom experimentation module, its interface to the TopGen
A B C
framework is of utmost importance. In section III it will be
described in more detail by presenting an example experi-
Fig. 1. Basic TopGen Architecture mentation module. By this means, the programming model
underlying TopGen will be exemplified.
At the very core of the TopGen framework stand its graph
facilities. In this component, thread-safe abstractions of ba- A. Simulation Mediation
sic graphs, vertices and edges as well as their visualizable So far, only TopGen’s basic simulation facilities have been
counterparts are implemented. In order to facilitate scalable described. Through extending TopGen with a special experi-
experimentation modules, special support to utilize the inherent mentation module, a hybrid setting can be achieved in which
parallelism often found in network algorithms is provided. TopGen additionally acts as a DVE server application. This
A special module allows to submit tasks that are processed special module - the so-called Simulation Mediation module
- provides a network-accessible message sink to which actual
2 http://www.mono-project.com 3D client applications can connect. The module’s main task
is to mediate between incoming messages and simulation TopGen
events to which other experimentation modules can subscribe. Simulated Network Topology
Furthermore, the Mediation module relays incoming messages
to other clients based on the currently simulated network Experimentation Modules
topology. By this means, any simulation implemented in an
Simulation
experimentation module (or a set of modules) can be enriched ... ...
Mediation
by actual clients that interact with the simulation in real-
time. A basic example of such a hybrid simulation which

Network
will be described in more detail in section III is a setting in
which a customizable number of simulated avatars (moving
e.g. according to SecondLife3 avatar trace files) together with
3D Browsing Client
a number of avatars controlled by actual users are used to Bus
simulate traffic from a P2P-based retrieval of 3D contents. A Mediation
network view of such a hybrid scenario is shown in Figure 2.
Message Bus
Here a number of simulated entities (denoted by the letter S)
is enriched with four real clients A, B, C and D which connect Extensions
to TopGen via its Mediation module. Content Avatar Input User
Visualization Visualization Processing Input
Mgmt
Simulated C
S 3D Engine
Network S D
Microsoft XNA Framework
Topology S S
S
A
B
S
S
Fig. 3. Basic Architecture of the 3D Browsing Client

B. 3D Browsing Client
The 3D browsing client is an important component in the
Internet overall evaluation system. It has been implemented on top of
TopGen the Microsoft XNA Framework4 . As described above, its main
Simulation Server purpose is to enable actual users to connect to a simulated
DVE. As such, it might also be considered an external 3D
visualization component which can be used to jump into
a running TopGen DVE simulation. In fact, the main idea
behind the 3D browsing client’s architecture is to provide
for its flexible usage. We tried to achieve this goal by not
implementing a particular 3D browser but rather providing a
A B C D
generic framework in which modules can be used to implement
various browsing applications. For this purpose custom-defined
modules can be used to customize every aspect like e.g. content
Client Client Client Client visualization, generated messages or user controls. Another
important aspect is that the browser shall not contain any
Fig. 2. Network Layout of the Hybrid Evaluation System. Simulated entities code specific to a certain DVE algorithm or topology. By this
in the network topology are labeled with S, while representations of actual means, no redeployment is required when other topologies
clients are labeled with their corresponding letter.
or algorithms needs to be used. Its basic functionality is to
provide an infrastructure that allows a translation of user-input
All messages received by the Simulation Mediation module
to messages sent to TopGen’s Simulation Mediation module as
are translated to graph events and potentially relayed to other
well as providing a 3D visualization of the actual environment
nodes via the simulated network. The decision to which
based on messages relayed by TopGen. The advantage of
other clients a message is relayed can be based on a user-
sending all messages to the central TopGen simulation server
implemented routing strategy. Since TopGen has a notion
is that the relaying of messages can be left to TopGen.
of router-level graphs and contains a module for generat-
By this means, the actual topology by which both real and
ing realistic router distributions, connectivity and latencies,
simulated clients are interconnected can be defined within a
artificial message propagation delays may be induced based
TopGen experimentation module. Due to the dynamism of
on a customizable router-level graph. For this purpose either
TopGen, this module can even be switched at runtime without
virtual overlay nodes can be connected to simulated routers or
having to interrupt the simulation or stop actual clients. Having
precomputed end-to-end latency matrices generated by TopGen
motivated its basic underlying principles, the basic architecture
or other tools can be used. So far, no technical details of the
of the 3D browsing client is shown in Figure 3.
3D browsing application have been given which can be used
to actually connect to and participate in a simulation. In the The so-called Message Bus is the browser’s basic commu-
following section, we intend to provide a better understanding nication abstraction by which all of its remaining components
of its architecture and its underlying principles. are interconnected to each other as well as to the TopGen
3 http://www.secondlife.com 4 http://www.xna.com/
server. All messages on this bus are typed via descriptive XML available from our project’s website 5 .
contracts and categorized into so-called Local and Remotable 1) Epidemic Hoarding: Hot spot regions arising from high
messages. In addition to being delivered locally, a special object densities represent a fundamental problem in distributed
Bus Mediation component automatically serializes Remotable virtual environments. One facet of this problem is the fact, that
messages and forwards them to the Simulation Mediation clients entering hot spot regions may be required to load a
module in a running TopGen simulation via a TCP connection. large body of data within a short time-frame. This problem is
As a general rule, messages that affect other avatars and com- independent from bandwidth available at the content provider.
ponents within the DVE should be defined as Remotable, an The limited bandwidth of the client’s network connection
example for this being messages related to avatar movement or represents a bottleneck that can prevent timely data availability
interaction. The decision whether a message type should be Re- and thus lead to notable load delays. In order to mitigate
motable is however left to the implementor of the type. Locally, this problem, a hot spot aware prefetching scheme has been
messages are transmitted between software components via developed. The basic idea is to constantly use a fixed fraction
the CLR’s efficient eventing mechanism. A special user-input of free network bandwidth for hoarding data selectively from
component translates input like e.g. pressing a key or moving crowded regions in order to mitigate load delays when the
the mouse to messages and transfers them to the Message user actually enters them. The problem with this approach
Bus. Another (customizable) component can subscribe to these is that clients need a constantly updated view of the DVE’s
events, e.g. locally visualize avatar movement and produce current load distribution. In [3], a self-organized and scalable
an appropriate Remotable message to notify nearby actual or solution based on fixed-size random information exchanges
simulated clients. with a single one-hop neighbor has been presented. This can
Regarding the visualization of content, again a customizable be achieved by considering objects and avatars as physical
path has been taken. Rather than predefining content types, particles having a certain mass that reflects the communication
custom modules can be used to visualize any XNA-renderable effort involved when loading the object. Each client can simply
content based on their current state and incoming messages. calculate the cumulative mass and the center of mass within
The source of this content can be chosen by the module, i.e. its own area of awarenesses. This information, along with
either a local resource (in scenarios where content is pre- a list of preliminary ”most crowded regions” based on the
distributed), the payload of a message routed via TopGen locally available information can then be exchanged with a
or an arbitrary network accessible external source. The full random neighbor. Based on the epidemic aggregation algorithm
power of this approach becomes apparent when looking at presented in [6], list entries quickly converge towards the actual
typical scenarios occurring in the DVE research community. global maxima. In [3] it has been shown that this hoarding
Here, often research prototype clients are deployed on a mechanism is able to efficiently mitigate load delays in hot
small scale either for the interested general public, the testing spot regions.
community or collaborating partners. With our centralized 2) Simulating Epidemic Hoarding: Based on the concept
hybrid simulation/testbed approach, such small-scale deploy- that has been described in the previous paragraph, in this
ments (in a range between dozens to a few hundred concurrent section we intend to describe how the scheme can be simulated
users) are simplified in the following ways: Testing different in TopGen. The starting point for the implementation of a Top-
network topologies and overlay topologies does not require Gen experimental module is to extend the abstract framework
a redeployment of the clients or even the interruption of class TopGen.Experiments.ExperimentalModule. Two special
the service or simulation. It is a mere matter of activating methods Start and Stop will be invoked by the framework
another experimentation module in the TopGen simulation when (a) a deterministic, event-based module is attached to or
server. Furthermore, results of an experiment can easily be detached from a simulation or (b) when a non-deterministic
collected and the state of the system can be visualized by a simulation module operating in a custom thread shall start
central instance in real-time. Video-recording facilities that are or stop operation. By assigning framework-defined attributes
integrated into TopGen’s visualization component can even be to the implemented module, the implementor can explicitly
used to record the global behavior and mobility of users. When tell TopGen which kind of threading mode is desired for the
addressing P2P topologies for DVEs, another important issue module. For the epidemic hoarding simulation, we seek to
is the handling of end-user firewalls. Another advantage of utilize the event-based mode which hooks into discrete events
a TopGen-based small-scale deployment of a DVE prototype generated outside the module. This mode’s main advantage is
is the fact that the clients’ only communication channels are being deterministic in the sense that the same random seed will
client-initiated TCP connections. Since the same connections generate exactly the same sequence of events. Accordingly in
are used by the TopGen server’s Simulation Mediation module our case we would assign the attribute
to relay messages from peers, firewall traversal is not an issue.
[ExperimentType(ExperimentType.Event)]
III. A C ASE S TUDY to the module implementation. The signature of the Start
So far the TopGen simulation framework has been de- method that can be implemented to perform custom module
scribed at a mere abstract architectural level. At this point, initialization reads:
the reader is probably interested in a more detailed case study Start(ref Graph g, Settings s, SimulationContext context);
exemplifying the programming model underlying TopGen’s
experimentation modules. For this, we describe a module that In the first argument, a graph is given on which the exper-
has been implemented in the HyperVerse context in order to imentation module can operate. In the case of the epidemic
simulate the speculative hoarding approach to content retrieval hoarding scheme we seek to implement, this is a collection of
that is described in [3]. In order to be able to follow the vertices representing clients along with their avatars’ position
implementation details, the scheme will be briefly described in the DVE. An edge between two vertices represents a con-
in the following paragraph. The source code of this module is
included in the freely accessible TopGen repository which is 5 http://hyperverse.syssoft.uni-trier.de
nection in the underlying network topology. It is not necessary frame by producing tick events in adjustable intervals. This
to define the exact topology in the epidemic hoarding scheme. time frame is the same for all modules attached to the same
The topology can either be implemented in a separate module simulation. Another important task of the SimulationContext
or TopGen’s integrated geometric graph abstraction can be is the provision of a transaction-based storage for arbitrary
used. In the latter case, avatars within a certain visibility range data that can be shared across modules. To retain determinism
are automatically interconnected by the framework. By default, in multi-threaded simulations, changed data are made visible
the graph that is passed to a simulation module can be of any to other modules not before all parallel processing threads
type inheriting the framework’s basic Graph class. Often a started in a certain simulation step have been completed. In
module implementor wishes to restrict the types of graphs on order to perform a computation in each discrete time step of a
which the module can be started to a certain subset. In our case, simulation, modules can subscribe to a special OnTick event.
we need a notion of a geographic position for each vertex. This For the epidemic hoarding module, in each step several actions
can be enforced by assigning the following attribute are performed for each vertex in the graph (and thus DVE
client):
[AcceptedGraphType(typeof(GeometricGraph))]
• A random neighbor is determined with which information
which restricts the application of the module to graph types on cumulative particle mass, center of mass and a list of
that are subclasses of the framework’s basic geometric graph. most-crowded spots is exchanged and aggregated.
To implement modules that initially generate graphs (like • A number of data blocks limited by the client’s per-step
in our case a geometric graph consisting of a customizable bandwidth is moved from the download queue to the
number of randomly placed vertices), one can explicitly allow cache. Blocks in the download queue are prioritized based
TopGen to pass an initial null reference by attaching the on the object’s distance to the client’s current position.
attribute [AcceptsNullGraph(true)]. A module can then create • In case there is remaining bandwidth, the hoarding of
an arbitrary graph and pass the result of this generation back to object data blocks from within predicted hot spot areas is
the environment where downstream modules can use them for simulated in the same way as described above.
simulations. A module for generating arbitrarily sized random • For both previous steps, a per-vertex (least recently used)
geometric graphs is included in TopGen and can be used to caching strategy is used
create the initial graph for our epidemic hoarding simulation. Finally, the Stop method needs to be implemented in order
In the remainder of this section, a vertex in the graph will to unsubscribe from the vertices’ OnMove and the Simulation-
be denoted as simulated client. Depending on a graph’s type, Context’s OnTick events.
the graph instance as well as the vertices provide certain 3) Composability with other modules: So far, the epidemic
events that can be used to react to simulation events. An hoarding scheme has been implemented to operate on a ge-
example for such an event in geometric graphs is the vertices’ ometric graph created outside the module and passed to it
OnMove event, which will be fired whenever the geographic when the simulation starts. It has not yet been described where
position of a vertex changes. The epidemic hoarding scheme either the network topology or the mobility of (simulated)
needs to subscribe to this event in order to update a list of avatars is implemented. At this point, it is important to note
objects in the client’s area of interest whenever the client’s that both is independent from the epidemic hoarding module.
avatar has changed position. Technically, by subscribing to an In a hybrid scenario, both might stem from the Simulation
event, a function pointer (or delegate) is passed to the eventing Mediation module, i.e. real clients joining the system will
infrastructure of the CLI that will be used to invoke a user- trigger the creation of vertices and actual user-input will be
defined handler whenever the event occurs. reflected in vertex mobility. In scenarios with simulated users,
The second argument in the signature of the Start method both needs to be done by other modules that can be attached
is an object containing custom settings for a module. The to the simulation and which share the same SimulationContext.
exact type of this object can be specified by the implementor Simulated mobility can be implemented by modules which
of a simulation module, all user-defined public fields serving - in each simulation step - change the position of each vertex
as simulation parameters that can be dynamically set and of the geometric graph according to a certain model. While
changed via the TopGen GUI. In the case of the epidemic custom models can be implemented in a similar fashion as
hoarding simulation module, parameters contain the number described in the previous paragraph, there are a couple of
of simulated objects, parameters influencing their distribution pre-defined models that can be attached to simulations right
and sizes, the clients’ bandwidths and so forth. Based on these away. In addition to the well-known random waypoint model,
initial settings, modules usually need to perform some kind and a (probably more realistic) model in which vertices move
of initialization before the simulation starts. In the epidemic towards crowded regions with higher probability, TopGen also
hoarding case, this e.g. involves the creation of a number of comes with a module that allows to replay avatar trace files
objects with their corresponding simulated transmission sizes. from SecondLife that have been recorded and made available
In order to simulate data transmissions, the module uses per- to the public in [7].
vertex download queues and caches that can take individual 4) Evaluation: Another important aspect of implementing
object data blocks. The notion of objects, corresponding data simulations in TopGen is data evaluation. For this, the frame-
blocks and the implementation of a caching scheme has been work provides a special component that can be used to log
implemented in a dedicated TopGen extension and can be used result data sets produced by experimentation modules. Logged
by simulations right away. A client’s downlink bandwidth can results can automatically be written to log files or plotted
be simulated on a per-step basis by moving a limited number in real-time. Technically experimentation modules can create
of object data blocks from the download queue of a vertex to custom-defined result structures and post them to the analysis
its cache. component where they will be buffered. In the epidemic
The third and final argument of the Start method is a hoarding example, the module can compute e.g. in each step
reference to the so-called SimulationContext, an object which the number of objects in the field of view for which not all
- amongst other things - provides the simulation’s unique time data blocks reside in the client’s cache, a number representing
perceivable load delays. At the end of the simulation step, the simulate a large back-end system while letting actual users
current result structure can be posted to the environment via a judge the perceived quality of service.
special call. The contained result data fields will be accessed • It allows to easily extract traces of real user mobility
by TopGen via the CLI’s reflection mechanism. and behavior as well as to study the interdependencies
of simulation mobility and user behavior. In this respect,
IV. R ELATED W ORK TopGen provides means to extract and validate new
PlanetLab and G-Lab 6 are research testbeds supporting the mobility and behavioral models.
development of new network technologies such as distributed Finally, in the following we intend to summarize some
storage, network mapping, P2P systems, distributed hash ta- technical and practical advantages of the proposed system.
bles, and query processing. However, just as in simulations,
user behavior can only be simulated since tests run in an • By disabling the possibility to attach real clients, TopGen
unattended fashion. Furthermore, access to those testbeds is can be used in a pure simulation mode with simulations
limited and the deployment is comparably laborious. being deterministic in the sense that any run can be
Our hybrid approach has been inspired by the JAVA-based reproduced exactly. In this mode, TopGen can handle
simulation platform JANE [8], which has been developed in network topologies with as much as several hundred
our group for a past research project. The JANE software can thousand simulated nodes.
be used in a pure simulation mode, in a hybrid setting with • Due to TopGen’s modular architecture, avatars using sim-
real devices being attached to a running simulation and, finally, ulated mobilities based on models (like e.g. random way-
in a setting using real devices only. Developed to support ad- point) can easily be combined with trace-driven avatars.
hoc network researchers in application and protocol design it • TopGen can be run in a pure testbed mode if no simulated
addresses a different application scenario though. Other hybrid entities are used. In this case, extensive logging, tracing
approaches in the domain of sensor networks include e.g. the and evaluation is facilitated by the fact that all clients
system described in [9]. connect to a central machine. When using P2P topologies,
The authors of [10] describe the Generic Visualization messages are routed within a virtual network controlled by
System (GVS), which combines physical simulations and a user-implemented modules. In this case, TopGen acts as
real-time visualization to display results in a realistic 3D gateway to the simulated network and performs routing
environment. For this, it can also merge inputs from different based on the simulated topology. While it is clear that
simulation programs running simultaneously. Developed with the centralized approach sacrifices scalability, for the
a clear focus on simulating military operations and without targeted number of attached real clients we argue that
notion of a customizable underlying network topology, it is the advantages outweigh this disadvantage.
however not applicable in our scenario. • In a hybrid mode, real clients can be combined with sim-
ulated entities (based on mobility models and/or mobility
V. C ONCLUSION traces).
• In all of the above modes, the same implementation of
In this article, we described a hybrid evaluation infrastruc-
ture that combines simulations of a large number of entities algorithms and topologies can be used.
• In all modes, simulated topologies, algorithms as well as
with a small-scale testbed deployment. At this point we must
emphasize that our evaluation system is not intended to be a any dependent parameters can be changed in real-time
scalable DVE hosting solution for large-scale deployments. Al- without interfering with users currently in the system.
• Clients can be deployed without dealing with complex
though we are currently looking into the possibility to support
a larger amount of users by adding cluster support, the current firewall traversal while at the same time utilizing simu-
centralized approach limits the number of connectible actual lated P2P topologies.
• The simulation framework provides support for multi-
clients to typically a few dozens or a few hundreds (depending
on the complexity of the simulated topology, data traffic and processor platforms while retaining the determinism of
used hardware). The main intention of our system is to provide single-threaded simulations by means of a transactional
a rapid prototyping and simulation tool for DVE implementors memory abstraction.
and the scientific community alike in which simulations of We conclude this article by pointing out some open issues.
massive numbers can be enriched with a moderate number of Although all components of the infrastructure (the TopGen
actual clients and vice versa. The main advantages of such a simulation environment, the Simulation Mediation module as
hybrid approach are the following: well as the 3D browsing client) have been implemented and
• It saves implementation efforts since modules imple- tested, an extensive performance evaluation must be considered
mented for simulation can be used to drive actual exper- future work. So far, only TopGen’s scalability on multi core
iments and vice versa. Furthermore, the implementation processors has been evaluated and found to be linear for typical
of topologies for testbeds is simplified by TopGen’s high- network algorithms. Further studies regarding the exact amount
level graph and communication abstractions. of supported users and simulated agents as well as its inter-
• Actual 3D clients can be used to get a first-hand view of dependency with the simulated network topology are required
a simulation in real-time. They might thus also be seen as to prove the suitability in practical scenarios. Regarding our
3D visualization clients that can be attached to a running hybrid approach to DVE evaluations, the interrelations between
simulation, possibly without interfering with it. simulated and real agents as well as their influence on realism
• Especially in the context of DVEs, there are issues that are require further attention.
hard to evaluate by metrics, like e.g. the users’ perception All components of our proposed infrastructure are available
of certain latency characteristics, inconsistent views, load both as source code and binaries from our project’s website.
delays, etc. A hybrid mode provides the possibility to At this stage we would like to encourage the community to
test and evaluate our toolkit. We also actively solicit feedback
6 http://www.german-lab.de and suggestions for its future development.
 R EFERENCES
[1] J. Botev, M. Esch, A. Höhfeld, H. Schloss, and I. Scholtes, “The hyperverse
- concepts for a federated and torrent-based ”3d web”,” The 1st International
Workshop on Massively Multiuser Virtual Environments (MMVE), 2008.
[2] I. Scholtes, J. Botev, M. Esch, A. Hoehfeld, H. Schloss, and B. Zech, “Topgen -
internet router-level topology generation based on technology constraints,” in Pro-
ceedings of the First International Conference on Simulation Tools and Techniques
for Communications, Networks and Systems (SIMUTools), February 2008.
[3] I. Scholtes, J. Botev, M. Esch, H. Schloss, and P. Sturm, “Minimizing load delays
in distributed virtual environments using epidemic hoarding,” in Proceedings of
the 4th International Conference on Collaborative Computing (CollaborateCom),
November 2008.
[4] I. Scholtes, J. Botev, M. Esch, A. Höhfeld, and H. Schloss, “Awareness-driven phase
transitions in very large scale distributed systems,” in Proceedings of the Second
IEEE International Conferences on Self-Adaptive and Self-Organizing Systems
(SaSo). IEEE, 2008.
[5] I. Scholtes, J. Botev, M. Esch, and P. Sturm, “Epidemic self-synchronization in
complex networks,” in Proceedings of the 1st International Conference on Complex
Sciences: Theory and Applications, 2009.
[6] M. Jelasity and A. Montresor, “Epidemic-style proactive aggregationin large
overlay networks,” in Proceedings of The 24th International Conference
on Distributed ComputingSystems (ICDCS 2004). Tokyo, Japan: IEEE
Computer Society, 2004, pp. 102–109. [Online]. Available: citeseer.ist.psu.edu/
jelasity04epidemicstyle.html
[7] H. Liang, I. Tay, M. F. Neo, W. T. Ooi, and M. Motani, “Avatar Mobility
in Networked Virtual Environments: Measurements, Analysis, and Implications”
arXiv:0807:2328v1, 2008.
[8] D. Görgen, H. Frey, and C. Hiedels, “Jane-the java ad hoc network development
environment,” in ANSS ’07: Proceedings of the 40th Annual Simulation Symposium.
Washington, DC, USA: IEEE Computer Society, 2007, pp. 163–176.
[9] S.-H. Lo, J.-H. Ding, S.-J. Hung, J.-W. Tang, W.-L. Tsai, and Y.-C. Chung, “Semu:
A framework of simulation environment for wireless sensor networks with co-
simulation model,” in Advances in Grid and Pervasive Computing. Springer
Berlin / Heidelberg, 2007, pp. 672–677.
[10] C. Holmes, J. Wolff, D. Challou, and P. Huang, “Generic Visualization System: The
Link Between Digital Simulations and a Virtual Environment,” in Virtual Concepts
2005, Biarritz, Frannce, 2005.

View publication stats