Distributed Processing

This paper presents the design philosophy and initial design decisions of Herald: a highly scalable global event notification system that is being designed and built at Microsoft Research. Herald is a distributed system designed to transparently scale in all respects, including numbers of subscribers and publishers, numbers of event subscription points, and event delivery rates. Event delivery can occur within a single machine, within a local network or Intranet, and throughout the Internet. Herald tries to take into account the lessons learned from the successes of both the Internet and the Web. Most notably, Herald is being designed, like the Internet, to operate correctly in the presence of numerous broken and disconnected components. The Herald service will be constructed as a set of protocols governing a federation of machines within cooperating but mutually suspicious domains of trust. Like the Web, Herald will try to avoid, to the extent possible, the maintenance of globally consistent state and will make failures part of the clientvisible interface.

The growing design complexity of today's embedded real-time systems requires new techniques aiming the raising of the abstraction level since earlier stages of design in order to deal with such complexity in a suitable way. This paper reports a case study, which provides an assessment of two well-know highlevel paradigms, namely Aspect-(AO) and Object-Oriented (OO) paradigms. Concepts of both paradigms were applied at modeling phase of a Distributed Embedded Real-Time System (DERTS). The handling of DERTS' functional and non-functional requirements (at modeling level) using AO and OO concepts is discussed. Both paradigms are compared using of a set of software engineering metrics, which were adapted to be applied at modeling level. The presented results show the suitability of each paradigm for DERTS specification in terms of reusability quality of model elements.

Introduction The purpose of this paper is to serve as an introduction to neural networks and to sug-gest that neural network architecture can be used for design and application of intelligent control systems [ 11. The neural network ap-proach is introduced by defining the problem ...

The importance of graphs as the fundamental structure underpinning many real world applications is no longer to be proved. Large graphs have emerged in various fields such as biological, social and transportation networks. The large volume of the network poses challenges to traditional techniques for storage and analysis of graph data. In particular, OLAP (On-Line Analytical Processing) analysis requires access to large portions of data to extract key information and to feed strategic decision making. OLAP provides multilevel, multiperspective views of the data. Most of current techniques are optimized for centralized graph processing. A distributed approach providing horizontal scalability is required in order to handle the analysis workload. In this paper, we focus on applying OLAP analysis on large, distributed graph data. We describe Distributed Graph Cube, our distributed framework for graph-based OLAP cubes computation and aggregation. Experimental results on large real-world datasets demonstrate that our method significantly outperforms its centralized counterparts. We also evaluate the performance of both Hadoop and Spark for distributed cubes computations.

We introduce the continuous convergence protocol for handling locally committed and possibly concurrent updates to replicated data objects in distributed real-time databases. The protocol is designed for systems where replicated database objects are used to share data between collaborating real-time applications. We define the properties of the protocol, and describe how it can be implemented using data structures such as version vectors and conflict sets. We also show how applications can exploit the protocol's properties, and how the protocol supports forward resolution of update conflicts.

new distributed algorithm is presented for dynamically determining weighted shortest paths used for message routing in computer networks. The major features of the algorithm are that the paths .defined do not form transient loops when weights change and the number of steps required to find new shortest paths when network links fail is less than for previous algorithms. Specifically, the worst case recovery time is proportional to the largest number of hops h in any of the weighted shortest paths. For previous loop-free distributed algorithms this recovery time is proportional to h2.

With the increasing complexity of multiprocessor and distributed processing systems, the need to develop efficient and accurate modeling methods is evident. Fault tolerance and degradable performance of such systems has given rise to considerable interest in models for the combined evaluation of performance and reliability [l], [2]. Markov or semi-Markov reward models can be used to evaluate the effectiveness of degradable fault-tolerant systems. Beaudry [l] proposed a simple method for computing the distribution of performability in a Markov reward process. We present two extensions of Beaudry's approach. First, we generalize the method to a semi-Markov reward process. Second, we remove the restriction requiring the association of zero reward to absorbing states only. We illustrate the use of the approach with three interesting applications.

A framework for measuring the performance of distributed programs is presented. This framework includes a model of distributed programs, a description of the measurement principles and methods, and a guideline for implementing these ideas. The author describes a measurement system called the Distributed Programs Monitor (DPM), which he has constructed on the basis of these concepts. DPM has been implemented and used for measurement studies on two different operating systems, DEMOS/MP and Berkeley Unix.

The technologies, architectures, and methodologies traditionally used to develop distributed applications exhibit a variety of limitations and drawbacks when applied to large scale distributed settings (e.g., the Internet). In particular, they fail in providing the desired degree of configurability, scalability, and customizability. To address these issues, researchers are investigating a variety of innovative approaches. The most promising and intriguing ones are those based on the ability of moving code across the nodes of a network, exploiting the notion of mobile code. As an emerging research field, code mobility is generating a growing body of scientific literature and industrial developments. Nevertheless, the field is still characterized by the lack of a sound and comprehensive body of concepts and terms. As a consequence, it is rather difficult to understand, assess, and compare the existing approaches. In turn, this limits our ability to fully exploit them in practice, and to further promote the research work on mobile code. Indeed, a significant symptom of this situation is the lack of a commonly accepted and sound definition of the term "mobile code" itself.

Distributed computing environments have evolved from in-house clusters to Grids and now Cloud platforms. We, as others, provide HPC benchmarks results over Amazon EC2 that show a lower performance of Cloud resources compared to private resources. So, it is not yet clear how much of impact Clouds will have in high performance computing (HPC). But hybrid Grid/Cloud computing may offer opportunities to increase overall applications performance, while benefiting from in-house computational resources extending them by Cloud ones only whenever needed. In this paper, we advocate the usage of ProActive, a well established middleware in the grid community, for mixed Grid/Cloud computing, extended with features to address Grid/Cloud issues with little or no effort for application developers. We also introduce a framework, developed in the context of the DiscoGrid project, based upon the ProActive middleware to couple HPC domain-decomposition SPMD applications in heterogeneous multi-domain environments. Performance results coupling Grid and Cloud resources for the execution of such kind of highly communicating and processing intensive applications have shown an overhead of about 15%, which is a non-negligible value, but lower enough to consider using such environments to achieve a better cost-performance trade-off than using exclusively Cloud resources.

The Reference Model of Open Distributed Processing (RM -ODP) was a joint effort by the international standards bodies ISO and ITU-T to develop a coordinating framework for the standardisation of open distributed p~ocessing (ODP). The ~odel describes an architecture within which support of distribution, interworking, interoperability and portability can be integrated. The RM-ODP framework defines ODP concerns using five "viewpoints" (abstractions), namely enterprise, 'information, computational, engineering, and technology. This tutorial introduces the reference model, describing the viewpoints and some of the ODP functions and transparencies. 1.1. The Goals and Deliverables of RM-ODP RM-ODP aims to achieve: portability of applications across heterogeneous platforms interworking between ODP systems, i.e. meaningful exchange of information and convenient use of functionality throughout the distributed system distribution transparency, i.e. hide the consequences of distribution from both the applications programmer and user K. Raymond et al. (eds.), Open Distributed Processing

Efficient operation of a submillimeter interferometer requires remote (preferably automated) control of mechanically tuned local oscillators, phase-lock loops, mixers, optics, calibration vanes and cryostats. The present control system for these aspects of the Submillimeter Array (SMA) will be described. Distributed processing forms the underlying architecture. In each antenna cabin, a serial network of up to ten independent 80C196 microcontroller boards attaches to the real-time PowerPC computer (running LynxOS). A multi-threaded, gcccompiled program on the PowerPC accepts top-level requests via remote procedure calls (RPC), subsequently dispatches tuning commands to the relevant microcontrollers, and regularly reports the system status to opticalfiber-based reflective memory for common access by the telescope monitor and error reporting system. All serial communication occurs asynchronously via encoded, variable-length packets. The microcontrollers respond to the requested commands and queries by accessing non-volatile, rewriteable lookup-tables (when appropriate) and executing embedded software that operates additional electronic devices (DACs, ADCs, etc.). Since various receiver hardware components require linear or rotary motion, each microcontroller also implements a position servo via a one-millisecond interrupt service routine which drives a DC-motor/encoder combination that remains standard across each subsystem.

Abstract This paper introduces a multi-agent framework for distributed multiplayer game development and simulations. The framework allows the incorporation and experimentation with different strategies of the Players, possibly including intelligent behaviors. The framework consists of a set of generic agent types that must be specialized and configured for a specific game. The framework uses a generic agent that enforces the rules of the game, as well as generic bot-agents that represent the game Players. To support our claims of ...

An intrusion-tolerant distributed system is a system which is designed so that any intrusion into apart of the system will not endanger confidentiality, integrity and availability. This approach is suitable for distributed systems, because distribution enables isolation of elements so that an intrusion gives physical access to only a part of the system. By intrusion, we mean not only computer breakins by non-registered people, but also attempts by registered users to exceed or to abuse their privileges. In particular, possible malice of security administrators is taken into account. This paper describes how some functions of distributed systems can be designed to tolerate intrusions, in particular security functions such as user authentication and authorization, and application functions such as file management.

Regression testing is expensive and may consume much of organizations' software development budgets. Thus, it is of interest to reduce the total time devoted to test execution by using test selection techniques. Many techniques have been proposed but few have been evaluated on real-world, large scale systems. In this paper we report on an empirical evaluation of using the class firewall regression test selection technique, in combination with scenario testing, on a large scale industrial software system using the Java byte code in the analysis. The study was performed on a large complex distributed software system in one of Sweden's largest banks. Effects of using scenario testing together with regression test selection are reported. The results are that not all test cases were selected by the class firewall selection technique. Using scenario testing, where test cases are dependent, affects the number of test cases selected, as do the location and number of changes in the system.

The Bsoft package [Heymann, J.B., Belnap, D.M., 2007. Bsoft: image processing and molecular modeling for electron microscopy. J. Struct. Biol. 157, 3-18] has been enhanced by adding utilities for processing electron tomographic (ET) data; in particular, cryo-ET data characterized by low contrast and high noise. To handle the high computational load efficiently, a workflow was developed, based on the database-like parameter handling in Bsoft, aimed at minimizing user interaction and facilitating automation. To the same end, scripting elements distribute the processing among multiple processors on the same or different computers. The resolution of a tomogram depends on the precision of projection alignment, which is usually based on pinpointing fiducial markers (electron-dense gold particles). Alignment requires accurate specification of the tilt axis, and our protocol includes a procedure for determining it to adequate accuracy. Refinement of projection alignment provides information that allows assessment of its precision, as well as projection quality control. We implemented a reciprocal space algorithm that affords an alternative to back-projection or real space algorithms for calculating tomograms. Resources are also included that allow resolution assessment by cross-validation (NLOO2D); denoising and interpretation; and the extraction, mutual alignment, and averaging of tomographic sub-volumes.

The paper presents a semantic logging framework which allows structured information logging in an agent-based distributed system for chemical incident response. The logging framework is "semantic" because it allows semantic interpretation of logs according to relationships defined between logging events. For example, this approach could help the reconstruction of the order of events that occurred during the response to an incident, thus giving a detailed view of system execution trace, as well as of agents' decisions taken at various decision points during the incident management workflow. We intend to use semantic logs (i) for helping experts to analyze and explain system actions and thus improving system response to future incidents, as well as (ii) for training stakeholders by setting the system to run replaylike simulations of past incident management workflows. Sorin Ilie, Mihnea Scafeş, Costin Bȃdicȃ, Thomas Neidhart, and Rani Pinchuk were supported by DIADEM project. Diadem project is funded by European Union under Information and Communication Technologies (ICT) theme of the 7th Framework Programme for R&D, ref. no: 224318. 1. DIADEM project -Distributed information acquisition and decisionmaking for environmental management: http://www.ist-diadem.eu/ -265 -978-1-4244-7433-2/10/$26.00 ©2010 IEEE

The Telecommunications Information Networking Architecture Consortium (TINA-C) is a worldwide consortium formed by network operators, and telecommunication and computer equipment suppliers. The consortium is aiming at defining and validating an "open" architecture for telecommunications services in the emerging broadband, multi-media and "information super-highway"/"information society" era. The architecture is based on distributed computing, object orientation, and other concepts and standards from the telecommunications and computing industries, e.g., Open Distributed Processing (ODP), Intelligent Networks (IN), Telecommunication Management Networks (TMN), Asynchronous Transfer Mode (ATM), and Common Object Request Broker Architecture (CORBA). The intention of this paper is to give an introduction to, and an overview of the TINA architecture. First, it presents an overall view of the TINA architecture, followed by a brief introduction to the main parts of the architecture, namely the computing, network, service, and management architectures of TINA. This is followed by a brief overview of interworking and migration scenarios for legacy systems. Finally, an overview of a tool-set to aid service specification and design is presented.

The newly developed Big Data oriented distributed systems such as Apache HBase have been proven effective in storing and analyzing the exponentially growing volume of wide variety of data such as sensors data, customer generated media, web logs, and so forth. In this paper, Apache HBase, a distributed scalable big data store, is used to store, process, and analyze a large amount of spatiotemporal data generated by shipboard AIS transponders. The objective is to predict the behavior of ships navigating through heavily trafficked fairways around the gates of busy harbors. For that purpose, experiments were conducted using tens of gigabytes of real world AIS data. The data were processed to form historical ships' tracks and were classified based on ships attributes such as type, draught, voyage destination and country of origin. Finally, a simple algorithm was implemented to predict the target ships behavior based on its attributes and movement characteristic. As a result, an acceptable prediction of ships movement is achieved. Furthermore, the experimental result also indicated that in the case of data processing speed, this technique remarkably outperformed the traditional GIS application software.

Networking together hundreds or thousands of cheap microsensor nodes allows users to accurately monitor a remote environment by intelligently combining the data from the individual nodes. These networks require robust wireless communication protocols that are energy efficient and provide low latency. In this paper, we develop and analyze low-energy adaptive clustering hierarchy (LEACH), a protocol architecture for microsensor networks that combines the ideas of energy-efficient cluster-based routing and media access together with application-specific data aggregation to achieve good performance in terms of system lifetime, latency, and application-perceived quality. LEACH includes a new, distributed cluster formation technique that enables self-organization of large numbers of nodes, algorithms for adapting clusters and rotating cluster head positions to evenly distribute the energy load among all the nodes, and techniques to enable distributed signal processing to save communication resources. Our results show that LEACH can improve system lifetime by an order of magnitude compared with general-purpose multihop approaches.

The Adaptive Safety Analysis and Monitoring (ASAM) system is a hybrid model-based software tool for assisting intelligence analysts to identify terrorist threats, to predict possible evolution of the terrorist activities, and to suggest strategies for countering terrorism. The ASAM system provides a distributed processing structure for gathering, sharing, understanding, and using information to assess and predict terrorist network states. In combination with counter-terrorist network models, it can also suggest feasible actions to inhibit potential terrorist threats. In this paper, we will introduce the architecture of the ASAM system, and discuss the hybrid modeling approach embedded in it, viz., Hidden Markov Models (HMMs) to detect and provide soft evidence on the states of terrorist network nodes based on partial and imperfect observations, and Bayesian networks (BNs) to integrate soft evidence from multiple HMMs. The functionality of the ASAM system is illustrated by way of application to the Indian Airlines Hijacking, as modeled from open sources.

Complex event processing has become increasingly important in modern applications, ranging from supply chain management for RFID tracking to real-time intrusion detection. The goal is to extract patterns from such event streams in order to make informed decisions in real-time. However, networking latencies and even machine failure may cause events to arrive out-of-order at the event stream processing engine. In this work, we address the problem of processing event pattern queries specified over event streams that may contain out-of-order data. First, we analyze the problems state-of-the-art event stream processing technology would experience when faced with out-of-order data arrival. We then propose a new solution of physical implementation strategies for the core stream algebra operators such as sequence scan and pattern construction, including stackbased data structures and associated purge algorithms. Optimizations for sequence scan and construction as well as state purging to minimize CPU cost and memory consumption are also introduced. Lastly, we conduct an experimental study demonstrating the effectiveness of our approach.

The Beowulf Distributed Process Space (BProc) is a set of Linux kernel modifications which provides a single system image and process migration facilities for processes running in a Beowulf style cluster. With BProc, all the processes running in a cluster are visible on the cluster front end machine and are controllable via existing UNIX process control mechanisms. Process creation is done on the front end machine and the processes are placed on the nodes where they will run with BProc's process migration mechanism.

Moment distribution continues to be a valuable structural analysis tool for spot-checking member-endmoments of continuous beams and frame structures from computer results. Prior to the widespread use of mainframe and, later, personal computers in the 1980s and 1990s, moment distribution provided a simple hand method for practicing civil engineers to analyze and design statically indeterminate structures, since its public introduction in the early 1930s by Hardy Cross, directly leading to new types of designs and construction. The chief benefits of this method of analysis over the other available methods are that (1) member-end-moments are found directly from the applied loads without first determining various displacement components, (2) no simultaneous equations need to be formed and solved, (3) it is easy to remember and apply, (4) an approximate solution can be obtained at any stage in the analysis and (5) a remarkable variety of statically indeterminate structures can be solved by hand. While the development and introduction of moment distribution represents a great advancement in the history of structural engineering, the distribution process can be tedious, especially if only a few final member-end-moments of a multi-redundant structure are desired. Additionally, when results are wanted to a greater accuracy, the number of distribution cycles required and the number of significant figures that must be carried through the process make it a daunting procedure. In this paper, a new method is presented that gives exact member-end-moments for continuous beams and bridge structures, without the need to distribute moments back and forth as in moment distribution or to set up and solve simultaneous equations, as with the stiffness method. It is anticipated that the primary application of the proposed method will be to spot-check computer results.

The authors present the design and implementation of an asynchronous remote operation execution facility, futures, that retains the benefits of the remote procedure call (RPC) abstraction but allows execution to proceed locally in parallel with remote execution and provides extensive support for managing replies. It is shown how this facility can be easily used to support many common interprocess communication

Distributed work is an increasingly common phenomenon in a number of technical and professional settings, and the complexity of this work requires high degrees of knowledge sharing and integration that move beyond assembly-line approaches to collaboration. Since participants in distributed-work settings rely almost exclusively on written and spoken language to mediate their collaborative relationships, professional communication faculty need educational approaches that empower students with language practices designed specifically to support effective teaming in these complex environments. To address this need, we employ discourse analysis and Speech Act Theory to identify these language practices in a case study of two cohorts of distributed, interdisciplinary, and cross-cultural student teams. The findings show correlations between language practices and successful collaboration. These correlations have significant implications for teaching and practice.

Integrated Modular Avionics (IMA) architectures have been defined for sharing communication and computation resources. The aim of this paper is to evaluate temporal consistency properties of functions implemented on IMA platforms. More specifically, the two contributions are : (1) a modeling approach for IMA platforms based on the tagged signal model and an abstraction of the network, (2) the definition of two evaluation methods for temporal consistency properties. The industrial applicability of the method is demonstrated on an Airbus A380-like platform. We also discuss the significance of the over-approximations induced by the network abstraction.

In recent years, the leasing concept has become increasingly popular in the field of distributed systems; main examples are JINI and the introduction of leasing to the CORBA context. Nevertheless, no detailed analysis of leasing variants and their fields of application has been done yet. In this paper, we give a systematic classification of possible leasing variants, discuss their advantages and disadvantages and point out their possible uses. In this context, we dispose of the restriction that resource claimants are not capable of waiting for a resource; an assumption made in hitherto literature for simplicity reasons. Furthermore, we consider possible mechanisms on which to base lease renewal decisions that go beyond the simple rules examined in previous works. In the process, we hint at how existing leasing approaches have to be enhanced to accomplish these relaxations.

Cloud computing is an emerging technology that allows users to utilize on-demand computation, storage, data and services from around the world. However, Cloud service providers charge users for these services. Specifically, to access data from their globally distributed storage edge servers, providers charge users depending on the user's location and the amount of data transferred. When deploying data-intensive applications in a Cloud computing environment, optimizing the cost of transferring data to and from these edge servers is a priority, as data play the dominant role in the application's execution. In this paper, we formulate a non-linear programming model to minimize the data retrieval and execution cost of data-intensive workflows in Clouds. Our model retrieves data from Cloud storage resources such that the amount of data transferred is inversely proportional to the communication cost. We take an example of an 'intrusion detection' application workflow, where the data logs are made available from globally distributed Cloud storage servers. We construct the application as a workflow and experiment with Cloud based storage and compute resources. We compare the cost of multiple executions of the workflow given by a solution of our non-linear program against that given by Amazon CloudFront's 'nearest' single data source selection. Our results show a savings of three-quarters of total cost using our model.

This paper presents important issues on the design and implementation of a Multimedia Digital Classroom (MDC) system with its applications. The MDC system is implemented in Java language. It aims at providing important tools to support a distributed interactive ...

Resumen. Este trabajo se halla enmarcado dentro de la teoría del procesamiento distribuido y la programación paralela, conjuntamente con técnicas basadas en los conceptos de dual, polar u ortogonal, que se asocian a inecuaciones lineales y pertenecen a las teorías de polaridad. En este trabajo se define cómo pueden ser aplicados los conjuntos ortogonales para resolver problemas matriciales, estos problemas son abordados desde una perspectiva puramente algebraica sin considerar los aspectos relativos al análisis numérico como la estabilidad, la aplicabilidad a matrices dispersas, la elección numérica de pivotes, etc.

Matchmaker, a language used to specify and automate the generation of interprocess communication interfaces, is presented. The process of and reasons for the evolution of Matchmaker are described. Performance and usage statistics are presented. Comparisons are made between Matchmaker and other related systems. Possible future directions are examined This paper Abstract Matchmaker, a language used to specify and automate the generation of interprocess communication interfaces, is presented. The process of and reasons for the evolution of Matchmaker are described. Performance and usage statistics are presented. Comparisons are made between Matchmaker and other related systems. Possible future directions are examined.

Today’s real-time systems (RTS) are characterized by managing large volumes of dispersed data making real-time distributed data processing a reality. Large business houses need to do distributed processing for many reasons, and they often must do it in order to stay competitive. So, efficient database management algorithms and protocols for accessing and manipulating data are required to satisfy timing constraints of supported applications. Therefore, new research in distributed real-time database systems (DRTDBS) is needed to investigate possible ways of applying database systems technology to real-time systems. This paper first discusses the performance issues that are important to DRTDBS, and then surveys the research that has been done so far on the issues like priority assignment policy, commit protocols and optimizing the use of memory in non-replicated/replicated environment pertaining to distributed real time transaction processing. In fact, this study provides a foundation for addressing performance issues important for the management of very large real time data and pointer to other publications in journals and conference proceedings for further investigation of unanswered research questions.

Abstract–In Cellular networks, number of users increase in an exponential manner. As a result of this exponential growth of users, overloading of the system takes place. Besides this, unacceptable delays and high computational cost create a strong prerequisite for efficient location management techniques. Motivated by this fact, a novel user profile based (UPB) scheme is proposed in this paper to track the location of the mobile users.

The 21st Century has transformed the world of science by breaking the physical boundaries of distributed organizations and inter-connecting them into virtual science environments, allowing for systems and systems of systems to seamlessly access and share information and resources across highly geographically distributed areas. This e-science transformation is enabling new scientific discoveries by allowing for greater collaboration as well as by enabling systems to combine and correlate disparate data sets. At the Jet Propulsion Laboratory in Pasadena, California, we have been developing science data systems for highly distributed communities in physical and life sciences that require extensive sharing of distributed services and common information models based on common architectures. The common architecture contributes a set of atomic functions, interfaces and information models that support sharing and distributed processing. Additionally, the architecture provides a blueprint for a software product line known as the Object Oriented Data Technology (OODT) framework. OODT has enabled reuse of software for science data generation, capture and management, and delivery across highly distributed organizations for planetary science, earth science and cancer research. Our experience to date shows that a well-defined architecture and set of accompanied software vastly improves our ability to develop roadmaps for and to construct virtual science environments.

Business rules provide an elegant solution to manage dynamic business logic by separating business knowledge from its implementation logic. The drawback of most existing business rule approaches is the lack of standardization and interoperability. The lack of service-orientation and remote accessibility of business rule engines makes it hard to use business rules in distributed environments. This paper contributes the design and implementation of VIDRE (Vienna Distributed Rules Engine), a service-oriented business rule engine based on RuleML. VIDRE enables enterprise applications to access business rules as easy as accessing a database, by exposing rules as Web services. VIDRE uses RuleML as an interlingua to represent facts, rules, and queries. One of the main contributions of the VIDRE approach is the ability to distribute rules and facts across various rule engines, therefore, enabling powerful ways of separating and executing business rules within intra-and interorganizational boundaries.