Abstract
In data stream applications, streams typically arise from a geographically distributed collection of producers and may be queried by consumers, which may be distributed as well. In such a setting, a query can be seen as a subscription asking to be informed of all tuples that satisfy a specific condition. We propose to support the publishing and querying of distributed data streams by a publish/subscribe architecture.
To enable such a system to scale to a large number of producers and consumers requires the introduction of republishers which collect together data streams and make the merged stream available. If republishers consume from other republishers, a hierarchy of republishers results.
We present a formalism that allows distributed data streams, published by independent stream producers, to be integrated as views on a mediated schema. We use the formalism to develop methods to adapt query plans to changes in the set of available data streams and allow consumers to dynamically change which streams they subscribe to.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Arasu, A., Babcock, B., Babu, S., Cieslewicz, J., Datar, M., Ito, K., Motwani, R., Srivastava, U., Widom, J.: Data-Stream Management: Processing High-Speed Data Streams. In: STREAM: The Stanford Data Stream Management System, Springer, Heidelberg (2005)
Carzaniga, A., Wolf, A.L.: Forwarding in a content-based network. In: Proceedings of ACM SIGCOMM, Karlsruhe (Germany), August 2003, pp. 163–174 (2003)
Cooke, A., Gray, A.J.G., Nutt, W.: Stream integration techniques for grid monitoring. Journal on Data Semantics 2, 136–175 (2005)
Cooke, A.W., Gray, A.J.G., Nutt, W., Magowan, J., Oevers, M., Taylor, P., Cordenonsi, R., Byrom, R., Cornwall, L., Djaoui, A., Field, L., Fisher, S.M., Hicks, S., Leake, J., Middleton, R., Wilson, A., Zhu, X., Podhorszki, N., Coghlan, B., Kenny, S., O’Callaghan, D., Ryan, J.: The relational grid monitoring architecture: Mediating information about the grid. Journal of Grid Computing 2(4), 323–339 (2004)
Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid information services for distributed resource sharing. In: 10th International Symposium on High Performance Distributed Computing, June 2001, pp. 181–194. IEEE Computer Society, Los Alamitos (2001)
Eugster, P.T., Felber, P.A., Guerraoui, R., Kermarrec, A.: The many faces of publish/subscribe. ACM Computing Surveys 35(2), 114–131 (2003)
Fiege, L., Mühl, G., Gärtner, F.C.: A modular approach to build structured event-based systems. In: Proc. 2002 ACM Symposium on Applied Computing, Madrid (Spain), March 2002, pp. 385–392. ACM Press, New York (2002)
Gray, A.J.G., Nutt, W.: Republishers in a publish/subscribe architecture for data streams. In: Jackson, M., Nelson, D., Stirk, S. (eds.) BNCOD 2005. LNCS, vol. 3567, pp. 179–184. Springer, Heidelberg (2005)
Halevy, A.: Answering queries using views: A survey. The VLDB Journal 10(4), 270–294 (2001)
Madden, S.R., Franklin, M.J.: Fjording the stream: An architecture for queries over streaming sensor data. In: Proc. 18th International Conference on Data Engineering, San Jose, CA, USA, pp. 555–566. IEEE Computer Society, Los Alamitos (2002)
Plale, B., Schwan, K.: Dynamic querying of streaming data with the dQUOB system. IEEE Transactions on Parallel and Distributed Systems 14(3), 422–432 (2003)
Stegmaier, B., Kuntschke, R., Kemper, A.: StreamGlobe: Adaptive query processing and optimization in streaming P2P environments. In: Proc. of the 1st International Workshop on Data Management for Sensor Networks, Toronto (Canada), August 2004. VLDB, pp. 88–97 (2004)
Tierney, B., Aydt, R., Gunter, D., Smith, W., Swany, M., Taylor, V., Wolski, R.: A Grid monitoring architecture. In: Global Grid Forum Performance Working Group (January 2002) (revised)
Wiederhold, G.: Mediators in the architecture of future information systems. IEEE Computer 25(3), 38–49 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gray, A.J.G., Nutt, W. (2005). A Data Stream Publish/Subscribe Architecture with Self-adapting Queries. In: Meersman, R., Tari, Z. (eds) On the Move to Meaningful Internet Systems 2005: CoopIS, DOA, and ODBASE. OTM 2005. Lecture Notes in Computer Science, vol 3760. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11575771_27
Download citation
DOI: https://doi.org/10.1007/11575771_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29736-9
Online ISBN: 978-3-540-32116-3
eBook Packages: Computer ScienceComputer Science (R0)