TutorialsProcessing Flows of Information: From Data Stream to Complex Event Processing
Alessandro Margara and Gianpaolo Cugola - Politecnico di Milano
An increasing number of distributed applications requires processing continuously flowing data from geographically distributed sources at unpredictable rate to obtain timely responses to complex queries. Examples of such applications come from the most disparate fields: from wireless sensor networks to financial tickers, from traffic management to click-stream inspection. These requirements led to the development of a number of systems specifically designed to process information as a flow according to a set of pre-deployed processing rules. Despite having a common goal, these systems differ in a wide range of aspects, including architecture, data models, rule languages, and processing mechanisms. In part, this is due to the fact that they were the result of the research efforts of different communities, each one bringing its own view of the problem and its background for the definition of a solution, not to mention its own vocabulary. After several years of research and development we can say that two models emerged and are today competing: the data stream processing model and the complex event processing model. We claim that neither of the two aforementioned models may entirely satisfy the needs of applications, which often require features coming from both worlds. In this tutorial, which is based upon our work "Processing Flows of Information: From Data Stream to Complex Event Processing" recently accepted for publication in ACM Computing Surveys, we draw a general framework to compare the results coming from the different communities, and use it to provide a review of the state of the art in the area. In doing this, we have two main goals: on the one hand, we aim at favoring the communication between different communities, thus reducing the effort to merge the achievements reached so far; on the other hand, we highlight a number of open issue that still need to be addressed in research.
Alessandro Margara is a third year Ph.D. student at the Dipartimento di Elettronica e Informazione (DEI) of the Politecnico di Milano. As part of his Ph.D. research, he is currently exploring solutions for bringing Complex Event Processing (CEP) to large scale scenarios, with focus on language abstractions, processing techniques, and communication protocols. His main research interests are in distributed systems, and more specifically in the area of event-based middleware, but also in parallel systems for high performance computing and in programming languages and abstractions.
Gianpaolo Cugola is Associate Professor at Politecnico di Milano where he teaches several courses in the area of Computer Science. In 1998 he received the Prize for Engineering and Technology for his Ph.D. thesis on Software Development Environments, while in 2007 his paper on “The JEDI event-based infrastructure and its application to the development of the OPSS WFMS” was awarded as “Most cited Software Engineering Paper in 2001”. He has been involved in several projects financed by the EU commission and by the Italian government. He is co-author of tens of scientific papers published in international journals and conference proceedings. His research interests are in the area of Software Engineering and Distributed Systems. In particular, his current research focuses on middleware technology for largely distributed and highly reconfigurable distributed applications, with a special attention to the issue of Content Based Routing and Complex Event Processing.
Non-functional properties of event processing
Opher Etzion, Ella Rabinovich, and Inna Skarbovsky - IBM Haifa
An important aspect of any system is the non-functional aspect, which is not concerned with what a system does, but rather with how well a system works. Since event processing applications are not monolithic, there are various considerations which vary among these applications. In this tutorial we survey the various non-functional properties of event processing systems and concentrate on the areas of: scalability and performance, availability, correctness and security. The tutorial is presented along three aspects for each topic: requirements, mapping of requirements to event processing application type and current state-of-the-art and research efforts to cope with the various requirements.
Opher Etzion Opher Etzion is IBM Senior Technical Staff Member and Event Processing Scientific Leader in IBM Haifa Research Lab, Previously he has been lead architect of event processing technology in IBM Websphere, and a Senior Manager in IBM Research division, managed a department that has performed one of the pioneering projects that shaped the area of "complex event processing". He is also the chair of EPTS (Event Processing Technical Society). In parallel he is also an adjunct faculty member in the rank of professor at the Technion - Israel Institute of Technology. He has authored or co-authored around 80 papers in refereed journals and conferences, on topics related to: active databases, temporal databases, rule-base systems, complex event processing and autonomic computing, he is completing a book on event processing called: Event Processing in Action (with Peter Niblett) and co-authored the book "Temporal Database - Research and Practice", Springer-Verlag, 1998. Prior to joining IBM in 1997, he has been a faculty member and Founding Head of the Information Systems Engineering department at the Technion, and held professional and managerial positions in industry and in the Israel Air-Force. He is a senior member of ACM. He has supervised 6 PhD and 19 MSc students.
Ella Rabinovich is a Research Staff Member in the Event-Based Systems department at the IBM Haifa Research Lab. She received a B.Sc. degree in Information Systems Engineering from Technion, Israel Institute of Technology, in 2006, and is currently a M.Sc. student in Information Management Engineering. Her past experience includes development of tools for verification of hardware systems, and her current research focuses on event-based technologies.
Inna Skarbovsky is a Research Staff Member in the Event-based systems group at the IBM Haifa Research Lab. Her current research focuses on event-based technologies. Her past experience includes development of high-availability high-performance distributed applications, and current interests include distributed applications, cloud computing and SOA.
Hybrid programming abstraction for e-science workflows and Complex event processing
Beth Plale and Chathura Herath - Indiana University Bloomington
The scientific event processing for research and development adds extra dimension to the complexities associated with scientific computing. In achieving this, experience gained building scientific computing infrastructures that could span and scale into super computing resources can be reused in significant ways. But there are many unresolved issues relating to managing event streams in such an environment that require attention. Over the years the volumes of events generated in scientific disciplines have steadily grown. The limiting factor of many of these systems have become the time and attention of scientist and with expertise to derive insight out of the high volumes of events generated by the sensors. In this tutorial we propose to share the motivating use cases, research issues and outcomes and tools and frameworks used for event processing in science gateways in conjunction with complex event processing. We would provide hands on experience to the programming model, framework and tools that had evolved as a result of research over the years and relate how the scientific workflow based programming paradigm can provide a cleaner abstraction for query based Complex event processing systems.
Professor Beth Plale is an experimental computer scientist whose research interests are in data management, data-driven computing, and scientific data preservation. Plale frequently engages in research problems involving interdisciplinary collaborations with the earth and atmospheric sciences, and in digital humanities. Plale's current research areas are data provenance and metadata, digital preservation of complex scientific data, service-oriented architectures, workflow systems in grids and clouds, and complex events processing in environmental applications.
Chathura Herath is a PhD candidate(ABD) in Center for Data and Search Informatics with thesis focus on programming model for distributed event processing in scientific computing. His area of research include e-science, complex event processing, grid/cloud computing involving inter disciplinary collaboration.
Grand Challenge: The Global Event Processing Fabric and its Applications
Pedro Bizarro - University of Coimbra
Nenan Stojanovic - FZI Research Center for Information Technology
Mani Chandy - California Institute of Technology (Caltech)
This tutorial introduces a grand challenge that serves as a common goal and mechanism for coordinating research across the spectrum of people working on event processing. Grand Challenge is a single, though broad, challenge that impacts society which the community can use as a basis for measuring progress. Tutorial road map: we start by describing the event processing fabric grand challenge and provide the basic building blocks for its efficient realization. Thereafter we list life or society changing applications which utilize this fabric. Finally, we will present some ongoing work in this area. This tutorial is a part of the outcomes of the working group for Grand Challenge established at Event Processing Dagstuhl Seminar 2010.
Pedro Bizarro is an Assistant Professor at the University of Coimbra and founder and Chief Scientist Officer at FeedZai. Pedro worked and is interested on adaptive query processing, data stream systems, event processing, benchmarking and performance of streaming systems, and virtualization. Pedro, a Fulbright Fellow (2001-2006), is also an Associate Visiting Teaching Professor at Carnegie Mellon University and a consultant for the European Commission.
Nenad Stojanovic is the project leader in IPE department at FZI. He published more than 70 technical papers in international journals, conferences, and workshops in the areas of applications of ontologies and machine learning. He is initiator and co-chair of several workshop serials in the area of Semantic Web and Complex Event processing on prominent research conferences. He was co-organizer of the AAAI 2009 Spring Symposia on Intelligent Event Processing. He is one of co-leaders of the Business Value of Event Processing working group in EPTS.
Professor K. Mani Chandy is the Simon Ramo Professor and Deputy Chair of Engineering and Applied Sciences at the California Institute of Technology in Pasadena, California. His Bachelors is from the Indian Institute of Technology, Madras and PhD from the Massachusetts Institute of Technology in Electrical Engineering. He was a professor at the University of Texas at Austin from 1970 to 1987, and has been at Caltech since then. He is a member of the U.S. National Academy of Engineering and has received several awards. He does research on distributed systems and systems that detect events in the environment and respond to them. He works on applications dealing with earthquakes, radiation detection, healthcare for the disadvantaged, and the smart grid.
Architectural and Functional Design Patterns for Event Processing
Paul Vincent - TIBCO Software Ltd
Alexandre Alves - Oracle Corporation
Catherine Moxey - IBM
Adrian Paschke - FU Berlin
The Tutorial introduces the EPTS Reference Architecture and its components, and covers some of the Reference Architecture subfunctions as design patterns against some implementation languages; for example:
- Filtering
- Enrichment
- Aggregation
- Routing
Paul Vincent is the fellow Co-chair of the EPTS Reference Architecture Working Group, Member of the British Computer Society, co-chair of OMG PRR rules standard, and CTO for CEP and Business Rules at TIBCO Software. He holds an MSc in Intelligence Systems, and has presented CEP and rule tutorials and presentations at various events over the past years (OMG Workshops, BRForum, Semantic Tech Conference, DAMA, etc) as well as providing customer training and guidance for event processing users in Fortune 100 organizations.
Alexandre Castro Alves is a computer scientist, who has worked in the past decade on middleware; particularly around network management, CORBA, real-time distributed systems, J2EE, Web-Services, BPEL, OSGi, and most recently on Complex Event Processing (CEP). He is a co-author of the WS-BPEL 2.0 specification, a member of the steering committee for the Event Processing Technical Society (EPTS), the author of the book 'OSGi in Depth', and holds several patents around BPEL and event processing. He is currently employed by Oracle, previously having worked for BEA Systems as the architect for the product WebLogic Event Server, re-branded Oracle CEP.
Catherine Moxey is an IBM Senior Technical Staff Member in CICS Strategy and Planning, based at IBM Hursley near Winchester. Catherine has more than 20 years' development experience with IBM, primarily in CICS and System z but also in WebSphere Application Server web services technologies. She is the architect for the Event Processing support in CICS Transaction Server, and an active member of the Event Processing Technical Society. Catherine has written a number of papers on topics including CICS and event processing, and frequently speaks at conferences around the world.
Professor Adrian Paschke is head of the Corporate Semantic Web chair (AG-CSW) at the institute of computer science at the Freie Universität Berlin (FUB). He is director of RuleML Inc., vice director of the Semantics Technologies Institute Berlin (STI Berlin) and leads the Berlin Semantic Web Meetup Group. He is Steering-Committee Chair of the RuleML Web Rule Standardization Initiative (RuleML), co-chair of the Reaction RuleML technical group, founding member of the Event Processing Technology Society (EPTS), co-chair of the EPTS Reference Architecture working group (EPTS RA), voting member of OMG, and active member of several W3C groups such as the W3C Health Care and Life Sciences group (W3C HCLS) and the W3C Rule Interchange Format working group (W3C RIF), where he is editor of the W3C RIF standard and is hosting the W3C HCLS KB in Berlin.
