Counting 3,834 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Author
Last Commit
May. 22, 2019
Created
Apr. 21, 2014

Siddhi - Cloud native stream processing

Jenkins Build Status GitHub (pre-)release GitHub (Pre-)Release Date GitHub last commit codecov License

Geting Started | Download | User Guide | Contribute

Siddhi is a cloud native Streaming and Complex Event Processing engine that understands Streaming SQL queries in order to capture events from diverse data sources, process them, detect complex conditions, and publish output to various endpoints in real time.

Siddhi can run as an embedded Java library, and as a micro service on bare metal, VM, Docker and natively in Kubernetes. It also has a graphical and text editor for building Streaming Data Integration and Streaming Analytics applications.

Distributions

Kubernetes Docker Binary Java Source

And more installation options

Overview

Siddhi supports:

  • Streaming Data Integration

    • Retrieving data from various event sources (NATS, Kafka, JMS, HTTP, CDC, etc)
    • Map events to and from multiple event formats (JSON, XML, Text, Avro, etc)
    • Data preprocessing & cleaning
    • Joining multiple data streams
    • Integrate streaming data with databases (RDBMS, Cassandra, HBase, Redis, etc)
    • Integrate with external services
    • Publish data to multiple event sinks (Email, JMS, HTTP, etc)
  • Streaming Data Analytics

    • Generating alerts based on thresholds
    • Calculate aggregations over a short windows (time, length, session, unique, etc) or a long time period
    • Calculate aggregations over long time periods with seconds, minutes, hours, days, months & years granularity
    • Correlating data while finding missing and erroneous events
    • Detecting temporal event patterns
    • Analyzing trends (rise, fall, turn, tipple bottom)
    • Run pretreated machine learning models (PMML, Tensorflow)
    • Learn and predict at runtime using online machine learning models
  • Adaptive Intelligence

    • Static rule processing
    • Stateful rule processing
    • Decision making through synchronous stream processing
    • Query tables, windows and aggregations
  • And many more ... For more information, see Patterns of Streaming Realtime Analytics

Siddhi is free and open source, released under Apache Software License v2.0.

Why use Siddhi ?

If you are a Siddhi user, we would love to hear more on how you use Siddhi? Please share your experience and feedback via the Siddhi user Google group.

Get Started!

Get started with Siddhi in a few minutes by following the Siddhi Quick Start Guide

Siddhi Development Environment

Siddhi Tooling

Siddhi provides siddhi-tooling that supports following features to develop and test stream processing applications:

  • Text Query Editor with syntax highlighting and advanced auto completion support.

  • Event Simulator and Debugger to test Siddhi Applications.

  • Graphical Query Editor with drag and drop query building support.

IntelliJ IDEA Plugin

Install IDEA plugin to get the following features:

  • Siddhi Query Editor with syntax highlighting and with basic auto completion
  • Siddhi Runner and Debugger support to test Siddhi Application

Siddhi Versions

  • Active development version of Siddhi : v5.0.0 built on Java 8 & 11.

    Find the released Siddhi libraries here.

    Siddhi Query Guide for Siddhi v5.x.x

    Architecture of Siddhi v5.x.x

  • Latest Stable Release of Siddhi v4.x.x : v4.4.8 built on Java 8. (Recommended for production use)

    Find the released Siddhi libraries here.

    Siddhi Query Guide for Siddhi v4.x.x

    Architecture of Siddhi v4.x.x

  • Latest Stable Release of Siddhi v3.x.x : v3.2.3 built on Java 7.

    Find the released Siddhi libraries here.

    Siddhi Query Guide for Siddhi v3.x.x

Latest API Docs

Latest API Docs is 5.0.0.

Contact us

How to Contribute

Find the detail information on asking questions, providing feedback, reporting issues, building and contributing code on How to contribute? section.

Roadmap

  • Support Kafka
  • Support NATS
  • Siddhi Runner Distribution
  • Siddhi Tooling (Editor)
  • Siddhi Kubernetes CRD
  • Periodic incremental state persistence
  • Support Prometheus for metrics collection
  • Support high available Siddhi deployment with NATS via Kubernetes CRD
  • Support distributed Siddhi deployment with NATS via Kubernetes CRD

Support

WSO2 provides production, and query support for Siddhi and its extensions. For more details contact via http://wso2.com/support/

Siddhi is joint research project initiated by WSO2 and University of Moratuwa, Sri Lanka.

Latest Releases
v4.5.9
 May. 13 2019
v4.5.8
 May. 8 2019
v4.5.7
 May. 2 2019
v4.5.6
 May. 1 2019
v4.5.5
 Apr. 17 2019