WSO2 Stream Processor confusion

Question

I have a little bit of confusion about the Stream Processor. I've previously used the CEP and now I'm using the Stream Processor.

if I'm not mistaken, the Data Analytics Server, the CEP and the Machine Learner merged into the Stream Processor, is it true?

Because I found some inconsistencies, for example the SP can't publish directly in the dashboard, while CEP could. So, my question is, all the feutures in the CEP and in the ML, are going to flow in the SP?

Answer 1

WSO2 Stream Processor is the latest WSO2 analytics offering. It has a super set of functionalities that WSO2 CEP had. Following is a comparison of capabilities of WSO2 CEP vs WSO2 SP.

General

• The core of SP 4.x is the latest siddhi 4.x which is more stable and has improved performance. While CEP is powered by Siddhi 3.x.
• SP is based on C5 and it's lean and light weight than CEP which was based on C4.
• SP is designed to be container friendly and could native. Where as CEP had some challenges when deployed in containerised environments.
• Everything is now contained in a Siddhi App, which is a single file which can be deployed and executed on it's own.

Incremental Analysis

• New siddhi has the incremental analysis feature which is designed to cater batch analytics. With this feature users can easily do time series aggregations without having to integrate with other platforms such as Spark.
• Incremental analysis smoothly federates real time analytics with batch analytics by allowing both forms of analytics to be done in the same message flow.

Distributed Deployment

• SP 4.x has a distributed architecture which is highly scalable. SP's container friendly nature let's it be scaled massively.
• The distributed deployment is fault tolerant and it supports exactly once processing with the aid of Apache Kafka.
• CEP distributed architecture was based on Apache Storm.
• Also, SP has in build support for Multi data center deployment. While CEP does not.

Tooling

• SP has a rich editor which supports auto completion, event simulation, debugging of siddhi queries, etc. CEP only has the query editor UI in the management console.
• Status Dashboard of SP let's users monitor their deployment with comprehensive set of statistics related to performance, resource consumption etc of Siddhi Apps and JVM. CEP had the carbon metric support which shows only JVM stats.

Business Rules

• SP has Business rules feature where non-tech users can build processing logics through a graphical wizard-like UI without having to rite queries.
• Developers can use this feature to present complex problems in a abstract manner which is understandable to business users.
• CEP did not have feature focusing on business users.

Answer 2

So, my question is, all the feutures in the CEP and in the ML, are going to flow in the SP?

I don't believe so. StreamProcessor has only subset of capabilies of CEP, DAS or ML. IMHO it t is promoted currently as it is new, more lightweight and faster

Answer 3

DAS, CEP and ML have not been completely merged into the Stream Processor.

In DAS, the real time analytics were handled by Siddhi and the batch analytics were done through Spark. However, in Stream Processor, only Siddhi acts as the core processor and Spark is not used.

Stream processor processes data in streaming manner through siddhi. In order to fulfill the requiremnts for batch analytics, incremental processing[1] which has been introduced to Siddhi 4.0.0 can be used.

Also ML support is provided through ml extentions written for Siddhi 4.0.0.

In das/cep it is required to define several artifacts like receivers, execution plans, publishers etc.. in order to create a analytic work flow.

But in Stream Processor,it is possible to define the whole flow in a single Siddhi-App.

For further clarification, please refer to the DAS to SP migration guide[2] and WSO2 analytics site[3].

[1] https://wso2.github.io/siddhi/documentation/siddhi-4.0/#incremental-aggregation

[2] https://docs.wso2.com/display/SP4xx/Upgrading+from+a+Previous+Release

[3] https://wso2.com/analytics