Streaming Predictions in Apache Flink

Question

Is it possible to make predictions on a dataStream in Apache Flink using a model that is already trained in batch?

The predict function from svm needs as input a dataset and does not take a datastream.

Unfortunately I am not able to figure it out how I can make it work with a flatpMap/map function.

I trained my SVM-model this way:

val svm2 = SVM()
svm2.setSeed(1)
svm2.fit(trainLV)
val testVD = testLV.map(lv => (lv.vector, lv.label))
val evalSet = svm2.evaluate(testVD)

and saved the model: val modelSvm = svm2.weightsOption.get

Then I have an incoming datastream in the streaming environment:
dataStream[(Int, Int, Int)]
which should be bininary classified using the svm model.

Thank you!

Answer 1

Flink's ML library only supports batch processing at the moment. If you want to make predictions using the DataStream API, you would need to implement your own flatMap/map function which takes the model and applies it to the incoming events.