I have a Azure Stream analytics job. I want to make it process output every 15 minutes

Question

I am getting some events from event hub and saving them in ADLS Gen 2 without performing any operation. Just saving the live events in ADLS Gen 2. I am not doing any kind of sum, average or filteration. I want my job to update my blob every 15 minutes. Like it should not process events as soon as event came, it should hold according to 15 minutes cycle amd then process the events. I want output to be updated every 15 minutes.

I am getting some events from event hub and saving them in ADLS Gen 2 without performing any operation. Just saving the live events in ADLS Gen 2.

I am not doing any kind of sum, average or filtration.

I want my job to update my blob every 15 minutes.

Like it should not process events as soon as event came, it should hold according to 15 minutes cycle amd then process the events.

I want output to be updated every 15 minutes.

Answer 1

Take a look at the output configuration. You can use the combination between Minimum rows and Maximum time to meet your demands.

Minimum rows:

The number of minimum rows per batch. For Parquet, every batch will create a new file. The current default value is 2,000 rows and the allowed maximum is 10,000 rows.

Maximum time

The maximum wait time per batch. After this time, the batch will be written to the output even if the minimum rows requirement isn't met. The current default value is 1 minute and the allowed maximum is 2 hours. If your blob output has path pattern frequency, the wait time can't be higher than the partition time range.

There is a quickstart that shows you how to configure an output using the ASA Tools extension for Visual Studio Code.