Azure Stream Analytics: "Output contains multiple rows …" warning

493 Views Asked by At

we're using a Stream Analytics component in Azure to send data (log messages from different web apps) to a table storage account. The messages are retrieved from an Event Hub, but I think this doesn't matter here.

Within the Stream Analytics component we defined an output for the table storage account including partition and row key settings. As of now the partition key will be the name of the app that sent the log message in the first place. This might not be ideal, but I'm lacking experience in choosing the right values here. However, I think this is a whole different topic. The row key will be a unique id of the specific log message.

Now when I watch the Stream Analytics Output within the Azure portal the following warning message pops up very frequently (and sometimes disappears for a couple of seconds):

Warning: Output contains multiple rows and just one row per partition key. If the output latency is higher than expected, consider choosing a partition key that splits output into multiple partitions while maintaining about 100 records per partition.

Regarding this message I have two questions:

  1. What does this exactly mean or why does it happen? I can see that a single new log message will always qualify as "just one row per partition key", simply because it's just one row. But looking at maybe hundreds of rows sent within a short period of time they all share just three partition keys (three apps logging to the Event Hub), pretty much equally divided. That's why I don't get the whole "Output contains multiple rows and just one row per partition key" thing.
  2. Does this in any way affect the performance or overall functionality of the Stream Analytics component or the table storage?

I also played with the "Batch size" setting of the table storage output, but this didn't change anything.

Thanks in advance for reading and trying to help.

1

There are 1 best solutions below

1
On BEST ANSWER

What does this exactly mean or why does it happen?

It is a warning not a error. It means that each row in your output has the unique partition key.

I can see that a single new log message will always qualify as "just one row per partition key", simply because it's just one row.

The warning is not suitable for a single message. I suggest you post a feedback on Azure feedback site which is used for accepting user voice and bugs.

https://feedback.azure.com/forums/34192--general-feedback

Does this in any way affect the performance or overall functionality of the Stream Analytics component or the table storage?

No, you could just ignore the warning.