Stream analytics LAST method Partition by

Question

When using Stream Analytics LAST built in method do you set a property which is "PARTITION BY", does this mean that you change the overall partitioning you set from the input stage or is this somehow locally.

For clarification, you can set partition key on a specific column when setting up the input in Stream Analytics. In this case do i use PARTITION BY on another key.

ex:

Input -> Partition by Category

Query -> LAST(value) OVER (PARTITION BY [FRUIT_ID] LIMIT duration(hour, 24) WHEN value is not null)

Answer 1

Hello! It seems like you have a question about using the LAST built-in method in Stream Analytics and how it interacts with partitioning.

When you use the LAST built-in method in Stream Analytics, you can use the PARTITION BY clause to specify the partition key for the window function. This means that the LAST function will be applied to the rows within each partition separately.

To clarify, the partition key that you set when setting up the input in Stream Analytics determines the initial partitioning of the data. However, when you use the PARTITION BY clause with the LAST function, you are specifying a local partitioning scheme for the window function. This means that the LAST function will only consider the rows within each partition specified by the PARTITION BY clause.

In your example, you have set the partition key for the input to be "Category". However, you are using the PARTITION BY clause with the LAST function to specify a local partitioning scheme based on the "FRUIT_ID" column. This means that the LAST function will only consider the rows within each partition of "FRUIT_ID" separately, regardless of the initial partitioning of the data based on "Category".

I hope this helps clarify how partitioning works with the LAST function in Stream Analytics. Let me know if you have any further questions!