Azure search indexer index entire Cosmos db data source

Evgeny 1 Reputation point
2020-06-30T15:44:53.64+00:00

Adding custom query to cosmos db data source causes my indexer to go through entire source, not only the newest elements.
Also, it's worth to mention that adding query like SELECT * FROM c WHERE c._ts >= @HighWaterMark ORDER BY c._ts doesn't break anything. But as soon as I add fields into select statement (e.g. SELECT c.id FROM c WHERE c._ts >= @HighWaterMark ORDER BY c._ts) my indexer ignores previous progress.

I noted that if I add @HighWaterMark as one of my search fields to query (SELECT c.id, @HighWaterMark as {some field} FROM c WHERE c._ts >= @HighWaterMark ORDER BY c._ts) this field becomes -1. I'm not sure if it is a correct way to get @HighWaterMark value.

Azure AI Search
Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
707 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. brtrach-MSFT 15,251 Reputation points Microsoft Employee
    2020-07-24T20:33:31.707+00:00

    @Evgeny-1093, If you are still having trouble, can you please reach out to us at azcommunity@microsoft.com with your Azure subscription ID? We'd like to provide you with a free support case to get you the support needed.

    If you found a solution, we would appreciate it if you shared the solution with the community to help others with similar issues.

    0 comments