Azure Cosmos DB 中的變更摘要 - 概觀Change feed in Azure Cosmos DB - overview

Azure Cosmos DB 中的變更摘要支援是藉由接聽 Azure Cosmos 容器進行任何變更來運作。Change feed support in Azure Cosmos DB works by listening to an Azure Cosmos container for any changes. 然後變更摘要會輸出已排序的文件清單,這些文件已依其修改的順序變更過。It then outputs the sorted list of documents that were changed in the order in which they were modified. 變更會保存,可進行非同步累加處理,而輸出可配送給一或多個取用者進行平行處理。The changes are persisted, can be processed asynchronously and incrementally, and the output can be distributed across one or more consumers for parallel processing.

Azure Cosmos DB 非常適合用於 IoT、遊戲、零售,以及操作記錄應用程式。Azure Cosmos DB is well-suited for IoT, gaming, retail, and operational logging applications. 這類應用程式常用的設計模式,是利用資料的變更觸發其他動作。A common design pattern in these applications is to use changes to the data to trigger additional actions. 其他動作的範例包括:Examples of additional actions include:

  • 於插入或修改項目時觸發通知或呼叫 API。Triggering a notification or a call to an API, when an item is inserted or updated.
  • IoT 的即時串流處理或在操作資料上的即時分析處理。Real-time stream processing for IoT or real-time analytics processing on operational data.
  • 藉由與快取、搜尋引擎、或資料倉儲進行同步處理,或將資料封存到冷儲存體,進行額外的資料移動。Additional data movement by either synchronizing with a cache or a search engine or a data warehouse or archiving data to cold storage.

Azure Cosmos DB 中的變更摘要可讓您針對每一個模式建置有效率且可調整的解決方案,如下圖所示:The change feed in Azure Cosmos DB enables you to build efficient and scalable solutions for each of these patterns, as shown in the following image:

使用 Azure Cosmos DB 變更摘要來提供即時分析和事件導向的計算案例

支援的 API 和用戶端 SDKSupported APIs and client SDKs

此功能目前受到下列 Cosmos DB API 和用戶端 SDK 支援。This feature is currently supported by the following Azure Cosmos DB APIs and client SDKs.

用戶端驅動程式Client drivers Azure CLIAzure CLI SQL APISQL API Cassandra APICassandra API 適用於 MongoDB 的 Azure Cosmos DB APIAzure Cosmos DB's API for MongoDB Gremlin APIGremlin API 資料表 APITable API
.NET.NET NANA Yes No No Yes No
JavaJava NANA Yes No No Yes No
PythonPython NANA Yes No No Yes No
節點/JSNode/JS NANA Yes No No Yes No

變更摘要和不同的作業Change feed and different operations

現在,您會看到變更摘要中的所有作業。Today, you see all operations in the change feed. 可讓您控制變更摘要的功能還無法使用,只能執行特定作業,例如更新,無法執行像插入等作業。The functionality where you can control change feed, for specific operations such as updates only and not inserts is not yet available. 在變更摘要中處理項目時,您可以在項目上新增「軟標記」以用於更新,並據以篩選。You can add a “soft marker” on the item for updates and filter based on that when processing items in the change feed. 目前變更摘要不會記錄刪除項目。Currently change feed doesn’t log deletes. 您可以像先前的範例那樣,在正在刪除的項目上新增軟標記,可以在項目中新增稱為 "deleted" 的屬性,並將它設定為 "true",然後在項目上設定 TTL,讓它可以被自動刪除。Similar to the previous example, you can add a soft marker on the items that are being deleted, for example, you can add an attribute in the item called "deleted" and set it to "true" and set a TTL on the item, so that it can be automatically deleted. 您可以讀取變更摘要以瞭解有哪些的歷程記錄項目,例如,五年前新增的項目。You can read the change feed for historic items, for example, items that were added five years ago. 如果項目未刪除,您可以讀取至原始容器的變更摘要。If the item is not deleted you can read the change feed as far as the origin of your container.

變更摘要中項目的排序順序Sort order of items in change feed

變更摘要項目會按修改時間順序排列。Change feed items come in the order of their modification time. 系統會依據邏輯分區索引鍵保障此排序順序。This sort order is guaranteed per logical partition key.

多重區域 Azure Cosmos 帳戶中的變更摘要Change feed in multi-region Azure Cosmos accounts

在多重區域 Azure Cosmos 帳戶,中,如果寫入區域進行容錯移轉,變更摘要會跨手動容錯移轉作業運作,而且會連續運作。In a multi-region Azure Cosmos account, if a write-region fails over, change feed will work across the manual failover operation and it will be contiguous.

變更摘要和存留時間 (TTL)Change feed and Time to Live (TTL)

如果項目上的 TTL (存留時間) 屬性設為 -1,變更摘要會永久保存。If a TTL (Time to Live) property is set on an item to -1, change feed will persist forever. 如果未刪除資料,即會一直保留在變更摘要中。If the data is not deleted, it will remain in the change feed.

變更摘要和 _etag、_lsn 或 _tsChange feed and _etag, _lsn or _ts

_etag 格式是作為內部之用,因為該格式可以隨時變更,請別依賴它。The _etag format is internal and you should not take dependency on it, because it can change anytime. _ts 是修改或建立時間戳記。_ts is a modification or a creation timestamp. 您可以使用 _ts,依時間順序進行比較。You can use _ts for chronological comparison. _lsn 是僅為變更摘要新增的批次識別碼;它代表交易識別碼。_lsn is a batch ID that is added for change feed only; it represents the transaction ID. 許多項目可能有相同的 _lsn。Many items may have same _lsn. FeedResponse 上的 ETag 與您在項目上看到的 _etag 不同。ETag on FeedResponse is different from the _etag you see on the item. _etag 是內部識別碼,而且用於並行控制,它會指示項目版本,而使用 ETag 來排序摘要。_etag is an internal identifier and is used for concurrency control tells about the version of the item, whereas ETag is used for sequencing the feed.

變更摘要使用個案和案例Change feed use cases and scenarios

變更摘要可有效處理具有大量寫入的大型資料集。Change feed enables efficient processing of large datasets with a high volume of writes. 變更摘要也提供了替代方式,以查詢整個資料集來識別已變更的項目。Change feed also offers an alternative to querying an entire dataset to identify what has changed.

使用案例Use cases

例如,您可以利用變更摘要有效率地執行下列工作︰For example, with change feed you can perform the following tasks efficiently:

  • 透過儲存在 Azure Cosmos DB 中的資料,來更新快取、更新搜尋索引,或更新資料倉儲。Update a cache, update a search index, or update a data warehouse with data stored in Azure Cosmos DB.

  • 實作應用程式層級的資料階層處理和封存,例如在 Azure Cosmos DB 中儲存「熱資料」,並將「冷資料」轉存到其他儲存系統,例如 Azure Blob 儲存體Implement an application-level data tiering and archival, for example, store "hot data" in Azure Cosmos DB and age out "cold data" to other storage systems, for example, Azure Blob Storage.

  • 執行零停機時間移轉至另一個 Azure Cosmos 帳戶或具有不同邏輯分割區索引鍵的另一個 Azure Cosmos 容器。Perform zero down-time migrations to another Azure Cosmos account or another Azure Cosmos container with a different logical partition key.

  • 使用 Azure Cosmos DB 實作lambda 架構,其中 Azure Cosmos DB 同時支援即時、批次和服務層,因此可支援具有低 TCO 的 lambda 架構。Implement lambda architecture using Azure Cosmos DB, where Azure Cosmos DB supports both real-time, batch and query serving layers, thus enabling lambda architecture with low TCO.

  • 接收與儲存來自裝置、感應器、基礎結構和應用程式的事件資料,並即時處理這些事件,例如使用 SparkReceive and store event data from devices, sensors, infrastructure and applications, and process these events in real time, for example, using Spark. 下圖顯示如何透過變更摘要使用 Azure Cosmos DB 實作 lambda 架構:The following image shows how you can implement lambda architecture using Azure Cosmos DB via change feed:

運用 Azure Cosmos DB 進行擷取和查詢的 lambda 管線


以下是您可以使用變更摘要輕鬆實作的一些案例:The following are some of the scenarios you can easily implement with change feed:

  • 在您的 serverless Web 和行動應用程式內,您可以追蹤像是對客戶設定檔、喜好設定或其位置等所做之所有變更的事件,並觸發特定動作,例如使用 Azure Functions 傳送推播通知到客戶裝置。Within your serverless web or mobile apps, you can track events such as all the changes to your customer's profile, preferences, or their location and trigger certain actions, for example, sending push notifications to their devices using Azure Functions.

  • 例如,如果您使用 Azure Cosmos DB 來建置遊戲,就可以根據完成遊戲的分數,使用變更摘要來實作即時排行榜。If you're using Azure Cosmos DB to build a game, you can, for example, use change feed to implement real-time leaderboards based on scores from completed games.

使用變更摘要Working with change feed

您可以搭配變更摘要使用下列選項:You can work with change feed using the following options:

容器內每個邏輯分割區索引鍵都可使用變更摘要,因此可以配送給一或多個取用者以進行平行處理,如下圖所示。Change feed is available for each logical partition key within the container, and it can be distributed across one or more consumers for parallel processing as shown in the image below.

Azure Cosmos DB 變更摘要的分散式處理

變更摘要的功能Features of change feed

  • 根據預設,所有 Azure Cosmos 帳戶都會啟用變更摘要。Change feed is enabled by default for all Azure Cosmos accounts.

  • 和任何其他 Azure Cosmos DB 作業一樣,您可以在與您的 Azure Cosmos 資料庫關聯的任何區域中,使用佈建輸送量從變更摘要讀取。You can use your provisioned throughput to read from the change feed, just like any other Azure Cosmos DB operation, in any of the regions associated with your Azure Cosmos database.

  • 變更摘要包含對容器內項目進行的插入與更新作業。The change feed includes inserts and update operations made to items within the container. 您可以擷取刪除項目,方法是在您的項目 (例如文件) 內設定「虛刪除」旗標來取代刪除動作。You can capture deletes by setting a "soft-delete" flag within your items (for example, documents) in place of deletes. 或者,您可以使用 TTL 功能為項目設定有限的逾期期限。Alternatively, you can set a finite expiration period for your items with the TTL capability. 例如 24 小時,並使用該屬性的值擷取刪除項目。For example, 24 hours and use the value of that property to capture deletes. 使用此解決方案,您必須在比 TTL 逾期期限更短的時間間隔內處理變更。With this solution, you have to process the changes within a shorter time interval than the TTL expiration period.

  • 對項目所做的每項變更皆會在變更摘要中出現一次,且用戶端必須管理檢查點邏輯。Each change to an item appears exactly once in the change feed, and the clients must manage the checkpointing logic. 如果您想要避免管理檢查點的複雜性, 變更摘要處理器會提供自動檢查點和「至少一次」的語義。If you want to avoid the complexity of managing checkpoints, the change feed processor provides automatic checkpointing and "at least once" semantics. 請參閱搭配變更摘要處理器使用變更摘要。See using change feed with change feed processor.

  • 變更記錄檔中只會包含指定項目的最新變更。Only the most recent change for a given item is included in the change log. 中繼變更可能無法使用。Intermediate changes may not be available.

  • 變更摘要會依照各個邏輯分割區索引鍵值內的修改順序排序。The change feed is sorted by the order of modification within each logical partition key value. 跨分割區索引鍵值順序不會是固定的。There is no guaranteed order across the partition key values.

  • 變更可以從任何時間點同步處理,也就是說,可用變更沒有固定的資料保留期限。Changes can be synchronized from any point-in-time, that is there is no fixed data retention period for which changes are available.

  • Azure Cosmos 容器所有邏輯分割區索引鍵的變更都是以平行方式提供使用。Changes are available in parallel for all logical partition keys of an Azure Cosmos container. 這項功能可讓大型容器的變更由多個取用者平行處理。This capability allows changes from large containers to be processed in parallel by multiple consumers.

  • 應用程式可以在相同的容器上同時要求多個變更摘要。Applications can request multiple change feeds on the same container simultaneously. ChangeFeedOptions.StartTime 可用來提供初始的開始點。ChangeFeedOptions.StartTime can be used to provide an initial starting point. 例如,尋找與指定時鐘時間相對應的接續權杖。For example, to find the continuation token corresponding to a given clock time. ContinuationToken 如經指定,會優先於 StartTime 和 StartFromBeginning 值。The ContinuationToken, if specified, wins over the StartTime and StartFromBeginning values. ChangeFeedOptions.StartTime 的精確度為 5 秒內。The precision of ChangeFeedOptions.StartTime is ~5 secs.

後續步驟Next steps

您現在可以在下列文章中繼續深入了解變更摘要:You can now proceed to learn more about change feed in the following articles: