什麼是 Azure Content Moderator?What is Azure Content Moderator?

Azure Content Moderator API 是一項認知服務,可檢查文字、影像和視訊內容是否含有可能具冒犯意味、有風險或不當的資料。The Azure Content Moderator API is a cognitive service that checks text, image, and video content for material that is potentially offensive, risky, or otherwise undesirable. 找到這類資料時,服務會對內容套用適當的標籤 (旗標)。When such material is found, the service applies appropriate labels (flags) to the content. 加上旗標的內容隨後可由您的應用程式處理,以遵循法規或維護使用者應有的環境。Your app can then handle flagged content in order to comply with regulations or maintain the intended environment for users. 請參閱 Content Moderator API 一節,以深入了解不同內容旗標的意義。See the Content Moderator APIs section to learn more about what the different content flags indicate.

使用位置Where it is used

以下是軟體開發人員或小組可能使用 Content Moderator 的幾個案例:The following are a few scenarios in which a software developer or team would use Content Moderator:

  • 對產品目錄和使用者產生的其他內容進行仲裁的線上市集Online marketplaces that moderate product catalogs and other user-generated content
  • 對使用者產生的遊戲成品和聊天室進行仲裁的遊戲公司Gaming companies that moderate user-generated game artifacts and chat rooms
  • 對使用者新增的影像、文字和影片進行仲裁的社交傳訊平台Social messaging platforms that moderate images, text, and videos added by their users
  • 對其內容實作集中式仲裁的企業媒體公司Enterprise media companies that implement centralized moderation for their content
  • 為學生和授課者篩選掉不當內容的 K-12 教育解決方案提供者K-12 education solution providers filtering out content that is inappropriate for students and educators

包含內容What it includes

Content Moderator 服務由數個可透過 REST 呼叫和 .NET SDK 使用的 Web 服務 API 所組成。The Content Moderator service consists of several web service APIs available through both REST calls and a .NET SDK. 此外也包含人工審核工具,可讓人工審核者輔助服務,以及改善或微調其仲裁功能。It also includes the human review tool, which allows human reviewers to aid the service and improve or fine-tune its moderation function.

顯示仲裁 API、審核 API 和人工審核工具的 Content Moderator 區塊圖

Content Moderator APIContent Moderator APIs

Content Moderator 服務包含適用於下列案例的 API。The Content Moderator service includes APIs for the following scenarios.

API 群組API group 說明Description
文字仲裁Text moderation 掃描文字中具冒犯性的內容、明顯色情或性暗示內容、粗話和個人資料。Scans text for offensive content, sexually explicit or suggestive content, profanity, and personal data.
自訂字詞清單Custom term lists 除了內建字詞外,也根據自訂字詞清單掃描文字。Scans text against a custom list of terms in addition to the built-in terms. 使用自訂清單可根據您自己的內容原則來封鎖或允許內容。Use custom lists to block or allow content according to your own content policies.
影像仲裁Image moderation 掃描影像中的成人或猥褻內容、使用光學字元辨識 (OCR) 功能偵測影像中的文字,以及偵測臉部。Scans images for adult or racy content, detects text in images with the Optical Character Recognition (OCR) capability, and detects faces.
自訂影像清單Custom image lists 根據自訂影像清單掃描影像。Scans images against a custom list of images. 使用自訂影像清單,篩選掉重複出現、而您不想再次分類的內容執行個體。Use custom image lists to filter out instances of commonly recurring content that you don't want to classify again.
影片仲裁Video moderation 掃描影片中的成人或猥褻內容,並傳回該內容的時間標記。Scans videos for adult or racy content and returns time markers for said content.
檢閱 APIReview APIs 使用作業審核工作流程作業來建立及自動化審核工具的人機互動工作流程。Use the Jobs, Reviews, and Workflow operations to create and automate human-in-the-loop workflows with the human review tool. .NET SDK 尚未提供工作流程 API。The Workflow API is not yet available in the .NET SDK.

檢閱工具Review tool

Content Moderator 服務也包含以 Web 為基礎的審核工具,此工具會裝載內容審核以供人力審核者處理。The Content Moderator service also includes the web-based Review tool, which hosts the content reviews for human moderators to process. 人工輸入不會訓練服務,但在服務與人工審核小組的搭配運作下,開發人員將可在效率和精確度之間取得適當的平衡。The human input does not train the service, but the combined work of the service and human review teams allows developers to strike the right balance between efficiency and accuracy. 審核工具也針對各種 Content Moderator 資源提供方便使用的前端。The Review tool also provides a user-friendly front-end for a variety of Content Moderator resources.

Content Moderator 的人工審核工具首頁

資料隱私權和安全性Data privacy and security

和所有認知服務一樣,使用 Content Moderator 服務的開發人員應該要了解 Microsoft 對於客戶資料的政策。As with all of the Cognitive Services, developers using the Content Moderator service should be aware of Microsoft's policies on customer data. 請參閱 Microsoft 信任中心上的認知服務頁面,以進行深入了解。See the Cognitive Services page on the Microsoft Trust Center to learn more.

後續步驟Next steps

請遵循在 Web 上試用 Content Moderator 中的指示,以開始使用 Content Moderator 服務。Get started using the Content Moderator service by following the instructions in Try Content Moderator on the web.