了解文字仲裁概念Learn text moderation concepts

使用內容仲裁的文字仲裁模型來分析文字內容。Use Content Moderator's text moderation models to analyze text content.

您可以根據您的原則和閾值來封鎖、核准或審核內容(請參閱評論、工作流程和作業,以瞭解如何設定人工審核)。You can block, approve or review the content based on your policies and thresholds (see Reviews, workflows, and jobs to learn how to set up human reviews). 使用文字仲裁模型,為合作夥伴、員工和取用者產生文字內容的環境,增強人工審核。Use the text moderation models to augment human moderation of environments where partners, employees and consumers generate text content. 這些環境包括聊天室、討論區、聊天機器人、電子商務目錄和文件。These include chat rooms, discussion boards, chatbots, e-commerce catalogs, and documents.

服務回應會包含下列資訊:The service response includes the following information:

  • 粗話:搭配內建的多語言粗話字詞清單進行字詞型比對Profanity: term-based matching with built-in list of profane terms in various languages
  • 分類:由電腦輔助分類來分成三種類別Classification: machine-assisted classification into three categories
  • 個人資料Personal data
  • 自動校正的文字Auto-corrected text
  • 原始文字Original text
  • LanguageLanguage

粗話Profanity

如果 API 偵測到以任何支援的語言表達的任何粗話字詞,這些字詞就會包含在回應中。If the API detects any profane terms in any of the supported languages, those terms are included in the response. 此回應也會包含它們在原始文字中的位置 (Index)。The response also contains their location (Index) in the original text. 以下範例 JSON 中的 ListId 係指在 自訂字詞清單 (如果有的話) 中找到的字詞。The ListId in the following sample JSON refers to terms found in custom term lists if available.

"Terms": [
    {
        "Index": 118,
        "OriginalIndex": 118,
        "ListId": 0,
        "Term": "crap"
    }

注意

針對 language 參數,請指派 eng 或將其保留空白,以查看電腦輔助分類 回應 (預覽版功能)。For the language parameter, assign eng or leave it empty to see the machine-assisted classification response (preview feature). 此功能僅支援英文This feature supports English only.

針對粗話字詞偵測,請使用本文中所列支援語言的 ISO 639-3 代碼或將其保留空白。For profanity terms detection, use the ISO 639-3 code of the supported languages listed in this article, or leave it empty.

分類Classification

內容仲裁的電腦輔助文字分類功能僅支援英文,並協助偵測可能不想要的內容。Content Moderator's machine-assisted text classification feature supports English only, and helps detect potentially undesired content. 所標幟的內容可能是依據上下文而被評估為不當的內容。The flagged content may be assessed as inappropriate depending on context. 它會傳遞每個類別的可能性,而且可能會建議人工審核。It conveys the likelihood of each category and may recommend a human review. 此功能使用定型模型來識別可能的濫用、毀謗性或歧視性語言。The feature uses a trained model to identify possible abusive, derogatory or discriminatory language. 這包括要供審核的俚語、縮寫單字、冒犯性及刻意拼錯的單字。This includes slang, abbreviated words, offensive, and intentionally misspelled words for review.

以下 JSON 擷取內容顯示一個範例輸出︰The following extract in the JSON extract shows an example output:

"Classification": {
    "ReviewRecommended": true,
    "Category1": {
        "Score": 1.5113095059859916E-06
    },
    "Category2": {
        "Score": 0.12747249007225037
    },
    "Category3": {
        "Score": 0.98799997568130493
    }
}

說明Explanation

  • Category1 指的是可能有在特定情況下被視為明顯色情或成人內容的語言存在。Category1 refers to potential presence of language that may be considered sexually explicit or adult in certain situations.
  • Category2 指的是可能有在特定情況下被視為具性暗示或成人內容的語言存在。Category2 refers to potential presence of language that may be considered sexually suggestive or mature in certain situations.
  • Category3 指的是可能有在特定情況下被視為具冒犯性的語言存在。Category3 refers to potential presence of language that may be considered offensive in certain situations.
  • Score 介於 0 到 1 之間。Score is between 0 and 1. 分數越高,模型預測為適用該類別的可能性就越高。The higher the score, the higher the model is predicting that the category may be applicable. 此功能須倚賴統計模型,而不是手動編碼的結果。This feature relies on a statistical model rather than manually coded outcomes. 建議您使用自己的內容進行測試,以判斷每個類別如何符合您的需求。We recommend testing with your own content to determine how each category aligns to your requirements.
  • ReviewRecommended 會是 true 或 false,視內部分數閾值而定。ReviewRecommended is either true or false depending on the internal score thresholds. 客戶應該評估是要使用此值,還是根據其內容原則決定自訂閾值。Customers should assess whether to use this value or decide on custom thresholds based on their content policies.

個人資料Personal data

「個人資料」功能會偵測這項資訊的可能存在:The personal data feature detects the potential presence of this information:

  • 電子郵件地址Email address
  • 美國郵寄地址US mailing address
  • IP 位址IP address
  • 美國電話號碼US phone number

以下範例顯示一個範例回應:The following example shows a sample response:

"pii":{
  "email":[
      {
        "detected":"abcdef@abcd.com",
        "sub_type":"Regular",
        "text":"abcdef@abcd.com",
        "index":32
      }
  ],
  "ssn":[

  ],
  "ipa":[
      {
        "sub_type":"IPV4",
        "text":"255.255.255.255",
        "index":72
      }
  ],
  "phone":[
      {
        "country_code":"US",
        "text":"6657789887",
        "index":56
      }
  ],
  "address":[
      {
        "text":"1 Microsoft Way, Redmond, WA 98052",
        "index":89
      }
  ]
}

自動校正Auto-correction

假設輸入文字是(「lzay」和「f0x」是故意的):Suppose the input text is (the 'lzay' and 'f0x' are intentional):

Qu! ck 棕色 f0x 會跳過 lzay 狗。The qu!ck brown f0x jumps over the lzay dog.

如果您要求自動校正,回應就會包含該文字的校正版:If you ask for auto-correction, the response contains the corrected version of the text:

快速棕色 fox 會跳過延遲狗。The quick brown fox jumps over the lazy dog.

建立及管理您的自訂字詞清單Creating and managing your custom lists of terms

雖然預設的全域字詞清單適用於大多數案例,但您可能會想要依據業務需求特定的字詞來進行過濾。While the default, global list of terms works great for most cases, you may want to screen against terms that are specific to your business needs. 例如,您可能會想要從使用者的文章中篩選掉任何競爭的品牌名稱。For example, you may want to filter out any competitive brand names from posts by users.

注意

上限是 5 個字詞清單,其中每個清單不可超過 10,000 個字詞There is a maximum limit of 5 term lists with each list to not exceed 10,000 terms.

以下範例顯示相符的「清單識別碼」:The following example shows the matching List ID:

"Terms": [
    {
        "Index": 118,
        "OriginalIndex": 118,
        "ListId": 231.
        "Term": "crap"
    }

Content Moderator 有提供一個字詞清單 API,其中含有可管理自訂字詞清單的作業。The Content Moderator provides a Term List API with operations for managing custom term lists. 請從字詞清單 API 主控台開始著手,然後使用 REST API 程式碼範例。Start with the Term Lists API Console and use the REST API code samples. 此外,如果您已熟悉 Visual Studio 和 C#,請一併參閱字詞清單 .NET 快速入門Also check out the Term Lists .NET quickstart if you are familiar with Visual Studio and C#.

後續步驟Next steps

使用文字審核 api 主控台來測試 api。Test out the APIs with the Text moderation API console. 另請參閱審查、工作流程和作業,以瞭解如何設定人工審核。Also see Reviews, workflows, and jobs to learn how to set up human reviews.