A segmenter class that is able to segment provided text into words or word stems (depending on the particular language).
public : sealed class WordsSegmenter
public sealed class WordsSegmenter
Public NotInheritable Class WordsSegmenter
var wordsSegmenter = new wordsSegmenter(language);
Windows 10 requirements
Windows 10 (introduced v10.0.10240.0 - for Xbox, see UWP features that aren't yet supported on Xbox)
Windows.Foundation.UniversalApiContract (introduced v1)
For languages that do not use spaces between words (such as Japanese, Chinese, Korean, and Thai), use of a segmenter is the only way to obtain individual words for textual processing scenarios such as keyword search.
The language supplied when this object is constructed is matched against the languages with word breakers on the system, and the best word segmentation rules available are used. The language need not be one of the app's supported languages. If there are no supported language rules available specifically for that language, the language-neutral rules are used (an implementation of Unicode Standard Annex #29 Unicode Text Segmentation), and the ResolvedLanguage property is set to "und" (undetermined language).
For keyword search scenarios, it is always recommended to request a segmenter in the language of the text contents.
For spellchecking scenarios, some language segmenters (such as German) may return multiple word stem segments for a single compound word. In contrast, the spellchecking APIs may expect the words to be kept together as a single word. For such languages, you may choose to force language-neutral segmenting rules by explicitly requesting the "und" (undetermined language) segmenter. However, doing so will greatly reduce the breaking quality of non-spaced languages. Therefore, it is recommended that you use the Language.Script API to determine if the content language uses one of the following non-spaced scripts:
|Talu||New Tai Lue|
If none of these scripts are found, then it should be safe to use "und" for spellchecking scenario segmentation.
|WordsSegmenter(String) WordsSegmenter(String) WordsSegmenter(String) WordsSegmenter(String) WordsSegmenter(String)|
|ResolvedLanguage ResolvedLanguage ResolvedLanguage ResolvedLanguage ResolvedLanguage||
Gets the language of the rules used by this WordsSegmenter object.
"und" (undetermined) is returned if we are using language-neutral rules.
|GetTokenAt(String, UInt32) GetTokenAt(String, UInt32) GetTokenAt(String, UInt32) GetTokenAt(String, UInt32) GetTokenAt(String, UInt32)||
Determines and returns the word or word stem which contains or follows a specified index into the provided text.
|GetTokens(String) GetTokens(String) GetTokens(String) GetTokens(String) GetTokens(String)||
Determines and returns all of the words or word stems in the provided text.
|Tokenize(String, UInt32, WordSegmentsTokenizingHandler) Tokenize(String, UInt32, WordSegmentsTokenizingHandler) Tokenize(String, UInt32, WordSegmentsTokenizingHandler) Tokenize(String, UInt32, WordSegmentsTokenizingHandler) Tokenize(String, UInt32, WordSegmentsTokenizingHandler)||
Calls the provided handler with two iterators that iterate through the words prior to and following a given index into the provided text.