WordsSegmenter WordsSegmenter WordsSegmenter WordsSegmenter Class

Creates a WordsSegmenter object. See the introduction in WordsSegmenter for a description of how the language supplied to this constructor is used.

public : WordsSegmenter(PlatForm::String language)public WordsSegmenter(String language)Public Sub New(language As String)// You can use this method in JavaScript.

Gets the language of the rules used by this WordsSegmenter object.

"und" (undetermined) is returned if we are using language-neutral rules.

public : PlatForm::String ResolvedLanguage { get; }public string ResolvedLanguage { get; }Public ReadOnly Property ResolvedLanguage As string// You can use this property in JavaScript.

Determines and returns the word or word stem which contains or follows a specified index into the provided text.

public : WordSegment GetTokenAt(PlatForm::String text, unsigned int startIndex)public WordSegment GetTokenAt(String text, UInt32 startIndex)Public Function GetTokenAt(text As String, startIndex As UInt32) As WordSegment// You can use this method in JavaScript.

Remarks

Note that some languages do not use spaces (such as Japanese or Chinese) and some languages may return multiple word stems for compound words (such as German).

Determines and returns all of the words or word stems in the provided text.

public : IVectorView<WordSegment> GetTokens(PlatForm::String text)public IReadOnlyList<WordSegment> GetTokens(String text)Public Function GetTokens(text As String) As IReadOnlyList( Of WordSegment )// You can use this method in JavaScript.

Remarks

Note that some languages do not use spaces (such as Japanese or Chinese) and some languages may return multiple word stems for compound words (such as German).

Calls the provided handler with two iterators that iterate through the words prior to and following a given index into the provided text.

public : void Tokenize(PlatForm::String text, unsigned int startIndex, WordSegmentsTokenizingHandler handler)public void Tokenize(String text, UInt32 startIndex, WordSegmentsTokenizingHandler handler)Public Function Tokenize(text As String, startIndex As UInt32, handler As WordSegmentsTokenizingHandler) As void// You can use this method in JavaScript.

Remarks

The iterators in WordSegmentsTokenizingHandler are lazy and evaluate small chunks of text at a time.

The handler is called at most once per call to Tokenize. The handler is not called if there are no selectable words in text.

See Also

WordSegmentsTokenizingHandlerWordSegmentsTokenizingHandlerWordSegmentsTokenizingHandlerWordSegmentsTokenizingHandler