WordsSegmenter.Tokenize(String, UInt32, WordSegmentsTokenizingHandler) Method

Definition

Namespace:

Windows.Data.Text

Calls the provided handler with two iterators that iterate through the words prior to and following a given index into the provided text.

public:
 virtual void Tokenize(Platform::String ^ text, unsigned int startIndex, WordSegmentsTokenizingHandler ^ handler) = Tokenize;

void Tokenize(winrt::hstring const& text, uint32_t const& startIndex, WordSegmentsTokenizingHandler const& handler);

public void Tokenize(string text, uint startIndex, WordSegmentsTokenizingHandler handler);

function tokenize(text, startIndex, handler)

Public Sub Tokenize (text As String, startIndex As UInteger, handler As WordSegmentsTokenizingHandler)

Parameters

text

String

Platform::String

winrt::hstring

Provided text containing words to be returned.

startIndex

UInt32

unsigned int

uint32_t

A zero-based index into text. It must be less than the length of text.

handler

WordSegmentsTokenizingHandler

The function that receives the iterators.

Remarks

The iterators in WordSegmentsTokenizingHandler are lazy and evaluate small chunks of text at a time.

The handler is called at most once per call to Tokenize. The handler is not called if there are no selectable words in text.

See also

• WordSegmentsTokenizingHandler