WordsSegmenter.GetTokens(String) Method

Definition

Namespace:

Windows.Data.Text

Determines and returns all of the words or word stems in the provided text.

public:
 virtual IVectorView<WordSegment ^> ^ GetTokens(Platform::String ^ text) = GetTokens;

IVectorView<WordSegment> GetTokens(winrt::hstring const& text);

public IReadOnlyList<WordSegment> GetTokens(string text);

function getTokens(text)

Public Function GetTokens (text As String) As IReadOnlyList(Of WordSegment)

Parameters

text

String

Platform::String

winrt::hstring

Provided text containing words or word stems to be returned.

Returns

IVectorView<WordSegment>

IReadOnlyList<WordSegment>

A collection of WordSegment objects that represent the words or word stems.

Remarks

Note that some languages do not use spaces (such as Japanese or Chinese) and some languages may return multiple word stems for compound words (such as German).