TokenType TokenType TokenType TokenType Enum


Specifies the different types of tokens that can be identified and returned from a language service scanner.

public enum class TokenType
public enum TokenType
type TokenType = 
Public Enum TokenType


Comment Comment Comment Comment 10

A block comment. For example, in C# or C++, a comment is bounded by / and /. In XML, the comment is bounded by <!-- and -->.

Delimiter Delimiter Delimiter Delimiter 7

A token that operates as a separator between two language elements. For example, in C#, the period "." between class name and member name. In XML, the angle brackets surrounding a tag, < and >.

Identifier Identifier Identifier Identifier 3

An identifier or name. For example, the name of a variable, method, or class. In XML, this could be the name of a tag or attribute.

Keyword Keyword Keyword Keyword 2

A language keyword, an identifier that is reserved by the language. For example, in C#, do, while, foreach, if, and else, are all keywords.

LineComment LineComment LineComment LineComment 9

A line comment (comment is terminated at the end of the line). For example, in C# or C++, a comment is preceded by a //. In Visual Basic, this is a single tick '.

Literal Literal Literal Literal 5

A literal value (a character or number). For example, in C# or C++, this is a character bounded by single quotes, or a decimal or hexadecimal number.

Operator Operator Operator Operator 6

A punctuation character that has a specific meaning in a language. For example, in C#, arithmetic operators +, -, *, and /. In C++, pointer dereference operator ->, insertion operator >>, and extraction operation <<. In XML, assignment operator =.

String String String String 4

A string. Typically defined as zero or more characters bounded by double quotes.

Text Text Text Text 1

General text; any text not identified as a specified token type.

Unknown Unknown Unknown Unknown 0

The token is an unknown type. This is typically used for any token not recognized by the parser and should be considered an error in the code being parsed.

WhiteSpace WhiteSpace WhiteSpace WhiteSpace 8

A space, tab, or newline. Typically, a contiguous run of any whitespace is considered a single whitespace token. For example, the three spaces in "name this" would be treated as one whitespace token.


This enumeration is used in the TokenType structure to identify the type of token parsed. The TokenType structure is used in the IScanner scanner as implemented in a language service.

The types specified in this enumeration cover tokens that can appear in all common computer languages. More importantly, these are the token types that the default managed package framework language service classes understand. You can add additional types that your scanner support but you should first adhere to the meanings of the types described here.

Applies to