The Default Babel Implementation in the Language Service Package

The default Babel implementation is provided in the Visual Studio Language Package solution. This implementation assumes that you are using a lexer and parser that are similar to lex and yacc (for example, Flex and Bison), and that you have already defined the tokens and grammar of your language.

For more information about the Language Service Package solution, see Walkthrough: Creating a Language Service Package. The discussion in the following sections is based on the code generated in that solution.

Implementation

Babel interfaces are defined in the babelservice.idl file, which you can find in the Language Service Package solution in the common project under Source Files. You can find a full description of these interfaces in Babel Interfaces.

The default Babel implementation of the IBabelService Interface is the StdService class in the stdservice_.cpp file in the common project under Source Files. The lexer is accessed through the standard yylex method, and the parser is accessed through the standard yyparse method.

The getTokenInfo method in the StdService class can be overridden to provide a list of tokens to support syntax highlighting and token triggers. This list is an array of TokenInfo objects that specify the token ID, colorable item class, token type, description, and triggers. The TokenInfo structure, the StdService and CommentService classes, and all other support classes are defined in stdservice.h.

You can override additional methods in the StdService class to add or modify various language features. For example, you may want to override the getMethodFormat method to return a different format for methods (which controls how method signatures are displayed in the IntelliSense Parameter Info ToolTip).

A Note About Lexers and Parsers

The Babel implementation must link to a lexer and parser to scan and parse a source file. You can provide your own lexer and parser, or you can use one of the lexer and parser generation tools that are generally available.

A lexer transforms raw text into tokens that are used for colorization and token identification. A lexer tool (such as Flex) creates a lexer from a lexical specification, which uses regular expressions to specify the text that constitute tokens.

A parser transforms a list of tokens into an abstract syntax tree and is used for error messages, brace matching, and so on. A parser tool (such as Bison) creates a parser from a grammar specification using Backus-Naur Form (BNF) expressions.

The Visual C++ library in the default implementation of IBabelService Interface has been tested with Flex version 2.5.4.a and Bison 1.24. When you create your language using the default Babel implementation, use version 1.24 or later of Bison (or version 1.9 or later of yacc), or version 2.5.4a or later of Flex. 

Change History

Date	History	Reason
July 2008	Rewrote and refactored project.	Content bug fix.