Enterprise Search and Bing Services – Part 1: The Bing Translator
(In May of this year, Microsoft launched its Bing search service for the Web. While Bing has shown steady growth in the Web search market, it’s not well known that Bing also includes a collection of services that can be accessed programmatically to enhance enterprise applications. This is the first of series of guest posts that explore how to combine some of these Bing services with Microsoft enterprise search offerings. nt)
If your organization has customers or employees in multiple regions around the world, chances are you have the need to search across content in multiple languages. Earlier this year Microsoft Research announced the Bing Translator AJAX API (http://www.microsofttranslator.com/dev/ajax/) – an interface that enables developers to integrate the translator into any Web-based application. In this article we will take a high level look at how to integrate the Bing Translator with enterprise search – specifically with the FAST Enterprise Search Platform (ESP).
The Bing Translator AJAX API is a remote service that currently supports 20 different languages. The API features include auto detection of language as well as translation between any two languages. For applications with secure data, the API supports the HTTPS protocol for secure connections over the Internet.
The screenshot below shows one example of a FAST ESP powered search application with results containing documents from different languages.
In this example, not only is the user’s query translated and expanded to include other languages (French, German, and Chinese), but the user has the ability to translate the teasers or the entire document using the Bing Translator. The search results also include query highlighting for each of the multiple translations of the query. Finally, the user can use the slider bar (or the visual navigator) to favor documents written in certain languages. Any slider action causes the result set to update automatically. The relevance control behind this slider widget is actually a feature of FAST ESP, but it shows another way of surfacing cross-lingual search.
There are various ways to display and expose query translation features to an end user, and the example above is just one. While this example applies query translation automatically, it’s better, in our experience, to allow the user to select it as an option. Alternatively, the application can display translated results in separate tabs.
Integrating with FAST ESP
This example implementation integrating the Bing Translator was done as a Query Transformer (QT) in the FAST Query Result Server (QRServer). Depending on the query, the query transformer can also suggest query translations to the user. Implementing the translator as a query transformer means it can be used with any of the supported ESP search API’s, across multiple UX implementations, and is platform independent.
query = office hours, translate = on, language = en
The query transformer changes the above input query to the multiple variations of the terms based on its original language. The input language is known since the users are usually coming from a regional portal or have a language preference set in their browser. The QT gets back multiple terms from the Bing Translator by connecting to the API through remote services (over HTTP). The original query is then expanded to search for all translated terms. All query terms are cached to minimize traffic going over the wire. Any other FAST query time linguistics features, like stemming, spell checking, and synonyms, will still apply on the translated terms.
This is just one example of integrating enterprise search with Bing services. If you’re interested in including this particular capability in your search application, or you have any more questions, please feel to reach out to me at Runar.Olsen@Microsoft.com.
Runar Olsen, Senior Architect | Microsoft Enterprise Search Group, Services