“Evil” way to federate search results through a password protected proxy

In real world environment, people sometimes use password protected proxy to make company employees to access the Internet. Most of the time, that is a basic authentication. So in this kind of environment, the federated search webpart of Microsoft Search Server 2008 will not work out-of-the-box because we only support non-password protected proxy.

But is there any way to workaround?

Yes, otherwise why I’m talking about it?

For the word “evil”, I’m not referring to the definition of the word from a “not evil” company. My “evil” is always some kind of tricks, or hacks, and you will love them because they can really solve problems. BTW – in MMO RPGs I’m always a chaotic neutral character, but I like evil ones - that’s my best description.

The theory is to make a data tunnel through the password protected proxy, so we can map external website to local port, and federate the search result. There’re some applications which can do the job, but here we will use HTTPort as an example.

Here’re the steps!

1. Get a copy of HTTPort from www.htthost.com, the newest version is HTTPort 3.SNFM. Install it.

2. In proxy configuration window, fill in your proxy server name and port, check “Proxy requires authentication” and then input your username and password for accessing this proxy.

3. Check the RSS feed website domain name you want to federate. In this example, we are using Live Search China. The domain name is “cnweb.search.live.com”.

4. Click Port mapping tab of HTTPort, and add a new port tunnel. Fill in a local port, for example, 991, then fill in remote host name and port.

5. Switch back to proxy tab. press start button in lower right corner.

6. Check if it works in your browser: replace domain name of your RSS feed with 127.0.0.1:991. If everything is going on well, and you are lucky enough, the RSS feed will be there and you can make Search Server federate it through this new local URL!

7. Just some note: Not every service can be federated like this. If the target website has more security check, for example Yahoo search, the RSS feed cannot be fetched through such tunnel. Therefore you have to consider other ways, or spend some time to imporve this evil hack:).

HTTPort is a free software written by Dmitry Dvoinikov.