span.sup { vertical-align:text-top; }

Going Places

What Can a Robot Teach You?

Mike Calligaro

Contents

I, WiMo
WiMo Sings the Blues
WiMo Text Home
You Put Your Left Tread In ...
WiMo's Electric Eye
WiMoFi
Do WiMos Dream of Electric Sheep?

What can a robot teach you? If you're a fan of the best show on television, you're probably thinking, "Well, it could probably teach me how to walk, talk, and beat the frak out of my opponents." If, however, you're also a fan of the best mobile operating system on the planet, you might be thinking, "Well, it could probably teach me how to programmatically take pictures, use SMS as a data channel, send data over Bluetooth or WiFi, and be notified at various points in a playing song." I can't help you with beating the frak out of your opponents. But if you're interested in the other stuff, read on.

Although I've been a software developer at Microsoft for more than 14 years, I started life as a hardware design engineer. I graduated with a degree in Electrical Engineering and spent my career's formative years designing mainframes. This was back in the days when we had to lay out the logic gates by hand—none of that fancy Verilog stuff those young whippersnappers use today. (We also walked miles to work in the smog. Uphill. Both ways. And we liked it.)

This is something of a contrast to my friend and coworker, Brian Cross. Brian's a software guy to the core, always has been and always will be. He's been at Microsoft his whole career and speaks C# almost as well as English. (It was Brian—and Charles Petzold, of course—who taught me the wonders of managed code.) So it's somewhat ironic that, between the two of us, Brian is the guy who built a robot.

You may have heard of WiMo, Brian's Windows Mobile®-based robot, an illustration of which is shown in Figure 1. WiMo has gained some notoriety for being a fully programmable robot that you don't need to be an Electrical Engineer to build. Brian, in classic Microsoft style, sees the whole thing as a software problem. In its simplest form, you can build a functional WiMo out of nothing more than a Windows Mobile phone and some Legos—no soldering iron required.

Figure 1 Artist's Rendering of WiMo

But, easy as that may be, this is a column about mobile development, not robotics. So today's column isn't really going to be about building a WiMo. Instead I'll explore some of the things that WiMo's code base does and discuss how you can apply those same things to your mobile apps. If, along the way, you're inspired to do something really off the wall with your phone, all the better.

First things first. WiMo is basically a simple set of motors and sensors whose main purpose is to carry around a Windows Mobile device that acts as its brain. All of the real work is done by the phone, and all of the code that does this work is available for free on WiMo's Web site, wimobot.com. If you're interested in building a WiMo, you'll want to follow the Building WiMo link to wimobot.com/Building.aspx. If you're just interested in the source code, though, go to wimobot.com/SourceCode.aspx. We're going to be spending a lot of time talking about that source code, so you might as well get it now.

Go ahead. I'll wait.

What's that? Yes, click the disclaimer that says you won't sue Brian if you use his technology to build a machine that eventually turns on its creator. That'll never happen. Trust us.

After you've unpacked the code, open the file WiMo.sln in Visual Studio®. Don't mind the XML document errors; Part of WiMo's charm is that it's being done by Brian the "We can do anything" developer, not Brian the "We can't even think about releasing something without a proper test plan" corporate suit. WiMo.sln has a number of projects, but you should start at Form1.cs in the WiMo project. In that file, you'll see the usage of many of the more interesting classes defined in the solution.

WiMo Sings the Blues

The Windows Mobile part of WiMo communicates with the wheels and sensors via Bluetooth. Humorously, Brian chose this connection style because he was worried about accidentally ruining his phone if he tried to solder wires to it. All the better, because now you can lift this code and use it for non-robotic things. If you're looking for a more official venue for source code, you can find Blue­tooth samples on MSDN ® at msdn.microsoft.com/embedded/aa714519. You'll find a lot of similarity between the WiMo Bluetooth classes and the MSDN samples, though.

Most Windows Mobile phones support Bluetooth, and if yours is one that uses the Microsoft® Bluetooth stack (unfortunately, some don't), the WiMo code will work on it. Because Bluetooth is so common, if you want to write an app that communicates with nearby phones, using Bluetooth as your communication channel is a reasonable way to do it.

The Brain behind WiMo—a Phone

So say you've copied over the Bluetooth classes from WiMo or the MSDN samples, and you've paired your phone with a device called "MyDevice." Here's the code to connect with the device:

using Microsoft.WindowsMobile.SharedSource.Bluetooth;

BluetoothRadio radio = new BluetoothRadio();
NetworkStream bthStream = null;
foreach (BluetoothDevice device in radio.PairedDevices) {
  if (device.Name == "MyDevice") {
    try {
      bthStream = 
        device.Connect(StandardServices.SerialPortServiceGuid);
    }
    catch (SocketException) {
      // do error recovery stuff here
    }
    break;
  }
}

At that point, you have a standard NetworkStream you can use to send and receive data with bthStream.Read and bthStream.Write. And as easy as that, you're now talking over Bluetooth. You could use this for any situation where you've got your app running on two phones and you want them to communicate with each other. But there are a ton of other possibilities as well.

This is the basic method for talking to any Bluetooth device that speaks serial. For instance, if you want to connect to a Bluetooth-enabled GPS unit, you'd do something very similar. A poster on the WiMo blog said that he used code like this to collect engine performance data from his Bluetooth-enabled car. Lego Mindstorms speaks Bluetooth, so this is how the Lego version of WiMo works. And, if you're not afraid of a little soldering, you can connect an inexpensive Bluetooth modem to any serial device. (Brian has the details on the WiMo site under the "Hardware to buy to build a WiMo" section.) There's a huge array of sensors available in the world that you could connect to one of these modems. As mobile developers, you know what to do with the data once you get it onto your phone. The problem was traditionally that getting it onto the phone was hard. Not anymore.

WiMo Text Home

Because WiMo's brain is a phone, it can receive commands from a number of different sources, including SMS. This means that WiMo can receive commands from most of the cell phones on the planet, Windows Mobile or otherwise, and can even talk to desktop IM clients that support SMS. (Windows Live™ Messenger is one of these.)

This is handled by the MessageInterceptor class, which you can see being used in WiMo's DesktopSms.cs file. This file is under the WiMo.Common.Device project. It's also documented in full on MSDN at go.microsoft.com/fwlink/?LinkId=120888. MessageInterceptor basically says, "Whenever the phone receives an SMS with these conditions, send it to my app." You can even have it launch your app if it's not already running.

Message interception, like most things C#, is easy to implement. Figure 2 shows the basic code you need.

Figure2 Message Interception

using Microsoft.WindowsMobile.PocketOutlook;
using Microsoft.WindowsMobile.PocketOutlook.MessageInterception;

const string smsCommandId = "MyAppName";
const string smsComparisonValue = "MyAppSmsIdentfier";

MessageInterceptor intercept = 
  new MessageInterceptor(InterceptionAction.NotifyAndDelete);
intercept.MessageCondition = new MessageCondition();
intercept.MessageCondition.CaseSensitive = true;
intercept.MessageCondition.ComparisonType = 
  MessagePropertyComparisonType.StartsWith;
intercept.MessageCondition.ComparisonValue = smsComparisonValue;
intercept.EnableApplicationLauncher(smsCommandId);

intercept.MessageReceived += 
  new MessageInterceptorEventHandler(intercept_MessageReceived);

void intercept_MessageReceived(object sender, 
  MessageInterceptorEventArgs e) {
  if (e.Message is SmsMessage) {
    SmsMessage sms = (SmsMessage)e.Message;
    // sms.Body contains the text of the message we received
  }
}

You may think of SMSs as short e-mail messages, but these classes allow you to use them as a full data channel. Imagine writing a turn-based game like Chess (or Shogi, the far superior Japanese version of Chess) where you move by sending and receiving SMS messages.

Or imagine having an app on your phone that can be remotely configured. Do you frequently misplace your phone? (I'll bet it ran off with your car keys.) You could log into your IM client and send it a command to play a WAV file that proclaims, "Hey Mike, I'm over here." (Although telling me where your phone is would be kind of weird.) Or, if you want to be really hip, you could have your phone proclaim, "WiMo 5 is alive!" No one would get the joke, but that's part of the appeal.

You Put Your Left Tread In ...

This is a little embarrassing but ... how do I put this? OK, WiMo dances the Hokey Pokey. There, I said it. Like the three laws of robotics, this is actually a protection for humanity. If WiMo ever evolves to the point where it can pick up a weapon and say, "I'm sorry, Brian I'm afraid I can't do that," Brian will respond, "WiMo, dance!" Then, after completing the Hokey Pokey, WiMo will slink off into a corner and pretend that the whole sordid affair never happened.

Not that I'd ever suggest you code your Windows Mobile apps to interact with the Hokey Pokey, but the idea of being notified at certain points in a song is pretty nifty. And the actual infrastructure that the WiMo Hokey Pokey uncovers is really very cool.

The infrastructure I'm talking about is the Windows Mobile State and Notifications Broker (SNAPI). The managed version is in the Microsoft.WindowsMobile.Status namespace, and the class you really care about is SystemState.

A good resource for additional detail on the SystemState class can be found at go.microsoft.com/fwlink/?LinkId=120893. Click the link for SystemState Properties. See that list of a hundred or so system properties that you can access? Signal strength, battery level, Bluetooth states, whether or not the user is talking on the phone, the device's phone number, and so on? What could you do with access to that kind of power? Use it wisely. SNAPI can notify you whenever one of these values changes. It can even launch your app if your chosen value changes while your app isn't running.

WiMo dances as a result of being notified whenever the MediaPlayerTrackTimeElapsed property changes. It knows what to do at each point in the song and acts accordingly. Here's the code:

using Microsoft.WindowsMobile.Status;

SystemState mediaTimeElapsed = 
  new SystemState(SystemProperty.MediaPlayerTrackTimeElapsed);
mediaTimeElapsed.Changed += 
  new ChangeEventHandler(mediaTimeElapsed_Changed);

void mediaTimeElapsed_Changed(object sender, ChangeEventArgs args) {
  int time = (int)args.NewValue;
  // Do something with the new value
}

If you're interested in having your app launch when one of these states changes, look at the EnableApplicationLauncher method in SystemState. There you'll find the WiMo code that does this in the main Form1.cs.

WiMo's Electric Eye

Most cell phones today have cameras, and WiMo makes good use of the one built into its brain. The part of WiMo that holds the phone is on a disk attached to a motor. One of the things WiMo knows how to do is spin the phone around to point the camera at you, take a picture, and then spin the phone back around so that you can see the screen. The phone can also act as a rudimentary video camera, allowing you to drive WiMo remotely and watch what it's seeing.

In Windows Mobile 5.0 (which WiMo is based on), the camera API is somewhat limited. There's no managed interface to it, and the native interface is heavily geared toward taking a still image and saving it on a storage card. To use it as a video camera, frame rate is limited by how fast the device can store and retrieve images from that storage card. No one would ever claim that this is ideal, but it gets the job done.

There are, however, some silver linings in this situation. First, if you've ever wanted to write a native DLL that you could call from your managed application, WiMo provides a very nice example of how to do it. Look at the WiMoNative project, and then look at how the rest of the WiMo solution interacts with it. If you've ever pInvoked something from managed code, you're already halfway there. Calling into a DLL that you wrote is just a matter of pInvoking the functions you exported. For example, look at the end of the Camera.cs file under the Wimo.Common.Device project. Here are two of the pInvoke declarations you'll find there:

[DllImport("WiMoNative.dll")]
private static extern int OpenCamera(int id, ref IntPtr hCamera);

[DllImport("WiMoNative.dll")]
private static extern int CloseCamera(IntPtr hCamera);

And here are the native declarations of the same functions in ­WiMoNative.cpp:

HRESULT OpenCamera(DWORD dwResolutionId, HANDLE* hCamera);
HRESULT CloseCamera(HANDLE hCamera);

In addition, WiMoNative.cpp is probably the simplest introduction to DirectShow that you'll ever find. The OpenCamera function builds a filter graph and hooks up the appropriate pins in around sixty lines of code. Over the years I've had multiple experts try to teach me how DirectShow works. And each time my eyes have glazed over in the first five minutes. But when it comes to understanding code, there's nothing like stepping through a solid example that does something useful.

WiMo started with Brian saying, "I feel like learning something new. I'll tackle electronics and build a robot." But that's really only half the story. Brian also said, "I wrote all these Windows Mobile managed classes. I should do something that shows people how to use them." If WiMo feels like it uses every technology under the sun, that's because it does. Intentionally.

TCP/IP is one of those technologies. WiMo can talk to a desktop PC via WiFi, which uses the same managed TcpClient that wired Ethernet connections use. The code is available in WimoNetTcp.cs under the Wimo.Common project. The MSDN page for the TcpClient is msdn.microsoft.com/system.net.sockets.tcpclient.

Communicating over TCP is similar to communicating over Blue­tooth. You first make a connection, and then you use a Network­Stream to send and receive data. If you're connecting to a server, this is as easy as:

using System.Net.Sockets;
using System.Net;

const string myServer = "www.myserver.com";
const int myPort = 11000;

TcpClient client = new TcpClient();
client.Connect(myServer, myPort);

NetworkStream stream = client.GetStream();

// From here you can call the stream.Read and stream.Write APIs

stream.Close();

If you are talking to another device, you can use the version of client.Connect that takes an IP Address instead of a host name. Of course, then you need some way to get that IP Address from the remote machine. Getting your own addresses is easy, though. For example, let's say you want your IPv4 address.

string myHostName = Dns.GetHostName();
IPHostEntry hostEntry = Dns.GetHostEntry(myHostName);
foreach (IPAddress ip in hostEntry.AddressList) {
  if (ip.AddressFamily == AddressFamily.InterNetwork) {
    // ip is the IPv4 address   
  }
}

If you're writing an app that communicates via TCP to the same app on another phone, you might get your IP address, send it to the other phone via SMS, and then have the other phone use that IP Address to connect to you. For that, you should look at the Tcp­Listener class at msdn.microsoft.com/system.net.sockets.tcplistener. The general idea is this:

TcpListener listener = 
  new TcpListener(IPAddress.Any, port);
listener.Start();
TcpClient client = listener.AcceptTcpClient();
listener.Stop();

That's a whirlwind tour of TCP/IP, but I believe it's enough to get you going.

Do WiMos Dream of Electric Sheep?

There's a lot more to WiMo's codebase than I touched on here. WiMo is actually a fleet of robots with a number of different wheel and sensor modules. The early ones were built from robot parts (motors, microcontrollers, and circuit boards). But later WiMos were based on other things, including a remote control monster truck and Lego Mindstorms. Brian's currently working on turning a Roomba vacuum cleaner into a WiMo (you'll find the start of that code in the sources). There's a lot of neat code there that abstracts these different bases, as well as some good stuff for talking to an inexpensive microcontroller.

My goal in this column was to show you some of the technologies you have available to you as you develop your Windows Mobile apps. But if you want to go beyond that, don't hold back. If you've ever wanted to dive into electronics, WiMo is a great place to start, even if you don't know the difference between an inductor and a capacitor. The Wimobot site will show you all steps needed, everything from which components to buy to how to hook them together to how to write the code that talks to them.

If you're not interested in a device that can dance the Hokey Pokey, you've still got a huge range of options before you. The typical Windows Mobile phone is made of hardware that is better in all ways than the PC I used when I started at Microsoft. It's got a faster processor, more RAM, more storage, and a screen that shows more pixels. And you can carry it around with you.

Did you replace a Windows Mobile 5.0 device with a Windows Mobile 6 one? What's your WM5 device doing now? You probably disabled the cell phone service, but everything else should still work. If this is the case, you've got a fully programmable mobile computer collecting dust in a closet—you should do something with it. Build a robot. Make a security camera (WiMo takes pictures and sends them to a PC, and it doesn't need any wheels or sensors to do it). Build or buy a weather station that talks Bluetooth, and use your phone to gather and analyze data from it. Maybe even start working on that Astromech Droid. ("Come along, WiMo.")

I'd say "The sky's the limit" but that's too ... limiting. If you're reading this, you're probably an engineer. You can do anything. You just have to get started. I hope that WiMo and this column have inspired you to do something exciting with your Windows Mobile devices. Happy coding.

Send your questions and comments to goplaces@microsoft.com.

Mike Calligaro is a Senior Development Lead with the Windows Mobile team at Microsoft and a contributor to the Windows Mobile team blog, which you can see at blogs.msdn.com/windowsmobile.