Interoperability Testing

 

Scott Seely
Microsoft Corporation

August 15, 2001

Download Interop.exe.

Introduction

By the time this column hits the Web, we should have a graphic showing what Matt and I look like. Given the size constraints for the graphic, we wound up choosing a picture that shows both of our faces as clearly as possible in a small graphic. Amongst the traditional shots, the graphic artist attending the photo shoot suggested that we try a fun picture as well. The idea: the column is called "At Your Service"; the original photo of Mary Kirtland had her holding a platter. What if we had Matt holding my head on a platter? Through the magic of Adobe Photoshop® and the hard work of Gary Spradling, we have it!

Figure 1. The picture that we just had to show you

Don't worry: no heads were actually severed to create this picture. No vegetables were harmed either—Gary had that available from an image library. With this little bit of fun out of the way, let's get to the topic at hand: Web Service interoperability.

I am also trying out something a little different this time around. This article assumes you are somewhat familiar with the Favorites Service and that you have read many of the articles. If you have not been following the project (and shame on you if you haven't!), I include a list of relevant links at the end of this article.

The one big reason many people have chosen to implement Web Services is the promise of cross-platform and cross-language interoperability. The skeptics out there all have one question in mind: "How true is this?" To answer this question, you have to look at how interoperability is attained. In practice, two Simple Object Access Protocol (SOAP) implementations will interoperate if they agree on the following things:

If you had to implement all of this on your own, the work alone would be too intimidating. Fortunately, the above specifications have proved so compelling that implementations exist for most devices and can be accessed through most common programming languages. Due to the pervasiveness of networking and the Web, you should have no problem finding a TCP/IP stack or HTTP implementation for your chosen platform.

You should also have no problems finding a working XML parser. XML itself has been stable for quite some time, and the major XML implementations (Microsoft, Apache, Perl, and so on) have all implemented the latest W3C XSD recommendation. The hard part comes in finding compatible SOAP and WSDL implementations. Fortunately, many people are interested in making their implementations talk to each other. The result of this desire can be seen on the soapbuilders mailing list. If you want to find a solid implementation of SOAP (and usually WSDL too), make sure that the implementation is represented on this list. To easily check for participation, go to the Round 1 home page or the SOAPBuilders Interoperability Lab Round 2 home page. Both pages list implementations at various stages in attaining interoperability. So far, I have experimented with four of the implementations listed on the Round 1 home page:

  1. Microsoft SOAP Toolkit 2.0 (COM-aware languages)
  2. SOAP::Lite (Perl)
  3. Apache 2.1 (and 2.2) (Java)

I also experimented with one that is not on the page (but should be—it's pretty good and easy to use): SOAPy 0.1 for Python. Because Perl, Java, and Python are all fairly portable languages, the implementations sampled work well on Windows, Unix, Mac, or any other architecture that has a valid interpreter or virtual machine.

Testing the Interoperability Promise

For our interoperability testing, I wrote clients for the Favorites Service using the non-Microsoft toolkits that most people seem to use. Of these, there are really only two: Apache SOAP and SOAP::Lite. I picked the Python library for two reasons: I wanted to use a third language, and I wanted an excuse to use Python. I will comment more on that experience in a bit.

With the clients, I developed the same application three times. I figured I should handle the most basic operations an end user would want to handle. This application allows users to perform the following actions:

  1. Display their favorites
  2. Add a favorite or category
  3. Rename a favorite or category
  4. Delete a favorite or category
  5. Quit (I have to let the user exit the application!)

I made some other "under-the-cover" assumptions about the setup. I assumed that only one user would want to use the simple application. I also assumed that the script is specific to a given installation of the Favorites Service so that I can hard code the licensee name and password in order to gain access to the service.

When writing the code for the three, mostly identical applications, I tried to keep function names similar across implementations to allow for easier comparisons and understanding. As I cover the implementations, I will comment on the good and bad parts of using the various toolkits.

In all fairness, not one of the toolkits worked well out of the box. All required some modification to the WSDL file, but the modification depended on the implementation. I also wound up disabling Secure Socket Layer (SSL) when logging on. I tried to use it and install the correct SSL transports for the various languages, but without luck. I contacted various people and have come to the conclusion that I have far too much beta software running on my machine. I fully believe that everyone else can and does have a successful SSL implementation. That said, if you need to disable SSL support on your own machine because you also suffer from PEBKAC (Problem Exists Between Keyboard And Chair) errors, here is how:

  1. Open up C:\Program Files\SSF\PhaseI\WebService\logon.asp

  2. Locate the line that checks for SSL.

    If Request.ServerVariables("HTTPS") <> "on" then
    
  3. Change to read like this:

If false and Request.ServerVariables("HTTPS") <> "on" then

Once this is done, you can access the Logon Web service without SSL.

The User Interface

The user interface is console based. I went retro, opting for text menus and keyboard input. This allowed me to pay a minimal amount of attention to user interface details and focus on the SOAP part of the applications. The menu and prompts for all interfaces is identical.

Figure 2. Standard menu for application

The application reads all input from the keyboard and displays the output on the screen. Almost no input validation is performed. Why? I wanted to make sure that various toolkits could talk to the Favorites Service, but I didn't want to spend a lot of time writing code that guaranteed valid input. Instead, I left that as an exercise for the reader.

The Apache Implementation

Of the three implementations, this one required the most work. Because I wanted to take advantage of WSDL, I went to IBM and downloaded the Web Services Toolkit. The toolkit allows you to generate a Java proxy object if you have a local copy of the WSDL file. Like all of the various toolkits, the source WSDL for the proxy had to be modified ever so slightly. When the Microsoft SOAP Toolkit 2.0 generates a WSDL file, it uses a namespace named wsdlns. The IBM proxy generator tool does not know what to do when it sees this namespace and, as a result, will not do what you want it to do.

To coax a proxy out of the tool, I opened up the local copy of the WSDL file and did a search and replace for "wsdlns:", replacing it with nothing. I ran the tool and it balked again, this time complaining that it doesn't understand the 2001 XSD namespace. I did a search and replaced 2001 with 1999 and tried again. I could do this because the fundamental types used by both Logon.WSDL and Account.WSDL have not changed between the 1999 XSD proposal and the 2001 recommendation. This time I typed in

c:\wstk-2.3\bin\proxygen.bat logon.wsdl 

and got a small error about the tool not knowing what to do with my parameters. I then went through the generated proxy (LogonSoapPortProxy.java) and filled in the function signatures so everything looked right and it compiled. You will find the edited versions in the download for this article.

I repeated the above process for Account.WSDL and then built the generated proxies. Once that was done, I needed to write the client application.

At this point, the first thing I did was to write the basic loop, menu, and Web Service logon code. I then wrote stubs for all the menu items (a process I would repeat for the Perl and Python versions). When it came time to make sure that the logon code worked, however, I got a bunch of errors from the Java code stating that it had no idea how to deserialize the returned string (even though the WSDL file said a string was coming back). When the Microsoft SOAP Toolkit 2.0 returned the key, they key came back just like this:

<Result>key-value</Result>

What happened was that Apache had no idea how to interpret the result. To handle the result, you need to tell Apache what object to use to deserialize the return value. If the return value is a complex type, you need to write your own handler. If it is a simple value, like in all of the Favorite APIs, life is a bit easier. Every proxy has a member variable named call of type Call. This class contains a variable of type SOAPMappingRegistry whose name is smr. To handle the return values coming back from the Web Service, tell the SOAPMappingRegistry the name of the value coming in and the object that will deserialize it. To deserialize a string, use the following:

Deserializer stringDser = new StringDeserializer();
smr.mapTypes ("https://schemas.xmlsoap.org/soap/encoding/", 
    new QName ("", "Result"), null, null, stringDser);

Other classes exist to deserialize other basic types, such as ints and floats. I took a final pass through the proxies, so that they would know how to deserialize the values returned by the Favorites Service. At that point, I was more or less done with the SOAP-specific part of the project.

All of the applications handle logon and capturing user information the same way. Somewhere, the application keeps track of the current key and user name. At startup, this information is set to the empty string. Two functions are then written to grab the user name and key: getUser and getKey. The Java functions are typical of the function as a whole:

public String getUser() {
    if ( currentUser.length() == 0 ) {
        // Ask the user for their name and store it.
        System.out.println( "What is your user name? " );
        try {
            BufferedReader stdin = new BufferedReader(
                new InputStreamReader( System.in ) );
            currentUser = stdin.readLine();
        } 
        catch(java.io.IOException e){
            System.out.println( "Failed to read in your input." );
        }
    }
    
    // The username should be filled in now.
    return currentUser;
}

public String getKey() {
    // Check to see if we have logged in yet.
    if ( key.length() == 0 ){
        
        try {
            // We haven't logged in yet, so get a key
            LogonSoapPortProxy client = new LogonSoapPortProxy();
            key = client.Logon( "TestSite", "UN9]M;UfII" );
        }
        catch( java.net.MalformedURLException e ) {
        }
        catch( org.apache.soap.SOAPException e ) {
            System.out.println( e.toString() );
        }
        System.out.println( "Key: " + key );
    }
    // The key should be filled in now, so return it.
    return key;
}

Of the versions of these functions, these are the most verbose. Java requires you to catch exceptions. The Perl and Python versions just error out if anything bad happens.

The rest of the code is a set of handlers for the menu items. They all follow the same mold: ask the user for some data, then execute the Web method. AddFavorite is typical of these methods.

public void addFavorite() {
    try {
        BufferedReader stdin = new BufferedReader(
            new InputStreamReader( System.in ) );
        System.out.println( 
            "What is the parent category for this favorite?" );
        Integer parentID = Integer.decode( stdin.readLine() );
        System.out.println( 
            "What would you like to name the favorite?" );
        String newName = stdin.readLine();
        System.out.println( "What is the associated URL?" );
        String newURL = stdin.readLine();
        AccountSoapPortProxy client = new AccountSoapPortProxy();
        client.AddFavorite( getKey(), getUser(),
            newName, newURL, parentID.intValue() );
        updateFavorites();
    }
    catch( java.net.MalformedURLException e ) {
        System.out.println( e.toString() );
    }
    catch( org.apache.soap.SOAPException e ) {
        System.out.println( e.toString() );
    }
    catch( java.io.IOException e ) {
        System.out.println( e.toString() );
    }
}

Again, exceptions are caught because Java mandates that I must either catch the exception or state that the method throws exceptions of the given type. The code simply prints the error message to the console and keeps on going.

To display the favorites to the user, I used a Simple API for XML (SAX) parser and a custom class. You can see the class in ClientHandler.java. This class is used within the Client.displayFavorites() method.

The SOAP::Lite Implementation

Of the three SOAP implementations, this one required the fewest changes to get it working. One of the good things about installing the SOAP::Lite library is that if you use the Comprehensive Perl Archive Network (CPAN), you will also get anything else you need installed when you install SOAP::Lite. Directions for installing SOAP::Lite are here. Outside of the SSL issues (which, as I previously stated, were most likely my fault), this ran fine with no modifications to the WSDL files (other than I had to change the Logon.asp URL to go through port 80 (HTTP) instead of port 443 (HTTPS)). Calling various operations on the SOAP endpoint was all very easy. The code to call addFavorite is more or less the same length when considering the error-handling code.

sub addFavorite {
    print "\nWhat is the parent category for this favorite? ";
    $parentID = <STDIN>;
    print "\nWhat would you like to name the favorite? ";
    $newName = <STDIN>;

    print "\nWhat is the associated URL? ";
    $theURI = <STDIN>;

    # Remove the trailing '\n' at the end of each string.
    chop( $parentID );
    chop( $newName );
    chop( $theURI );

    SOAP::Lite
        -> service( $AccountWSDL )
        -> AddFavorite(getKey(), getUser(), $newName, 
              $theURI, $parentID );
    getFavorites();
}

The location of the Account.WSDL file is stored in a global variable named $AccountWSDL. This way, when I change the location of the file to read from a Web server or new local location, I only need to make the change in one location.

All the other item calls to SOAP methods worked equally well.

The SOAPy Implementation

While the difficulty of getting everything setup for SOAPy lies somewhere between that for Perl and that for Java, this one was the most fun for me to write. Why? Python is an object-oriented scripting language that comes with lots and lots of libraries. The language makes sense, and because of that, I was able to go from not knowing anything about Python to writing yet another implementation of the Favorites client application in about two days.

Let's take a look at what addFavorite looks like in Python, then go over what had to be done to make everything work. This is the shortest implementation of addFavorite.

def addFavorite(self):
    client = soap.get_proxy( self.accountWSDL )
    print"What is the parent category for this favorite?"
    parentID = sys.stdin.readline()[:-1]

    print"What would you like to name the favorite?"
    newName = sys.stdin.readline()[:-1]

    print"What is the associated URL?"
    newURL = sys.stdin.readline()[:-1]

    client.AddFavorite( key = self.getKey(), Username = self.getUser(),
                        FriendlyName = newName, uri = newURL,
                        ParentID = int( parentID ) )
    self.updateFavorites()

In case you are unfamiliar with Python, its big thing is indentation. Items at the same scope are indented the same amount. The idea behind this is to enhance readability. The idea looks like a winner. Again, most of the functions were equally easy to implement. What did I have to do to make all of this work?

The first problem I ran into with the SOAPy library is that it only supports 1999 XSD. As in the case of Java, I had to change the version of WSDL that the Python library loaded to only reference the 1999 namespace. The other problem I ran into was a bit more complex. The SOAPy library is more literal than any of the other implementations when it comes to reading the WSDL and matching the return values with the final results. The generated WSDL creates the part names as Object.methodNameResult. Guess what? The SOAP toolkit returns the parts as methodNameResult and SOAPy cannot find the matching elements. Other than this, it all worked just fine.

Conclusion

If you develop a Web Service and make it publicly available, you will wind up with many potential clients. These clients will be using many different libraries to connect to you. It behooves you to at least take a look at some of the more popular toolkits and see what alterations need to be made in order for those clients to connect to you. Then you will be able to offer the edited files for the client proxies. If necessary, as with Apache, you may need to provide alternative endpoints as well. Also, implementing these different clients will help you offer feature-rich samples for your clients to draw from.

In our next column, Matt will be back to talk about defending your Web Services against attacks.

Further Reading

Perhaps you have been a fan of the Favorites Service, but in case you are just now getting interested, here is list of links that will help get you up to date on the service. Some of the links also contain pointers to other helpful information.

  • Perl home page links to books and resources for Perl.
  • Python home page links to books and resources for Python.
  • Java home page is Sun's Java home page.
  • Round 1 Interop testing gives an overview of who is participating in the Round 1 interoperability testing, and describes what passing the interoperability tests means.
  • Round 2 Interop testing gives an overview of who is participating in the Round 2 interoperability testing and describes what passing the interoperability tests means.

 

At Your Service

Scott Seely is a member of the MSDN Architectural Samples team. Besides his work there, he has published two books through Prentice Hall: SOAP: Cross Platform Web Service Development Using XML and Windows Shell Programming. He wrote and maintains a small C++ based SOAP library (http://www.scottseely.com/soap.htm), published under the LGPL.