September 2013

Volume 28 Number 9

The Working Programmer - Exploring NSpec

By Ted Neward | September 2013

Ted NewardReaders of my column will know that one of the mechanisms I like to use to explore or demonstrate a new technology (library or otherwise) is what a colleague described once as an “exploration test”: a suite of unit tests designed not to test the technology, but explore it. It allows me—the developer/user—to validate some assumptions I’m making about the technology. More important, it gives me a regression suite to run when a new version of that technology comes out, so that if the new version makes some kind of critical breaking change, I can find out about it right away, and not after I’ve made the upgrade (and had to discover the breaking change at run time, surrounded in the context of my program, rather than the much lighter-weight context of a test suite).

On the subject of testing, however, there’s been some angst and consternation within the community over different “modes” of testing: unit testing versus Test-Driven Development (TDD) versus Behavior-Driven Development (BDD), usually accompanied with lots of shouting and screaming. Because of the religious nature of such debates, I’ve tended to avoid that subject matter, but recently (as part of doing some research for this column, in fact), I’ve discovered a testing tool—NSpec—that comes from the BDD crowd. Regardless of how much you like or dislike the BDD approach to testing, NSpec is worth exploring.

Getting Started

As with so many of the other Microsoft .NET Framework packages these days, getting started with NSpec is just a NuGet “Install-Package” command away. As a stalking horse for it, I begin by creating a Class Library solution (in C#, because it won the rock-paper-scissors-lizard-Spock contest that I had between it, F# and Visual Basic for rights to be the code to test) to build a Reverse Polish Notation (RPN) calculator class that I’ll call “NiftyCalc” for lack of anything more original. NiftyCalc is going to be a bit of a deviant from traditional RPN calculators, however, mostly so I can have something a little more complex to test.

I’ll begin with bare-bones basics. After creating a new solution and Class Library project in Visual Studio, I’m going to create a simple class that has a single, auto-generated property member, named Current, to reflect the current value held in the calculator’s memory:

namespace Calculator
{
  public class NiftyCalc
  {
    public NiftyCalc()
    {
      Current = 0;
    }
    public int Current { get; set; }
  }
}

It ain’t much, but it gives enough scaffolding to get NSpec installed so I can look at the beginnings of using it.

After doing the “Install-Package nspec” in the Package Manager Console, a couple of things will have appeared. First, NSpec inserts another file into the Class Library project called DebuggerShim.cs. I won’t go into much detail on that now, but it lets you run NSpec tests via TDD and ReSharper, along with providing for continuous integration. More important, NSpec hasn’t created a standalone project as a peer to the Class Library project, like a standard Microsoft Test or NUnit test scenario would—the intent is that NSpec tests (or, as the company prefers to call them, specifications) are living inside the same project as the code they’re specifying.

Which means, then, that to create a test to test this, you can just create a class that extends the base class “nspec”:

using NSpec;
namespace Calculator
{
  class my_first_spec : nspec
  {
    void given_the_world_has_not_come_to_an_end()
    {
      it["Thunderhorse should be Thunderhorse"] =
        () => "Thunderhorse".should_be("Thunderhorse ");
      }
  }
  // ...
}

This clearly isn’t a traditional unit test: it lacks the custom attribute tags; the naming convention has all those underscores in it; there’s some kind of string-to-lambda dictionary at work here (“it”); and everything is essentially private/internal. To further make things significantly different, NSpec installed a new test-runner tool into the PATH inside the Package Manager Console, and that’s where you turn in order to execute the tests. In the Package Manager Console (which, remember, is basically a Windows PowerShell console running inside Visual Studio), run NSpecRunner.exe with the full path to the compiled class library:

PM> install-package nspec
Successfully installed 'nspec 0.9.65'.
Successfully added 'nspec 0.9.65' to Calculator.
PM> NSpecRunner.exe .\Calculator\bin\debug\Calculator.dll
my first spec
  given the world has not come to an end
    Thunderhorse should be Thunderhorse
1 Examples, 0 Failed, 0 Pending
PM>

Now the odd naming conventions become clear: NSpec is using reflection to discover the classes and methods to execute as part of the test suite, and it uses the underscores in the names as spaces when printing the output. The intent here is to get to more natural-­language-like syntax, so that I can write in “My first spec” that there’s a specification, “Given the world has not come to an end, Thunderhorse should be Thunderhorse”; so long as the lambda expression associated with that last part of the spec yields true, everything is good. Should the spec/test be changed slightly such that the two strings aren’t equal, NSpecRunner will give a different output (just as any test runner would), as shown in Figure 1.

Figure 1 A Test with Two Unequal Strings

PM> NSpecRunner.exe .\Calculator\bin\debug\Calculator.dll
my first spec
  given the world has not come to an end
  Thunderhorse should be Thunderhorse - FAILED - Expected string   length 12 but was 6. Strings differ at index 1.,   Expected: "Thunderhorse", But was: "Hello ", -----------------^
FAILURES ****
nspec. my first spec. given the world has not come to an end. Thunderhorse should be Thunderhorse.
Expected string length 12 but was 6. Strings differ at index 1., Expected: "Thunderhorse", But was: "Hello ", -----------------^
   at Calculator.my_first_spec.<Thunderhorse_should_be_Thunderhorse>b__2() in        c:\Projects\Publications\Articles\MSDN\WorkingProg\NSpec\NSpec\   Calculator\   Class1.cs:line 19
2 Examples, 1 Failed, 0 Pending
PM>

As might be inferred, the actual “test” part of the spec is the should_be extension method that NSpec introduces onto anything that derives from System.Object. Further, there are a number of similarly named overloads on the idea, such that if you want to make sure 2+2 is greater than 3, you can use the should_be_greater_than method instead. Another half-dozen or so overloads of similar vein are available; anyone familiar with any unit-test framework should be comfortable with these.

One drawback to NSpec, at least the version (0.95) I tested, is that NSpec isn’t deeply integrated with Visual Studio, so that you can’t just hit F5 and build/run the projects; instead, you have to explicitly build the library, then find the Package Manager Console and up-arrow to the NSpecRunner line to kick off the tests. The Web site NSpec.org has a solution to this, called specwatchr, which is a Ruby script that keeps an eye on the files in the directory and kicks off NSpecRunner whenever a source file is saved. Although I don’t mind it, some developers and some shops will turn up their nose at installing the Ruby platform just to have this functionality, useful as it may be. Personally, I think whatever distaste you might have for installing Ruby on your machine is vastly offset by having a tool that automatically runs tests like this, particularly as it has proven to be a highly useful technique in other language and platform environments, with the .NET Framework as the lone contrarian on this point. I recommend taking the time to give it a whirl—in a virtual machine (VM) that you don’t care about, if you must.

Testing NiftyCalc

Meanwhile, with some of the NSpec basics out of the way, I can turn to putting some tests around NiftyCalc and extending it. For starters, I want to make sure that when a NiftyCalc is created, its current contents start at 0; when I push a number onto the NiftyCalc stack, that number is what’s displayed as Current; and if I push a number, push another number and pop the stack, the first number is retained, as shown in Figure 2.

Figure 2 Testing NiftyCalc Functionality

using NSpec;
namespace Calculator
{
  class nifty_calc : nspec
  {
    void calculator_basics()
    {
      it["New NiftyCalc should have a Current of 0"] =
        () => new NiftyCalc().Current.should_be(0);
      it["Pushing 5 should show Current as 5"] =
        () =>
        {
          NiftyCalc nc = new NiftyCalc();
          nc.Push(5);
          nc.Current.should_be(5);
        };
      it["Push, push, pop, should have Current set to first pushed value"] =
        () =>
        {
          NiftyCalc nc = new NiftyCalc();
          nc.Push(5); nc.Push(7); nc.Pop();
          nc.Current.should_be(5);
        };
      }
    }
  public class NiftyCalc
  {
    private Stack<int> theStack = new Stack<int>();
    public NiftyCalc()
    {
      theStack.Push(0);
    }
    public int Current
    {
      get { return theStack.Peek(); }
    }
    public void Push(int value)
    {
      theStack.Push(value);
    }
    public int Pop()
    {
      return theStack.Pop();
    }
  }
}

So far, so good. In fact, it’s almost redundant to describe in prose the test I want to write, then show the code for the test, because the very syntax of the NSpec test registry (the “it” instance inside of the NSpec-derived class) really allows for as verbose or as terse a description as I’d like. This is, as mentioned earlier, by design, so that tests and specifications are easy to read and understand—which, considering how often test suites are being used as documentation these days, is a valuable thing.

Exceptions

Of course, there’s also the matter of some edge cases to consider: What should happen in the event a user tries to pop one more value off of the NiftyCalc than was ever pushed? It’s a common problem with stacks, but NiftyCalc isn’t trying to be a stack, but rather a functional calculator. Toward that end, both the Pop method and the Current property’s get method should check to see if there’s anything in the stack to retrieve, and if not, yield a value of zero.

However, should the business prefer that NiftyCalc toss an exception when popping an empty stack, the NSpec code can check for that, like so:

it["Should throw a NullRef if the NiftyCalc is null"] =
  expect<NullReferenceException>( () => {
    NiftyCalc nc = null;
    nc.Push(5);
  });

Again, this isn’t that far away from what a unit-test framework (such as Microsoft Test Manager or NUnit) looks like.

Context

There’s a fair amount of repetition with the preceding NSpec code, by the way; it would be nice if you could set up a “context” in which the test is run, so you don’t have to constantly allocate the NiftyCalc and push a few values in order to test. (This will be invaluable in the next set of tests, because I want to test the functionality of add, subtract and so on.) NSpec allows you to create such a context, as shown in Figure 3.

Figure 3 Setting Up a Test Context

using NSpec;
namespace Calculator
{
  class nifty_calc : nspec
  {
      void before_each()
      {
        calc = new NiftyCalc(); 
      }
    void calculator_basics()
    {
      // ...
      context["When NiftyCalc has a 2 and 3 pushed"] = () =>
      {
        before = () =>
        {
          calc.Push(3);
          calc.Push(2);
        };
        it["should Add to 5"] = () =>
        {
          calc.Add();
          calc.Current.should_be(5);
        };
        it["should Subtract to 1"] = () =>
        {
          calc.Subtract();
          calc.Current.should_be(1);
        };
      };
    }
    private NiftyCalc calc;
  }
  public class NiftyCalc
  {
    private Stack<int> theStack = new Stack<int>();
    public NiftyCalc() { }
    public int Current
    {
      get { return (theStack.Count != 0) ? theStack.Peek() : 0; }
    }
    public void Push(int value) {
      theStack.Push(value);
    }
    public int Pop() {
      return (theStack.Count != 0) ? theStack.Pop() : 0;
    }
    public void Add() { Push(Pop() + Pop()); }
    public void Subtract() { int top = Pop();     
    int bot = Pop(); Push(bot - top); }
  }
}

This test also shows how to make use of the “before” hook, to set up each individual test exercise before executing it—this gives you the chance to create the NiftyCalc instance and push two values to it before handing it into the test for execution.

Growing Usage

There’s a whole slew more about NSpec—and the larger subject of BDD—beyond what’s been discussed here, but this gets some of the basics in play. BDD enthusiasts will criticize the style and approach I’ve taken to writing my tests, but that’s a stylistic and aesthetic debate I neither want to participate in nor try to capture in this article. NSpec itself is an interesting approach to testing—and more important, a component that’s appearing in more and more open source projects. If you want to learn more about NSpec, Amir Rajan, codeveloper of the framework, has kindly volunteered to answer questions and provide guidance. You can reach him on Twitter at twitter.com/amirrajan or via his GitHub site at github.com/amirrajan.

Happy coding!


Ted Neward is the principal of Neward & Associates LLC. He has written more than 100 articles and authored and coauthored a dozen books, including “Professional F# 2.0” (Wrox, 2010). He’s an F# MVP and speaks at conferences around the world. He consults and mentors regularly—reach him at ted@tedneward.com if you’re interested in having him come work with your team, or read his blog at blogs.tedneward.com.

Thanks to the following technical expert for reviewing this article: Amir Rajan (Improving Enterprises)