How do you test nondeterministic code?
Jay posted some of the code we wrote on the lazy loader and weak references. An astute reader pointed out that there was a flaw in my implementation. Specifically I was messing up how I interacted with “null” and “None<A>.Instance”. None<A>.Instance is both a singleton and an example of Fowler's “Null Object” (see Introduce Null Object). However, I didn't catch it because it passed all testing that I put it through. The issue was non of the testing ever hit the WeakReference when it had actually released it's hold on the object. I wanted to add tests to make sure that works, but I wasn't sure how. Not only are weak references out of my control as the class is provided by the BCL but its highly dependent on how the GC works which is basically a non deterministic beast that i don't have enough control enough. (Even if the BCL provided a “collect all weakreferences method”, I'd just extend this question to any non-deterministic code). So how do I test this? How do you test code like this?
Note: this also applies to multi-threaded code. The best I've been able to come up with is doing something like spawning tons of threads and having tehm go nuts and then verifying validaity of state every so often. Bleagh.