How to: Fail a Build When Tests Fail
Got an email from Martin Woodward this morning asking:
What would be your preferred way to fully fail the build on test failure rather than partially succeed the build?
A bit of background - Team Build 2008 marks builds as Succeeded if no errors are encountered during the build process, Partially Succeeded if compilation succeeds but an error is encountered at some other stage of the build process (e.g. a unit test fails), and Failed otherwise. Depending on where you are in your development process, this logic may or may not make sense to you. For example, if you are two weeks away from shipping your product, a unit test failure might be just as important to you as a compilation failure.
The design of Team Build 2008 doesn't really allow for setting the overall status of the build directly - the setting of the build status is what indicates to TFS that the build is "complete", which triggers the build queue logic to start up the next build, etc. As such, you really shouldn't ever set this property of the build yourself!
You can get around this by setting the properties Team Build uses to determine the overall status of the build, however - CompilationStatus, TestStatus, and an internal logger property that keeps track of whether any errors have occurred. Note the above algorithm for determining status one more time - Succeeded if no errors are encountered, Partially Succeeded if CompilationStatus is Succeeded but an error occurs at some other stage, and Failed otherwise. So - if you want the overall status of the build to be Failed, you'll need to set CompilationStatus to Failed.
So - here is my recommended approach:
<Target Name="AfterTest"> <!-- Refresh the build properties. --> <GetBuildProperties TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Condition=" '$(IsDesktopBuild)' != 'true' "> <Output TaskParameter="TestSuccess" PropertyName="TestSuccess" /> </GetBuildProperties> <!-- Set CompilationStatus to Failed if TestSuccess is false. --> <SetBuildProperties TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" CompilationStatus="Failed" Condition=" '$(IsDesktopBuild)' != 'true' and '$(TestSuccess)' != 'true' "> </Target>
The idea here is to check whether or not tests have succeeded (see my previous post here on this topic) and then update the CompilationStatus (and thus the overall Status) of the build accordingly. If you want the build to halt immediately in addition to marking the status as Failed, you can use an Error task, and you'll probably also want to move the logic to the AfterTestConfiguration target (to avoid running tests for each configuration before halting the build).