Sarbanes-Oxley and Team Foundation Server 2010

Northwest Cadence

April 2011

This paper provides an overview of the Sarbanes-Oxley Act of 2002 (also known as SOx) as it relates to software development practices and the associated risks to financial systems. Visual Studio Team Foundation Server 2010 can be used to reduce many of those risks and provide evidence as proof of compliance for auditors.

Applies To

Team Foundation Server 2010

What is SOx?

How does SOx apply to software development?

How does Team Foundation Server help?

There are many scenarios which may apply to SOx compliance and in order to meet the requirements of SOx a tool must have a comprehensive and flexible set of capabilities.

What is SOx?

In 2002, in response to several major corporate accounting scandals the U.S. Government enacted the Sarbanes-Oxley Act. SOx contains 11 sections (or titles) related to accounting and financial disclosure and applies only to publicly held companies. Of key interest to the information technology divisions of companies required to abide by SOx regulations is Title IV (Enhanced Financial Disclosures), Section 404 (Management assessment of internal controls). This section states that with regard to annual financial reports an internal control report will also be submitted and which will contain the following:

• (1) state the responsibility of management for establishing and maintaining an adequate internal control structure and procedures for financial reporting; and

• (2) contain an assessment, as of the end of the most recentfiscal year of the issuer, of the effectiveness of the internalcontrol structure and procedures of the issuer for financialreporting.

The full text of SOx Title IV, Section 404 is available on the following website: Public Law 107-204 on page 45 of the PDF document, page 789 of the statute.

This loosely defined section has sprung up a cottage industry of auditing teams for SOx compliance. But what does it mean to you or your organization who has to implement these controls and what are controls anyway?

Compliance Terminology

This article uses several terms that have specific meanings, as they relate to corporate compliance. This section introduces some of this terminology:

• Framework—An overall collection of defined risks and associated mitigation activities (or "controls") that span the business and have a specific compliance focus, such as corporate financial reporting.

• Risk-based—The framework is based on an up-front, clearly defined risk appraisal in the context of Sarbanes-Oxley 404. The risk appraisal defines a set of specific financially related risks that become the focus of control objectives (as described later).

• Performed in accordance to a formally defined framework—The framework has a significant degree of rigor: It is usually defined by a qualified, authorized appraiser and customized for each business; it contains checkpoints and tests; and senior managers must attest to the resulting data that appears on corporate financial reports, and so on.

• Evidence-based—The framework is substantive, that is, it generates evidence that actions are occurring in the business to demonstrate the management of identified risks. Without such evidence, controls cannot be demonstrated as working.

  참고

  When we use the term "evidence" in this article with Team Foundation Server 2010, we generally mean "information," and do not mean to imply that information that is generated by Team Foundation Server is sufficient to demonstrate compliance. You should work with your own counsel and Sarbanes-Oxley 404 experts to determine that question.

• Continuous—The actions within the framework have frequency and continuity, which means that it does not "happen" just at the time of audit; it is an ongoing process. Auditors will typically attempt to establish that the processes that are described occur on an ongoing basis, and will drill into ad-hoc samples to confirm this.

• Action-oriented—Compliance is not achieved by a tool, but by defined actions that are performed regularly by people (for example, the project board must periodically review risks and document their decisions against each item). It is a characteristic of successfully implemented frameworks that tool support and human activity combine to deliver the level of compliance that is required. Frequently, activities flow across departmental boundaries, between personnel, and between systems—requiring a coordinated approach to achieve compliance.

• Customer-related—A customer (or business owner) is considered to be any group—either internal or external—for whom contracted software-application development work is performed, where this work is considered of significance for auditing purposes to Sarbanes-Oxley 404 requirements.

• Control – A required process which produces evidence which provides feedback as to the effectiveness of the control. Controls and the evidence they produce are subject to audits and must withstand the scrutiny of such audits.

Outline of a Regulatory Framework in Action

While there is commonality among many businesses—such as order processing, accounts payable, manufacturing, R&D, distribution, sales and marketing, and so on—it is nevertheless necessary for a business to conduct an individual risk assessment for its own enterprise. The purpose for the business is to establish a set of specific risks that are relevant to Sarbanes-Oxley 404, and then to establish procedures that manage these risks successfully over time. Figure 1 shows the importance of the initial risk assessment and how the management of each specific risk relates to this:

Figure 1. Regulatory framework for a business

The initial risk assessment is made by the business, in conjunction with its compliance expert. The scope of this assessment is broad—potentially, touching many areas across the organization. This might be a phased exercise, with the more significant business areas and risks being established first.

For each applicable risk that is identified by the business, a regulatory control objective is written that declares the intention of the risk management. For example, the specific risk might be that "changes to production data cannot be traced to the approver," and the corresponding control objective would be "all changes to production data have a documented approval process which is traceable." A control activity is then put in place; this is a procedure with steps that fulfill this control objective—for example, a document which contains a description of the change, why the change is necessary, who approved the change and who implemented the change. The completed document forms the evidence for this control.

How does SOx apply to software development?

In the previous sections you have seen a general description of how SOx relates to financial reporting and how regulatory frameworks are structured. However, SOx 404 does not specifically say “if developers are building an application of type A then do this…” Understanding what types of controls typically apply to the software development process is the key to understanding whether your tool of choice can manage the risks being audited for. The following list shows some common risks present in the software development process (as it relates to financial applications) that auditing firms examine when they are determining SOx compliance.

Common risks when you are developing financial applications

• How do you know that the requirements of the system have been met?

• How do you know that no unauthorized changes have been made to the system?

• Who approved the requirements that went into the system?

• Who approved the release of the software into production?

• Was the application tested to verify correct functionality?

• Who made changes to the system?

These risks represent two categories – traceability and quality. Traceability is a key issue because auditors have to know who requested and approved changes to systems involved in financial reporting (this includes financial systems which provide numbers used in reporting such as an inventory or sales system) to make sure that unauthorized changes (or changes requested and approved by individuals not authorized to approve those changes) have not been made. The quality aspect is there to make sure that the numbers that are reported out of the system are accurate. Because software does so much of the work that in the past multiple sets of eyes reviewed manually, the accurate processing of data is even more critical in today's environment.

Once the risks are understood, the process of turning these risks into controls must be done. As an example, take the third risk – “Who approved the requirements that went into the system?” A control for this may be that every requirement requires a physical signature on a document and that no changes to the document may be made after it is signed. This of course leads to a more complex situation relating to how change management on a system is performed. But it is a start and leads into how Team Foundation Server can help meet these risks through a variety of built-in features and extensibility points.

How does Team Foundation Server help?

While Team Foundation Server is a general purpose software development and work tracking tool it provides features uniquely suited to documenting information that helps eliminate the risks that this topic describes.

참고

The information provided here documents features of Team Foundation Server 2010 that can be used to meet common controls related risks such as those described earlier in this topic. However, as with anything related to Sarbanes-Oxley 404 compliance, your company auditor must approve the specific controls and the evidence that is provided to verify that a control has been met. Therefore, discuss the information presented in this document together with your auditor before using Team Foundation Server as your control tracking system.

Team Foundation Server provides three main pieces of functionality as it relates to SOx – Work Item Tracking, Version Control and Automated Build. These three features working together provide the capability to eliminate many of the risks associated with SOx 404 compliance.

Traceability

At the heart of SOx is the ability to know who made what change when, what requirement was the change made for and who authorized the change to be made. Was the requirement tested to make sure that it works correctly and who authorized the release of that requirement to production? Or, to state it another way, is the requirement traceable?

Many technical solutions require multiple tools to provide this kind of traceability and generally they break the traceability chain at some point because they are not integrated at all required points. With Team Foundation Server 2010, full traceability is baked in.

Requirements to Code

The most basic traceability is from a requirement to the code that implements that requirement. The reverse of this is also true – someone can examine the source code and determine how each line of code made it into the final version of any particular source file. Figure 2 shows a Requirements work item which contains a description of the requirement and a full history of all changes that were made to the requirement.

Figure 2: Requirement Work Item

From a traceability perspective this is a must have and is supported by all work items, not just a requirement. There are three other pieces of information in the requirement work item which also apply to traceability: the Implementation, Change Request and Test Cases tab. Figure 3 shows the implementation tab and a list of all tasks that you need to complete the requirement.

Figure 3 - Requirement Work Item - Implementation Tab

The ability to link work items using a specific type of link provides for complete traceability and reporting on these relationships. Using Team Foundation Server, you can go one step further which is to trace down to the code level. Figure 4 shows the relationships between tasks and code.

Figure 4 - Traceability - Tasks to Code

When code is checked in it can be associated with a task which is in turn associated with a requirement which leads to full traceability from code to requirements and vice-versa. In this example, Changeset 37 is associated with Task 5 which is a child of Requirement 1.

Figure 5 - Traceability from Code to Requirements (Annotate)

Taking this one step further a developer (or auditor) can examine the code and trace from the code back to the requirement as shown in Figure 5.

Requirements to Test Cases and Results

New in Team Foundation Server 2010 is the ability to manage test cases and results and, again, provide full traceability from those test cases to the requirements that they are testing. Looking at the All Links tab of the requirement shown previously, Figure 6 shows the test cases which test the requirement.

Figure 6 - Tracing Requirements to Test Cases

In in this manner, auditors can view the requirements and see all of the test cases which have been created to test those requirements. Drilling into the test case provides a view of the steps that you need to validate that the requirement is ready for production. Figure 7 shows the steps of a sample test case.

Figure 7 - Test Cases

Using Microsoft Test Manager, as soon as the test case has been executed, the results are associated with the test case and are directly traceable to any requires that the test case validates. All of this information comes together in a single report called the Overview (Requirement or Stories) report shown in Figure 8.

Figure 8 - Stories Overview report

Very quickly an auditor can view the list of requirements, determine how many test cases exist and what the results of those tests has been. In other words, the question of whether a requirement has been tested before being released is absolutely clear with a minimal amount of work. And test results can be output into a hard copy as additional evidence of compliance.

The relationship between work items and code is very simple to use because it works out of the box. There are no additional tools to install or pieces of software to learn – it is an integrated solution which reduces the amount of work that is required to provide evidence of compliance to an auditor.

Customizing Traceability

While the out of the box features of Team Foundation Server 2010 provide great flexibility there are also situations where you may want to add more traceability and the ability to report on certain types of links between items. In Team Foundation Server there are a defined set of links as noted in Table 1.

Table 1 – Built-in link types

Link Type	Description
Parent/Child	Hierarchical – a recursive relationship between work items. A child can have only one parent and a parent can have many children.
Affects/Affected By	Shows a relationship where one item (such as a Change Request) affects multiple other items.
Tests/Tested By	Indicates where a requirement, change request or bug is tested by a particular test case.
Predecessor/Successor	Indicates an order of precedence as it relates to work that must be performed.
Related	A flat relationship that shows a bi-directional relationship of no particular type.
Hyperlink	A link to an external URL or URI.
Versioned Item	A link to an item stored in Team Foundation Server Version Control.
Changeset	Links work items to source code check-ins

While these links are helpful and can be used in many different ways, consider the following situation: The SOx team wants to quickly be able to report on the authorization of all items for a particular release. On its surface this seems fairly simple to do in a variety of ways. You could add a field to each Requirement or User Story work item called “Release #” and “Approved By” and have someone fill it in for every item (fairly simple to do) but this approach is fairly inflexible if you have to add more information and it is repetitive. Another approach is to create a Release work item type that has information related to the release which can then be updated in one location as necessary. This approach has the merits of being simple, easily extensible and is not repetitive.

How then do you link this Release work item type with all of the work items going into a specific release? You could use an Affects/Affected by but reporting on that relationship might mix in other items like change requests as well. In this situation, creating a new link type called “Approved/Approved By” or “Releases/Released By” might be a better solution and in Team Foundation Server 2010 this is easily done. Figure 9 shows a new Release work item type.

Figure 9 - A Custom Release Work Item Type

Certain fields have been added to make the release item beneficial. The Release Information section contains an Approved By field in addition to an actual release date. The approved by field here is free form but the person who fills in the name there is recorded in the history. You can make this field only editable by people in the appropriate security group to restrict access to this field. Also, notice that it “Releases” Requirement 1. Looking at Requirement 1 you can see that it is “Released By” Release 4.3.2 (Figure 10).

Figure 10 - A Requirement traceable to an approved release

This is the custom “Releases/Released By” link type created specifically for this purpose. Teams can now report specifically on this relationship.

Managing Change

Change Management is an area of particular importance to being compliant with SOx. Change management is simply the ability to accept changes, properly authorize the changes, implement (or not implement) the change and verify that there have been no negative effects (regression bugs) from the change. Team Foundation Server allows you to implement as simple or as complex a change management scheme as your teams need. The MSF for CMMI template has a Change Request work item built in but for the MSF for Agile template you will have to improvise a bit because change is managed as part of the regular process. From a compliance perspective it is okay to improvise – just document the process the team is using, follow it and you will be compliant.

중요

Managing Change on an Agile project: Agile development does not really have the concept of a change request. Agile methodologies assume there will always be change and handle it as just the addition of a new feature. In other words, you do not create a change request, you just create a new user story and add it to the backlog. It may be that from an auditing perspective the simple maintenance of the Product Backlog (since this involves reviewing user stories and prioritizing them) and Iteration Planning (accepting user stories into an iteration or authorizing them) will satisfy the needs for managing changes. It is always best to check with your auditor.

The Change Request work item, out of the box, contains a description of the change, analysis and justification of the change. In addition to this, all work items share the concept of a State. By default the states are Proposed > Active > Resolved > Closed. In this configuration Proposed is exactly what it means, Active means that it is being worked, Resolved means it is ready to be tested and Closed means that the tests passed and it is ready to go. However, you may need more, such as an actual “Approved” state to a) record the fact that the change request is approved and b) record the authorized person who approved the change request. This may also extend to requirements as well.

There are two capabilities which support making this change – first, the work item system in Team Foundation Server is highly extensible allowing for many different changes and configuration to suit team needs. Different work items can be created, fields can be modified (or added or removed), and different business rules can be applied to those fields. Along with these changes is the second capability – the ability to modify the workflow for each work item and to apply rules to the state changes.

For example, in the previous scenario a new state can be added so that the states are Proposed > Approved > Active > Resolved > Closed. A rule can then be applied to the state transition from Proposed to Approved so that only a specific individual or group can move the change request from Proposed to Approved. Because everything is recorded as part of the work item history, the specific individual who changed the state is recorded.

Reporting and Auditing

The ability to report on almost any information in Team Foundation Server makes up for the ability to control everything. In almost every situation you will find that the only control mechanism is process which is not always easy to follow. It is important that any system involved in SOx compliance be able to provide reports so auditors can inspect the data to determine whether things such as unauthorized changes have been made. So what does an unauthorized change look like? In the first place, it is a change that is not associated with a Task or possibly a Task that is not associated with a requirement or maybe a requirement that has not yet been approved.

Backing Team Foundation Server (in most installations, although this is not a requirement) is a SQL Server Analysis Services data cube which enables teams to report, over time, on the progress of their projects. It provides detailed reporting on all aspects of work items and source control. The main mechanism for reporting on this data is through Microsoft Excel (you can also use SQL Server Reporting Services, SQL Server Report Builder, PerformancePoint or any other tool which can access a data cube).

Figure 11 shows a simple Excel report to validate that all code that has been checked in is related to a work item (the assumption is that the work item is traceable to a requirement).

Figure 11 - Auditing Traceability

This report shows how many lines of code were modified as part of each changeset (row labels) and which work items the check-in is associated with (column labels). Note that changesets 70, 71 and 72 are not associated with any work item and are therefore not traceable. This ability to quickly and easily see this information enables teams to perform self-audits before they have to go through an official audit. Items such as this can be corrected before any official audit (code can be related to work items after it has already been checked in – and even this change is audited so the date the link is made (instead of when the code is checked in) can be checked).

Other reporting is a function of how the tool works. For example, knowing what source code changed in each build and which work items were included in each build helps auditors understand what went into a release. The build report provides evidence of what is included in each build. Figure 12 shows an example of the build report.

Figure 12 - Build Reports

The build report notes the day the build was executed, who executed it, where it executed, what tests were run as part of the build and what work items and code were associated with the build. To help make releases easier you can create a “promotion” build. The build engine gathers the details of this report by looking at the last successful build and noting all of the differences made to the code (and the related work items) between then and the current successful build. A promotion build is a build that is executed only when moving to production. In this way the build report will gather all of the source code changes and all of the work item changes from the last time that you promoted code to production. And in one report you have everything that went into a particular release.

Team Foundation Server provides a comprehensive and flexible set of capabilities through traceability, test case management and automated builds. The rich extensibility of both the built-in features and a third-party ecosystem gives you the tools you need in order to meet the requirements of SOx.

See Also

Concepts

Technical Articles for Visual Studio Application Lifecycle Management

Northwest Cadence is a national leader in Microsoft ALM and Software Lifecycle solutions. Recognized by Microsoft as a Gold ALM Partner, Northwest Cadence focuses exclusively on Application Lifecycle Management with clients across the globe. With experience providing consulting services on Microsoft ALM Tools dating back to product inception (Visual Studio 2005 Team System), Northwest Cadence has actively worked with clients and Microsoft on product development, implementation, and process incorporation. Northwest Cadence has a solid commitment to honesty and excellence. This commitment coupled with the vast experience means Northwest Cadence clients know to expect an exceptional experience, every time.