Sharepoint

Building a Powerful Survey Infrastructure

Jim Bradley

At a Glance:

• Planning and implementing a survey solution
• Gathering and processing survey results in SharePoint libraries
• Managing performance and security issues

You can't make good decisions without information. Hunches just don't cut it—not when your decisions will impact your business. This is true whether you're deciding where to host a team

party, or if you're contemplating the reallocation of 50 percent of your company's resources to launch a new product. But how do you get this information?

Surveys provide an effective—and cost-effective—way to collect feedback on everything from how satisfied customers are with your product offerings to whether or not employees liked the sandwiches at your latest team meeting. Surveys yield the substance needed for future product, system, or process improvements and development. The question is, how do you, the IT professional, implement a survey system that can collect data and store it so the information can be used in meaningful ways?

You probably already have one or more of the tools needed to gather, structure, and analyze this sort of data. The 2007 Microsoft® Office system provides these tools and makes them quite easy to use. In fact, there are numerous ways you can go about performing a survey. The key is to match the approach to your particular needs. For instance, e-mail-based surveys are a good choice for impromptu, real-time data gathering when dealing with a small number of respondents and non-critical data. For more information, see the sidebar below "Using E-Mail for Quick Surveys". Database-driven surveys, on the other hand, are a better choice for more complex initiatives that include a larger group of respondents and pertain to mission-critical data. See the sidebar below "Build a Database-Driven Survey Solution" for more information on this topic .

In this article, I'll focus on another, more comprehensive and flexible solution: SharePoint®-based online surveys. This approach is suitable for both mission-critical and non-critical surveys of any level of complexity. Because SharePoint surveys are Web-based, they can be completed by anyone who has access to a Web browser—even mobile devices are supported. With a SharePoint survey, responses can be named or anonymous, real-time results are available, and you even have access to analysis tools.

This survey type is implemented using either Windows® SharePoint Services 3.0 (WSS) with Forms Server and InfoPath® or Microsoft Office SharePoint Server 2007 (MOSS) with Forms Services and InfoPath® (see Figure 1 for a description of each of these technologies). Because both scenarios offer essentially the same functionality, I will focus this discussion on the latter combination. For a comprehensive comparison of SharePoint products and their functionality, see Microsoft Office SharePoint Server 2007 Products Comparison Download at office.microsoft.com/en-us/sharepointserver/HA101978031033.aspx.

Figure 1 Microsoft technologies that enable SharePoint surveys

Plan and Implement

When designing a survey, you need to consider a number of factors up front. This takes some advance analysis, so be sure to take the time to develop a well-thought-out survey plan. In the initial planning stage, you'll want to define the problem space and the sort of data you are looking to obtain, determine the technologies you need to use, and establish the budgetary and administrative requirements. Then you can move on to the implementation. The general workflow is to design a survey form, publish the form, collect and validate responses, aggregate and analyze the data, and then report the results.

A survey implementation workflow typically includes at least a survey designer, respondents, and an analyst (as shown in Figure 2). Of course, survey workflows can vary greatly in complexity. A company-wide survey, for instance, would likely go through several rounds of design edits and approvals. It would probably utilize automated reminders to respondents, offer various forms of support and troubleshooting, provide real-time views into the results for managers, and have a clear process for determining and delivering the final analysis. A complex workflow of this sort really requires a Web-based system that supports workflow and reporting capabilities.

Figure 2** A survey workflow from form design to data analysis **(Click the image for a larger view)

What to Look for in a Comprehensive Solution

There is a long list of key requirements you should expect a comprehensive survey solution to satisfy. The solution should allow individual teams or business units to create, disseminate, and collect survey results with minimal involvement from the IT department—and it should not require any programming skills. The forms creation tool should offer an easy-to-use WYSIWYG interface, as well as a rich set of features that facilitate piping, branching, and conditional logic. And, it goes without saying, the survey-form creation tool should accommodate all surveys, regardless of size and complexity.

Moreover, the survey should run on a Web server and integrate with a centrally maintained SQL Server® database, eliminating the need for individual departments to maintain dedicated SQL databases. Any respondent with Internet access should be able to complete the survey from any standards-compliant browser.

The survey process should be easily linked to workflows and it should meet business needs without compromising security. And, for many organizations, the survey solution should provide multi-language support.

When used together, MOSS 2007, WSS 3.0, and InfoPath 2007 create an integrated solution that addresses all of these requirements. Figure 3 shows how the various components fit together in the SharePoint stack. But before I discuss this complete survey solution, I want to take a quick look at what you can achieve with WSS alone. Then I will show you the added benefits you get when adding MOSS and InfoPath to the mix.

Figure 3** Components of the SharePoint stack **(Click the image for a larger view)

Working with WSS Alone

Even without the other components, you can use WSS to create and implement a survey. In fact, WSS includes a built-in survey template that makes the process easy. To create a survey, click Start | All Programs | Administrative Tools and select SharePoint 3.0 Central Administration. From the dropdown list, select Site Actions and then click Create. The Create page is displayed, showing you headings for Libraries, Communications, Tracking, Custom Lists, and Web Pages. Under each heading, there are template options. Under Tracking, click Survey.

At this point, WSS leads you step-by-step through the survey creation process. You can create open or closed questions and specify whether a question can be left unanswered. You can even create branching logic surveys, which will lead respondents down different paths, depending on their answers. Surveys can be anonymous or, if necessary, identify the respondent. You also have the option of assigning a workflow to the survey.

The survey is browser-based, so no special software is required to create or respond to the survey. To complete the survey, respondents simply go to the SharePoint site and complete the form. Access permissions are inherited from the parent site but can be edited directly from the SharePoint Actions menu. Survey responses are saved to the SharePoint survey site, and results can be viewed in list form or as graphical summaries or exported to Excel®.

To reiterate, this solution (with the process shown in Figure 4) can be achieved with WSS and no additional software. Assuming you already have a well-designed SharePoint site in place, an individual department should be able to create and implement a survey with little to no assistance from IT.

Figure 4** A survey based on Windows SharePoint Services **(Click the image for a larger view)

One limitation of the WSS-only process becomes apparent when a survey needs customization. WSS creates survey lists using various default ASPX pages (AllItems.aspx, DispForm.aspx, EditForm.aspx, NewForm.aspx, overview.aspx, and summary.aspx). While page customization beyond the parameters inherent in WSS is possible, most features come from Web Parts that are not readily customizable. Moreover, the WSS-only solution is best suited to surveys that do not need to integrate external data sources, such as your company's ERP system.

When you do need to expand the survey solution to include a custom UI or integrate with additional data sources, you should seriously consider the solution that combines MOSS, WSS, and InfoPath.

The Complete Survey Solution

MOSS includes many features that extend WSS, but for now I want to focus on a single component—InfoPath Forms Services. With InfoPath Forms Services, only the survey designer needs InfoPath installed on her computer. Others can access the survey through a Web browser.

In InfoPath, the designer selects the option that enables the form to be filled out using a browser. InfoPath then creates a form that can be viewed on any standards-compliant Web browser. This Web-based form performs in the same way as an InfoPath form, with the exception of some advanced InfoPath features (such as user roles, vertical text, dialog box actions, and advanced controls). The form is then published to a SharePoint library or list. Meanwhile, another InfoPath feature, called Design Checker, ensures that the survey form is compatible with InfoPath Forms Services.

There is no difference between an InfoPath forms library and a browser-enabled forms library. In each case, a form template is an .xsn file. Either InfoPath Forms Services displays the form in a browser or the file is downloaded to the client and displayed directly in InfoPath. If a survey designer has at least Contributor permissions for a SharePoint site, he can use SharePoint document libraries to publish form templates.

A respondent with a standards-compliant browser can complete the InfoPath Forms Services survey, and his responses are then seamlessly returned to the SharePoint server. Within SharePoint, the data can be securely stored, parsed, shared, and analyzed (using Excel, SQL, or other tools available on the network). All this can be accomplished with minimal IT administration assistance.

Postbacks and Performance Issues

There are potential performance issues, however. For example, browser-enabled forms run in the context of a system account on the SharePoint server, which means that if a form includes code or data connections (as shown in Figure 5), the code or data connections are executed on the server, not the client. And complex forms may need to frequently post data back to the server, which increases the workload on the server.

Figure 5** InfoPath Forms Services-based survey solution with additional data connections **(Click the image for a larger view)

The more that a survey uses postback, the higher the load on the front-end Web servers. However, there are times when postback is unavoidable, such as when implementing response branching. To minimize potential adverse effects on the system, survey designers must be aware of postback issues and understand how to best implement surveys when postback can't be avoided. For instance, designers should use wizard-like surveys that post data back to the server when the user clicks the Next button. This helps reduce the number of postbacks.

InfoPath Forms Services must maintain the state of each currently active form on the server. The default session timeout is 60 minutes, which means that if a complex survey takes longer than 60 minutes to fill out and there are no postbacks during that time, the session will be closed on the server. Data that is entered but not submitted is lost, and the respondent must start again.

Such issues are magnified when a large number of respondents are working concurrently on a survey form and when surveys have large data sources or include file attachments. For more information, see "Improving the Performance of InfoPath 2007 Forms" at msdn2.microsoft.com/bb380251.

Security Considerations

While an in-depth security discussion is beyond the scope of this article, it is important to highlight a few security considerations. For starters, SharePoint and InfoPath 2007 adhere to the Trustworthy Computing Initiative that Microsoft adopted in early 2002. InfoPath forms have three possible security levels: Restricted, Domain, and Full Trust. And, by default, InfoPath automatically determines and applies a recommended security level to the form.

Digital signature functionality can help insure that a form is created or filled out by a specific user and that it is not altered. And Information Rights Management (IRM) can restrict access to templates and completed forms. To prevent malicious users from uploading forms that contain harmful code or using the SharePoint platform to launch attacks against other systems through data connections, InfoPath distinguishes between user-deployed and administrator-deployed forms. Users can upload forms as long as the forms do not contain any custom code and use only the Domain security level, which limits cross-domain data connections. These parameters are sufficient for the majority of survey solutions.

When a survey design requires the Full Trust security level, in order to permit unrestricted access to resources, form publishing requires IT administration approval. By default, IT administration approval is required when a form contains managed code, when cross-domain data connections are defined in the form template, when the form uses data connections defined in the centrally managed data connection library, and when the option to support rendering on mobile devices has been enabled.

For the Full Trust security level, the survey designer creates an InfoPath form and saves the form template. The template or .xsn file is sent to the SharePoint administrator as an e-mail attachment or passed along through a network share. The IT administrator then checks the survey features and any code it contains before making the survey available on the production system. The IT administrator completes deployment by uploading the survey template to a site collection and activating the survey template. Both procedures are completed on the Application Management page of the WSS Central Administration console. The end result is that an IT administrator can delegate forms publishing to individual departments and still maintain oversight of the publishing process if forms go beyond basic data-gathering activities.

IT administrator oversight is necessary in certain common scenarios: when forms must be populated with default data and when survey results are submitted to multiple data sources. By default, Domain Trust form templates cannot establish cross-domain data connections, but there are a few actions you can take to resolve this limitation including:

• Granting the form Full Trust permissions.
• Creating Domain Trust using data connections from the data connection library.
• Using Domain Trust with the centrally managed connection library.

Full Trust and the centrally managed connection library options require administrator approval during forms publishing. However, the data connection library (DCL) allows user-published forms to cross the domain boundaries using the DCL maintained at the site-collection level. This DCL can be under the control of the individual department. It's important to understand, though, that allowing individual departments to define their own server-based data connections can be a security issue.

The most secure option is to define data connections in the centrally managed connection library and then have an IT administrator deploy advanced survey forms that use these data connections. The centrally managed connection library is advantageous because it is available across all site collections and the entire server farm. This allows the administrator to define central authentication settings for accessing data sources that are not on the local SharePoint server. For additional information, read the article titled "About Data Connections, Authentication, and Alternate Access Mapping," online at msdn2.microsoft.com/ms771995.

Wrapping Up

Surveys provide a powerful way to gather mission-critical (and not so mission-critical) information for every business, at every level. Creating an effective survey, however, requires the right tools, a sound plan, and a balanced mix of art and science. Thoughtful planning upfront will help you build a solution that allows each department within your organization to create, send, receive, store, and analyze survey data.

The most effective surveys are those that yield timely, relevant information. In order to gather information when it is needed, your solution must make it easy to create and manage surveys at the department level—without compromising security. InfoPath empowers information workers to create high-quality surveys by providing an intuitive interface that can be used with little training. And InfoPath lets you gather high-quality survey data without requiring a lot of IT administration.

InfoPath 2007 seamlessly works with WSS and MOSS to facilitate rich, complex survey solutions that are cross-platform and cross-browser compatible. And this allows your business to harness one of its most important assets: information.

Using E-Mail for Quick Surveys

Impromptu surveys are an essential tool for quickly collecting data from a small group of respondents. E-mailbased surveys are perfect for this sort of real-time research—they can be created quickly and don't require dedicated servers or IT assistance. One of the simplest approaches, voting buttons embedded inside a Microsoft® Outlook® message, lets you send, receive, and tabulate a one-dimensional survey.

After launching a voting buttons survey, your recipients receive an e-mail message that contains a voting band, which displays a dropdown list of options. The recipient responds by clicking one of the options. A popup window then displays a message, "You have chosen to respond: <your choice>", and offers the choice to Send or Edit the response.

The survey designer, or an assigned delegate, receives the individual responses in e-mail and manually tabulates the results. If the survey includes a large number of recipients, you should create a rule in Outlook (using the Rules Wizard) that will automatically route the responses to a dedicated folder, as demonstrated in Figure A.

Figure A** Survey responses automatically routed to a dedicated folder **(Click the image for a larger view)

Voting buttons, however, are quite limited. Survey forms inserted into e-mail messages can collect much more data, but manually retrieving and tabulating this data can be very unmanageable. On the more complex side of e-mail-based surveys is the use of VBScript to customize Outlook surveys. Customized forms can make it easier to gather and analyze larger quantities of more complex data. To access developer functionality in Outlook 2007, you must first display the Developer tab, shown in Figure B.

Figure B** Developer tab in Outlook 2007 **(Click the image for a larger view)

Creating custom Outlook forms is beyond the knowledge of the typical Outlook user. But InfoPath® 2007 provides new features that make it easy to implement customized forms. With no special expertise, survey designers can create InfoPath forms and templates in three ways:

• By importing existing Microsoft Word documents or Excel® spreadsheets.
• By downloading predesigned InfoPath templates and modifying them to meet specific needs.
• By designing templates from scratch using the form design functionality of InfoPath.

In all three cases, templates can be published to Outlook and distributed via e-mail. With InfoPath 2007 and Outlook 2007, a few hours, and little to no assistance from IT, an information worker can create a twenty-question survey, e-mail it to a group, receive the results in Outlook, and analyze the data in an Excel spreadsheet.

The only real requirement in this scenario is that all participants have InfoPath and Outlook installed on their computers. And since InfoPath 2007 actually extends the functionality of Outlook 2007, the recipients can respond to the survey right from within the Outlook interface.

But InfoPath supports much more than just e-mail-based surveys. You can create templates that query and submit data to Web services and SQL Server® databases, or you can use an existing XML document or XML Schema as the data source. In other words, with minimal training and little or no added code, a user can tap into the power and flexibility of XML.

E-mail as a survey solution has some notable disadvantages. For example, these surveys are not anonymous, as the respondents e-mail address is available. This can reduce candor and skew the results. There are also security concerns, such as phishing attacks, so there is a limit to the type and depth of information you can gather using e-mail-based surveys.

Build a Database-Driven Survey Solution

E-mail-based surveys aren't suited to more complex studies that involve large numbers of respondents—using a folder in Outlook® just isn't sufficient for managing and analyzing all the data. A more robust solution is to direct survey responses to a database. The advanced storage, indexing, processing, and reporting capabilities you get with a database can be very helpful. Microsoft® SQL Server® 2005, for example, includes SQL Server Reporting Services, which can be used to process survey results automatically.

To create an InfoPath form template that will submit responses to a database, you must first create an Access® or SQL Server database within your network. Then, in InfoPath®, you can start the template creation process using the Design a Form Template wizard. This opens the Data Connection Wizard, which steps you through the process of linking your form to the database. InfoPath uses the information in your database to build the data source query and data fields.

Most workers will have enough knowledge to complete and submit the survey with little training or support. However, in this scenario (where data is passed directly to the database), respondents must have InfoPath and Outlook installed, and they must be behind the corporate firewall or have VPN access to the corporate network.

The InfoPath Data Connection Wizard interfaces only with Microsoft SQL Server 2000 or later and Access databases that are natively using ADOXML. Direct submission to the database is not supported for other databases. Other limitations are that InfoPath does not allow rich text controls to be bound to database fields, and large binary data types are not supported.

As the network topology becomes larger and more complex, it becomes increasingly advantageous to submit data to an intermediate Web service and then pass that data to a database. Decoupling the survey form front end from the database back end, by means of a Web service, also facilitates implementation of business logic on the Web server. And since the Web service is performed over HTTP or HTTPS connections, participants can submit data through firewalls, as shown in Figure C. Note, however, that respondents still must have InfoPath 2003 or 2007 installed.

The downside to this scenario is that the implementation requires IT participation. Describing this process in detail is beyond the scope of this article, but I will warn you that building a database solution from scratch is a significant undertaking. It requires advance planning, good survey design, programming of the Web service, and design of the database back end. InfoPath 2007 simplifies the forms design aspect of the process, but the end-to-end solution requires solid programming and database design skill. And adding advanced functionality, such as workflow or multi-language support, quickly increases the cost of implementation.

Figure C** InfoPath data flow through a Web service **(Click the image for a larger view)

Jim Bradley is the owner of CoyoteTech LLC, a technical communications company that specializes in user assistance documentation for Microsoft server products. Documentation projects include "Microsoft Exchange Server 2007 Edge Transport and Messaging Protection", "Going 64-Bit with Microsoft Exchange Server 2007", and "Microsoft Exchange 2003 Technical Reference Guide".

© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.