Understanding and Creating Customized and Uncustomized Files in Windows SharePoint Services 3.0

Summary: Learn how you can create and use customized and uncustomized files to meet the scalability and high demands of dynamic customization in Windows SharePoint Services 3.0 sites. (8 printed pages)

Andrew Connell, Andrew Connell Inc. (Microsoft MVP)

April 2008

Applies to: Windows SharePoint Services 3.0, Microsoft Office SharePoint Server 2007, Microsoft Visual Studio 2008

Contents

  • Introduction to Customized and Uncustomized Files

  • Understanding Windows SharePoint Services 3.0 Architecture

  • Creating New SharePoint Sites

  • Understanding Customized vs. Uncustomized Files

  • Creating Custom Pages in SharePoint Sites

  • Conclusion

  • Additional Resources

Introduction to Customized and Uncustomized Files

Windows SharePoint Services 3.0 sites, like typical ASP.NET 2.0 Web sites, consist of ASP.NET 2.0 pages. To meet the scalability and the high demands of dynamic customization on a site-by-site basis, Microsoft introduced the concepts of customized and uncustomized files. This article explores the details of these concepts, including their advantages and disadvantages, and it demonstrates how you can create new customized and uncustomized files.

Understanding Windows SharePoint Services 3.0 Architecture

To fully understand the concept of file customization in Windows SharePoint Services 3.0 sites, you should have a good understanding of the underlying architecture. First, consider the architecture of a typical ASP.NET 2.0 Web site. When a new ASP.NET 2.0 site is created, a new Web application (also known as a Web site) is created in Internet Information Services (IIS). This IIS Web application points to a specific folder on the file system that acts as the site's Web root. It is in this folder that the files and code that implement a new site are found. These general steps are usually implemented by a server administrator instead of the developers or users of the site. If the Web site is load balanced across multiple servers, the same setup must be duplicated on all servers.

Although Windows SharePoint Services 3.0 is built on top of ASP.NET 2.0, the process of creating new sites is very different between the two platforms. At the code level, Windows SharePoint Services is a site-provisioning engine and Web-based collaboration framework. One of the goals is to facilitate a delegation of empowerment in that people within the organization are granted the ability to manage their sites without having to go back to an administrator to do manual setup work on the server every time a new site is created. If a new collaboration site is needed for a project, someone with appropriate permissions can create the new site by using the Web interface without any involvement from the administrator.

In addition to the delegation of empowerment, another capability of Windows SharePoint Services is to support the creation of hundreds, thousands, or even tens of thousands of sites within a SharePoint farm. Although ASP.NET 2.0 also supports creating multiple Web sites on the same hardware, it can quickly become a challenge to manage all the sites across servers on a load balanced farm because they all must contain the same code base.

Creating New SharePoint Sites

All SharePoint sites live within site collections. Site collections are logical structures that are used for many things that fall into management and administration categories. But to the point of this article, they are just containers of sites. Site collections are available through Web browsers through the HTTP protocol. IIS Web applications are used to provide the HTTP entry point to a site collection. Because of this fact, before you can create a site collection to host SharePoint sites, a server farm administrator must first create a new Web application by using the SharePoint Central Administration Web site. When Windows SharePoint Services creates a new Web application in IIS (or extends an existing Web application), it does a few extra things. It first loads a custom HTTP application and then replaces all installed HTTP handlers and modules with Windows SharePoint Services–specific ones. These handlers and modules essentially tell IIS to route all file requests through the ASP.NET 2.0 pipeline. This is because most files in a SharePoint site are stored in a Microsoft SQL Server database.

After a Web application has been created and configured for Windows SharePoint Services, administrators can then create site collections within the Web application. Web applications can contain multiple site collections. Each site collection is stored in a SQL Server database called the content database. This content database contains all the files, lists, and data for all sites in the site collection. One thing to note is that Windows SharePoint Services does not create any files or folders on the file system when the site collection or sites are created; everything is created in the content database.

SharePoint sites consist of numerous Web pages, just like any ASP.NET 2.0 application. When a site is created, these pages are created as instances in the content database. The instances refer to the actual file on the file system. This is known as a virtualized file system.

Understanding the Virtualized File System

Why did Microsoft decide to use a virtualized file system rather than the traditional ASP.NET 2.0 Web site approach that keeps everything on the server's hard disk? There are a few reasons for this approach.

First, Windows SharePoint Services is a site-provisioning engine, which makes it very easy for those with appropriate permissions to create new sites. In addition, the architecture is set up to support hundreds, thousands, tens of thousands, even hundreds of thousands of sites. If each site that was created resulted in a new Web root created on the file system, an organization of any size would quickly accumulate a large number of unmanageable Web roots on the server.

Second, consider a load-balanced environment. In a load-balanced environment, each SharePoint front-end Web server is essentially a mirror of other front-end Web servers. This means if an organization has hundreds of sites, they also have hundreds of Web roots, not to mention all the supporting files and assemblies for the site, on each server. This type of implementation would be very vulnerable to becoming out of sync with other servers or experiencing file corruption.

This approach of storing Web pages within a database deviates from the typical ASP.NET 2.0 site structure. How does the Microsoft .NET Framework know to pull the source of a page from the file system or from the site's content database? Before addressing how the sources of pages are resolved and retrieved, this article first explains the subject of customized and uncustomized files, and also the different types of pages, because they have a significant role in a Windows SharePoint Services environment.

Understanding Customized vs. Uncustomized Files

As previously outlined, some of the pages in Windows SharePoint Services sites are not found on the file system as they are in a typical ASP.NET 2.0 Web site. However, other pages are. These two types of pages are site pages (also known as content pages) and application pages. Site pages are the ones stored as instances in the site collection's content database. These pages fall into one of two categories: customized files or uncustomized files.

Uncustomized Files

When you create a new SharePoint site in a site collection, anywhere in a site collection, Windows SharePoint Services provisions instances of files into the content database that resides on the file system. The files on the file system are called templates. On their own they are not very useful because they cannot be accessed by using a browser. This is because they are not mapped to any specific URL. The instance of the file in the content database merely adds it to the site. When ASP.NET receives a request for the file, it first finds the file in the content database. This entry in the content database tells ASP.NET that the file is actually based on a file on the file system and therefore, ASP.NET retrieves the source of the file on the file system when it constructs the page.

This is how most files start in a SharePoint site. They are called uncustomized files. In the earlier version of SharePoint products and technologies (Windows SharePoint Services 2.0 and SharePoint Portal Server 2003) these files were called ghosted files, a term which has been deprecated. An uncustomized file is one in which the source of the file lives on the file system and not in the content database. The only thing that lives in the content database is the uncustomized instance that registers the file in the site.

One thing to note is that there can be multiple instances of a file in a site collection's content database, or content databases, all referencing the same file. This is how the out-of-the-box master page, default.master, is implemented. Every SharePoint site starts with an uncustomized instance of the default.master page. If an environment contained 100 SharePoint sites created with the Blank Site template, and no customizations had been made, there will be 100 uncustomized instances of the default.master page in the various content databases, yet there would be only a single default.master page on the file system.

Customized Files

The other type of file is a customized file. A customized file is one in which the source of the file lives exclusively in the site collection's content database. In earlier versions (Windows SharePoint Services 2.0 and SharePoint Portal Server 2003) these files were called unghosted files, a term which has been deprecated. How can a file become customized? If you modify the file in any way through the SharePoint API, the source of the new page is saved in the content database. This includes SharePoint Designer 2007, which uses the SharePoint API via RPC and Web service calls to change files in sites. When the file is requested, ASP.NET first finds the file in the content database. The entry in the database tells ASP.NET whether the file is customized or uncustomized. If it is customized, it contains the source of the file, which is used by ASP.NET in the page contraction phase.

A file can start as an uncustomized instance and become customized. For example, after a site has been created, someone can open the site in SharePoint Designer 2007 and make a few changes to the default.master page for the site. When the file is saved, it has changed from being uncustomized to being customized. This enables Windows SharePoint Services to store only one instance of a file template on the file system regardless of how many sites implement it, while at the same time providing for site-by-site customizations of the file. Even if the file is customized, you can delete the customizations and put the file back into an uncustomized state by either right-clicking it in SharePoint Designer and selecting Reset to Site Definition as shown in Figure 1, or by entering the full URL of the file on the following page (as shown in Figure 2).

Figure 1. Resetting to site definition to delete customizations

Figure 2. Resetting specific URL to site definition version

Files can also start their life as customized files. You can do this by creating the file using the Windows SharePoint Services API or a tool such as SharePoint Designer. When this happens, the file stays exclusively in the site collection's content database and can never be associated with a file on the file system.

How Windows SharePoint Services Handles File Requests

By default, ASP.NET 2.0 looks for requested files on the file system, and not in a database. So how does Windows SharePoint Services introduce the concept of page instances in a content database? ASP.NET 2.0 provides the pluggable System.Web.Hosting.VirtualPathProvider class, which enables developers to change how the source of the page is retrieved. The Windows SharePoint Services team implemented a custom VirtualPathProvider object that first looks to the site collection's content database for the requested file. Upon finding the file, VirtualPathProvider checks whether the file is customized or uncustomized. If the file is customized, VirtualPathProvider retrieves the source of the file from the database and returns it back to ASP.NET 2.0. Otherwise, if the file is uncustomized, it retrieves the file from the location on the file system that the uncustomized instance is based on. This process is shown in Figure 3.

Figure 3. VirtualPathProvider process flow

Two Types of Pages

Although Windows SharePoint Services is built on top of ASP.NET 2.0, it is unique because it has two types of pages: site pages and application pages. As a SharePoint developer, it is important that you understand the uniqueness between the two types of pages in addition to where you can use them and what you can or cannot do with each.

Site pages are the ASP.NET 2.0 pages that most SharePoint Server end users are familiar with. These are the pages that support customization in the sense that they can be themed and they can host Web Parts. In addition, site owners and designers can customize these pages by using tools such as SharePoint Designer 2007. Site pages are virtualized in the site collection's content database and therefore can be classified as customized or uncustomized.

On the other hand, application pages are much more like traditional ASP.NET 2.0 pages. These pages reside in a special virtual directory (http://www.litware.com/\_layouts/\*.\*) that all sites have access to. This virtual directory points to the [..]\12\TEMPLATE\LAYOUTS folder on the file system. All Windows SharePoint Services sites on a front-end Web server use the same pages. These pages are not provisioned as instances in the site collection's content database. Instead, they act just like traditional ASP.NET 2.0 pages. Each is coded to analyze the current context of the HTTP request such as which site collection, site, and user to determine what it should and should not do. These pages do not appear in SharePoint Designer 2007 when you browse a site because they exist only on the file system. Therefore you cannot customize them.

Determining File Customization State and Reverting to the File Template

You can determine the status of a customized file in a few different ways. One way is to use SharePoint Designer 2007. A blue circle icon with a white "i" in it indicates that the file is customized.

Developers can also check the customization status of a file by using the Windows SharePoint Services API. Each file in a Windows SharePoint Services site is represented as an SPFile object. The SPFile object contains the CustomizedPageStatus property, which returns an enumeration of type Microsoft.SharePoint.SPCustomizedPageStatus.

So, after you customize a file, can you uncustomize it (also known as reghosting in the previous version of Windows SharePoint System)? Yes, you can, and there are several ways to accomplish this. As demonstrated earlier, Figures 1 and 2 show how you can do this by using SharePoint Designer 2007 and the browser interface. In addition, just like everything else in Windows SharePoint Services, if it can be done through the browser interface, it is also possible using the API. The SPFile object contains the RevertContentStream method, which deletes the customizations of the file instance in the site collection content database. This returns the file to an uncustomized state so that all future requests are served by the file template on the file system, as shown in the following code.

using (SPSite siteCollection = new SPSite("http://litware.com")) {
  using (SPWeb site = siteCollection.RootWeb){
    SPFile file = site.GetFile("defalt.aspx");
    
    // If the file is customized, revert to the site definition
    //   (the uncustomized instance on the file system).
    if (file.CustomizedPageStatus == SPCustomizedPageStatus.Customized)
      file.RevertContentStream();
  }
}

Creating Custom Pages in SharePoint Sites

Similar to working with ASP.NET 2.0 Web sites, you are free to create new pages within Windows SharePoint Services sites, including site and application pages, and those that are subject to the same customization rules outlined earlier. You can create pages in one of two ways: either by using the Windows SharePoint Services API, or by provisioning uncustomized instances using SharePoint Features. The following sections cover these two approaches and the advantages and disadvantages of both.

Creating Customized Page Instances by Using the SharePoint API

One available option for creating custom pages in SharePoint sites is to use the Windows SharePoint Services API. This approach includes creating pages by using tools such as SharePoint Designer. SharePoint Designer interacts with SharePoint sites through the API by using RPC calls and Web services. Because these pages are created using the API, they live exclusively in the content database and thus are considered customized files.

Creating pages directly by using the API involves creating a new file as a memory stream. After you create the contents of the file, you create the file in the SharePoint site by calling the Add method on the Files collection of the site, as shown in the following code.

using (SPSite siteCollection = new SPSite("http://litware.com")) {
  using (SPWeb site = siteCollection.RootWeb) {
    MemoryStream fileStream = new MemoryStream();
    StreamWriter fileWriter = new StreamWriter(fileStream);
    
    // Write the source of the page (include meta:progid so SharePoint Designer understands this file).
    fileWriter.WriteLine("<%@ Page MasterPageFile=\"~masterurl/default.master\" 
      meta:progid=\"SharePoint.WebPartPage.Document\" %>");
    fileWriter.WriteLine("<asp:Content runat=\"server\" ContentPlaceHolderID=\"PlaceHolderMain\">");
    fileWriter.WriteLine("<h1>MSDN</h1>");
    fileWriter.WriteLine("</asp:Content>");
    fileWriter.Flush();

    // Save the file to Windows SharePoint Services.
    site.Files.Add("ApiGeneragedPage.aspx", fileStream);

    // Cleanup.
    fileWriter.Close();
    fileWriter.Dispose();
    fileStream.Close();
    fileWriter.Dispose();
  } // SPWeb using statement
} // SPSite using statement

Creating files by using SharePoint Designer essentially does the same thing. SharePoint Designer streams the new content directly into the site through the API, as shown in the previous code example.

To create a new page in a SharePoint site by using SharePoint Designer, click New on the File menu, and then select the type of file to create.

Advantages of Creating Customized Pages

Creating pages by using the SharePoint API, including in SharePoint Designer, has many advantages. First, there are plenty of resources available in creating new pages available in the form of white papers, articles, documentations, and other various formats. In addition, creating new pages by using a tool such as SharePoint Designer make it very simple because of its familiar what-you-see-is-what-you-get (WYSIWYG) environment. Finally, it is very easy to make quick changes to pages in an environment by using tools such as SharePoint Designer.

Disadvantages of Creating Customized Pages

As with most things, there is not only an upside but also a downside to creating customized pages. First, after you create a page in one environment, it is not very easy to move that page to another environment because it resides exclusively in the site collection's content database; it is not as easy as exporting and importing the page. So how can you move a page from one environment to another? The obvious approach is to manually re-create it. Another option is to create a script that can extract the contents of an existing page and create a copy in a target environment. Although these are both possible options, they raise additional concerns when you are maintaining changes to pages.

Another challenge is that you cannot easily incorporate these pages into a source control management solution such as Microsoft Visual Studio Team Foundation Server.

Creating Uncustomized Page Instances by Using SharePoint Features

Instead of creating customized pages, you can also create page templates and a SharePoint Feature that would in turn create a new uncustomized instance of the page within a SharePoint site. This approach enables developers to stay within the bounds of the Microsoft Visual Studio environment. Unfortunately, some things that you might be used to when you create ASP.NET 2.0 Web sites do not work when you are creating pages for SharePoint sites. In these cases, you should take a step back and think about what Visual Studio is doing in a pure ASP.NET 2.0 environment.

A common example of this is debugging because many people think that debugging custom code is not possible in Windows SharePoint Services. Consider what happens in a pure ASP.NET 2.0 environment. Visual Studio compiles the project, starts a new Web process that requests the page, and attaches the Visual Studio debugger to the process. At this point, breakpoints that are set by the debugger can be hit, variables can be inspected, and so on. But when you are developing Windows SharePoint Services pages, pressing F5 does not work—but why? It is because Visual Studio has no native hooks into the Windows SharePoint Services environment. This does not mean that debugging is impossible when you are creating Windows SharePoint Services pages; it just means that some manual work is necessary. A developer would compile the project, fire off the Web process requesting the page in question, and manually attach the debugger.

To create uncustomized pages, you must first create a page template (the ASPX file) that can optionally use a code-behind file and the SharePoint Feature that will provision the uncustomized instance of the page based on the template upon activation. First create the page template as follows.

<%@ Page Language="C#" MasterPageFile="~masterurl/default.master" 
  meta:progid="SharePoint.WebPartPage.Document" %>
<asp:Content  ContentPlaceHolderID="PlaceHolderMain">
  <h1>MSDN</h1>
  Custom page creates as an uncustomized instance.
</asp:Content>

With the ASPX page created, the next step is to create the Feature. First, the Feature definition is quite straightforward, as shown in this example.

<?xml version="1.0" encoding="utf-8" ?>
<Feature xmlns="http://schemas.microsoft.com/sharepoint/"
         Id="597D8309-C964-48ef-8D24-D5C60C94A832"
         Title="MSDN Uncustomized Page Feature"
         Scope="Web">
  <ElementManifests>
    <ElementManifest Location="elements.xml"/>
    <ElementFile Location="PageTemplate.aspx" />
  </ElementManifests>
</Feature>

Finally, create the element manifest file that provisions the uncustomized instance of the page into the SharePoint site as follows.

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Module Url="SamplePages">
    <File Url="PageTemplate.aspx" Name="SamplePage.aspx" Type="Ghostable" />
  </Module>  
</Elements>

After the Feature has been installed and activated, a new uncustomized instance of the page is provisioned into the target site. You open this page in tools such as SharePoint Designer and customize it just like any other page; it is no different from, for example, the default.aspx page found at the root of all SharePoint sites.

Advantages of Creating Uncustomized Pages

Creating uncustomized pages has many advantages over creating customized ones. First, keeping the files uncustomized on the server makes it much easier to update many files between various environments because the source of the files is always being pulled from the file on the file system. In addition, because Features are used as the vehicle for adding these files to the site, you can use Windows SharePoint Services solution packages (WSP files) to package many files for deployment from one environment to another. This includes existing files that must be updated. Because these files are uncustomized, updating the files on the file system changes the page itself.

Another advantage to this approach is that the source of the files stays on the file system, and they can easily be added to any source control management solution because all understand files on the file system. This approach also makes it much easier for organizations with a strict change control process to implement changes across environments because it an easily repeatable process.

Disadvantages of Creating Uncustomized Pages

Just like creating customized pages, creating uncustomized pages also has its disadvantages. As shown earlier, not only do you have to create the actual page template, but you also have to create the Feature that handles the provisioning of the page. Unfortunately, this results in double development. In addition, Features are tedious to build because they require knowledge of Collaborative Application Markup Language (CAML), and they provide minimal debugging support when things go wrong.

Another disadvantage to this approach is that the Feature that provisions the instances does not remove the instances upon deactivation. This is by design, but you can easily lessen the effects by using a custom Feature receiver that deletes the created files when the feature is deactivated.

Conclusion

Windows SharePoint Services 3.0 sites, just like typical ASP.NET 2.0 Web sites, are made up of ASP.NET 2.0 pages. To meet the scalability and high demands of dynamic customization on a site by site basis, Microsoft introduced the concepts of customized and uncustomized files and also instances and templates. This article defines these terms and demonstrates how you can create new customized and uncustomized files for your SharePoint sites.

Additional Resources

For more information, see the following resources: