Creating a Custom Importer and Processor

Article
09/29/2011

Describes how a content importer adds support for a new art asset file format.

You will need to write a new content importer to add support. Also, you may need to write a custom processor, writer, and reader for the new art asset type after it has been imported.

Complete Sample

This topic provides a simple example to demonstrate the steps needed to write an importer and processor, and also the writer and reader needed to save and load the results. You can download a complete code sample for this topic, including full source code and any additional supporting files required by the sample.

Download CPExtPixelShader_Sample.zip

Overview

Most of the art, sound, or data assets that your game will use are likely in one of the commonly used file formats supported by the Standard Importers and Processors. A third party also may provide the custom importers and processors you need to support additional formats. In such cases, it is unnecessary to write your own custom importer or processor. However, if you need to support a new file format or data type in the Content Pipeline, writing your own importer and processor can be fairly straightforward.

The goal of the example in this tutorial is to import a pixel shader (in a file with the .psh extension), and to produce compiled effect data that can be assigned to an instance of Effect.

The sections below describe each step.

Creating a Content Pipeline Extension Library
Implementing a Simple Importer
Implementing a Processor
Implementing a Writer
Implementing a Reader for the Pixel Shaders
Using the Output of the New Processor in Your Game

Creating a Content Pipeline Extension Library

The first step in writing an importer and processor is to create a new project for them. You need to do this because your importer and processor are used by the Content Pipeline when your game is being built. They are not part of the game itself. As a result, you need to provide them as a separate library assembly that the Content Pipeline can link to when it needs to build the new file format you are supporting.

To create a Content Pipeline extension library

In XNA Game Studio, load a game solution you are developing (the sample uses "CPExtPixelShader").
In Solution Explorer, right-click the Solution node, click Add, and then click New Project.

The Add New Project dialog box opens.
In the Projects type pane under the Visual C# node, select XNA Game Studio 4.0 Refresh .
In the Templates pane, select Content Pipeline Extension Library (4.0 Refresh).
Next, in the Name box, enter PSProcessorLib as the project name, and then click OK.
In Solution Explorer, right-click the ContentProcessor1.cs item, and then click Delete.

The remaining steps create a reference to the PSProcessorLib content extension project.
In Solution Explorer, right-click the Content node of the CPExtPixelShader project, and then click Add Reference.
On the Projects tab page, select your content extension project, and then click OK.

The new project is now ready for your custom importer and processor implementation.

Implementing a Simple Importer

The objective of the importer is to read asset data from the file in to memory for further conversion by the processor. The importer may also perform some intermediate data transformation if necessary.

In this example, the importer reads the pixel shader from the file. It also appends a technique declaration, which is given the same name as the file, as required to transform the shader into an effect.

Follow these steps to add a content importer to your processor.

To implement a simple importer

Create a class to hold the input data you are importing.

In this case, it takes the form of a string of high-level shader language (HLSL) source code.
Add a new C# class named PSSourceCode to the processor project.

The first thing to do in the file containing your new class definition is to add the following using statement at the beginning of the file:
```
using Microsoft.Xna.Framework.Content.Pipeline;
```

Next, define the class:

class PSSourceCode
{
    const string techniqueCode = "{ pass p0 { PixelShader = compile ps_2_0 main(); } }";

    public PSSourceCode(string sourceCode, string techniqueName)
    {
        this.sourceCode = sourceCode + "\ntechnique " + techniqueName + techniqueCode;
    }

    private string sourceCode;
    public string SourceCode { get { return sourceCode; } }
}

Write an importer class to import the HLSL source code.

This class must be derived from ContentImporter and implement the Import method.

The importer reads a text file containing HLSL source code into your PSSourceCode class. It also appends a technique declaration, which is given the same name as the file, as required to transform the shader into an effect.
In the New Item dialog box, add a new Content Importer item named PSImporter to the processor project.

Define the class next:

[ContentImporter(".psh", DefaultProcessor = "PSProcessor",
      DisplayName = "Pixel Shader Importer")]
class PSImporter : ContentImporter<PSSourceCode>
{
    public override PSSourceCode Import(string filename, 
        ContentImporterContext context)
    {
        string sourceCode = System.IO.File.ReadAllText(filename);
        return new PSSourceCode(sourceCode, System.IO.Path.GetFileNameWithoutExtension(filename));
    }
}

The ContentImporter attribute applied to the PSImporter class provides some context for the user interface of XNA Game Studio. Since this importer supports files with a .psh extension, XNA Game Studio automatically selects the PSImporter importer when a .psh file is added to the project. In addition, the DefaultProcessor argument specifies which processor XNA Game Studio selects when a .psh file is added.

Note

To specify multiple file types, separate with a comma the file extensions listed in the ContentImporterAttribute. For example, [ContentImporter (".bmp",".dds",".tga")] declares an importer that accepts .bmp, .dds, and .tga file types. Normally, an importer that accepts multiple file formats is specialized to generate one particular kind of output type, such as textures. Aside from difficulties of maintenance, there is nothing to prevent a single importer from being written to handle many different content types.

When the game is built, the ContentImporter.Import function is called once for each XNA content item in the current project.

When invoked against an input file in the appropriate format, a custom importer is expected to parse the file and produce as output one or more content objects of appropriate types. Since an importer's output is passed directly to a Content Pipeline processor, each type that an importer generates must have at least one processor available that can accept it as input.

Tip

An importer that generates Document Object Model objects may also automatically generate an intermediate XML cache file that serializes these objects. For this to happen, the importer must be implemented with the CacheImportedData attribute flag set to true. This flag is false by default. To set the attribute flag to true, begin the implementation of your importer class like this:

 [ContentImporter( ".MyExt", CacheImportedData = true )] class PSImporter :         ContentImporter<PSSourceCode> { ... }

Implementing a Processor

After the new importer has read in the pixel shader source code from a text file, your content processor compiles the shader into binary form.

To write the processor

Create a class to store the compiled output, which in this case takes the form of an array of bytes.

Add a C# class named CompiledPS to the processor project, and then define the new class:

class CompiledPS
{
    public CompiledPS(byte[] compiledShader)
    {
        this.compiledShader = compiledShader;
    }

    private byte[] compiledShader;
    public byte[] CompiledShader {
        get { return (byte[])compiledShader.Clone(); } 
    }
}

Now you are ready to write the processor class, which converts a PSSourceCode object into a CompiledPS object.

In the New Item dialog box, add a new Content Processor item named PSProcessor to the processor project.

Define the class:

[ContentProcessor(DisplayName = "Pixel Shader Processor")]
class PSProcessor : ContentProcessor<PSSourceCode, CompiledPS>
{
    public override CompiledPS Process(PSSourceCode input, 
        ContentProcessorContext context)
    {
       // Load the shader into an effect and compile it
        EffectContent content = new EffectContent();
        content.EffectCode = input.SourceCode;
        EffectProcessor compiler = new EffectProcessor();
        CompiledEffectContent compiledContent = compiler.Process(content, context);

        // Return the compiled effect code portion only
        return new CompiledPS(compiledContent.GetEffectCode());
    }
}

The processor uses the EffectProcessor class to compile the shader that our importer transformed in to an effect. The processor then returns the resulting binary code that can be used in an Effect class.

The context.TargetPlatform argument targets the platform. If an error occurs during compilation, PSProcessor throws an InvalidContentException. The error appears in the Error List window of XNA Game Studio.

Note

The example processor could, and perhaps should, return the full Effect class, making use of the standard writer and reader for that class that is part of the Content Pipeline library. However, because we wish to demonstrate writing a custom writer and reader, we have chosen to return only the compiled binary result within the Effect class.

Implementing a Writer

The final design-time class to implement is a writer that saves the compiled pixel shader effect produced by your processor as a binary .xnb file.

To implement the writer for the compiled shader effect

In the New Item dialog box, add a new Content Type Writer item named PSWriter to the processor project.

Define the new class:

[ContentTypeWriter]
class PSWriter : ContentTypeWriter<CompiledPS>
{
    protected override void Write(ContentWriter output, CompiledPS value)
    {
        output.Write(value.CompiledShader.Length);
        output.Write(value.CompiledShader);
    }
    public override string GetRuntimeType(TargetPlatform targetPlatform)
    {
        return typeof(CompiledPS).AssemblyQualifiedName;
    }
    public override string GetRuntimeReader(TargetPlatform targetPlatform)
    {
        return "CPExtPixelShader.PSReader, CPExtPixelShader," + 
            " Version=1.0.0.0, Culture=neutral";
    }
}

The GetRuntimeType method identifies the type of object your game should load from the .xnb file written by the writer object. In this instance, the .xnb file contains the binary array from your custom CompiledPS type.

The GetRuntimeReader method specifies what reader should be invoked to load the .xnb file in your game. It returns the namespace and name of the reader class, followed by the name of the assembly in which that class is physically located.

In your code, change the assembly name to match the actual name of your game and its assembly, since that is where you will be loading the shaders.

At this point, the code for your PSProcessorLib library is complete.

Implementing a Reader for the Pixel Shaders

Now move from the PSProcessorLib library project back to your game project, and write the class that your game uses to load the .xnb files that your processor creates. This is the class that your writer previously specified as its reader.

To implement a reader for the pixel shaders

In your game project, add a C# class named PSReader to your game project.

Add the using statements you will need at the top of the file:

using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.Graphics;

Deriving from the ContentTypeReader generic class , override the Read method, and define your class:

class PSReader : ContentTypeReader<CompiledPS>
{
    /// <summary>
    /// Loads an imported shader.
    /// </summary>
    protected override CompiledPS Read(ContentReader input,
        CompiledPS existingInstance)
    {
        int codeSize = input.ReadInt32();
        byte[] shaderCode = input.ReadBytes(codeSize);

        IGraphicsDeviceService graphicsDeviceService =
          (IGraphicsDeviceService)input.ContentManager.ServiceProvider.
          GetService(typeof(IGraphicsDeviceService));
        return new CompiledPS(shaderCode);
    }
}

At this point, build the processor project.

When the build is complete you are ready to use the new importer and processor to build pixel shaders into your game.

Using the Output of the New Processor in Your Game

Try adding a test HLSL source file with a .psh extension to your game project and see how it works.

To test your output

Copy into a folder in your game project a simple HLSL source file that you know is free of bugs, and rename the file "Ripple.psh."
In Solution Explorer, right-click your game project, click Add, click Existing Item, and then click Ripple.psh.
Right-click Ripple.psh, and click Properties.

In the Properties dialog box, you should now see entries assigning PSImporter as its content importer and PSProcessor as its content processor. Next time you build your game, Ripple.psh will be built into TestShader.xnb in a form appropriate for your target platform.

To use the resulting compiled pixel shader effect in your game, load it using ContentManager.Load as follows:

protected override void LoadContent()
{
    CompiledPS shader = Content.Load<CompiledPS>("Ripple");
    Effect shaderEffect = new Effect(graphics.GraphicsDevice, shader.CompiledShader);
}