March 2010
Volume 25 Number 03
UI Frontiers - MIDI Music in WPF Applications
By Charles Petzold | March 2010
Every PC contains a built-in 16-piece band ready to play some music. The members of this band probably feel much neglected, for they represent perhaps the most underutilized component of the array of sound and video features supported by Windows.
This 16-piece band is an electronic music synthesizer implemented in either hardware or software that conforms to the standard known as MIDI—the Musical Instrument Digital Interface. In the Win32 API, playing music through the MIDI synthesizer is supported through functions beginning with the words midiOut.
MIDI support is not part of the .NET Framework, however, so if you want to access this MIDI synthesizer in a Windows Forms or Windows Presentation Foundation (WPF) application, you’ll need to use either P/Invoke or an external library.
I was very pleased to find MIDI support in the NAudio sound library available on CodePlex that I discussed in my last column. You can download that library with source code from codeplex.com/naudio. For this article I used NAudio version 1.3.8.
A Brief Example
You can think of MIDI as a high-level interface to waveform audio in which you’re working with musical instruments and notes.
The MIDI standard was developed in the early 1980s. Manufacturers of electronic music synthesizers wanted a standard way to connect electronic music controllers (such as keyboards) with synthesizers, and they came up with a system to transmit small messages (mostly one, two or three bytes in length) through a cable with a 5-pin connector at the pokey rate of 3,125 bytes per second.
Two of the most important of these messages are called Note On and Note Off. When a musician presses a key on a MIDI keyboard, the keyboard generates a Note On message indicating the note that was pressed and the key’s velocity. The synthesizer responds by playing that note, generally louder for higher key velocities. When the musician releases the key, the keyboard generates a Note Off message, and the synthesizer responds by turning the note off. No actual audio data goes through the MIDI cable.
Although MIDI is still used to connect electronic-music hardware, it can also be used entirely within a PC through software. Sound boards can include MIDI synthesizers, and Windows itself emulates a MIDI synthesizer entirely in software.
To access that synthesizer in your WinForms or WPF application using the NAudio library, add NAudio.dll as a reference and include this using directive in your source code:
using NAudio.Midi;
Suppose you want your application to play a single one-second note that sounds like the middle C of a piano. You can do that with the following code:
MidiOut midiOut = new MidiOut(0);
midiOut.Send(MidiMessage.StartNote(60, 127, 0).RawData);
Thread.Sleep(1000);
midiOut.Send(MidiMessage.StopNote(60, 0, 0).RawData);
Thread.Sleep(1000);
midiOut.Close();
midiOut.Dispose();
A PC might have access to multiple MIDI synthesizers; the argument to the MidiOut constructor is a numeric ID to select the one to open. The constructor will raise an exception if the MIDI output device is already in use.
A program can obtain information about the MIDI synthesizers by first using the static MidiOut.NumberOfDevices property to discover how many synthesizers are present. The numeric IDs range from 0 to one less than the number of devices. The static MidiOut.DeviceInfo method accepts a numeric ID and returns an object of type MidiOutCapabilities that describe the synthesizer. (I won’t be using these features. For the remainder of this article I’ll simply use the default MIDI synthesizer available with an ID of zero.)
The Send method of the MidiOut class sends a message to the MIDI synthesizer. A MIDI message comprises one, two or three bytes, but the Win32 API (and NAudio) wants them packed into a single 32-bit integer. The MidiMessage.StartNote and MidiMessage.StopNote methods do this packing for you. You can replace the two arguments to Send with 0x007F3C90 and 0x00003C80, respectively.
The first argument to StartNote and StopNote is a code ranging from 0 through 127 indicating the actual note, where the value 60 is middle C. An octave higher is 72. An octave lower is 48. The second argument is the velocity that the key is pressed or released. (Release velocities are usually ignored by synthesizers.) These can range from 0 to 127. Lower the second argument to MidiMessage.StartNote to make the note softer. (I’ll discuss the third argument shortly.)
The two calls to Thread.Sleep suspend the thread for 1,000 milliseconds. This is a very simple way of timing the messages, but should be avoided in a user-interface thread. The second Sleep call is necessary to let the note die off before it is abruptly truncated by the Close call.
What About Polyphony?
That’s how you can play one note. What about multiple notes at the same time? That’s possible as well. For example, if you wanted to play a C-major chord rather than just a single C note, you can replace the first Send message with the following:
midiOut.Send(MidiMessage.StartNote(60, 127, 0).RawData);
midiOut.Send(MidiMessage.StartNote(64, 127, 0).RawData);
midiOut.Send(MidiMessage.StartNote(67, 127, 0).RawData);
midiOut.Send(MidiMessage.StartNote(72, 127, 0).RawData);
Then replace the second Send message with:
midiOut.Send(MidiMessage.StopNote(60, 0, 0).RawData);
midiOut.Send(MidiMessage.StopNote(64, 0, 0).RawData);
midiOut.Send(MidiMessage.StopNote(67, 0, 0).RawData);
midiOut.Send(MidiMessage.StopNote(72, 0, 0).RawData);
If you want the various notes to start and stop at various times, you’ll probably want to abandon the use of Thread.Sleep and get an actual timer involved, particularly if you’re playing the music on a user-interface thread. More on this shortly.
There is a MIDI file format that combines MIDI messages with timing information, but these files require specialized software to create and I won’t be discussing them here.
Instruments and Channels
So far I’ve been playing only piano sounds. You can switch the synthesizer to play sounds of other instruments using the MIDI Program Change message, implemented in NAudio with the ChangePatch method:
midiOut.Send(MidiMessage.ChangePatch(47, 0).RawData);
The first argument to ChangePatch is a numeric code ranging from 0 to 127 to indicate a particular instrument sound.
Back in the early days of MIDI, the actual sounds coming out of the synthesizers were entirely controlled by the performer through dials and patch cables. (That’s why a particular synthesizer setup or instrument sound is often referred to as a “patch.”) Later on, creators of MIDI files wanted a standard set of instruments so the files would sound pretty much the same regardless of the synthesizer they played on. This led to a standard called General MIDI.
A good reference for General MIDI is the Wikipedia entry en.wikipedia.org/wiki/General_midi. Under the heading “Melodic sounds” are 128 instrument sounds with codes ranging from 1 to 128. You use zero-based codes in the ChangePatch method, so code 47 in the previous example is instrument 48 in this list, which is a Timpani sound.
I mentioned at the outset that the MIDI synthesizer is equivalent to a 16-piece band. The MIDI synthesizer supports 16 channels. At any time, each channel is associated with a particular instrument based on the most recent Program Change message. The channel number ranges from 0 through 15 and is specified in the final argument of the StartNote, StopNote and ChangePatch methods.
Channel 9 is special. This is a percussion channel. (It’s often referred to as Channel 10, but that’s if the channels are numbered beginning at 1.) For channel 9, the codes passed to the StartNote and StopNote methods refer to particular non-tonal percussion sounds rather than pitches. In the Wikipedia entry on General MIDI, see the list under the heading “Percussion.” For example, the following call plays a cowbell sound, which is indicated by a code of 56:
midiOut.Send(MidiMessage.StartNote(56, 127, 9).RawData);
There is much more to MIDI, but those are the essentials.
XAML-Based MIDI
In keeping with the spirit of WPF and XAML, I thought it would be fun to develop a string-based format for embedding short pieces of music directly in XAML files and playing them back. I call this format a MIDI string—a text string of notes and timing information. All tokens are separated by white space.
Notes are capital letters A through G, followed by any number of + signs or # signs (each raises the pitch one semitone) or – signs or the letter b (to lower the pitch one semitone) followed by an optional octave number, where the octave beginning at middle C is octave four. (This is a standard way of numbering octaves.) Thus, the C# below middle C is:
C#3
The letter R by itself is a rest. A note or a rest can be optionally followed by a duration, which indicates the period of time until the next note. For example, this is a quarter note, which is also the default if no duration is indicated:
1/4
Durations are sticky—that is, if a duration does not follow a note, the last duration will be used. If the duration begins with a slash, the numerator is assumed to be 1.
That duration indicates the time until the next note. This duration is also used for the length of the note—that is, the time until the note is turned off. For a more staccato sound, you may want the note’s length to be less than its duration. Or you might want successive notes to overlap somewhat. You indicate a note’s length the same way as the duration, but with a minus sign:
–3/16
Durations and lengths always appear after the note to which they apply, but the order doesn’t matter. Lengths are not sticky. If a note length does not appear, the duration is used for the length.
Notes can also be preceded by tokens. To set an instrument voice, follow the letter I by the zero-based patch number. For example, this indicates a violin for the successive notes:
I40
The piano is the default patch.
To set a new volume (that is, a velocity) for successive notes use V, such as:
V64
For both I and V, the number that follows must range from zero through 127.
By default, the tempo is 60 quarter notes per minute. To set a new tempo for the following notes, use T followed by the number of quarter notes per minute, for example:
T120
If you’d like a group of notes to be played with all the same parameters, you can put them in parentheses. Here’s a C-major chord:
(C4 E4 G4 C5)
Only notes may appear in parentheses. The vertical bar | separates channels. The channels are played simultaneously, and they are entirely independent, including the tempos.
If a particular channel contains a capital P anywhere within, that channel becomes the percussion channel. That channel can contain notes or rests in the normal notation, but also allows percussion voices to be indicated numerically. For example, this is the cowbell:
P56
If you go to en.wikipedia.org/wiki/Charge_(fanfare), you’ll see the “Charge!” tune often played at sporting events. That can be expressed in the MIDI string format as:
"T100 I56 G4 /12 C5 E5 G5 3/16 -3/32 E5 /16 G5 /2"
The MidiStringPlayer
The MidiStringPlayer is the only public class in the Petzold.Midi library project included with the downloadable source code. It derives from FrameworkElement so you can embed it in the visual tree in a XAML file, but it has no visual appearance. Set the MidiString property to a string in the format shown in the previous example and call Play (and, optionally, Stop to stop the sequence before it’s completed).
MidiStringPlayer also has a PlayOnLoad property to play a sequence when the element loads, and a get-only IsPlaying property. The element generates an Ended event when it’s completed playing a sequence, and a Failed event that’s fired if there’s an error in the syntax of the MIDI string. The event includes an offset in the text string indicating the problematic token and a text explanation of the error.
Two WPF programs are also included in the downloadable code. The MusicComposer program lets you interactively put together a MIDI string. The WpfMusicDemo program encodes some simple sequences in a MIDI file, as shown in Figure 1.
Figure 1 WpfMusicDemo.xaml Encodes Several Simple MIDI Strings
<Window x:Class="WpfMusicDemo.Window1"
xmlns="https://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="https://schemas.microsoft.com/winfx/2006/xaml"
xmlns:midi="clr-namespace:Petzold.Midi;assembly=Petzold.Midi"
Title="WPF Music Demo"
Height="300" Width="300">
<Grid>
<midi:MidiStringPlayer Name="player"
PlayOnLoad="True"
MidiString="{Binding ElementName=chargeButton, Path=Tag}" />
<UniformGrid Rows="2"
ButtonBase.Click="OnButtonClick">
<UniformGrid.Resources>
<Style TargetType="Button">
<Setter Property="HorizontalAlignment" Value="Center" />
<Setter Property="VerticalAlignment" Value="Center" />
<Style.Triggers>
<DataTrigger
Binding="{Binding ElementName=player, Path=IsPlaying}"
Value="True">
<Setter Property="IsEnabled" Value="False" />
</DataTrigger>
</Style.Triggers>
</Style>
</UniformGrid.Resources>
<Button Name="chargeButton"
Content="Charge!"
Tag="T100 I56 G4 /12 C5 E5 G5 3/16 -3/32 E5 /16 G5 /2" />
<Button Content="Bach D-Minor Toccata"
Tag="T24 I19 A5 /64 G5 A5 5/32 R /32 G5 /64 F5 E5 D5 C#5 /32 D5 /16 R 4/16 A4 /64 G4 A4 5/32 R /32 E4 F4 C#4 D4 /16 R 4/16 | T24
I19 A4 /64 G4 A4 5/32 R /32 G4 /64 F4 E4 D4 C#4 /32 D4 /16 R 4/16 A3 /64 G3 A3 5/32 R /32 E3 F3 C#3 D3 /16 R 4/16"/>
<Button Content="Shave & a Haircut"
Tag="T130 I58 C5 G4 /8 G4 Ab4 /4 G4 R I75 B4 C5" />
<Button Content="Beethoven Fifth"
Tag="T200 I71 R /8 G4 G4 G4 Eb4 7/8 R /8 F4 F4 F4 D4 5/4 | T200 I40 R /8 G4 G4 G4 Eb4 7/8 R /8 F4 F4 F4 D4 5/4 | T200 I40 R /8 G4
G4 G4 Eb4 7/8 R /8 F4 F4 F4 D4 5/4 | T200 I41 R /8 G3 G3 G3 Eb3 7/8 R /8 F3 F3 F3 D3 5/4 | T200 I43 R /8 G2 G2 G2 Eb2 7/8 R /8 F2 F2 F2 D2
5/4 | T200 I43 R /8 G2 G2 G2 Eb2 7/8 R /8 F2 F2 F2 D2 5/4"/>
</UniformGrid>
</Grid>
</Window>
A crucial part of any piece of music-playing software is the timer, but for MidiStringPlayer I used the very simple DispatcherTimer, which runs on the UI thread. This is certainly not optimum. If another program is hogging the CPU, the music playback will become irregular. DispatcherTimer also cannot generate Tick events faster than about 60 per second, which is satisfactory for simple pieces, but doesn’t provide the necessary precision of for more rhythmically complex music.
The Win32 API includes a high-resolution timer specifically for playing MIDI sequences, but this has not yet made it to the NAudio library. Perhaps at some later time I’ll replace the DispatcherTimer with something a little more precise and regular, but for now I’m happy that it works as well as it does with this simple solution.
Charles Petzold is a long-time Contributing Editor to MSDN Magazine*. His most recent book is “The Annotated Turing: A Guided Tour through Alan Turing’s Historic Paper on Computability and the Turing Machine” (Wiley, 2008). Petzold blogs on his Web site charlespetzold.com.*
Thanks to the following technical expert for reviewing this article: Mark Heath