Tutorial: Using the F# PowerPack and the F# MathProvider

Article
09/12/2011

Applies to: Functional Programming

Authors: Yin Zhu

Referenced Image

Get this book in Print, PDF, ePub and Kindle at manning.com. Use code “MSDN37b” to save 37%.

Summary: The F# PowerPack provides types for vectors and matrices, and the F# MathProvider supports linear algebra operations on these types. This article presents an introduction to the two libraries.

This topic contains the following sections.

Introducing the F# PowerPack Math
Creating Vectors and Matrices
Performing Linear Algebra Using the F# MathProvider
Additional Resources
See Also

This article is associated with Real World Functional Programming: With Examples in F# and C# by Tomas Petricek with Jon Skeet from Manning Publications (ISBN 9781933988924, copyright Manning Publications 2009, all rights reserved). No part of these chapters may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, electrostatic, mechanical, photocopying, recording, or otherwise—without the prior written permission of the publisher, except in the case of brief quotations embodied in critical articles or reviews.

Introducing the F# PowerPack Math

The F# PowerPack is a set of additional F# libraries developed by the F# team. It contains libraries for lexing, parsing, and accessing data using LINQ and libraries related to math. This article shows how to use its vector and matrix types. The types implement only basic linear algebra operations, but it is possible to use a more efficient implementation using the F# MathProvider, which is a wrapper for BLAS and LAPACK libraries. It was originally a part of the F# PowerPack, but was turned into a standalone project hosted at CodePlex and is regularly maintained by the F# community.

The F# PowerPack implements types for vectors and matrices, including both dense and sparse versions. These types are designed for F#, which makes them very easy and convenient to use. The F# MathProvider is a linear algebra library that uses the vector and matrix types in The F# PowerPack. It is very natural to use these two libraries together—for the F# PowerPack to hold the data and for the F# MathProvider to perform the linear algebra operations.

This article first introduces the types for representing the vectors and matrices provided by the F# PowerPack and then demonstrates how to perform linear algebra operations using the F# MathProvider.

Creating Vectors and Matrices

The F# PowerPack includes several independent .NET projects. The source code is available under a permissive Apache 2 license, which makes it possible to copy relevant parts of the source code to your project to simplify the deployment. Vector and matrix types are implemented in the FSharp.PowerPack.dll assembly. When creating a project, this library needs to be referenced using the Add Reference dialog. In an F# script file, it can be referenced using the #r directive. The following snippet shows how to create two 3×3 matrices and one vector:

#r "FSharp.PowerPack.dll"

let a = matrix [ [ 1.0; 7.0 ];
                 [ 1.0; 3.0 ]; ]

let b = matrix [ [ 10.0; 70.0 ];
                 [ 10.0; 30.0 ]; ]

let c = vector [ 5.0; 8.0 ];

The code in the listing looks as if it were syntax directly supported by the F# language. However, vector and matrix are just simple F# functions. They take standard F# lists (or lists of lists) and turn them into values of type vector and matrix, respectively. These two types are just aliases for generic Vector<float> and Matrix<float> types.

Calculating with Matrices

The F# PowerPack library comes with a basic implementation of linear algebra operations. Although it is not very efficient, it may be suitable for many standard tasks. Better performance can be obtained using the F# MathProvider (discussed later). The following snippet uses F# Interactive to demonstrate standard operations on matrices and vectors:

> a + b;;
val it : Matrix<float> = matrix [[11.0; 77.0]
                                 [11.0; 33.0]]
> a - b;;
val it : Matrix<float> = matrix [[-9.0; -63.0]
                                 [-9.0; -27.0]]
> a * b;;
val it : Matrix<float> = matrix [[80.0; 280.0]
                                 [40.0; 160.0]]
> a .* b;;
val it : Matrix<float> = matrix [[10.0; 490.0]
                                 [10.0; 90.0]]

Basic linear algebra operations for matrices are provided using standard (overloaded) F# operators. The operators + and - implement pointwise addition and subtraction. There are two different multiplication operators. The standard * symbol is used for matrix multiplication and .* is used for pointwise multiplication. It is also possible to multiply matrices by vectors or scalars:

> a * 2.0;;
val it : Matrix<float> = matrix [[2.0; 14.0]
                                 [2.0; 6.0]]
> -a;;
val it : Matrix<float> = matrix [[-1.0; -7.0]
                                 [-1.0; -3.0]]
> a * c;;
val it : Vector<float> = vector [|61.0; 29.0|]

> let a' = a.Transpose;;
val a' : Matrix<float> = matrix [[1.0; 1.0]
                                 [7.0; 3.0]]

The first command shows how to multiply a matrix by a scalar. The unary minus operator essentially behaves as multiplication by a scalar -1.0. The next example shows that multiplying a matrix by a vector gives a vector. The last command demonstrates how to calculate transposition using a member named Transpose. Other members provided by the Matrix<'T> type are listed in Table 1.

Figure 1. A brief list of Matrix<'T> members

Name	Description
Dimensions	Returns a tuple containing the dimensions (number of rows and columns) of the matrix.
NumRows and NumCols	Returns the number of rows or the number of columns as integers.
Copy	Creates a copy of the matrix. When the copy is modified, it doesn't affect the original matrix.
ToArray2D	Creates a two-dimensional array of type 'T[,] containing a copy of the matrix data.
ToVector and ToRowVector	Return the first column or row of a matrix as a vector. These functions are mainly used to convert a matrix with 1 column or 1 row to a vector type.

Matrix and vector types are mutable and their elements can be accessed directly using indices. The following section shows several examples.

Accessing Vector and Matrix Elements

The elements of F# matrices can be accessed using their indices. The rows and columns of a matrix are indexed from 0, which means that the first column of the first row can be accessed by writing a.[0, 0]. Matrices also support a slicing syntax that makes it possible to access a part of the matrix. This is a very useful feature that exists in many numerical languages. The following F# Interactive session shows several examples:

> let a = matrix [ [ 1.0; 2.0; 3.0 ]
                 [ 10.0; 20.0; 30.0 ] ];;
val a : matrix = (...)

> a.[1, 1];;
val it : float = 20.0

> a.[0, 0] <- 0.9;;
val it : unit = ()

> a.[0 .. , 0 .. 1];;
val it : Matrix<float> = matrix [[0.9; 2.0]
                                 [10.0; 20.0]]

> a.[0 .., 2 .. 2] <- matrix [ [4.0]; [40.0] ];;
val it : unit = ()

> a;;
val it : matrix = matrix [[0.9; 2.0; 4.0]
                          [10.0; 20.0; 40.0]]

The first command creates a matrix with two rows and three columns. The indexer notation can be used for both reading and modifying the element's value. As already mentioned, the index starts from zero. The notation used by later examples is more interesting. The command a.[0 .., 0 .. 1] gets a block of the matrix. The first argument specifies that the result should contain rows starting from index 0 to the end. Similarly, the second specifies that the result should contain first two columns.

The splicing syntax can be used for creating a copy of a part of the matrix. If the snippet stored the returned matrix and modified it, the change would not affect the original matrix. However, the syntax is also useful for modifying a matrix. The next command shows how to replace the last column of the matrix with new values. Note that the two matrices need to have the same dimensions (2×1, in this case).

The matrix type also provides four members for accessing columns and rows:

> a.Column(1);;
val it : Vector<float> = vector [|2.0; 20.0|]

> a.Row(1);;
val it : RowVector<float> = rowvec [|10.0; 20.0; 40.0|]

> a.Columns(1, 2);;
val it : Matrix<float> = matrix [[2.0; 4.0]
                                 [20.0; 40.0]]

The Column member can be used to get a single column of the matrix as a vector. Note that the returned value will always be a vector, so the return type is Vector<float>. The Row member behaves similarly, but returns a row vector. F# uses a different type to represent column and row vectors, so the type of the result is RowVector<float>. Finally, the last line demonstrates the Columns member. It can be used to get a certain number of columns (in the example 2) starting from a specified index (in the example 1). There is also a member Rows that behaves similarly.

Every standard collection type in F# has a corresponding module (such as List, Array or Seq) with various helper higher-order functions such as map and fold. The matrix type has a similar module, which can simplify number of common tasks:

> let b = Matrix.init 3 3 (fun i j -> float i + 10.0 * float j);;
val b : matrix = matrix [[0.0; 10.0; 20.0]
                         [1.0; 11.0; 21.0]
                         [2.0; 12.0; 22.0]]

> Matrix.sum b;;
val it : float = 99.0

> let id3 = Matrix.identity 3;;
val id3 : matrix = matrix [[1.0; 0.0; 0.0]
                           [0.0; 1.0; 0.0]
                           [0.0; 0.0; 1.0]]

> let twos = id3 |> Matrix.map (fun n -> n * 2.0);;
val twos : matrix = matrix [[2.0; 0.0; 0.0]
                            [0.0; 2.0; 0.0]
                            [0.0; 0.0; 2.0]]

> Matrix.trace twos;;
val it : float = 6.0

The listing starts by creating a new matrix using the Matrix.init function. It takes the dimensions of the matrix and a lambda function that is called to generate a value of each element. The next command calculates the sum of all elements using Matrix.sum. A product of all elements can be obtained using Matrix.product.

Another way to create a matrix is to use Matrix.identity, which creates a square matrix with 1.0 on the diagonal (and 0.0 elsewhere). The snippet then uses Matrix.map to multiply all of the elements of the matrix by two, which gives a matrix of twos that contains 2.0 on the diagonal. Finally, the Matrix.trace function calculates a trace (the sum of elements on the diagonal).

Creating Sparse Matrices

The F# PowerPack also supports sparse matrix and vector types. Sparse matrices are not mutable, which means that they cannot be changed using the indexer and the <- operator. However, sparse matrices can be manipulated using the functions from the previous section. A sparse matrix can be created using the Matrix.initSparse function:

> let d = Matrix.initSparse 100 100 
            [ (30, 19, 1.0); (3, 99, 2.0); (18, 21, 3.0); ];;
val d : matrix = (...)

> d.IsSparse;;
val it : bool = true

> Matrix.sum d;;
val it : float = 6.0

The function Matrix.initSparse takes two numeric arguments representing the dimensions of the matrix and a list that provides values for some of the matrix elements. The list contains tuples of type int * int * float. The first two numbers specify the index and the last number is the value.

A sparse matrix has the same type as dense matrix and the two can be mixed. It is possible to find out whether a matrix is sparse using the IsSparse member. Finally, the last command shows that functions such as Matrix.sum can be used with sparse matrices too.

The implementation of matrices provided by the F# PowerPack is relatively slow compared to other numerical libraries that can be used from F#. However, it is possible to use a more efficient implementation by using the F# MathProvider.

Performing Linear Algebra Using the F# MathProvider

The F# is an F# wrapper for native BLAS and LAPACK libraries. It uses the matrix type provided by the F# PowerPack, so a script or project needs to reference both FSharp.PowerPack.dll and MathProvider.dll. By default, the provider is not started. When a linear algebra routine is called, the MathProvider uses the managed F# implementation.

Configuring the MathProvider to use a native implementation involves two steps. The first step is to configure the current directory so that blas.dll and lapack.dll can be located. The second step is to call the startProvider function:

> #r @"FSharp.PowerPack.dll"
  #r @"C:\MathProvider\Net 4.0\MathProvider.dll";;
(...)
> let dir = @"C:\MathProvider\LapackRuntime"
  System.Environment.CurrentDirectory <- dir;;

> module LA = MathProvider.LinearAlgebra;;
> LA.startProvider();;
Fatlab: dense linear algebra service = "Native Lapack"

The snippet starts by declaring an alias LA for functions in the LinearAlgebra module. This module contains wrappers for BLAS and LAPACK functions. The startProvider function returns a Boolean value indicating whether the native libraries loaded correctly. A false return value usually means that the library wasn't able to locate the native DLL files. After loading the library, it is possible to run some of the functions from the LA module:

> let a = matrix [ [12.0; -51.0; ]; [6.0; 167.0 ] ];;
val a : matrix = (...)

> LA.det a;;
val it : float = 2310.0

> LA.inv a;;
val it : matrix = matrix [[0.07229437229; 0.02207792208]
                          [-0.002597402597; 0.005194805195]]
> LA.inv a * a;;
val it : Matrix<float> = matrix [[1.0; -4.440892099e-16]
                                 [0.0; 1.0]]

The listing starts by creating a matrix using the standard function matrix from the the F# PowerPack. Then, it uses functions from the LA module to calculate the determinant of the matrix and an inverse matrix. As demonstrated by the last command, multiplying a matrix by its inverse yields a diagonal matrix (with a small error caused by the imprecision of floating point numbers).

The following listing demonstrates two functions for matrix decomposition:

> let v, s, ut = LA.svd a;;
val v : matrix = (...)
val ut : matrix = (...)
val s : Vector<float> = (...)

> v * Matrix.initDiagonal s * ut;;
val it : Matrix<float> = matrix [[12.0; -51.0]
                                 [6.0; 167.0]]

> let a', b' = LA.cov a a  |> LA.eigenSym;;
val b' : Matrix<float> = (...)
val a' : vector = (...)

The first part of the listing shows how to perform SVD decomposition of the matrix. This is done using the LA.svd function, which returns two matrices (ut is a transposed matrix) and a vector representing diagonal of the s component. The original matrix can be recovered by multiplying the three components. The vector s can be turned into a matrix with original values on the diagonal using the Matrix.initDiagonal function.

The last part of the snippet shows how to perform eigenvalue decomposition (it could be extended to perform PCA analysis). It calculates a covariance matrix and then performs the eigenvalue decomposition using LA.eigenSym.

Additional Resources

This article introduced mathematical libraries available in the F# PowerPack. However, there are numerous other numerical and charting libraries that can be used from F#. The following articles review several other options:

As a functional programming language, F# differs in many ways from languages like R, Matlab, or Python. The following articles discuss the key functional concepts and their effect when implementing parallel computations:

To download the code snippets shown in this article, go to https://code.msdn.microsoft.com/Chapter-4-Numerical-3df3edee