How to: Write a Parallel.ForEach Loop with Thread-Local Variables

The following example shows how to write a ForEach method that uses thread-local variables. When a ForEach loop executes, it divides its source collection into multiple partitions. Each partition will get its own copy of the "thread-local" variable. (The term "thread-local" is slightly inaccurate here, because in some cases two partitions may run on the same thread.)

The code and parameters in this example closely resemble the corresponding For method. For more information, see How to: Write a Parallel.For Loop with Thread-Local Variables.

To use a thread-local variable in a ForEach loop, you must call one of the method overloads that takes two type parameters. The first type parameter, TSource, specifies the type of the source element, and the second type parameter, TLocal, specifies the type of the thread-local variable.


The following example calls System.Threading.Tasks.Parallel.ForEach<TSource,TLocal>(IEnumerable<TSource>, Func<TLocal>, Func<TSource,ParallelLoopState,TLocal,TLocal>, Action<TLocal>) overload to compute the sum of an array of one million elements. This overload has four parameters:

  • source, which is the data source. It must implement IEnumerable<T>. The data source in our example is the one million member IEnumerable<Int32> object returned by the System.Linq.Enumerable.Range method.

  • localInit, or the function that initializes the thread-local variable. This function is called once for each partition in which the System.Threading.Tasks.Parallel.ForEach operation executes. Our example initializes the thread-local variable to zero.

  • body, a Func<T1,T2,T3,TResult> that is invoked by the parallel loop on each iteration of the loop. Its signature is Func\<TSource, ParallelLoopState, TLocal, TLocal>. You supply the code for the delegate, and the loop passes in the input parameters, which are:

    • The current element of the IEnumerable<T>.

    • A ParallelLoopState variable that you can use in your delegate's code to examine the state of the loop.

    • The thread-local variable.

    Your delegate returns the thread-local variable, which is then passed to the next iteration of the loop that executes in that particular partition. Each loop partition maintains a separate instance of this variable.

    In the example, the delegate adds the value of each integer to the thread-local variable, which maintains a running total of the values of the integer elements in that partition.

  • localFinally, an Action<TLocal> delegate that the System.Threading.Tasks.Parallel.ForEach invokes when the looping operations in each partition have completed. The System.Threading.Tasks.Parallel.ForEach method passes your Action<TLocal> delegate the final value of the thread-local variable for this thread (or loop partition), and you provide the code that performs the required action for combining the result from this partition with the results from the other partitions. This delegate can be invoked concurrently by multiple tasks. Because of this, the example uses the System.Threading.Interlocked.Add(Int32, Int32) method to synchronize access to the total variable. Because the delegate type is an Action<T>, there is no return value.

 using System;
 using System.Linq;
 using System.Threading;
 using System.Threading.Tasks;

 class Test
     static void Main()
         int[] nums = Enumerable.Range(0, 1000000).ToArray();
         long total = 0;

         // First type parameter is the type of the source elements
         // Second type parameter is the type of the thread-local variable (partition subtotal)
         Parallel.ForEach<int, long>(nums, // source collection
                                     () => 0, // method to initialize the local variable
                                     (j, loop, subtotal) => // method invoked by the loop on each iteration
                                         subtotal += j; //modify local variable
                                         return subtotal; // value to be passed to next iteration
             // Method to be executed when each partition has completed.
             // finalResult is the final value of subtotal for a particular partition.
                                     (finalResult) => Interlocked.Add(ref total, finalResult)

         Console.WriteLine("The total from Parallel.ForEach is {0:N0}", total);
// The example displays the following output:
//        The total from Parallel.ForEach is 499,999,500,000
' How to: Write a Parallel.ForEach Loop That Has Thread-Local Variables

Imports System.Threading
Imports System.Threading.Tasks

Module ForEachThreadLocal
    Sub Main()

        Dim nums() As Integer = Enumerable.Range(0, 1000000).ToArray()
        Dim total As Long = 0

        ' First type paramemter is the type of the source elements
        ' Second type parameter is the type of the thread-local variable (partition subtotal)
        Parallel.ForEach(Of Integer, Long)(nums, Function() 0,
                                           Function(elem, loopState, subtotal)
                                               subtotal += elem
                                               Return subtotal
                                           End Function,
                                                Interlocked.Add(total, finalResult)
                                            End Sub)

        Console.WriteLine("The result of Parallel.ForEach is {0:N0}", total)
    End Sub
End Module
' The example displays the following output:
'       The result of Parallel.ForEach is 499,999,500,000

See Also

Data Parallelism
How to: Write a Parallel.For Loop with Thread-Local Variables
Lambda Expressions in PLINQ and TPL