Exercise 4: Parallelize LINQ Queries using PLINQ

Developers can optimize their LINQ queries to execute in a parallel environment by utilizing Parallelized LINQ (PLINQ).

The Parallel Extensions library offers many different ways to implement parallelism in LINQ queries. PLINQ provides you with the System.Linq.ParallelEnumerable class which offers functionality similar to the System.Linq.Enumerable class.

Task 1 – Using the ParallelEnumerable Class’ Static Methods to Parallelize LINQ

In this task, you will continue to use the same solution as the previous exercises.

1. Open Microsoft Visual Studio 2010 from Start | All Programs | Microsoft Visual Studio 2010 | Microsoft Visual Studio 2010.
2. Open the solution file ParallelExtLab.sln located under Source\Ex04-PLINQ\begin (Choosing the folder that matches the language of your preference.) Optionally, you can continue working the solution you created in the previous exercise.

3. Replace the current method calls from Main(), with a call to Ex4Task1_PLINQ().

   C#

   static void Main(string[] args)

   FakePre-13dc01572e4149398062056196276ce8-8d38fcdbb73f42ea9f1342b8b1f34ffeFakePre-7b4f4dda8980485e880942921d67f7a0-4af591ab8c734370be09408813448753FakePre-089ad0859d3343408e39f7fdf61ba33f-74207284a3074c428d8acd6f0d9e6cde Ex4Task1_PLINQ();FakePre-d67e2c502d6e441988c7978563630546-ce4ba1a1644f4d95808c96b9621cb9e7FakePre-f3ee9402a2ef4b5cb53ac8f4d5a18438-6a03aeb2a8a142b29e61adab2682ea50

   Visual Basic

   Sub Main(ByVal args() As String)

   FakePre-dc39b030c8e04cba872b80e4188f9292-ae0d781a07e543cf93e682629fb7183bFakePre-2a788101a8f54bbcab7e9448695669ef-1ea53cd12df7445ba0e2e637849d49ed Ex4Task1_PLINQ()FakePre-23852a8fb24f4e36a88c65473a008f3f-69b4f31e974e434db6362177f2123c7aFakePre-fa3ef20373dc4453acc934e5ceed04fd-53c081db79bd4d04880dae24d59d414f

4. Add the Ex4Task1_PLINQ() method to Program.cs (C#) or Module1.vb (Visual Basic):

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task1_PLINQ CSharp)

   C#

   private static void Ex4Task1_PLINQ() { var q = Enumerable.Select( Enumerable.OrderBy( Enumerable.Where(employeeData, x => x.EmployeeID % 2 == 0), x => x.EmployeeID), x => PayrollServices.GetEmployeeInfo(x)) .ToList(); foreach (var e in q) { Console.WriteLine(e); } }

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task1_PLINQ VB)

   Visual Basic

   Private Sub Ex4Task1_PLINQ() Dim q = Enumerable.Select( Enumerable.OrderBy( Enumerable.Where( employeeData, Function(x) x.EmployeeID Mod 2 = 0), Function(x) x.EmployeeID), Function(x) PayrollServices.GetEmployeeInfo(x)) _ .ToList() For Each e In q Console.WriteLine(e) Next e End Sub

   Note:

   The Select(), OrderBy(), and Where() methods are extension methods off of the IEnumerable generic class, however you are accessing them in a static manner here. You will try a more succinct usage later.

   The ToList() call is for illustrative purposes and is not always necessary in production code. It is used here because you want to immediately fire the LINQ query to collect all of the Employee Info strings, and then write them out to screen later.

   If you were to leave the ToList() off, the query will still fire in order of the Employee ID but each call to GetEmployeeInfo() wouldn’t fire until the IEnumerable generic is iterated through during the foreach loop. This is known as Delayed Execution.

   See Scott Wisniewski’s article at https://msdn.microsoft.com/en-us/magazine/cc163378.aspx for more information.

5. Build and run the application.

6. You should observe that the LINQ query performs the operations in order of the Employee ID. Also observe the total amount of time required to complete the work (the exact time required will vary):

   Figure 13

   Output from a non-parallelized LINQ query

7. It is easy to parallelize this query by making use of the static ParallelEnumerable class’s version of the same LINQ methods. Additionally you’ll need to add an AsParallel() call to the query’s data source. Modify the Main() method just to call the PLINQAsParallel method.

   C#

   static void Main(string[] args)

   FakePre-07038e1da0454b82aab1930a9f8d838e-43936e6227e841049acb06ec4d89b8deFakePre-be750876001d4323a85e1c39720a07a2-8b3ecba4d97248f8a7046098f9d44461FakePre-ac754b55fbc14284abf012ff59650d08-4d6d7587a0484eb5aa874f40b293ace6 Ex4Task1_PLINQAsParallel();FakePre-60c3f5f032d24c0bb2c063267694a2fb-974b3a6271914d2d8cab9b335a07de7fFakePre-51a6442487c942ba901cf19aa17ff9b0-c6151a92a76c481280e372d8e28ce899

   Visual Basic

   Sub Main(ByVal args() As String)

   FakePre-0128d65cd81e4294b22f355d7aca91f3-1faa4b8efb0246a8b3c1a2804de8e2c4FakePre-44fbde1cf26d4c1e92845b504f4f0f78-9ee1c5abd15541b1a6120885237b9692 Ex4Task1_PLINQAsParallel()FakePre-39bcf404f5af4e08b03d68c9b309a465-7c2a9f9a325f4126a816da5374cc3b53FakePre-840221f053a0437cb757abba89a4af6f-6c112aa83b5d47bcae08cdfdfc3b7e33

8. Add the Ex4Task1_PLINQAsParallel() to Program.cs (C#) or Module1.vb (Visual Basic):

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task1_PLINQAsParallel CSharp)

   C#

   private static void Ex4Task1_PLINQAsParallel() { var q = ParallelEnumerable.Select( ParallelEnumerable.OrderBy( ParallelEnumerable.Where(employeeData.AsParallel(), x => x.EmployeeID % 2 == 0), x => x.EmployeeID), x => PayrollServices.GetEmployeeInfo(x)) .ToList(); foreach (var e in q) { Console.WriteLine(e); } }

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task1_PLINQAsParallel VB)

   Visual Basic

   Private Sub Ex4Task1_PLINQAsParallel() Dim q = ParallelEnumerable.Select( ParallelEnumerable.OrderBy( ParallelEnumerable.Where( employeeData.AsParallel(), Function(x) x.EmployeeID Mod 2 = 0), Function(x) x.EmployeeID), Function(x) PayrollServices.GetEmployeeInfo(x)) _ .ToList() For Each e In q Console.WriteLine(e) Next e End Sub

   Note:

   The calls to the Select(), OrderBy(), and Where() methods, which were previously being made on Enumerable, are now being made on ParallelEnumerable. Also notice that a call to AsParallel() has been added to the data source.

9. Build and run the application.

10. You should observe the LINQ query no longer performs the operations in a particular order. Also note that in this example the parallelized version completes in less time than the non-parallelized version (in this case, it completed in roughly half the time due to it being run on a dual-core machine; your results will vary based on the hardware you run this example on).

    Figure 14

    Output from a parallelized LINQ query

    Note:

    The operations are executed in parallel with as many operations occurring concurrently as the number of physical cores will allow.

Task 2 – Using the ParallelEnumerable Class’ Extension Methods to Parallelize LINQ

As mentioned earlier, a more succinct way to take advantage of the Enumerable and ParallelEnumerable classes’ static LINQ methods is to use them as Extension methods.

1. Converting a non-parallelized LINQ query implemented using extension methods to a PLINQ query is straight forward. Replace the PLINQ query in the Main() method to match the following LINQ query:

   C#

   static void Main(string[] args)

   FakePre-c9ab03b9cfb54db9a7ee8f8492eb900e-576beb931433409c9430af04f033b671FakePre-d7c6ea48d6a64ba1986835a6d45d917e-f9289b217f18470b9b7bfb0cc6da852dFakePre-f1b0a11e097d4e52a6c27cf35c164bcb-bdb94267768244cd82d1985cb26eaefb Ex4Task2_Extensions();FakePre-b8b0f343395b413ea282bbb7f9209370-ac6bd5cef10c487296c5bf1b77fa8344FakePre-4e49a0e4f22e4b89a12637c6f3ac673c-5dbabfba18a84f08926536c5ba1771d0

   Visual Basic

   Sub Main(ByVal args() As String)

   FakePre-4222cb2ceca74c3d981b4d04459f260c-b33d7de103e94607a621ec0f4b568316FakePre-7509ad0d4e47416ab90385066b13739b-8ca57584ab01480c9562ea310dbba4e6 Ex4Task2_Extensions()FakePre-93660a8ae5fd4f7d8ce686f87141fc67-b6fa7d77de124a9c8f83a9c8ce86befaFakePre-9e30da8c8966432e885d0aa573c5fcae-9e6f6d7994a0483789858a9404571774

2. Add the Ex4Task2_Extensions() method to Program.cs (C#) or Module1.vb (Visual Basic):

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task2_Extensions CSharp)

   C#

   private static void Ex4Task2_Extensions() { var q = employeeData. Where(x => x.EmployeeID % 2 == 0).OrderBy(x => x.EmployeeID) .Select(x => PayrollServices.GetEmployeeInfo(x)) .ToList(); foreach (var e in q) { Console.WriteLine(e); } }

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task2_Extensions VB)

   Visual Basic

   Private Sub Ex4Task2_Extensions() Dim q = employeeData _ .Where(Function(x) x.EmployeeID Mod 2 = 0) _ .OrderBy(Function(x) x.EmployeeID) _ .Select(Function(x) PayrollServices.GetEmployeeInfo(x)) _ .ToList() For Each e In q Console.WriteLine(e) Next e End Sub

   Note:

   Again, the ToList() used to execute the LINQ query immediately rather than waiting for it to execute when the IEnumerable<T> returned by Select() is iterated over during the foreach that comes later. You are avoiding Delayed Execution.

3. Build and run the application.

4. You should observe that the LINQ query performs the operations in order of the Employee ID. Also observe the total amount of time required to complete the work (the exact time required will vary):

   Figure 15

   Output from non-parallelized LINQ query with extension methods

5. To parallelize this LINQ query just replace the current method call from Main(), with the following one.

   C#

   static void Main(string[] args)

   FakePre-c48f3c20c4dc44089b7c351cbc65bd97-72cdcd5a199a4137b65477d3ca39759bFakePre-f14525b08dcf488485c860290ca46ab8-b287fac1d31945918d3f45a3935cfd64FakePre-3fbe97561de44f978867550bc52a49b4-307b637ac96742529e3c3e1895fba622 Ex4Task2_ConvertToParallelExtensions();FakePre-a40ba1a3dde44ce4b24fd57613aa9ecf-b8b7f6fc2a37498b9a851fb6e7d28fe9FakePre-bd5eadb60d7f42c2a64ebaa8a9173114-8bfcfc25216240b6ae28ae3680f7aeb0

   Visual Basic

   Sub Main(ByVal args() As String)

   FakePre-e5db2d4b98e14cb6b860d76fd4c3c0f7-09654dc304d148efa99ec35406646d71FakePre-bcbc3280d5a5466b8e605cf555a490ea-7ed2884250c2444280ee6a56184217c3 Ex4Task2_ConvertToParallelExtensions()FakePre-4c84af9fb1e14e23922c547ca64c405d-b747456c7f2244009ea60a2757ef9523FakePre-308400ef1cfd498785ae33dcfe86a73c-087a37db4f29496b97f6f38c9c84eeb9

6. Add the Ex4Task2_ConvertToParallelExtensions() method to Program.cs (C#) or Module1.vb (Visual Basic):

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task2_ConvertToParallelExtensions CSharp)

   C#

   private static void Ex4Task2_ConvertToParallelExtensions() { var q = employeeData.AsParallel() .Where(x => x.EmployeeID % 2 == 0).OrderBy(x => x.EmployeeID) .Select(x => PayrollServices.GetEmployeeInfo(x)) .ToList(); foreach (var e in q) { Console.WriteLine(e); } }

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task2_ConvertToParallelExtensions VB)

   Visual Basic

   Private Sub Ex4Task2_ConvertToParallelExtensions() Dim q = employeeData.AsParallel() _ .Where(Function(x) x.EmployeeID Mod 2 = 0) _ .OrderBy(Function(x) x.EmployeeID) _ .Select(Function(x) PayrollServices.GetEmployeeInfo(x)) _ .ToList() For Each e In q Console.WriteLine(e) Next e End Sub

7. Build and run the application.

8. You should observe that like the LINQ query based on the ParallelEnumerable static class, the PLINQ query implemented as extension methods no longer performs operations in order by EmployeeID. Instead the operations are executed in parallel with as many operations occurring concurrently as the number of physical cores will allow. Also note that as in the previous parallelize LINQ example, the parallelized version completes in roughly half the time as the non-parallelized version.

   Figure 16

   Output from parallelized LINQ query with extension methods

Task 3 – Using AsParallel() With Query Comprehension Syntax

In this task you will use the Parallel Extensions library and the AsParallel() method to create parallelized LINQ queries using the query comprehension syntax.

1. Replace the LINQ query in the Main() method with the following query comprehension syntax:

   C#

   static void Main(string[] args)

   FakePre-ea60337f40ee49d98ea3f099ec516033-2b44c75f6963495fad7d1faa05e02e09FakePre-ac6a8d6235104108baa724ba477baee9-762c058a69754e0d9ec50aa5e0a8f61aFakePre-b19f4413b1e6483f80575e2a6c22a9cf-9d83e157a3e94edb8fa306e96e2bfbe6 Ex4Task3_PLINQComprehensionSyntax();FakePre-5dff31965dbf47ae94150b243d24c640-46b47163b7dd49f6b6933f3e6e72f45aFakePre-2e89a7bf264f4b1a9d63f89443de125d-105f2b8fe9c4427193eae4e72e69d34d

   Visual Basic

   Sub Main(ByVal args() As String)

   FakePre-a361e8944f9c41deaf2389bba22fb0d6-7a5ee82d93a04ba49a49585230aca26dFakePre-430c0a5b28ca41d8bc2301f07ea25c9f-f87c7249acc24dd596c37a47385c7e8f Ex4Task3_PLINQComprehensionSyntax()FakePre-6ee0ce1cddd9455c83d36f799c53fa38-e2c60a4c668e40c889e0ede623a0c77bFakePre-a2b9da698e3d47c5ba9d57aa095617c0-07fdb9c1ac824900b8666b4dcac2b1e1

2. Add the Ex4Task3_PLINQComprehensionSyntax() method to Program.cs (C#) or Module1.vb (Visual Basic):

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task3_PLINQComprehensionSyntax CSharp)

   C#

   private static void Ex4Task3_PLINQComprehensionSyntax() { var q = from e in employeeData.AsParallel() where e.EmployeeID % 2 == 0 orderby e.EmployeeID select PayrollServices.GetEmployeeInfo(e); foreach (var e in q) { Console.WriteLine(e); } }

   (Code Snippet – Intro to Parallel Extensions Lab - Ex4 Ex4Task3_PLINQComprehensionSyntax VB)

   Visual Basic

   Private Sub Ex4Task3_PLINQComprehensionSyntax() Dim q = From e In employeeData.AsParallel() Where e.EmployeeID Mod 2 = 0 Order By e.EmployeeID Select PayrollServices.GetEmployeeInfo(e) For Each e In q Console.WriteLine(e) Next e End Sub

3. Build and run the application.

4. You should observe that although the LINQ syntax was changed, the data was processed in the same parallel manner as it was with the ParallelEnumerable extension methods.

   Figure 17

   Output from parallelized LINQ query using the query comprehension syntax

Next Step:

Summary