question

KaushikKapasi-9972 avatar image
0 Votes"
KaushikKapasi-9972 asked AgaveJoe edited

Updating dictionary from multiple threads

I have a 2-level nested dictionary:

Dictionary<int, Dictionary<int, string>> MyOriginalDict

I need to update the above string values.
I created a copy of the above dictionary and I iterate through this copy using foreach and for every level-2 key, I call a function which updates the value corresponding to it in the main dictionary.

I use ThreadPoolLoader to call the above function. Around 500 threads are created which independently update the original dictionary using ContainsKey() to update the correct value. No new values are added and no two threads update the same key-value.

There are no errors thrown and function calculation is correct. However, it seems the string values updated in the Original dictionary are randomly assigned. It's like ContainsKey() is not working - the values are updated randomly.

Where am I going wrong? Note: If I do the above in a single thread and simply update each string value in a foreach loop, it all works fine but takes several minutes to run.

Thanks.



dotnet-csharp
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.


According to documentation, Dictionary is not thread-safe, though it supports multiple readers concurrently. Maybe your threads will only work if they access different Dictionary<int, string> objects but never update the main dictionary. Show some details.

You can introduce a corresponding locking or use correctly the ConcurrentDictionary class, which is thread-safe.

0 Votes 0 ·
AgaveJoe avatar image
0 Votes"
AgaveJoe answered AgaveJoe edited

Your sample code is using i at the time of execution which is 50. It has to do with closures. Task.Run() is the recommended approach.

     public class MyClass
     {
         ConcurrentDictionary<int, int> _numbers = new ConcurrentDictionary<int, int>();
         public MyClass() { }
         public async Task CallMeAsync()
         {
             _numbers.Clear();
             _numbers.TryAdd(99, 100);
             int threadCount = 50;
    
             Task[] tasks = new Task[threadCount];
    
             for (int i = 0; i < threadCount; i++)
             {
                 int j = i;
                 tasks[j] = Task.Run(() => AddToDict(j));
             }
    
             await Task.WhenAll(tasks);
    
             foreach (KeyValuePair<int, int> item in _numbers)
                 Console.WriteLine("Key: " + item.Key.ToString() + "; Value: " + item.Value.ToString());
    
     
         }
         private void AddToDict(int number)
         {
             if(!_numbers.TryAdd(number, number + 1))
             {
                 Console.WriteLine("Not added - Key {0} Value {1} ", number, number + 1);
             }
         }
     }


Implementation (.NET 5)

     class Program
     {    
         static async Task Main(string[] args)
         {
             MyClass myClass = new MyClass();
             await myClass.CallMeAsync();
         }
     }


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KaushikKapasi-9972 avatar image
0 Votes"
KaushikKapasi-9972 answered AgaveJoe edited

WoW! You not only showed the right way to use multi-threading but also corrected careless mistakes. Thanks!

Q. Why is (int j =i) required?

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Why is (int j =i) required?

The closure is able to access the free variable scoped in the for...loop block. You can think of the concept in terms of a class instance with a local variable.

 public class Sample
 {
     public int i { get; set; } = 0;
     public void Increment()
     {
         i++;
     }
 }

But, you should take the time to learn closures and lambdas in C#.






0 Votes 0 ·
KaushikKapasi-9972 avatar image
0 Votes"
KaushikKapasi-9972 answered Viorel-1 edited

Here is my Console App (Please copy-paste in new VS console app and run):

// I create multiple threads in a loop and add values to a ConcurrentDictionary from within each thread. Once all threads
// have completed, I print the contents of the dictionary to screen - doesn't work!!! How do I make this work?

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;

namespace MultThreadTest
{
class Program
{
static void Main(string[] args)
{
MyClass myClass = new MyClass();
myClass.CallMe();
}
}

 public class MyClass
 {
     ConcurrentDictionary<int, int> _numbers = new ConcurrentDictionary<int, int>();

     public MyClass(){}


     public void CallMe()
     {
         _numbers.Clear();
         _numbers.TryAdd(99, 100);

         int threadCount = 50;
         var doneEvent = new CountdownEvent(threadCount);

         for (int i = 0; i < threadCount; i++)
         {
             Task.Factory.StartNew(() => AddToDict(i));
         }
         Task.WaitAll();

         foreach (KeyValuePair<int, int> item in _numbers)
             Console.WriteLine("Key: " + item.Key.ToString() + "; Value: " + item.Value.ToString());
     }

     private void AddToDict(int number)
     {
         _numbers.TryAdd(number, number + 1);
     }
 }

}

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.


This sample seems to include at least two issues.

The first issue is similar to problems like https://social.msdn.microsoft.com/Forums/en-US/a88e0d74-2f3b-4fe7-8ee7-60e3f7b617a2. To fix it, introduce a local variable.

The second issue is related to Task.WaitAll, which does not seem to have any effect.

Try this modification:

 var tasks = new List<Task>( threadCount );
    
 for( int i = 0; i < threadCount; i++ )
 {
    var i2 = i;
    tasks.Add( Task.Factory.StartNew( ( ) => AddToDict( i2 ) ) );
 }
    
 Task.WaitAll( tasks.ToArray( ) );
    
 Console.WriteLine( $"Total keys: {_numbers.Count}" );
    
 foreach( KeyValuePair<int, int> item in _numbers.OrderBy( p => p.Key ) )
    Console.WriteLine( $"Key: {item.Key}; Value: {item.Value}" );



0 Votes 0 ·
KaushikKapasi-9972 avatar image
0 Votes"
KaushikKapasi-9972 answered Viorel-1 commented

I changed my code to use ConcurrentDictionary. No luck.

I also changed the code to create a new ConcurrentDictionary at the class level and have multiple thread add objects to it. However, still when I look at it after all threads are completed, it has fewer objects than there should be!!! I even use lock{} before adding new KeyValue.

I think, I am missing something w.r.t. how items are added to this dictionary from simultaneously running multiple threads.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.


Since it is not clear how to reproduce the problem that corresponds to your scenario, you can prepare a simple project that illustrates the issue, and publish it on OneDrive or GitHub, etc.


0 Votes 0 ·