question

McAninchRobin-9786 avatar image
0 Votes"
McAninchRobin-9786 asked stevefred-5067 commented

When to use TPL Dataflow

We have code that uses parallelism (Parallel for loops) that is mostly in our services and scheduled tasks. We also have api code using concurrency (await/async). Ideally we would like to be able to share common Business logic between the two types of apps but when combined it has a tendency to create intermittent locking errors at various points so I am looking at ways to learn how to do better.

One thought was to look at TPL Dataflow. I am looking at TPL Dataflow tutorials and am trying to determine if this is a good fit for a web api and if so where would it get inserted into the pipeline? Obviously not in the controller but but beyond that I am struggling to return data to the client. One of the tutorials I found showed a TransformBlock<Request,Response> followed by an ActionBlock<Response> (here is one I followed) in a class which is fine but it writes to the console but I am not sure how use this pattern in the context of an api. https://www.blinkingcaret.com/2019/06/05/tpl-dataflow-in-net-core-in-depth-part-2/.

Essentially I am wondering if TPL Dataflow is a tool that I would consider using instead of a Parallel.ForEach in a task/service because using only concurrency it might cut down on our async/await method errors called by the Parallel threads but isn't appropriate. We didn't have much of a problem until a developer tried to do bulk operations (looping over our atomic actions) that were slow and then tried to mitigate that using a Parallel For loop which seems to cause timeouts.

Any direction as it relates to TPL Dataflow as a replacement for Parallel foreach, if they play better with aysnc/await and lastly in the case where we had bulk processing inside an API is this a bad idea?

Thank you.

dotnet-runtime
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi,@McAninchRobin-9786
TPL Dataflow is suitable for Batch processing, it allows us to assign different amount of thread to of block that really needs it.You can find more details here:
what-is-tpl-dataflow-in-net-and-when-should-we-use-it


0 Votes 0 ·

well that was a great non informational answer rofl
That is typical for microsoft when it comes to real life issues and questions
Microsoft has that United States Congress person mentally when comes to dealing with real life issues
and that is
Pass the buck to some other entity

0 Votes 0 ·

0 Answers