Apps

Implementing multithreading in .NET: Two major factors to consider

Justin James shares an interesting reader question about multithreading in .NET. Check out Justin's detailed response, which includes a downloadable PowerPoint presentation about the Parallel Extensions Library and sample code.

 

A reader recently e-mailed me a really good question about multithreading, and I thought other developers might find our exchange useful. Here's the reader's question:

I recently learned about multithreading and found it's extremely interesting. I came across your articles about Multithreading applications and think they are very useful. However, I want more complicated examples and projects so that I can really explore this subject and can make full use of its functionality. Actually, I have several past projects that were programmed single-threaded, but would be much better off if programmed multi-threaded. I have a feeling that it's not easy to implement but would be easier if I had better examples. So would you like to share some of yours with me?

Here's my response (it's slightly edited, since the e-mail was in plain text format):

You are right that it is not easy to implement. However, it does not have to be as hard as it used to be, either.

You may want to check out my slide deck and example code from a presentation I have given on the Parallel Extensions Library. It contains some sample code and explanations in a format that is fairly generic and may help you to get going.

While the examples in there are simple, they represent the most common patterns of code that can easily be turned into effective, efficient multithreaded code.

In general, your initial challenges are going to fall into two major categories: thread management and data integrity. Thread management is getting easier and easier. In the .NET Framework, you can pull threads from the ThreadPool. ThreadPool is a factory that generates threads, where calls to it block if it has created a certain number of threads that have not yet been destroyed. But how to make sure that there are not too many threads running at any given time? After all, if each thread is able to consume 100% of a CPU core, having more threads running than there are CPU cores, will simply cause the OS to start timeslicing the threads, which will cause context switching and inefficiency. In other words, two threads on the same core will not take twice as long to finish; they will probably take two times plus another 10% or so. Three threads on the same core trying to be at 100% CPU usage will probably take 3.25 - 3.5 times as long to finish as one thread. My experience has been that the curve is pretty exponential... more than a few threads per core trying to get 100% CPU, and none of them will ever finish.

So, how to manage how many threads are running?

One way to do this is to have a Semaphore object shared amongst the threads. Before the thread starts to run, it tries to call the WaitOne method of the Semaphore, and when it is finished, it releases the Semaphore. Set the Semaphore's limit to the number of cores on the CPU (using the Environment.ProcessorCount property to determine it); this will keep your system from running more threads at a time than you have cores. At the same time, pulling the threads from the ThreadPool will ensure that you are not creating too many threads at the same time. Creating too many threads at once, even if they are not running, is an easy way to waste system resources since each thread consumes resources. Using a Semaphore, the general pattern will look like this:

static Semaphore threadBlocker;
static void Execute(object state)

{

    threadBlocker.WaitOne();

    //Do work

    threadBlocker.Release();

}
static void RunThreads()

{

    threadBlocker = new Semaphore(0, Environment.ProcessorCount);

    for (int x = 0; x <= 2000; x++)

    {

        ThreadPool.QueueUserWorkItem(new WaitCallback(Execute));

    }

}

There are other ways of going about this as well. One approach that I tried some time ago was to maintain a List<T> of objects. Each object represented the full state of a "worker" object. The worker object would be populated with data for when it executed, and when it finished, it would set a property to indicate that it was done. The main thread would scan that list of objects, and if the number of running threads was low enough, it would start another. To be honest, while this system worked, it was a nightmare to code and debug, and I do not recommend it in the slightest.

There is now an even easier way to accomplish this. Microsoft has the Parallel Extensions Library in CTP. This library makes using these types of patterns very simple. Instead of worrying about the thread management yourself, it takes care of that for you. However, you will still need to handle data integrity issues on your own. An article in the October 2008 edition of MSDN Magazine has some very good information on the topic.

Overall, the big issues that you will need to worry about with data integrity are race conditions and deadlocks. Race conditions are caused by multiple threads trying to update the same object at the same time, which will cause problems. Imagine the following piece of code:

int x = 5;
x = x + 10;

Now, what happens if Thread A and Thread B run this code at the same time? It could work just fine, or it could work incorrectly. How would it work incorrectly? Well, each thread does not execute the entire statement at once and keep the other thread from doing so. So, we could have the following order of operations:

  1. Thread A retrieves the value of x (5).
  2. Thread B retrieves the value of x (5).
  3. Thread A assigns x + 10 (15) to x.
  4. Thread B assigns x + 10 (15) to x.
  5. x is now equal to 15.

Hmm. Alternatively, the exact same code could follow a different sequence:

  1. Thread A retrieves the value of x (5).
  2. Thread A assigned x = 10 (15) to x.
  3. Thread B retrieves the value of x (15).
  4. Thread B assigns x + 10 (25) to x.
  5. x is now equal to 25.

The easiest, most common way to work around race conditions in the .NET Framework is to use "Critical Sections." In VB.NET, the statement is "SyncLock" and in C# it is "lock". Both statements take an Object as a parameter; and other critical sections (including this one in a different instance) trying to lock using the same Object instance will block until the lock is released, allowing only one critical section to run at a time. Our previous piece of code now looks like:

int x = 5;

object lockObject = new object();

lock (lockObject)

{

    x = x + 10;

}

Now, Thread A (or Thread B) has to 100% finish executing the contents of the block before any other thread can enter the block. There are other approaches to race conditions, and you may find a need for them as your projects become more advanced, but the critical section approach is more than adequate for probably 80% of your needs.

Of the remaining 20%, 19% of it can be covered by the Monitor object. Monitor has a method called Enter, and once the Monitor object has been entered, any calls to Enter will block until the thread that called Enter calls Exit. Like the critical sections, Monitor requires an object to be passed to it. So again, our code would look like this:

int x = 5;

object lockObject = new object();

Monitor.Enter(lockObject);

x = x + 10;

Monitor.Exit(lockObject);

What does Monitor give us that critical sections do not? Nothing, unless you need more fine-grained control over when the lock is ended. Some pieces of complex code can either need a lock for a long time or a short time, depending upon conditions not known until run time, such as a variable value. In those circumstances, Monitor is a better choice than critical sections.

The other really big concern in data integrity is a deadlock, which is when multiple threads have locked resources in a way than none of them can continue. For example:

Thread A:
Monitor.Enter(object1);

Monitor.Enter(object2);

//Do work

Monitor.Exit(object1);

Monitor.Exit(object2);
Thread B:
Monitor.Enter(object2);

Monitor.Enter(object1);

//Do work

Monitor.Exit(object1);

Monitor.Exit(object2);

If Thread A and Thread B both call their first statements and complete simultaneously, neither of them will ever be able to call their second statements -- that's a deadlock. Being careful while writing your code and really thinking hard about how you are writing your code are beneficial. Deadlocks are frequently caused (in my personal experience) from a novice at threading trying to over engineer their locking strategy and getting too granular. Having code with nested locks usually indicates code that needs to be seriously examined.

Hope this helps!

J.Ja

Disclosure of Justin's industry affiliations: Justin James has a working arrangement with Microsoft to write an article for MSDN Magazine. He also has a contract with Spiceworks to write product buying guides.

---------------------------------------------------------------------------------------------------------------

Get weekly development tips in your inbox Keep your developer skills sharp by signing up for TechRepublic's free Web Developer newsletter, delivered each Tuesday. Automatically subscribe today!

About

Justin James is the Lead Architect for Conigent.

16 comments
un_chalo
un_chalo

Nice post. However, most people involved with Microsoft are evangelists that talk about Parallel Extensions. However, Parallel Extensions are always changing from CTP to CTP. They are a beta version we cannot use in serious projects. The problem is that we need to improve performance now! I think that two books are covering this topic as far as I can see from their table of contents: Look at http://www.multicoreinfo.com/books/ C# 2008 and 2005 Threaded Programming: Beginner?s Guide Author: Gast?n C. Hillar 395 pages, Expected January 2009 ISBN 1847197108, ISBN 13 978-1-847197-10-8 http://www.packtpub.com/beginners-guide-for-C-sharp-2008-and-2005-threaded-programming/book Concurrent Programming on Windows [Amazon Link] Author: Joe Duffy Publisher: Addison Wesley, Pages: 1,008, November 2008 ISBN: 0-3214-3482-X https://www.amazon.com/dp/032143482X We must learn multithreading. Microsoft will be changing the Parallel Extensions until Visual Studio 2010 Service Pack 1 is on the street. I remembered coding for Microsoft Transaction Server and then they changed EVERYTHING with COM+.

BobMRP
BobMRP

Thanks for getting Moore's Law correct in the slide deck. What Moore really said was the cost of a transistor is declining due to doubling the number of them on a peice of silicon every x months. The number of transistors is not the clock speed. Moore's law has not ran out.

Saurondor
Saurondor

Well not that they became unsafe. More like we discover they're not thread safe. Some implementations work well when they're run single thread, but soon fail in a multithread scenario. Class scoped variables vs Method scoped variables For example: class InvoiceTotalCalculator { Integer total; public Integer calculateTotal(List invoiceList) { total=0; for (int i=0; i=invoice.size; i++) { total+=invoice[i].total; } return total; } } BTW this is an overly simplified class that could be instantiated every time, but lets imagine we have more configuration in it and it has been left out for the sake of post simplicity. So in our application it is instantiated only once and used every time we need to calculate totals. This will work fine in a single thread environment because calculateTotal is called only once at any one given time. Now say we want to use a multicore system to calculate various invoice totals at the same time. Even locking won't help: lock (total) { total=0; for (int i=0; i=invoice.size; i++) { total+=invoice[i].total; } return total; } This will give the proper result, but it is the same as single thread. Because I can only use the total property once at a time only one thread can be actually calculating at any given point. I might get more granular and lock only when the total property is accessed: lock (total) { total=0; } for (int i=0; i=invoice.size; i++) { lock (total) { total+=invoice[i].total; } } return total; But this will give unreliable results. Thread A can enter the method, set total=0 and start adding. Thread B can then enter the method and reset the total to 0 while A is adding. Since for loop control variable i is declared inside the method calculateTotal we have two of them. One for thread A and one for thread B. The variable invoice is also specific to the thread. We have two invoices that can have different lengths. So thread B that entered after A might finish before A. Taking with it the result of the sum of its elements in invoice plus some possible elements in A. Then again if its list is short it might finish before A gets a chance to wake up. Thus you might get the right answer for B once or twice and think it actually works well or wonder why it calculates ok for B, but fails for A. The best solution is to include total inside the method declaration: class InvoiceTotalCalculator { public Integer calculateTotal(List invoiceList) { Integer total=0; for (int i=0; i=invoice.size; i++) { total+=invoice[i].total; } return total; } } This makes calculateTotals thread safe and doesn't require any locking at all. Variables declared inside a method are thread safe and need not be locked. Although it may seem like a very trivial example I've lost count of the number of times I've detected this issue on new programmers to servlets. Where the servlet is ran in multiple threads and programmers keep using servlet properties to control state. Another problem you might encounter when multithreading is stale data. Take for example this simplified example from a PBX application. We have a list of calls to process (be dial, answer, etc) and they're on a list. We used to: for (int i=0; i

Justin James
Justin James

Over the last few years, multithreaded code is gaining a lot more attention, thanks to the recent generations of multi-core CPUs from Intel and AMD. However, multithreaded code has traditionally been difficult to write. As a result, many developers still are not writing code like this, even on projects that could benefit from it. Has the difficulty in writing this kind of code been the reason you have not been using it? If there were easier, less confusing, less difficult ways of writing and debugging this kind of code, would you start doing so? Let us know! J.Ja

Justin James
Justin James

No problem on that! It is important to make the distinction, and a lot of people don't. The simple fact of Moore's Law showing itself in core count more than clock speed is the major impetus behind this stuff. If clock speeds just kept going up, in most cases, it would be faster to use a single thread with no timeslicing than multithreading, other than doing background processing. J.Ja

Justin James
Justin James

"Justin, could you post something on multithreading in GUI environments under .NET. You know the updating the GUI's progress bar by a very long calculation." Yup, not a problem at all! In fact, I have an application that I tinker with every now and then that not only does just that, but does some *intense* calculations as a parallel process. The thread management system mentioned in the article that used a list of objects that had a "finished" flag was originally written for this application; I then converted it to use the Parallel Extensions Library, and not only is the code much more maintanable (I dropped the LOC count by a bit during the conversion, but that only tells half the story since I made it much more robust in the process and added a lot of error handling), but it runs a touch faster too (my previous algorithm would have periods where it was running less threads than optimal). What I will do, is make a generic version of the application (without any of the internal processing, since I have been toying with commericalizing this code), and write an article in the near future with it, and have the code available as a download. :) J.Ja

jck
jck

I haven't gotten into threading in YEARS. I write apps that basically hold single DB connections, deal with one dataset or two in memory (but only do one thing at a time). I haven't taken the time to write threaded apps because I don't see where anything I do would benefit. And now that I've gotten professionally out of the MS software profession (as a livelihood), I don't really need to worry. Although if I find a need for it in the apps I'm working on freelance, I will look into it for sure. :)

Tony Hopkinson
Tony Hopkinson

process and print data, etc. More rarely you'll see something that will allow you to to parallelise it. However that's almost certainly going to be back end processing. Most attempts to multithread client side, break down on synchronsiing with the UI thread. Not so much that they can't be done, but that the UI has nothing else to do until they are. Aside from obvious scenarios, you have to do a good bit of rethinking, before you can make use of parallelisation within one product. Most front ends are linear processes. Adding the extra level of complexity just to show you can, is never a good idea. I know of one person who did some parallelisation, chopped a minute off a button click. It took three minutes anyway, so the user used to click and then read an email, make a brew etc. They were down to making a brew, machine was hammering away, barely usable for anything else...... At the front end there's a lot more of don't need it, than don't want it.

Saurondor
Saurondor

If you're developing web applications more certainly than not you're doing multithread related work. The considerations Justin mentions come into play big time in those scenarios. Particularly as you application becomes more complex.

Justin James
Justin James

I agree, there is a world of difference between applications which are well suited to a "multithreaded" apporach, like background processing or printing, and tasks that are good for a "parallel processing" conversion. The good PP candidates (in my experience) are those which have a loop who's contents can saturate the CPU (either per-iteration, or a lot of "easy" iterations), and at the same time, each iteration has little to do with each other, other that sharing an input and output queue, and maybe a few other basic items like a flag to cancel processing. In other words, a batch process. I may note, that stuff is EXTREMELY easy to convert to parallel processing using the Parallel Extensions Library, and the slide deck/code mentioned in the article shows exactly that (as well as comparisons to traditional-style parallel processing code!). J.Ja

Saurondor
Saurondor

If it isn't necessary the cost of redesigning a running app to multithread could be large. Specially if the elements sent into the thread are not thread safe and complex.

Tony Hopkinson
Tony Hopkinson

It implies threading isn't necessary to achieve the requirement. If it isn't, what's the business case for the cost of the extra complexity. I can come up with chapter and verse of the technical benefits, and potential business benefits. I may even get my way, it happens, every now and then. Usually when the paint moon blue process completes at the same time as some manager looks up at night, in a clear sky, when it's full enough.....

Saurondor
Saurondor

Clearly we can't run the two parts in parallel as they are required to run in series. Never the less the whole batch can be run in a thread so you can run multiple batch jobs concurrently. Regarding the status message. I don't see how that implies no threading. You can very well thread it. Update the status message instead of the progress bar and still enjoy the benefits of threading. For example a non locked GUI. Simple two serialized thread mode -start new thread -run part one -thread ends and message gets updated -new user input -start new thread -run part two More complex one thread sleep mode -start worker thread -run part one until done -thread notifies call back and is put on sleep -message gets updated -new user input -thread is woken up to continue with part two

Tony Hopkinson
Tony Hopkinson

Equally, you are talking a really trivial application where you couldn't do some. Whether it's worthwhile, from a business point of view and to a certain extent a technical one, is very much open to question. I don't see any difficulty in start collecting versus starts printing, in terms of the UI app. A printerserver is multi threaded, usually, why can't a data collection server be. Of course print servers come off the shelf.... I'm not arguing against progress bars, splash screens, even animated cursors, but they are pretty trivial. What I was saying was if you had a two stage batch process, and the UI spec said Do Part 1 Display Status Message Do Part 2 Display Status Message It implies no threading If the spec changes to Display Status with Progress bar, then you'd thread, call back , or poll and update the bar. If you could start Do Part 2 while Part One is in progress, then you can parallelise. But if the business says spec one is good enough ( ie there's nothing else you could sensibly do in the app), how are you going to justify option 2, never mind 3? There are classes of problems where the extra complexity of threading , simply can't be justified to a Business.

Saurondor
Saurondor

Sorry I had non html web apps in mind when I typed that. Two were in my mind. One was a PBX control software for an Asterisk PBX. I used the initial prototype code as an example for the stale data comment I posted. In this application there is no GUI, there is no web server and no html files. Requests come from the PBX over the AGI protocol and the application takes control of the call progress. Obviously you don't know when a call will come in, at what time will the caller respond with the DTMF tone selection nor how long the data queries will take. Multithread is the only way to go. The other example is a GUI based app that collects data over TCP/IP. You can't block the GUI after the "Start Collecting" button is hit, because how do you stop then? You need to update the display to let the user know there's activity too. All that needs a multithread environment that can not be delegated to some third party software in the way printing or a database query can be handed over.

Tony Hopkinson
Tony Hopkinson

In the system as a whole there is lots of multi threading. Sevice Containers, web services, printing, transfering, querying etc. Those however you can view more as an SOA type approach. Throw the request at the server wait for the 'answer'. Whether you block the UI thread (functionally, if not display wise) depends more on the functionality coupled with the perception of the UI. I wanted to thread / parallelise a batch process, but they asked me to block in the UI thread, because they wanted to display a process status, and get another UI request to continue. Doing the work would have been fun, it would have improved the code, if may have even shaved off a small percentage of processing time. But there's no way I could justify the required effort to do that based on user perception of the improvement. I don't like being laughed at by bean counters, at least you can explain to a peer, why you didn't go down the 'obvious' route.

Editor's Picks