Processors

How important is multithreading in application development?


I think multithreading in application development is the wave of the future -- even though not all applications lend themselves to being multithreaded.

In a blog post on MSDN about symmetric multicore processing being a 'dead end,' the author makes a lot of excellent points, not the least of which is that all too many applications still need one big core to work effectively. However, the author fails to see where the CPU market is headed and that the days of Moore's Law applying to individual cores is over for the time being.

I have been working on a very nifty little application for an article I am writing that will take me about a month's worth of weekends. The application does some image editing, but the editing it performs is rather CPU intensive. The CPU crunch shows a rather intriguing pattern.

As you can see in this Task Manager screenshot (Figure 1), one of my cores (this is a Core 2 Duo system) is cranking pretty hard. The other core is spinning but not so hard. The difference between the two is that the hard working core (the one on the right) is running my application. The "lazy" core is me doing the rest of my work on the PC, as well as Visual Studio handling the overhead of my application running within it (I am still debugging). For example, the two big spikes on the "lazy core" are two times where UAC kicked in (I was moving items on the Program menu).

figure-1.png

Figure 1 (Click image for full size)

What can I do? My first option is to leave the situation as it currently stands. Sure, the application is taking 10+ hours to run through a relatively lightweight operation, but it is rock solid on reliability, and the RAM usage is insanely stable, which reduces a ton of page file hits. My next choice is to try multithreading some portion of the work. As you can see, I have one core working hard, but one of them is not doing a thing to help me -- there is a lot of untapped power in my hardware. The system is still extremely responsive, and I am not seeing any slowdown. Looking at the graph (and the process list), my application is hovering around 48% or so of CPU, which is nearly 100% of the core that it is on. Popping off a second thread would make the system really slow (since I would be taking 100% on both cores), but it may allow me to finish the task in half the time. That is a tough trade off to consider.

I think there is a middle road in this case. Looking at the workload, there is no one particular portion of the workflow that could be performed asynchronously. Even if there was, none of the operations are long enough for the cost of context switching to be offset. Now that being said, the overall operations are performed in near total isolation of one another, and while they work with a common point of memory, that access is fairly quick and is only a small portion of the functionality. In other words, it is a low contention area. So having identified only one potential contention area (and being a rare case of a block occurring), we have found our middle road.

I am going to try to split up the loop, following my general rule on the number of threads: single thread CPU usage * number of logical cores. So, if it takes 50% of one logical core to run the processing thread, you do not want more than four threads running on a system with two logical cores; this puts your workload at 100% of all cores. I often decrement that number by 1 to ensure that the base OS and other applications are not completely squeezed. After all, does your customer care that your application is as efficient as possible if it effectively renders his/her PC useless while it is running? Probably not.

In this case, however, I am going to try something different. I will usually split the loop evenly and have each loop do its work in perfect parallel. This time, I am going to have one thread marked as a special "stutter thread" (I am making up words here -- I am not sure if there is a proper term for this). The work will not be evenly divided; the "stutter thread" will have only half of its "fair share" of work to do, and the other threads will have the other half evenly divided amongst themselves. In exchange for having less work to do, the "stutter thread" will only "work" 50% of the time. The other 50% of the time, it will remain idle, freeing up a touch more CPU for the OS and other applications. I will most likely use a semaphore tied to another thread (say, thread 0 signals the "stutter thread" on every other iteration). I could also use a monitor (thread 0 enters the monitor on every other iteration, and the "stutter thread" also tries entering the monitor). I think either approach would be viable.

I will let you know in a few weeks how this goes. (I am going out of town this weekend, so I will not be working on it.)

J.Ja

About

Justin James is the Lead Architect for Conigent.

51 comments
jslarochelle
jslarochelle

..stuff that we do. Our application interacts with external hardware (valves and sensors) and we absolutly need multiple threads to optimize data acquisition and processing. However, it is a necessary evil. The source of lots of complication and potential bugs. The code must be top notch and this is not always obvious when the pressure to ship the next version rises. Fortunatly the multithread support in Java got a good boost in version 1.5 and static analyzer are also available to help validate the code. JS

Q360
Q360

It is always important to offset added complexity (and therefore decreased maintainability and increased testing load) with the need to perform. That being said I have yet to run accross a system (of good size) that cannot make use of some dilligent multithreading. I think image processing is certainly one of them. You might even be able to run a few more threads, but since your processing is so intensive, I might recommend running it through the threadpool and configuring the threadpool, then using a semaphore to limit how many you queue up in he worker process queue at a time. Something like: queueThrottle = new Semaphore(throttelingQueueLimit, throttelingQueueLimit); with WaitOne and Release. As for strategies for running the 2 cores .... if you were doing your own custom threads, you could have each one have an affinity to the other core .... then on the lazy thread, just drop the priority of that thread and maybe tell it to spin its wheels once in a while with a sleep here and there. Looking forward to hearing your results.

Justin James
Justin James

Have you ever tried to do some parallel processing in a loop? Why or why not? If you did, how did you do it and what did you do? J.Ja

Justin James
Justin James

You are right on the money to bring up the maintenance issue! Fact is, most programmers cannot (or do not bother to try to) wrap their heads around MT. Writing an MT apps makes it fairly un-maintainable. J.Ja

reiki33
reiki33

A very interesting company, Alliant, http://en.wikipedia.org/wiki/Alliant_Computer_Systems with whom I interviewed in the 1980s had some interesting technology. Their FORTRAN compiler could unroll FOR loops so each processor could handle an iteration. They had far less success with C, as it was hard to unroll the logic of the pointer arithmetic.

reiki33
reiki33

In the 1970s, I wrote a backup scheme in Data General FORTRAN with threads, or multitasking, as they called it. There was a task to read the file, and one to write the records qualifying for backup.

SnoopDougEDoug
SnoopDougEDoug

Multithreading will never be widely used until we move away from explicit constructs with explicit locks, and into languages with built-in support for multithreading. If I'm not mistaken, Erlang is supposedly the numero uno language for multithreading? Have you tried it? In what language are you writing your app? Check out the discussion about this very topic on O'Reilly, http://radar.oreilly.com/archives/2007/03/concurrent_prog_1.html. My first experience with multithreading was when I worked at Sequent in Portland OR in the late '80s. (IBM bought them out years ago). Anita Ousterhous (sp?) wrote a "Guide to Parallel Programming" book. I maintained their version of the Unix Programmer's Manual. At that time the issue was how to transmogrify C to parallelize existing loops. I believe they got some technical oomph from Kuck from UofIllinois. doug in Seattle

john
john

I do lots of parallel processing and use multi-threading as a result. It's absolutely required if you want to squeeze performance out of your app (given that you can split the work into multiple threads of execution). Many of the problem you run into are not so obvious but the benefits are worth it.

Mike Page
Mike Page

It has been my experience under Windows that allowing a single applications to tap all the CPU power of the system does not make the system unresponsive. This is because the Windows scheduler shares processing power among all processes and threads concurrently running. The foreground application is biased with more execution time to improve responsiveness. The only way to lock up a system is to give a thread real-time priority. Then it soaks up all processing power.

Java Guru
Java Guru

You hit on some important points. It should be clear that some of the hoops you are going to have to jump through are not "normal" programming for the masses. It takes a very gifted programmer to develop an application that is efficiently multi-threaded. Luckily most applications (not web applications) do not need to be multi-threaded. The example you are playing with is one that would be written by a group of developers with very specific expertise. It is much easier to develop a system to be used by multiple users than it is to develop a system designed to be used by one user for several CPU intensive tasks.

Q360
Q360

The way I usually mitigate it is to write a well contained base class that other developers then just have to inherit from and override a method or two. It also means that if we want to do something MT, we need to put in the extra time to define the pattern and build a decent base class. However, once the base class is working, it makes it fairly quick and easy to build additional implementations on it (so you save time and defects in the end). My favorite one so far is one for MSMQ.

reiki33
reiki33

Probably the most important part of the maintenance issue with MT is providing documentation - preferably in the code so it survives - of the state space of the tasking. Without that, the maintenance programmer will have to deduce it, and that is fraught with peril. OTOH, that seems to be the normal software maintenance mode!

apotheon
apotheon

I haven't actually had a whole lot of call to deal with concurrency yet -- though I expect I will before much longer. I look forward to learning more about maintaining multithreaded code. No, really. I'm not being sarcastic at all. I love learning new things.

Justin James
Justin James

You are right that pointers (and any type of direct memory reference) system makes MT very unpredictable for something rules based. Another is OO, and global/shared variables. Thread safety is impossible for a compiler to predict, and therefore automatically cause to happen with those conditions in place! Any kind of compiler optimization needs to be founded upon mathematically provable principles, otherwise you have something cranking out code which breaks randomly or under certain circumstances, and features/functionality of most (if all) modern, mainstream make the code impossible to mathematically prove. J.Ja

SnoopDougEDoug
SnoopDougEDoug

I will probably expose myself as the dilettante I am, but in many cases folks implement multithreading as message-based systems. You issue a message and it is sent, received, and handled asynchronously. If the sender and receiver exist on the same CPU, I would think the overhead would be negligible. I worked on a project at a large software company (gee, I wonder who Doug, since you live near Seattle) which was trying to implement a massive distributed computing mesh. The rule of thumb was that until you reached about 50 KLOC, there was no advantage to firing off a remote request. I'm curious how one would test an algorithm to see when it makes sense to go through the pain of multithreading. If I were running a company where MT was under consideration, I would bring in consultants. A poorly-conceived MT application could run worse than a well-conceived mono-threaded app. doug

Justin James
Justin James

Doug - I am in general agreement; there are also many simple MT things that the compiler could do as an optimization. I have never worked in Erlang, but I know that Sun's Fortress language attempts (from what I can tell) to make MT very easy/seamless, which makes sense when you look at the Niagara CPU. This current project is in VB.Net; while .Net's MT stuff is capable, it is certainly no fun to work with, and inter-thread communications is a headache with callbacks and such. J.Ja

apotheon
apotheon

"[i]Multithreading will never be widely used until we move away from explicit constructs with explicit locks, and into languages with built-in support for multithreading.[/i]" There will be a need for significant adoption of "real" multithreading capability in programming languages in the near future. There are a great many languages already incorporating such capabilities, however. For instance, as I understand things the current stable Ruby release (1.8.6 or 1.8.8, I think, off the top of my head) has support only for "green" threads -- but 1.9 (the development branch for the upcoming Ruby 2.0) supports real threading. As more languages move to supporting real threads, the languages that don't will find themselves falling further and further behind the power curve. This will probably lead to some significant changes in the popularity of certain languages for certain types of programming projects. Languages like C will of course still be important for their core niche -- low-level systems programming -- but many language will find their own niches taken over by other languages if they don't get up to speed on threading support.

steve
steve

Agree with Justin in terms of direction of the industry. Just look at the direction Sun is headed with 8 and 16 cores per box. Granted this is the server, but still the direction. Also, check out this one - a company called Azul (azulsystems.com) has a server with 768 Cores and 768 GB of memory!!! Might take some serious programmers to take advantage of this processing power. Only draw back I can see is it is Java based. But if I read this correctly, they have one of the best buzz phrases I've heard in a long time. Called Optimistic Thread Concurrency. Basis is their appliance has the power to aid in managing the locks, thus freeing the developer from time worrying about locks. My point - just interesting stuff and agree with Justin's take on the direction of the industry.

davidsaylor
davidsaylor

I may be mis-understanding your use of the stutter thread, but it sounds like you are trying too hard. It sounds like you are trying to create an additional thread that doesn't to as much work so that it will have some 'free time' - as though it will be getting an equal share of CPU time, and can then 'donate' half of it's share of CPU time to other processes. I don't think you should make any assumptions about the number of CPUs/Cores available. Just user the appropriate number of threads for the work. And don't try to 'grab CPU cycles' that you don't need to create a 'processing reserve'. Just prioritize your heavy, time-consuming work below normal and let the windows scheduler take care of how CPU cycles are divied up between your threads and between your app and others that may be running - it will anyway. I think your stutter thread will either have no effect, or may simply result in some unused CPU cycles. It may make your CPU usage graph look like cylces are available, but that doesn't mean another process/app could use them to do work. Granted thread scheduling adds overhead to any multi-threaded project. I'd bank on the provided thread scheduler doing that work more efficiently and more effectively than what you are building. And it isn't like you can replace the thread scheduler anyway. What you build will run 'in addition to' the thread scheduler. -D

Nico_Baumgarten
Nico_Baumgarten

I totally agree. From my experience, Windows is quite capable of handling multiple threads, meaning that you will not need to consider available CPU cores really. Just split up the workload in as many threads as viable and let the OS do the splitting of these threads on available CPU cores. For my applications I tend to usually give the GUI thread normal or elevated priority - since this thread usually has little or no work to do and drastically improves responsiveness of the application (as felt by the user) While on the other hand for all working processes I assign "below-normal" priority. This way your application workload can finish as fast as possible while leaving room for any other applications to use some CPU power.

aureolin
aureolin

Multithreading on PC's has been around since OS/2 and the techniques to do it right haven't really changed since then. You do have to understand that it is a distinct programming skill. And, like so many other areas of programming, there are ways to do it right and many more ways to do it wrong. You don't need to be an "elite" programmer, but you do need to respect designing and implementing multithreading as a distinct skill set. Steve G.

Justin James
Justin James

Yes, multithreading is *not* for the easiest task in the world, and most "normal" programmers may get it to work, but it will often be buggy. Sadly, most of the tools that allow better and/or easier MT work (like the Intel CT compiler) tend to be for langugages like C++ and Fortran, which doesn't help the average developer too much. MT in .Net and Java is definitely un-fun. J.Ja

Justin James
Justin James

That is a very solid approach. It is a shame that more developers do not work that way! J.Ja

Justin James
Justin James

Doug - You are correct that in the strict, English meaning of the phrase "object oriented", noun-i-ness (speaking of strict English...) is guaranteed. As Chad points out, Ruby is OO and verb-y too, and he confirms my suspicion on Smalltalk (I never used it, but I've read a lot about it lately). I think it is possible, but only if OO gets thought of differently. For example, take the following (very dinky) class definitions: Public Class Dinky1 Private bFoo As Boolean Property Foo() As Boolean Get Return bFoo End Get Set (ByVal value As Boolean) bFoo = value End Set End Property Public Function Bar() As String If Me.Foo Then Return "Tarbash!" Else Return "Flarp" End Function End Class Public Class Dinky2 Private sWassap As String 'Blame the variable name on a post I just read by Chad... Property Wassap() As String Get Return sWassap End Get Set (ByVal value As String) sWassap = value End Set End Property Public Function Bar() As String If Me.Wassap.Length > 10 Then Return "True, true." Else Return "Feed me!" End Function Public Function Bar(ByVal JoinedText As String) As String If JoinedText = "Game Over" Then Return Me.Wassap Else Return "Windex makes a great aftershave!" End Function End Class Now, with these two classes, let's look at how my fantasy language can work: Dim d1Temp As Dinky1 Dim d2Temp As Dinky2 d1Temp.Foo = False d2Temp.Wassap = "Joined at the hip." Bar d1Temp 'Returns "Flarp" Bar d2Temp 'Returns "True, true." Bar d1Temp, "Game Over" 'Error! d1Temp does not implement this! Bar d2Temp, "Game Over" 'Returns "Joined at the hip." See how it works? The basic OO priciples of polymorphism, encapsulation, and (the example didn't go into it, but you can be sure it would be there) inheritance are still there. What has changed is merely the syntax. Internally, a function call passed on the class (the verbiness) simply finds a function in the class with the same prototype. In terms of total typing, it is the same (verb-space-noun instead of noun-period-verb). But the readability (and writeability) of it has dramatically changed. It is now a verb-oriented language! I may add, it seems to be a lot like the Ruby examples posted elsewhere in this forum, but with a VB.Net syntax. Now if we take this to the next step (defining a ton of functions outside of a class, and putting functions and objects on the same footing), we've now fully re-entered procedural code and reinvented C. :) J.Ja PS: that was Alan Kay who said that, of the OLPC project, Squeak, Xerox PARC, etc. etc. etc. :)

apotheon
apotheon

On the other hand, Smalltalk set the standard for good OOP, and it was initially intended as a more verb-oriented language. One of Smalltalk's creators -- who coined the term "object oriented programming" in the first place, though the techniques were already somewhat in use by then -- has since then lamented that he wished he'd called it something like "method oriented programming" or "message oriented programming" because the emphasis should be on the verbs of the language, and not the nouns. In other words, if you want verb-centric programming, you don't necessarily have to use a functional paradigm language. You can use an OO paradigm language, as long as you avoid the OO languages that are too entrenched in a noun-centric tradition. Ruby is really well-suited to a more verb-centric style of OOP, for a number of reasons. Among them is the way blocks are used, essentially acting as ephemeral verb neologisms. The ability to pass around procs as a higher-order function so easily is another case of Ruby lending itself to a verb-centric style.

SnoopDoug
SnoopDoug

But aren't you asking the impossible? You want OO and a verb-oriented language? OO demands a noun orientation. If a language is strictly OO, you cannot get "draw(circle)", you can only get "circle.draw". Typing is a separate issue. Compile-time binding versus run-time binding has vexed CS researchers for years. I go back and forth. For learning I prefer early binding. Once I know what I am doing, I like late binding. As far as proving correctness, I would think early binding would be more of a help than a hindrance to proving correctness. doug

Justin James
Justin James

When I say "OO", I used it synonymously with "noun orientedness" in the context of that "Kingdom of Nouns" piece everyone has (or should have) read. I have, sadly, never had the opportunity to work with a verb-oriented object language (although Smalltalk seems to fit the bill as well). Strictly speaking, "object" oriented necessitates "noun oriented", BTW! The problem is, it is insanely difficult to write a cohesive verb oriented language that is also strongly typed *and* supports objects, *unless* it is mandatory that all objects have ways of casting themselves to a "common language" that all verbs act upon. Think of Perl's way of everything ending up a scalar, hash, or array, and the three being interchangable in many situations, and you get the picture. It is much easier to write a verb oriented language with only three base types, than a verb oriented language with zillions of items directly decendent from type "object". J.Ja

apotheon
apotheon

Ruby's OO syntax for regular expressions is actually quite easy to use. It's very clean and easy to read -- I just have to think about it differently than I think about Perl regexen. OO syntax can be made very nice to use with regexen, as Ruby has proven. The problem isn't with an OO approach, but with a noun-centric, obtuse OO approach. Ruby's standard OO syntax for regex handling is a bit more verb-centric (where "method" == "verb") and involves implicit instantiation of a regex object in a method call, where the object to which the message is being passed is the string on which the method operates. For instance: [b]string_var = 'Java is my language. Java Rules!' puts string_var.gsub(/Java/, 'Ruby')[/b] . . . with output: [i]Ruby is my language. Ruby Rules![/i] You can also do this by passing a block to the String#gsub method: [b]string_var = 'ruby is my language. ruby rules!' puts string_var.gsub(/r/) { |char| char.upcase }[/b] . . . with output: [i]Ruby is my language. Ruby Rules![/i] Putting it all together (same output): [b]string_var = 'Java is my language. Java rules!' puts string_var.gsub(/Java/, 'Ruby').gsub(/r/) { |char| char.upcase }[/b] Compare with Perl's syntax: [b]my $string_var = 'Java is my language. Java rules!' $string_var =~ s/Java/Perl/; $string_var =~ s/r/R/; print "$string_var\n";[/b] Personally, I think they're different -- but about equally readable and easy to use. Of course, I can understand where the impression that OO syntax for regexen is obtuse and annoying -- because with [b]most[/b] OO syntaxes, it [b]is[/b] obtuse and annoying. Python comes to mind (largely because regex functionality exists outside the core language), and Java is even worse.

Justin James
Justin James

MT work is, by its very nature, "verb oriented". OO is, by design, "noun oriented". It's the same reason why OO languages are wretched for regex work, despite using the Perl regex syntax itself. Creating a regex object, configuring it, passing it a pattern and then a string, and iterating through .Matches is a lot dumber than saying "=~". A *LOT* dumber. MT is the same way. It's the same reason why SQL is not OO, and every attempt at OO data access feels a lot more miserable than SQL itself. Indeed, ADO.Net is so headache prone, the brought SQL-like syntax to .Net with LINQ, and then extended it to work for a lot more than just databases. Monday's blog will go a lot deeper in depth with this idea. J.Ja

Justin James
Justin James

Sadly, my employer's network does not allow IM clients (other than the Ipswitch one which we use for internal use), and when I am home, I never get onto IM. Of course, you have my email address, and when I am home and in front of the PC, I tend to answer email in near-real time. :) J.Ja

apotheon
apotheon

I'll keep you in mind as a potential source. If you're on IMs, I may even add you to my IM contact list at some point just so I can pester you in real-time if I run into trouble working with multithreaded code.

Justin James
Justin James

... to you learning about it too! Let me know if you need a hand or some tips. You might want to check out the series I did last summer showing off the basics. It's in VB.Net, which means that while the language is different from what you will be using, the concepts are spelled out in painfully verbose code. :) J.Ja

Justin James
Justin James

The 50 KLOC rule is pretty darned dumb. It's not the X LOC that matters, it's the *latency* in those LOC! I can fire off a single method that could take 10 minutes to run and could/should be run asynchronously (like an HTTP request to a big file); likewise, unrolling loops that may only have a few items in them on an SMP machine can make a ton of sense. In terms of an algorithm... fairly impossible. Common sense is your best ally here. For example, an optimizing compiler would have a hard time optimizing any code that calls another library... who knows if the work can really be run in parallel, or if the underlying code is thread safe? As far as I am aware, there are no code-level thread safety assertations, the best you can hope for is a mention of it in the documentation. These are the kinds of things that make an algorithm impossible. CPU architecture is another. 2, 3 years ago, I never used MT except to pop off a simple thread from the UI, so the UI could continue to work and handle a "Cancel" button. It just was not worth it because of single core and CPU architecture, except for asyncronous I/O. Today, I consider MT for just about anything I do that takes more than 10 seconds to run for a desktop apps (Web servers are another story entirely). Why? Multiple cores mean that spiking 100% CPU in one thread still leaves a lot of silicon unused. Now, for hard processing, MT is the only way to let the user get the most from their hardware for lengthy or CPU intensive tasks. J.Ja

Justin James
Justin James

All good information, thanks! I don't touch C/C++ in my day-to-day work (and neither do most developers anymore), but anything that helps the C/C++ folks helps everyone, as far as I am concerned. J.Ja

etiennefalcon
etiennefalcon

Hi all, I've had a class last semester where we used many MT library/language. OpenMP is by far the easy way to go. We've had slitghly faster result with CWinThreads, but the code was way more complicated. In openMP you don't have to design your code in a MT point of view, you simply make a sequential code. Once it's running smoothly, you add the OpenMP code over it. OpenMP takes care of mostly all the semaphore and timing handling you would have to do by hand on any other MT language. Pthreads and MPI are mostly for cluster configuration, not your usual home PC. Haven't tried CT yet... I doubt it could get any easier than OpenMP but hey! If you want to look into OpenMP it's built in visual 2007... get a walktrough on the web, and have fun!! Etienne Falcon

Justin James
Justin James

I read astoundingly few computer related books. As in "less than 10 over the course of my lifetime." Don't ask me why, I have no clue. I own over 10. Some of them I literally have never opened. I keep meaning to finish "Code Complete" (I'm 1/3rd through, but my Safari membership ran out) and then read the classic "designs and patterns" book, I think those two are fairly big gaps in my knowledge. J.Ja

SnoopDougEDoug
SnoopDougEDoug

I'm sure others got this also, but just after my previous posting I got an email from Intel about a Web seminar on multithreading. I haven't looked at it, so YMMV: http://softwaredispatch.intel.com/?t=3&pin=IS2LAUR3&xid=08G&lid=1815 According to a blurb on the page: Attend three or more live sessions and get the latest O'Reilly nutshell book, Intel? Threading Building Blocks: Outfitting C++ for Multi-core Processor Parallelism by James Reinders. You must be registered to view these events. Ooh, a free book. They sure know how to suck me in! doug

apotheon
apotheon

. . . yet. I probably will at some point. I'm waiting for threading in Objective Caml. That'll be a good day.

Justin James
Justin James

I don't work with any of those languages, but I have a very cursory understanding of OpenMP... but from some info I have, Intel's new CT stuff blows it away. I saw one examples where litertally hundreds of lines of OMP were reduced to a few dozen or so, and the code ran quicker and more efficiently. Then again, it was Intel marketing information, so it is wholly possible that I was looking at a "cherry picked" example and that reality is usually different. But from what I have read about CT, I am very impressed. Along those lines, Sun's Fortress language (a version of Fortran explicitly designed for MT, perfect for their Niagara CPUs) looks interesting as well. J.Ja

frylock
frylock

Have you looked at OpenMP? I've only given it a cursory look so far, but it sounds promising for simply adding threading support to C/C++/Fortran apps.

Justin James
Justin James

You understand perfectly what I am hoping to do with the "stutter thread". I agree that the scheduler may indeed be adequete. The only reason why I am considering such an approach, is due to the amazing about of time this application takes to run in the first place. Right now, with it sucking up; 100% of a 2.4 GHz core, a relative small job has been running for about 24 hours now, and it is just over 70% done (it does speed up as it progresses, since it reuses a lot of data from previous iterations)! That is a really long time to ask for someone's system to be really slow if I were to have a thread on every core taking 100% CPU with an average priority. "I don't think you should make any assumptions about the number of CPUs/Cores available. Just user the appropriate number of threads for the work." These are contradictory statements for this particular project. The work is discrete enough for it to be infinitely threaded out, with little concurrency issues. In fact, the lower into the work I take the point to thread (in order to reduce contention), the more thread I could have! If I really wanted to, I could easily write this to fire off up to a few million threads and let them sit in the thread pool. Which would not necessarily be a terribly bad thing, and would/could probably work well, and if someone had a super PC with a million cores, then all of the threads could run for 2 seconds each, everything would join, and the total execution time would be under 10 seconds. It is not too often you get to work on something like that. With all of that being said, the logical place to do the multithreading (to reduce the overhead of managing a zillion threads) is in the main work loop, to fork the overall job into slices. And since each slice can/will take 100% CPU, having more threads than cores is absolutely not a good idea, because then you will have two threads contending for the same core, which is a waste. So in this case, those two sentences do not go together, since the number of threads that is approapriate is entirely dependent upon the number of cores availble (at least as an upper cap). Since the majority of desktop PCs (this is a desktop app) have, at most, 2 cores, watching the task take twice as long as it needs to for the lack of multithreading is killing me. :) I think I may just keep it simple, and use fork() join() from the Microsoft Research TPL, which is brand new, getting ready for release, and rocks! J.Ja

Justin James
Justin James

I agree as well, with one caveat. That being that for something like this, where you know a single thread can consume 100% of a core's capacity, the "number of threads that is viable" is not going to be more than the number of cores, simply because otherwise the timeslicing is going to introduce more inefficiencies. That being said, Windows' timesharing has gotten *remarkably* better in Vista compared to XP, and XP was much better than previous Windows editions. That's why I would consider the "stutter thread" idea, but I also beleive that in Vista (and to a large extent, XP), letting the OS handle the re-allocation as needed is probably fine. J.Ja

Justin James
Justin James

Yeah, we've been over this ground before, and we are indeed in agreement. Indeed, .Net Framework is *so* Java-like, I found myself, more often than not, know what methods/properties to use when I first started, since I knew Java first. And now, I understand an awful lot of the J2EE Framework, simply because it is identical in many areas to .Net Framework. Above and beyond that though, the #1 offender is that languages like F# (hopefully not IronRuby or IronPython) simply have an object model *so* different from Java, VB.Net, or C#, that the *only* way to use the .Net Framework in those languages is to suddenly "thunk" to a C#-y syntax for accessing the Framework, unless the class you are working with can cleanly be translated to a literal in that language. I am betting big bucks that .Net Smalltalk actually works very well. J.Ja

apotheon
apotheon

I was mostly kidding. Of course, I tend to think of languages like Java, VB, and C# as un-fun, and I tend to think of the very Java-like library calls of .NET as very un-fun as well, but I also realize that there are other languages that can make working with .NET considerably less onerous. After all, they've got F# and IronRuby (or even IronPython, if you swing that way) in the .NET family now. edit: . . . but even with those languages, the .NET parts of working with them in a .NET environment just strike me as terribly un-fun. The F#-related discussion we had earlier with regard to a previous weblog article of yours made that point pretty well, with the very Java-like library calls in the middle of otherwise OCaml-like code.

Justin James
Justin James

See my sister post to this one... "intellectually challenging" is dealing with the MT logic. "Cut my heart out with a spoon" is working through VB.Net's MT system. I know the root problem, of why MT is such a hassle in most languages. I just figured it out. It will be Monday's post. Let's just say I hold OO accountable for it. J.Ja

Justin James
Justin James

Not "unfun" because it is hard, "unfun" because working with MT in most languages is like trying to rap with Jello in your mouth. It is a challenge I would enjoy on it's own, but the conditions I work under to do it (how many languages handle MT) takes the joy out of it. OI love thinking of new MT schemes for efficiency where possible (like the "stutter thread" idea), but implementing it in VB.Net will be boring, unfun, and a ton of code that will end up looking as close to spaghetti code as possible in an OO language. :( J.Ja

Justin James
Justin James

The .Net Framework itself is the least miserable of the kind, from my experience... and in a sad way, I prefer dealing with the consistent nature of it as opposed to cobbling together various libraries from all over the place. And it covers enough of the blindingly boring/frustrating ground to make it useful. I like the concept of the CLR. Visual Studio is a top notch IDE, IMHO. The .Net universe is very comfortable. "Fun" rarely applies to it, I will agree. VB.Net and C# as languages offer little satisfaction to me, and using them always makes me think of what, say, Russians must feel like trying to talk in English in the first week in the US... definitely not my native language! Java, I wholeheartedly agree with your version of it. Java was the least enjoyable programming experience I have ever had in my life. :) J.Ja

apotheon
apotheon

"Intellectually challenging" would be writing multithreaded applications in Ruby or Erlang. Doing so in a language like VB is more like "incredibly obtuse", "highly frustrating", or even "eye-stabbingly painful". I like intellectual challenges. I don't like intellectual oppression. edit: I changed the second use of the word "Ruby" to "VB" so that it said what I meant to say in the first place, rather than being self-contradictory. Sorry if anyone found it confusing.

SnoopDougEDoug
SnoopDougEDoug

Bah, this is one of the reasons I got into the field--the intellectual challenge. It is also a case where my computer science education has helped tremendously. Deadlocks, race conditions, critical sections. All damned interesting CS topics and challenges. Yes, it is HARD. That's why some folks are called software engineers and others are called programmers. (Ducking for cover). doug

apotheon
apotheon

[b][i]Programming[/i] in .Net and Java is definitely un-fun.[/b] There. Fixed it for you. /me ducks.

Editor's Picks