Web Development

Ruby and Python threading: What you need to know

Ruby and Python's standard implementations make use of a Global Interpreter Lock. Justin James explains the major advantages and downsides of the GIL mechanism.

Multithreading and parallel processing projects are of special interest to me because they present a unique challenge -- that is, as a developer, I focus on the code and not peripheral issues like UI design. When I wrote the Mongoose engine that powers my Rat Catcher application, I used the .NET Parallel Extensions Library (PFx) in order to speed the execution of searches. While Mongoose is not computationally intensive, it does perform a large number of network requests that can have a long wait period, so running them in parallel can deliver substantial performance gains.

I recently looked into rewriting the Mongoose engine in a language other than C#. Even though I'm very happy with the existing C# implementation, I've been thinking about moving Mongoose onto a cloud provider, and I'm seeing enough of a price difference between Linux and Windows cloud servers to justify the rewrite. While using Mono is an option now that 2.8 has been released with support for PFx, I also thought it would be a nice opportunity to try Python or Ruby for this purpose. Along the way, I learned a bit about how threading works in these languages, and how it may affect a developer's decision to use them for a project.

In Ruby and Python's standard implementations (MRI and CPython, respectively), the languages make use of a Global Interpreter Lock (GIL). The GIL mechanism performs timeslicing and scheduling of threads. Here's how it works: Each thread has exclusive access to the GIL for a bit of time, does some work, and releases it. Meanwhile, every other thread is on hold, waiting to get a chance to access the GIL. When the GIL is released, a random thread will get to access it and start running.

There are two major advantages to using this system. The first is that you can write code in these languages that use threading, and it will run on an operating system that does not natively support threading with no modifications needed. The second is that, because only one thread is running at a time, there are no thread safety issues, even when working with a non-thread safe library.

There are some major downsides, though. The biggest one is that multiple threads will never run at the same time. While your application may look like it is running in parallel, in reality, the process never has more than one thread, and it is just wildly bouncing around within one thread doing different things. This brings us to our second issue, which is speed. You will not see any speed advantage on multicore or multiprocessor machines because only one thread is running at a time; you will see a slowdown due to the context switching costs.

The use of the GIL makes a threaded application a bad idea in many (if not most) cases. Fortunately, there are options. For one thing, the GIL is not mandated by the language specifications. There are some implementations that do not use the GIL (JRuby and IronRuby, for example). Also, you can easily fall back on the process model that Ruby and Python both support, using the traditional fork/join mechanisms. While it may not be ideal (or possible) to use a different implementation or write your application to rely upon forking, it is good that there are alternatives to make truly parallel programs possible in Ruby and Python.

J.Ja

Disclosure of Justin's industry affiliations: Justin James has a contract with Spiceworks to write product buying guides; he has a contract with OpenAmplify, which is owned by Hapax, to write a series of blogs, tutorials, and articles; and he has a contract with OutSystems to write articles, sample code, etc.

About

Justin James is the Lead Architect for Conigent.

15 comments
chaimn
chaimn

Can you explain the following (what IS the price difference? How did you calculate it?): " Even though I?m very happy with the existing C# implementation, I?ve been thinking about moving Mongoose onto a cloud provider, and I?m seeing enough of a price difference between Linux and Windows cloud servers to justify the rewrite. "

jnativ47
jnativ47

A question: how does it relate to using Python with Google App Engine?

Mark Miller
Mark Miller

A few years ago Guido van Rossum wrote an article basically saying that running multiple processes (for Python) was the only viable option if you wanted to run things in parallel, which meant that you had to do some sort of message passing between processes (like, say, using XML and a web service), or set up a pipe to a daemon, or fork, in order to truly run anything in parallel on multiple cores. It's the same story with Ruby. Using web services would seem to be the most flexible solution. You could thereby write distributed modules that could be called upon as needed. The web service enables flexibility in terms of the number of copies of processes generated, and it lets the underlying OS handle load balancing between the cores. The thing is native threads are lighter on the system than spawning processes. So the above scheme is not that efficient. I don't know. Has anyone come up with an alternative scheme to web services for distributed processes (I mean, besides Microsoft, which has come up with multiple process/object invocation schemes)?

Justin James
Justin James

If so, what kind of work were you doing, and how did it come out? Did the GIL affect it at all? J.Ja

Justin James
Justin James

I looked at AWS, and what they charged for instances. Their Linux instances are significantly cheaper than their Windows instances for the same size box. On the smaller instances it works out to not that much money, but if you need to scale across many instances or need the bigger ones (or are paying for "on demand" instead of "pre-reserved"), the price gap gets bigger. Also of note, Microsoft Azure costs the same as AWS Windows instances, which bothers me. It should be the same as AWS Linux instances, since Microsoft doesn't need to pay for licenses, support, etc. But that's another topic... J.Ja

Justin James
Justin James

I really don't know... it depends on what Python implementation Google is using. I'd contact their support and find out. J.Ja

mhsemcheski
mhsemcheski

While it may be true that threads are lighter-weight than processes on Windows, there's no significant difference on Unix. See "The Design and Implementation of the FreeBSD Operating System." (But this is applicable to Linux and other Unixes.) Unix was designed with lots of processes talking together through pipes in mind. So, they made processes lightweight, and threads are just a special kind of process.

Sterling chip Camden
Sterling chip Camden

... is not that much less efficient on Unix systems than threads. I'd say a web service interface between them would be the main inefficiency. As long as you know you're running on the same system, why not just open a pipe to communicate? Or if you prefer sockets, at least skip the XML parsing and use something lighter.

mattohare
mattohare

I do wonder what mongrel and other web server applications are using.

Sterling chip Camden
Sterling chip Camden

Has been to implement logical concurrency where true concurrency isn't that important. For instance, an input loop that has a worker thread doing something else, neither of which are performance-critical. If I had to implement performance-critical concurrency in Ruby, I'd fork. Or not use Ruby -- Haskell has great concurrency support and will even utilize multiple cores without any special code.

jpellant
jpellant

It is still just processes leveraging IPC. Why go off the reservation? If you want SMP, why not use a language that natively supports it (e.g., java)? Use the right tool for the right job! $0.02 Jon

rogerdpack
rogerdpack

AFAIK mongrel uses threads, so that's GIL-bound in MRI and "real" threads in jruby.

Justin James
Justin James

In theory, they could use whatever interpreter they want. In reality, there are only a few Ruby implementations that seem to have any real adoption: MRI, JRuby, and Rubinius. You'd have to check with your specific server vendor, or see if the Ruby interpreter will cough up the data itself. J.Ja

Editor's Picks