There is a simple solution to most major threading problems: have no shared state. By shared state, I mean any data or resource (such as file handle, socket, graphics context, queue, buffer, etc) that is used by more than one thread at the same time. If two threads are truly independent, they can safely run concurrently without care or consideration. We wouldn't need sophisticated mechanisms for synchronisation if there's nothing to sync over. Unfortunately, this restriction is not at all practical or realistic. And as soon as you introduce shared state, you have to worry about atomicity, consistency, race conditions, and all sorts of issues.

So one of the first design considerations for concurrent systems is to try to minimise the amount of shared state between threads. The less shared state, the less complexity there is to manage, and the less overhead imposed. Locks that protect shared resources can harm performance: each lock is a region that enforces sequential access, and thus reduces the opportunities for concurrent scheduling.

Simply put, any resource shared between more than one thread needs to be protected by a mutex, such that access is serialised.

When considering scalability, the theoretical maximum speedup of N cores is always less than N times, due to scheduling and synchronisation overheads. Thus the most efficient algorithm designs consider these issues up front.

Problem: Atomicity

An operation is atomic if the operation completes without interruption. It is never partially complete, which may leave the system or data in an inconsistent or invalid state. Atomic operations come up all over the place, and even things that we might think are atomic (ie. a single line in your source) probably isn't. In databases, we use SQL transactions to ensure operations are atomic, such that a related set of changes are never partially applied.

The classic example of an atomic operation in a database is a bank transfer. You decide to pay your landlord by direct deposit, and transfer $1200 to their account. The normal sequence of operations is:

1. Ensure you have sufficient funds
2. Ensure the receiving account number is valid
3. Withdraw the $1200 in funds from your account
4. Deposit the $1200 in the landlord's account

Now the first two steps are really preconditions, and while they are necessary for the entire operation's success, if the operation was cancelled after step 1 or step 2, there would be no harm done, and nothing changed. But once the money comes out of your account after step 3, you absolutely want step 4 to complete, otherwise you are $1200 out of pocket and you still owe the rent! So steps 3 and 4 must either both happen successfully, or not at all. If the network goes down between steps 3 and 4 such that the deposit cannot complete, you want to undo the withdrawal and try again later. Steps 3 and 4 must be performed atomically.

While this example traditionally applies to database operations (and is thus resolved on-disk), the same concept applies to in-memory operations that must be atomic. A mutex (for "mutual exclusion") or critical section can be used to serialise access to resources within a region of code that must run as an atomic operation. We would lock the mutex before the transaction, and unlock it afterwards.

Problem: Race Conditions

A race condition is a general term for a class of problems, whereby the result of an operation is at the mercy of the timing of concurrent events (typically unknown or effectively random). This is bad because your program becomes non-deterministic - it may not always produce the correct result! They represent one of the most subtle and difficult to debug problems in software development, which is one of the main reasons why multi-threaded programming is considered difficult.

Example: A Counter

Even a simple increment operation, which is only a single operation in C++, may actually be the source of a race condition. Consider:

m_SequenceNumber++;

which actually translates (on the x86 architecture) to something like:

movl -12(%ebp), %eax incl %eax movl %eax, -12(%ebp)

which is a load, increment, then store. So there are actually two potential points that a race condition could occur - between the load and the increment, and between the increment and the save.

Now imagine two threads, which are both using a shared object. If both threads happen to call the method that performs the increment method at the same time, one thread could be preempted in the middle, such that the instructions are interleaved. So for threads A and B incrementing the sequence number, thus:

So, instead of the sequence number being incremented twice, and being 236 as we would expect, the sequence number is only 235. This is a subtle, nasty and rare bug and could be very difficult to track down for the unprepared.

Fortunately, there is a nice, simple solution to this particular problem. Operating systems provide atomic functions for safely performing an increment (with a minimum of overhead), as well as a few other common operations. Under Windows, you would use the InterlockedIncrement(), while under Mac OS X you would call OSAtomicIncrement32(). Under Linux, gcc provides __sync_fetch_and_add() and friends.

Example: Accessing a Queue

An illustration of a race condition is where multiple threads are removing work items from a queue. Imagine some code that looks something like this:

void unsafeWorkerThread() { while ( unsafeWorkerThreadRunning ) { // Warning: this is not thread-safe! if ( !queue.isEmpty() ) { work_t item = queue.pop(); process(item); } } }

This is not thread-safe! Even if we assume that the queue operations themselves are atomic (which is usually not the case for most container classes!), there is a race condition waiting to trigger a failure. Can you spot it?

Imagine there are two threads running, and there is one work item in the queue. When thread_A calls queue.isEmpty(), it goes into the if block, as there is more data to process. But then the thread is pre-empted, and thread_B gets a chance to run. This time, it removes the work item and processes it. But now the queue is empty, and thread_A thinks it isn't! When thread_A resumes, it will call queue.pop() and trigger a stack underflow exception. (And here, without a try-catch block, an exception would also cause the thread to exit without notice.) Now this code could run perfectly well for thousands of iterations, and never trigger these particular failure conditions. But every so often, the timing will be just right (or rather wrong!) and your application will mysteriously fail. You can see why many people say threading should be avoided at all costs! The code is valid, usually works but every so often it fails in an unusual way.

The simplest solution is to protect the queue with a mutex, and lock it around the inner block. (There's another improvement we can make after we discuss exceptions.)

void workerThread() { while ( workerThreadRunning ) { queueMutex.lock();   if ( !queue.isEmpty() ) { work_t item = queue.pop(); process(item); }   queueMutex.unlock(); } }

(There are more esoteric solutions to this problem, such as lock-free data structures, but those are beyond the scope of this series.)

Problem: Deadlocks

In a non-trivial application, there may be several threads and numerous mutexes. When more than one thread locks more than one mutex, there arises the potential for a condition known as a deadlock. This is where one thread is holding a lock while waiting for another to become available, while a second thread is holding the second lock and waiting for the first lock to become available. Since they are both waiting for each other to finish, neither can run. This deadlocked situation can be more involved, whereby several threads form a ring of holding and waiting for locks. This is illustrated as follows, first in code then as a diagram:

void threadA() { while (running) { mutexOne.lock(); mutexTwo.lock(); processStuff(); mutexOne.unlock(); mutexTwo.unlock(); } } void threadB() { while (running) { mutexTwo.lock(); mutexOne.lock(); processStuff(); mutexTwo.unlock(); mutexOne.unlock(); } }

Fortunately, the problem of deadlocks can largely be avoided by consistently locking mutexes in the same order, and unlocking them in reverse order.

Problem: Exceptions

Exceptions in C++ can be a very effective mechanism for error handling. However, like many C++ features, care must be taken, especially in threads. Because a thread is destroyed when the thread function returns, an uncaught exception can cause the entire thread to exit, without warning or notice. So if you have a thread that mysteriously disappears, an uncaught exception is a likely culprit.

As with non-threaded code, you should always catch the most specific exception type that you are able to handle. But if you wish to avoid your thread being killed, you should add a top-level try-catch block. At this point, depending on the application, you may simply log the uncaught exception then exit, or you may resume thread processing (but only if safe to do so!).

A skeleton thread function might look something like:

void processThread() { while(keepProcessing) { try { // Do actual work } // Catch more specific exceptions first if you can catch (std::exception& exc) { log("Uncaught exception: " + exc.what()); // Maybe return here? } } }

Just as an errant exception can cause havoc with threads running, they can also cause problems with mutexes. The next article shows how to use a lock guard to ensure mutexes are unlocked, even if an exception is thrown.

Other Problems

Just to reinforce the idea that multithreaded programming is not a trivial undertaking, there are even more traps for the unwary, for which there is not sufficient time to delve into here. Issues such as starvation, priority inversion, and high scheduling latency are all considerations for concurrent programming. It is definitely worth consulting a good book or three.

Closing thoughts

Fortunately, there are well-known solutions to multi-threaded design issues, so it is entirely possible to write robust threaded code, provided one is careful and thoughtful. Much of it starts with:

• Minimise shared state

In days gone by, a multi-threaded system would often still only be running on a single core. Certain classes of concurrency bugs were far less likely to happen. But now multicore systems are standard, and we have truly concurrent execution, which brings into the fray problems such as memory consistency, cache latency, write-throughs, and so on.

Especially now that multicore systems are shipping as standard on desktops and laptops, concurrent programming is required to take advantage of the available processing resources. So a thorough understanding of multithreading is increasingly important.

Some takeaway points to remember:

• Minimise shared state
• Program defensively
• Assume your threading code can be preempted at any time
• Identify shared resources and protect them with mutexes
• Identify algorithms that need to be atomic
• Identify code with dependent results that needs to be atomic
• Minimise the number of resources shared between threads
• Minimise the duration of locks
• Ensure exceptions cannot disrupt synchronisation flow
• Always acquire and release mutexes in the same order

The next article looks at the types of mutexes, and how to use them.

The Boost documentation is substantial, but can still be daunting to new users. So this article is the first of a series on using Boost, starting with basic threading. It aims to provide an accessible introduction, with complete working examples.

This article looks specifically at the different ways to create threads. There are many other techniques necessary for real multi-threaded systems, such as synchronisation and mutual exclusion, which will be covered in a future article.

Overview

A boost::thread object represents a single thread of execution, as you would normally create and manage using your operating system specific interfaces. For example: on POSIX systems, a Boost thread uses the Pthreads API, and on Win32 it uses the native CreateThread and related calls. Because Boost abstracts away all the platform-specific code, you can easily write sophisticated and portable code that runs across all major platforms. A thread object can be set to a special state of not-a-thread, in which case it is inactive (or hasn't been given a thread function to run yet).

A boost::thread object is normally constructed by passing the threading function or method it is to run. There are actually a number of different ways to do so. I cover the main thread creation approaches below.

Code Examples

All the code examples are provided in a single download below, which you can use for any purpose, no strings attached. (The usual disclaimer is that no warranties apply!) And just as Boost runs on many platforms (ie. Windows, Unix/Linux, Mac OS X and others) the example code should be similarly portable.

• Download boost_threads_eg1.zip (4kB)

There is a separate example program for each section below, and a common Bjam script to build them all (Jamroot). Bjam is the Boost build system, a very powerful (but notoriously difficult to learn, and worthy of a whole series of articles).

Having said that, you are certainly not obliged to use Bjam. It is still worth knowing how to build applications manually before relying on scripts, so here is an example command line for manual compilation on my system (Mac OS X with Boost installed from MacPorts):

g++ -I/opt/local/include -L/opt/local/lib -lboost_thread-mt -o t1 t1.cpp

All this does is add an include path (with the -I option) pointing to the root of the boost headers, add the library search path (with the -L option) and link in the threading library (boost_thread-mt). You can use the above as the basis for writing our own Makefile if you prefer, or creating build rules in your IDE of choice.

Doing 'Real' Work

And before we dive in, a quick note on doing "real work"... In the examples below, a simple sleep call is used to simulate performing actual work in the threads. This is simply to avoid cluttering up the examples with code that would take some finite time to execute but would otherwise be irrelevant.

The simplest way to sleep for a given duration using Boost is to first create a time duration object, and then pass this to the sleep method of the special boost::this_thread class. The this_thread class gives us a convenient way to refer to the currently running thread, when we otherwise may not be able to access it directly (ie. within an arbitrary function). Here's how to sleep for a few seconds:

// Three seconds of pure, hard work! boost::posix_time::seconds workTime(3); boost::this_thread::sleep(workTime);

In the main() function, we then wait for the worker thread to complete using the join() method. This will cause the main thread to sleep until the worker thread completes (successfully or otherwise).

The observant reader will wonder then what the advantage is in spawning a thread, only to wait for it to complete? Surely that serialises the execution path and places us firmly back in sequential world? What is the point of spawning a thread? Well, until we figure out how to use the synchronisation mechanisms, this is the most straightforward approach to illustrate thread creation. And knowing ow to 'join' threads is also very important. To synchronise the completion of a thread, we wait for it to finish by calling workerThread.join().

The general structure of the examples below is shown in the following sequence diagram:

The application starts, and the main thread runs at (a). Then at (b), the main thread spawns the worker thread by constructing a thread object with the worker function. Right after, at (c), the main thread calls join on the thread, which means it will go to sleep (and not consume any CPU time) until the worker thread has completed. As soon as the worker thread is created at (b), it will start execution. At some point later at (d), the worker completes. Since the main thread was joining on its completion, main wakes up and continues running. It finishes at (e) and the process terminates. Each of the examples below follow this general scheme - the difference lies in how the threads are created.

Type 1: A Thread Function

The simplest threading scenario is where you have a simple (C-style) function that you want to run as a separate thread. You just pass the function to the boost::thread constructor, and it will start running. You can then wait for the thread to complete by calling join(), as shown above.

The following example shows how. First, we include the correct Boost thread header, create the thread object and pass in our worker function. The main thread in the process will then wait for the thread to complete.

#include <iostream> #include <boost/thread.hpp> #include <boost/date_time.hpp>   void workerFunc() { boost::posix_time::seconds workTime(3);   std::cout << "Worker: running" << std::endl;   // Pretend to do something useful... boost::this_thread::sleep(workTime);   std::cout << "Worker: finished" << std::endl; }   int main(int argc, char* argv[]) { std::cout << "main: startup" << std::endl;   boost::thread workerThread(workerFunc);   std::cout << "main: waiting for thread" << std::endl;   workerThread.join();   std::cout << "main: done" << std::endl;   return 0; }

When you run the program, you should see output similar to the following:

% ./t1
main: startup
main: waiting for thread
Worker: running
Worker: finished
main: done

It's as simple as that! This created a thread and ran it, and you would have seen a pause while the worker thread was running (ok, busy sleeping!). Later on, we'll do something a bit more substantial. But this example shows the absolute minimum code required to start a simple thread. Simply pass your function to the boost::thread constructor.

Type 2: Function with Arguments

So the above function wasn't terribly useful by itself. We really want to be able to pass in arguments to the thread function. And fortunately, it's very easy - you simply add parameters to the thread object's constructor, and those arguments are automagically bound and passed in to the thread function.

Let's say your thread function had the following signature:

void workerFunc(const char* msg, unsigned delaySecs) //...

You simply pass the arguments to the thread constructor after the name of the thread function, thus:

boost::thread workerThread(workerFunc, "Hello, boost!", 3);

This example is called 't2' in the source examples.

Type 3: Functor

A functor is a fancy name for an object that can be called just like a function. The class defines a special method by overloading the operator() which will be invoked when the functor is called. In this way, the functor can encapsulate the thread's context and still behave like a thread function. (Functors are not specific to threads, they are simply very convenient.)

This is what our functor looks like:

class Worker { public:   Worker(unsigned N, float guess, unsigned iter) : m_Number(N), m_Guess(guess), m_Iterations(iter) { }   void operator()() { std::cout << "Worker: calculating sqrt(" << m_Number << "), itertations = " << m_Iterations << std::endl;   // Use Newton's Method float x; float x_last = m_Guess;   for (unsigned i=0; i < m_Iterations; i++) { x = x_last - (x_last*x_last-m_Number)/ (2*x_last); x_last = x;   std::cout << "Iter " << i << " = " << x << std::endl; }   std::cout << "Worker: Answer = " << x << std::endl; }   private:   unsigned m_Number; float m_Guess; unsigned m_Iterations; };

This worker functor calculates a square root of a number using Newton-Rhapson's method, just for fun (ok, I got bored putting sleep in all the threads). The number, a rough guess and the number of iterations is passed to the constructor. The calculation (which is in fact extremely fast) is performed in the thread itself, when the operator()() gets called.

So in the main code, first we create our callable object, constructing it with any necessary arguments as normal. Then you pass the instance to the boost::thread constructor, which will invoke the operator()() method on your functor. This becomes the new thread, and runs just like any other thread, with the added benefit that it has access to the object's context and other methods. This approach has the benefit of wrapping up a thread into a convenient bundle.

int main(int argc, char* argv[]) { std::cout << "main: startup" << std::endl;   Worker w(612, 10, 5); boost::thread workerThread(w);   std::cout << "main: waiting for thread" << std::endl;   workerThread.join();   std::cout << "main: done" << std::endl;   return 0; }

Note: A very important consideration when using functors with boost threads is that the thread constructor takes the functor parameter by value, and thus makes a copy of the functor object. Depending on the design of your functor object, this may have unintended side-effects. Take care when writing functor objects to ensure that they can be safely copied.

Type 4: Object method I

It is frequently convenient to define an object with an instance method that runs on its own thread. After all, we're coding C++ here, not C! With Boost's thread object, this only slightly more work than making a regular function into a thread. First, we have to specify the method using its class qualifier, as you would expect, and we use the & operator to pass the address of the method.

Because methods in C++ always have an implicit this pointer passed in as the first parameter, we need to make sure we call the object's method using the same convention. So we will pass the object pointer (or this depending on whether we are inside the object or not) as the first parameter, along with any other actual parameters we might have after that, thus:

Worker w(3); boost::thread workerThread(&Worker::processQueue, &w, 2);

As an aside, take care in your own code that you don't accidentally allocate an object on the stack in one place, spawn a thread, then have the object go out of scope and be destroyed before the thread has completed! This could be the source of many tricky bugs.

The full listing is essentially the same. To pass additional parameters to the method, simply add them in the constructor to the thread object after the object pointer.

Type 5: Object method II

You may want to create a bunch of objects that manage their own threads, and can be created and run in a more flexible manner than keeping around a bunch of objects along with their associated threads in the caller. (I think they call this encapsulation.)

So our final example places the thread instance within the object itself, and provides methods to manage them. Since the thread object exists as an instance member (as opposed to a pointer), what happens in the constructor? In particular, what if we don't want to run the thread at the same time as we create our object? Fortunately, the default constructor for the thread creates it in an "invalid" state, called not-a-thread, which will do nothing until you assign a real one to it (in our start method, for example).

So now our class declaration has the following data member added:

//... private: boost::thread m_Thread; //...

The Worker::start() method spawns the thread which will run the processQueue method. Notice how we pass in this as the first bound parameter? Because we are using an instance method (and not a class method or regular function), we must ensure the first parameter is the instance pointer. The N parameter is the first actual parameter for the thread function, as can be seen in its signature.

void start(int N) { m_Thread = boost::thread(&Worker::processQueue, this, N); }

The join method is very simply:

void join() { m_Thread.join(); }

which means our main function becomes no more than:

Worker worker;   worker.start(3);   worker.join();

This encapsulation of the threading can be very useful, especially when combined with patterns such as the Singleton (which we will look at in a future article).

Conclusion

We have seen a variety of techniques for creating threads using the Boost threading library. From simple C functions to instance methods with parameters, the thread class permits a great deal of flexibility in how you structure your application.

Future Articles

In later installments, we will look at synchronisation methods, mutexes, and all sorts of other interesting techniques.

Please leave a comment below, with any questions or feedback you may have. Too long? Too brief? More details required? Something confusing? Let me know.

Now most of the idiomatic C++ code I've read that uses STL iterators uses the prefix operator++ to move the iterator forward. And so I had long ago adopted this too, with a vague recollection of having read somewhere that it performed better. But why? Good question... (Updated: fixed formatting, got numbers the right way around.)

for ( unsigned i = 0; i < array_length; i++ ) { // do something with array[i] // ... }

When the indexing variable is an ordinal type (usually an integer), the incrementing order makes no difference. The expressions ++i and i++ are identical in this case. And so many people developing in C++ simply use the same style in C++ with their iterator code. Why would it matter?

Well, it turns out that it does. If the indexing variable is a class, such as an iterator, there are some subtle differences between the pre-increment and post-increment operators that may have a big impact on performance.

Referring back to Scott Meyer's wonderful More Effective C++ book, I tracked down the explanation (item 6, page 31), which I will attempt to reproduce here in simplified form. The signature of the pre-increment operator++ for a class T is:

T& operator++(); // prefix

while the post-increment operator had a dummy parameter added to make the signature unique:

const T operator++( int ); // postfix

But that isn't the only difference - notice that the prefix operator returns a reference to the object itself (permitting chaining of method calls), while the postfix operator returns a const object, semantically defined as the previous value. (This is for consistency with the behaviour of ordinal types.)

The result is that the pre-increment operator modifies the object in-place, whereas the post-increment operator will result in temporaries being created, invoking the constructor and destructor. So something as simple as ++it versus it++ turns out to have some significant side-effects when applied to an object with overloaded operators. Since this is the case with virtually all iterators in the STL, as well as many other similar objects, it is worth investigating.

So I decided to quantify the difference, to see how much of a difference it made. I wrote a test program that created a very large (5 million) array of integers, and iterated over the array. I used a high-resolution timer to time two versions of the loop, identical save for one being pre-increment and one being post-increment. After a warmup of 3 runs, I ran the test 10 times, collected the timings and compared them.

The results were very interesting: on an Intel workstation, the pre-increment code took an average of 14.1ns per call, while the post-increment code took 27.6ns per call. That's a significant difference of 49%, or nearly twice as fast! A plot of 10 iterations each is shown below:

So - it really does improve the performance of your code to write:

mylist::const_iterator it; for ( it = numbers.begin(); it != numbers.end(); ++it ) { // do something with *it }

For small arrays, it may not make much of a noticeable difference, but every cycle counts, and a little consistency goes a long way. Using the prefix increment will enable your application to scale better.

Using the prefix form won't make your actual loop run twice as fast, as the iterator increment is only one component contributing to the performance, and the looping overhead may indeed be swamped by the execution time of the body. So as always, YMMV!

I have provided the test source in C++, along with an R script to summarise and plot the output. You can download it via the attachment linked at the end of this article. The code should compile on any POSIX-conforming system with a decent C++ compiler. And if you do your own performance tests, please drop me a line and let me know what you found.

As one final note, the wonderful Boost++ library provides the foreach module, which provides a powerful wrapper to iterate over containers with a nice clean simple syntax:

int total = 0; foreach( int num, numbers ) { total += num; }

And yes, it uses the pre-increment operator for maximum performance.

Download the source used for this article:

iterperf-0.1.zip