Qt同步框架

http://labs.trolltech.com/blogs/2007/03/08/making-asynchronous-function-calls-with-qfuture/

QtConcurrent::run() runs a function in a worker thread. It returns a QFuture, which is then used to synchronize with the result:

QString foo();QFuture<QString> f = QtConcurrent::run(foo);...QString string = f.result()

Calling f.result() will block the current thread until foo() has returned. The QFuture template argument must match the return type of foo().

If the function you want to run takes arguments, use QtConcurrent::bind to supply values for them:

QString foo(const QString &string);QFuture<QString> f = QtConcurrent::run(QtConcurrent::bind(foo, QLatin1String("Hello World")));qDebug() < < f.result();

(bind is based on the excellent boost::bind package.)

It’s usually not a good idea to block the GUI thread to wait for results, so when writing GUI applications it’s possible to use signals and slots instead. The QFutureWatcher class is used to make the connections:

QFutureWatcher *watcher  = new QFutureWatcher();QObject::connect(watcher, SIGNAL(resultReady(const QVariant&)), myObject, SLOT(handleResult(const QVariant &)));QString foo();QFuture<QString> f = QtConcurrent::run(foo);watcher->setFuture(f);

When foo() returns, QFutureWatcher calls the handleResult slot using a queued signal-slot connection.

 

MapReduce was originally developed by Google to simplify writing parallel algorithms for computer clusters. The basic idea is that you divide your algorithm into two parts: one part that can be run in parallel on individual pieces of the input data (’map’), and one sequential part that collects the map results and produces the final result (’reduce’). Your program then sends the map and reduce functions along with your input data to the MapReduce framework which automatically parcels out the data to each cluster node and collects the results afterwards.

MapReduce in Qt Concurrent is implemented to work on shared-memory systems, so instead of managing cluster nodes it manages threads on a single computer. This also means we can drop some of the features that the Google version has, such as fault tolerance. (we assume that processors don’t fail)

The API looks like this:

QFuture<T> mappedReduced(list, mapFunction, reducefunction);

As an example, let’s say we want to do a word frequency count on the contents of several documents. Here the map function will count the word occurrences in each document in parallel, and the reduce function will combine the them into a final frequency count.

The input for the function is a list of text strings that contains the documents:

QList<QString> list;

The map function takes one document and produces a hash that stores the frequency count for each word in the document. This function will be called in parallel by several threads, so it can’t have any side-effects such as modifying global data (or more accurately: any side-effcts must be thread-safe). The number of threads used will be scaled according to the number of CPU cores on the system.

QHash<QString, int> mapFunction(const QString &document);

The reduce function takes one intermediate result hash and aggregates it into the final result. Qt Concurrent will make sure only one thread calls this function at a time. This has two implications: there is no need to use a mutex lock when updating the result variable, and the system can be smarter about how it manages threads. If a thread tries to become the reducer thread while another thread is reducing, the first thread doesn’t have to block but can put the result on the to-be-reduced queue and then call the map function on a new piece of data instead.

void reduceFunction(QHasht<QString, int> &finalResut, const QHash<QString, int> &intermediateResult);

Finally we put it all together like this:

QFuture<QHash<QString, int> >counting =  mappedReduced(list, mapFunction, reduceFunction);

Since mappedReduced returns a QFuture we have several options on how to synchronize with the result. The simplest thing is to just call QFuture::result() which will block until the result is ready. If blocking is inappropriate (say we are in the gui thread) we can use signal-slot connections to get progress and result notifications instead. It’s also possible to cancel mappedReduced by calling QFuture::cancel().