Lambda closures and parallelism

So now and then I spend time at work explaining the somewhat trickier parts of programming to the junior and medior developers. I’ve decided to post these on my web site, such that everyone can read them.

I assume that it is known what a lambda expression is. Otherwise the Wikipedia site about anonymous functions is a good place to start. Lambda expressions in combination with parallelism can be tricky things depending on the way closures are implemented.

Take for example the following piece of C# code:

static void Main(string[] args)
{
  for (int i = 0; i < 10; ++i)
    Task.Run(()
      => Console.Out.WriteLine("Task {0}", i));

  Console.Out.WriteLine(
    "Press any key to quit");
  Console.ReadKey();
}

The result of this piece of code on my Windows 8 machine results in:

Press any key to quit
Task 10
Task 10
Task 10
Task 10
Task 10
Task 10
Task 10
Task 10
Task 10
Task 10


On Windows 7 the first line stays “Task 2”. Both are perhaps not what you would expect. The way C# handles closures is the cause of the above result.

All the free variables in the lambda expression (like the variable i in the example) are copied to an object on the heap representing the closure. The free variables in the lambda expression then become references. This way, all the tasks use exactly the same variable.

In the example, the variable i lives on the heap and has been incremented to 10 when the loop finishes. The loop executes faster than the tasks can start. By the time the tasks are executed i is set to 10.

If the intended behavior was printing the numbers 0 to 9, then the code needs to be changed into the following:

static void Main(string[] args)
{
  for (int i = 0; i < 10; ++i)
    f(i);

  Console.Out.WriteLine(
    "Press any key to quit");
  Console.ReadKey();
}

static void f(int i)
{
  Task.Run(()
    => Console.Out.WriteLine("Task {0}", i));
}

The output is now:

Press any key to quit
Task 0
Task 3
Task 4
Task 7
Task 8
Task 1
Task 2
Task 6
Task 5
Task 9


C++11 does a much nicer job. It allows to explicitly specify if variables need to be referenced or copied. Take the following example of C++ code, with a little help of TBB.

#include <iostream>
#include "tbb/task_group.h"
using namespace std;
using namespace tbb;

int main(int argc, char** argv)
{
  cout << "Starting tasks" << endl;
  task_group g;
  for (int i=0; i < 10; ++i)
    g.run([&] { cout << i << endl; });

  g.wait();
  return 0;
}

Running this application gives by now an expected mess

Starting tasks
10
10
10
10
10
10
110101
0
0

This is because the outer variables are passed by reference. The ‘&’ sign in the lambda brackets ([&]), called the capture list clearly specify this. Replacing the capture list by [=] (copy all values) or [i] (copy i) produces:

Starting tasks
9
8
7
6
5
4
3
2
1
0