There are two ways to go about debugging MPI applications using Microsoft Visual Studio. There’s the easy way, and the hard way. Unfortunately for many, the easy way is only available for the professional versions of Microsoft Visual Studio. If you right click on your project, select properties, select the debugging page, you can simply choose to use the MPI Cluster Debugger. If you don’t have a professional version of visual studio, don’t panic, you can still debug your applications.

This article is a continuation of my introduction to MPI with visual studio.

If you have a hello world application, chances are that when you run it, it takes only a split second to execute. If only there was a good way to slow down all of the threads and give you time to attach to your application processes for debugging! The good news is that there is a great way to do this, and learn a new MPI concept called a barrier.

The barrier

A barrier is a way to synchronize all threads in a program. This is extremely useful in multi-threaded programs, especially when different threads need to communicate with each other. Suppose you place a barrier in your code. When a thread reaches the barrier, it checks if all the other threads have reached the barrier. If every other thread has reached the barrier, then they will all start executing code after the barrier. However, if not all of the threads have gotten to the barrier, then the current thread will simply wait. As soon as all threads reach the barrier, the thread will execute code once again. Using barriers, you can make sure that each thread has processed a certain amount of data before communicated, which is invaluable. Placing a barrier in your source code is easy, simply call the function MPI_Barrier.

MPI_Barrier(MPI_COMM_WORLD);

Debugging

So how does the barrier help us debug our MPI application? The idea is simple. When the program first starts to execute, at the very beginning of our program, we can as the user to type some sort of input. This can be a number, a character, a string, anything. The idea is to simply stall the application until you give it some sort of input manually. This way, you can start your program with Visual Studio, then go the the Debug menu, select “Attach to Process…” and you can attach to all the different threads in your program. There will be one thread process for each MPI thread, so unless you’re only interested in one particular thread, it’s best to select all your processes and attach for debug. The first time you do this, Visual Studio will warn you that it doesn’t have a symbols file for mpiexe.exe. You can ignore this warning. After you successfully attach to all your processes, you can go to your application, type anything, press enter, and the program will continue like normal. You can use breakpoints, and debug it just like a normal application. This is a slightly annoying way to debug MPI, but again, this is just for the express and standard versions of Visual Studio.

At the very top of your main() function, you should have some code similar to this:

int rank, size, temp;
MPI_Init( &argc, &argv );
MPI_Comm_size( MPI_COMM_WORLD, &size );
MPI_Comm_rank( MPI_COMM_WORLD, &rank );

if (rank == 0)
{
  cout << "Please input number: ";
  cin >> temp;
}

MPI_Barrier(MPI_COMM_WORLD); // All threads will wait here until you give thread 0 an input

Thereare a couple things I would like to point out about the above code. Since I use cout and cin, it’s necessary to have an “#include <iostream>” and “using namespace std;” somwhere at the top of the source file, otherwise it won’t compile. The cin >> temp, coupled with the MPI_Barrier is what allows me to have enough time to attach to the processes in order to debug, then move on when I’m ready. Notice how the input will only go to thread 0. This is because you will encounter errors if you have multiple threads using cin at the same time.

That’s all there is to it! There’s nothing else that’s special about debugging MPI applications. Sometimes, you’ll find that only some of the threads in your program aren’t acting right because of a bug somewhere in your code. When this is the case, using conditional breakpoints that only trigger if the rank equals a certain number is a great way to debug specific threads easily.

Next Tutorial: Thread Communication with MPI

Back to the MPI tutorial landing page