FAQ: Derived Datatypes

Cornell Theory Center

FAQ
Derived Datatypes

6/96

Goals and Prerequisites
Why Would I Use Derived Datatypes?

2.1 Basic MPI Datatypes
2.2 Motivation

What Are Derived Datatypes?

3.1 General Datatypes and Typemaps
3.2 Extent of a Datatype

How and When Do I Use Derived Datatypes?

4.1 When to Use
4.2 How to Use
4.3 Matching Rules

Key Points
Exercises
References

1. Goals and Prerequisites

There are no questions on this section at this time.

2. Why Would I Use Derived Datatypes?

There are no questions on this section at this time.

2.1 Basic MPI Datatypes

There are no questions on this section at this time.

2.1 Motivation

There are no questions on this section at this time.

3. What Are Derived Datatypes?

There are no questions on this section at this time.

3.1 General Datatypes and Typempas

There are no questions on this section at this time.

3.2 Extent of a Datatype

There are no questions on this section at this time.

4. How and When Do I Use Dervied Datatypes?

There are no questions on this section at this time.

4.1 When to Use

There are no questions on this section at this time.

4.2 How to Use

There are no questions on this section at this time.

4.3 Matching Rules

There are no questions on this section at this time.

5. Key Points

There are no questions on this section at this time.

6. Exercises

Questions:

Can I use the MPI_Type_struct derived datatype if my underlying C struct contains a pointer to dynamically allocated memory? For example, one of the solutions to the exercises, program completed.c, has the following C struct, which is passed between two processes.
```
struct {
  double mass ;
  int force[FMAX] ;
} packet ;
```
But when I substitute:
```
struct {
  double mass ;
  int *force ;
} packet ;

packet.force = (int *) malloc (FMAX * sizeof(int)) ;
```
I find the program doesn't work. Specifically, the mass component of the struct passes successfully from process to process, but not the force. Why not?
The last argument of the MPI_Recv function should be &status instead of status, since status is of type MPI_Status, so it is not a pointer.
The verification at the receiving side doesn't verify what it is supposed to, since the special values are set before the two processes' job become different. The correct way would be to set the special values only at the sender.
I'm not sure whether it is necessary, but the derived datatypes are not freed at the end.

Answers:

Good question! The answer is in the form of a program, which we'll refer to as malloc.c. You may want to review the original program-- in particular the values for the variable array_of_displacement--before reading the explanation below. The answer revolves around one line:

 array_of_displacements[1] = (char *) packet.force - (char *) &packet.mass ;

which replaces these lines in the original program:

 MPI_Type_extent (MPI_DOUBLE, &extent) ;
 array_of_displacements[1] = extent ;

When the force array is statically allocated by the compiler, the displacement of the second element in the struct is "extent"; however, when the force array is dynamically allocated with malloc, it is necessary to compute the force array's displacement. Afterall, who knows where in memory malloc will allocate the space for the force array? Its relationship to the beginning of the struct is not known as it is for the case the compiler statically allocates the space. So, we compute the displacement by subtracting the beginning of the struct (&packet.mass) from the dynamically allocated address for the force array (packet.force).

Casting the addresses as (char *) is important for two reasons. First, packet.force is a (int *) and &packet.mass is a (double *). C uses the underlying type (int and double in this case) to do its pointer arithmetic, so you want to have the same type when subtracting. The question then remains what underlying type to use. This brings us to the second important point. There is no requirement in C that any of the integral types be large enough to represent a pointer, although people often assume that type long or even int is large enough. It is dangerous to assume (int *) will work. One the other hand, any data pointer can be converted to type (char *) and back safely.

You may note that the array of blocklengths and the array of types remains the same.

Thanks to one of CTC's excellent staff members, John Zollweg, for his impressive insights that lead to the solution of your problem. He suggested running the program through the xldb debugger and see what was really happening. You might find running xldb a good further exercise. If so, check out the VW module titled Parallel Processing Performance Tools.

If you have not already looked at the solution program, malloc.c, here it is:


/* -----------------------------------------------------------------------
 * Code:   malloc.c
 * For:    MPI Derived Datatypes VW module
 *         This program sends data of mixed type (a C struct)
 *         from one process to another.  It shows how to use
 *         the struct derived datatype with a component of the struct
 *         being dynamically allocated.  malloc.c is a modified version
 *         of completed.c, a lab exercise code for the MPI Derived 
 *         Datatype module.
 * Usage:  malloc
 *         Runs on two nodes.
 * Author: Mike Hammill with many thanks to John Zollweg for his insights
 * Last revised: 6/28/96 MWH
 * ------------------------------------------------------------------------ */

#include "mpi.h"

#define PACKET_BLOCKS 2    /* number of elements in struct */
#define FMAX 14            /* length of array of forces */
#define PACKETTAG 99       /* MPI tag for packet to be sent and received */
#define DESTRANK 1         /* rank of destination process */
#define SOURCERANK 0       /* rank of source process */
#define MYMASS 150.0       /* arbitrary constant */

main( argc, argv )
     int argc ;
     char **argv ;
{
  int array_of_blocklengths[PACKET_BLOCKS] 
      = {1, FMAX},                          /* for MPI struct */
    i,                                      /* loop index */
    myrank ;                                /* rank in communicator of process */

  MPI_Aint 
    array_of_displacements[PACKET_BLOCKS] ; /* for MPI struct */

  MPI_Datatype 
    array_of_types[PACKET_BLOCKS]          
      = {MPI_DOUBLE, MPI_INT},              /* for MPI struct */
    packettype ;

  MPI_Status status ;                       /* status of received message */

  /* We want to send packet, a contiguous C struct with mixed types. */
  /* Declare packet prior to constructing a new derived datatype. */

  struct{
    double mass ;                           /* an object's mass */
    int *force;                             /* ptr to array of forces */
  } packet ;

  /* ------------------------------------------------------------------------ */

  /* Initialize packet to zero. */

  packet.mass = 0.0 ;

  packet.force = (int *) malloc ( FMAX * sizeof(int)) ;

  for (i=0; i < FMAX; i++)
         packet.force[i] = 0.0 ;

  /* Setup the MPI environment. */

  MPI_Init( &argc, &argv ) ;
  MPI_Comm_rank( MPI_COMM_WORLD, &myrank ) ;
  printf ( "Task %d initialized\n", myrank ) ;

  /* Note that all the following variables are constants */
  /* and depend only on the format of the C struct. */

  array_of_displacements[0] = 0 ; 

  /* This is the new code needed to determine the displacement of your
     dynamically allocated force array.  Care must be taken not to cast
     addresses as (int *).  There is no guarentee in C that an address will
     fit into an int */

  array_of_displacements[1] = (char *) packet.force - (char *)  &packet.mass ;

  /* Use these variables to create a new derived datatype and commit. */

  MPI_Type_struct (PACKET_BLOCKS, 
		   array_of_blocklengths,
		   array_of_displacements,
		   array_of_types,
		   &packettype) ;

  MPI_Type_commit (&packettype) ;

  /* ------------------------------------------------------------------------ */

  if ( myrank == SOURCERANK ) {

    /* Give packet values so we can verify they get sent. */

    packet.mass = MYMASS ;

    for (i=0; i < FMAX; i++)
      packet.force[i] = (i+1) * 100 ;

    /* Send packettype with count=1. */

    MPI_Send( &packet, 1, packettype, DESTRANK, 
	     PACKETTAG, MPI_COMM_WORLD ) ;
  }

  /* ------------------------------------------------------------------------ */

  else if ( myrank == DESTRANK ) {

    /* Task with DESTRANK will receive packettype with count=1 */

    /* Print packet before the receive. */

    printf ("Value of packet before receive:\n") ;
    printf ("Mass =  %.3f \n", packet.mass) ;
    printf ("Array of forces = " ) ; 
    for ( i=0; i < FMAX; i++ ) 
      printf ("%d  ", packet.force[i] ) ; 
    printf ( "\n" ) ; 

    MPI_Recv( &packet, 1, packettype, SOURCERANK, 
	     PACKETTAG, MPI_COMM_WORLD, &status ) ;

    /* Verify that structure got passed. */

    /* Print packet before the receive. */

    printf ("\nValue of packet after receive:\n") ;
    printf ("Mass =  %.3f \n", packet.mass) ;
    printf ("Array of forces = " ) ; 
    for ( i=0; i < FMAX; i++ ) 
      printf ("%d  ", packet.force[i] ) ; 
    printf ( "\n" ) ; 

  }

  /* ------------------------------------------------------------------------ */

    MPI_Finalize() ;
}

Thank you for pointing this out. You're right: status should have been &status. This has been fixed in the current lab exercises.
A better means of verifying that data has been passed from one task to another via derived datatypes has been implemented in the current lab exercises. In all exercises, the datatype is now printed out before and after the MPI_Recv.
Freeing the derived datatype at the end of the program would be a good programming habit, since in larger programs, you save system resources by freeing. In the short sample programs given here, it is not necessary, but perhaps we should add it at a later point.

7. References

There are no questions on this section at this time.

Return to VW home page

Cornell Theory Center