Distributed memory systems using the example of MPI

For distributed memory systems, MPI — the Message Passing Interface — is widely used.

 Others are used: PMV (Parallel Virtual Machine) or SHMEM.

MPI is based on the message passing mechanism.

MPI is a common medium for creating and executing parallel programs.

Message passing

 Method of transferring data from the memory of one processor to the memory of another

Data sent as packets

The message may consist of one or several packets.

 Packages usually contain routing and management information

Process :

 Set of executable commands (program)

The processor can run one or more processes.

Processes exchange information only via messages.

To improve performance, it is desirable to run each process on a separate processor.

 On a multi-core PC, processes can be performed on separate cores.

Message PassingLibrary

 Collection of functions used by the program

 Designed to send, receive and process messages

 Environment of parallel program execution

Send / receive

In the transmission of data, the interaction of two processors, the transmitter and receiver, is required.

Transmitter determines data position, size, type, receiver

Receiver must match the transmitted data.

Synchronous / Asynchronous

The synchronous transfer is completed only after the receiver confirms receipt of the message.

 Asynchronous transfer is performed without quitt (less reliably)

Applicationbuffer

Address space containing data to be transmitted or received (for example, a data memory area storing some variable)

Systembuffer

System memory for storing messages

Depending on the type of message, the data from the ApplicationBuffer can be copied to the SystemBuffer and vice versa.

BlockingCommunication

The function is completed only after some event

Non-blocking Communication

The function is completed without waiting for any communication events.

Using the system buffer after performing a non-blocking transfer operation DANGER!

CommunicatorsandGroups

Special objects that determine which processes can exchange data.

Process may be part of a group.

The group communicates using the communicator

 Within the group, the process gets a unique number (identifiers)

Process can belong to several groups (different identifiers!)

Messages sent in different communicators do not interfere with each other

Supports 2 programming styles:

MIMD -MultipleInstructionsMultipleData

• Processes execute different program code

• It is very difficult to write and debug.

• Difficult to synchronize and manage the interaction of processes

SPMD -SingleProgramMultipleData

• The most common option

• All processes run the same software code.

MPI is both a collection of libraries and a runtime environment.

 Compiling and linking with mpicc script

mpiccMPI_Hello.co MPI_Hello

 Run the program for execution using 5 processes

mpiexec-n 5 MPI_Helloс

Adding host addresses is done with the parameters “-hosts”, “-nodes”, “-hostfile”:

mpiexec – hosts 10.2.12.5 10.2.12.6

MPI feature set for data exchange between process groups

Features:

 in the reception-transfer mode all processes work

The collective function works simultaneously on reception and transmission.

 values ​​of all parameters in all processes (except the buffer address) must match

Include:

MPI_Bcast () - message broadcast

MPI_SCATTER () - distribution of data to different processes

MPI_Gather () - collecting data from all processes into one process

MPI_Allgather () - collecting data from all processes in all processes

 intMPI_Barrier () - point sync

MPI_Bcast (* buffer, count, datatype, root, comm)

buffer data buffer

count - data transfer counter

datatype data type

root - data source process

comm-communicator

MPI_Barrier ( comm )

 Suspends the process until the moment when all group processes do not reach the barrier (synchronization point)

Processes are waiting for each other.

MPI_Scatter (* sendbuf, sendcnt, sendtype, * recvbuf, recvcnt, recvtype, root, comm)

Transfers data from the source process array to all process drives

sendbuf-buffer source with an array of broadcast data

sendcnt-data transfer counter

sendtype data type

recvbuf-receive buffer

recvcnt-received data counter

recvtype type of received data

root-number of the process sending the data

comm-communicator

MPI_Gather (* sendbuf, sendcnt, sendtype, * recvbuf, recvcnt, recvtype, root, comm)

Collects data from the buffers of all processes in the accumulator of the collector process.

sendbuf-buffer source of broadcast data

sendcnt-data transfer counter

sendtype data type

recvbuf-receive buffer for data collection

recvcnt-received data counter

recvtype type of received data

root-collecting process number

comm-communicator

MPI function set for exchanging data between separate processes

One branch calls the transfer function and the other the receive function.

The bottom line:

Task 1 transmits:

intbuf [10];

MPI_Send (buf, 5, MPI_INT, 1, 0, MPI_COMM_WORLD);

Task 2 accepts:

intbuf [10];

MPI_Statusstatus;

MPI_Recv (buf, 10, MPI_INT, 0, 0,

MPI_COMM_WORLD, & status);

Create an application from N processes

Create and fill with a large array of data in the process 0

 Transfer the part of the array to each process

 In each process, we will sort the local array in ascending order.

In process 0, copy the local array to the output array

In process 0, in turn, we take local arrays from other processes and merge with the output array

Secondary functions:

• Buffer creation and filling with numbers

• Sort ascending specified buffer

• Print buffer items in the terminal window

• Merge two ordered arrays in ascending order

Функций Function Prototypes

double * generate1 (int);

double * processIt1 (double *, int);

voidshowIt1 (double *, int);

double * merge (double * arr1, double * arr2, intl1, intl2);