Basics Concepts of Parallel Programming

What is parallel computing It’s a form of computation where many calculations can be carried out simultaneously. In parallel computing we can divide large programs into smaller one and perform computation simultaneously.

3/24/20193

4

Save time and/or money Solve larger problems Provide concurrency 3/24/20195

Physics - applied, nuclear, particle, condensed matter, high pressure, fusion, photonics Bioscience, Biotechnology, Genetics Chemistry, Molecular Sciences Geology, Seismology Mechanical Engineering - from prosthetics to spacecraft Electrical Engineering, Circuit Design, Microelectronics Computer Science, Mathematics 3/24/20196

Threads “ A Thread is single stream of control in the flow of a program” Example 1: Consider the following code which computes product of 2 dense matrice of size n x n. for(row=0;row<n;row++) for(column=0;column<n;column++) c[row][column]=dot_product(get_row(a,row), get_col(b,col));

 The for loop in this code fragment has n 2 Iteration, each which can be executed independently. Such independent sequence of instructions is referred as thread.  In above example there are n 2 threads one for each iteration of for loop. Since each thread can be executed independently of others, they can be scheduled concurrently on multi processors. Example 2:here we are using function called create_thread to provide mechanism for specifying c function as thread for(row=0;row<n;row++) for(column=0;column<n;column++) c[row][column]=create_thread(dot_product(get_r ow(a,row),get_col(b,col)));

Thread –Basic Unit of execution –Light weight process –Process can have multiple thread Why thread –To improve the CPU Utilization Note: all threads share same shared memory. 3/24/20199

An Application Program Interface (API) that may be used to explicitly direct multi-threaded, shared memory parallelism. OpenMP is an API that can be used with FORTRAN, C and C++ for programming shared address space machines. OpenMP directives provides support for concurrency, synchronization and data handling. The OpenMP directives in C and C++ are based on the #pragma compiler directives. 3/24/201910

OpenMP programs execute serially until they encounter the parallel directive. This directive is responsible of creating group of threads. The main thread that encounters the parallel directive becomes the Master of this group of thread and the thread is assigned with ID 0 within this group

Fork - Join Model: –OpenMP uses the fork-join model of parallel execution: –All OpenMP programs begin as a single process: the master thread 3/24/201912

Format: Ex: –#pragma omp parallel default(shared) private(beta,pi) 3/24/ #pragma ompdirective-name[clause,...]newline Required for all OpenMP C/C++ directives. A valid OpenMP directive. Must appear after the pragma and before any clauses. Optional. Clauses can be in any order, and repeated as necessary unless otherwise restricted. Required. Precedes the structured block which is enclosed by this directive.

A parallel region is a block of code that will be executed by multiple threads When a thread reaches a PARALLEL directive –It creates a team of threads –Becomes the master of the team. –Master thread ID is 0. Starting from the beginning of the parallel region, –the code is duplicated –all threads will execute that code If any thread terminates within a parallel region, all threads in the team will terminate 3/24/201914

The number of threads in a parallel region is determined by the following factors –Setting of the NUM_THREADS clause –Use of the omp_set_num_threads() library function –Setting of the OMP_NUM_THREADS environment variable –Implementation default - usually the number of CPUs on a node 3/24/201915

#pragma omp parallel It’s a parallel directive that creates groupp of threads. Using this we can compute operations parallelly.

tid=omp_get_thread_num() Used to give unique ID for each thread created.

L11.Design, develop and execute a parallel program in C to add, element wise, two one- dimensional arrays A and B of N integer elements and store the result in another one- dimensional array C of N integer elements.

#include int main() { int a[10],b[10],c[10],i, n; printf("\nenter the number of elements"); scanf("%d",&n); printf("\nEnter the element of 1st array"); for(i=0;i<n;i++) scanf("%d",&a[i]);

printf("enter the elements of 2nd array"); for(i=0;i<n;i++) scanf("%d",&b[i]); printf("\nthe contents of array A\n"); for(i=0;i<n;i++) { printf("\na[%d]=%d",i,a[i]); }

printf("\nthe contents of array B\n"); for(i=0;i<n;i++) { printf("\nb[%d]=%d",i,b[i]); }

#pragma omp parallel for shared(c) for(i=0;i<n;i++) { #pragma omp critical (c_lock) c[i]=a[i]+b[i]; printf("\nc[%d]=%d,thread id=%d\n",i,c[i],omp_get_thread_num()); }

printf("\nthe new array is===="); for(i=0;i<n;i++) { printf("%d ", c[i]); }

L14.Design, develop and execute a parallel program in C to determine and print the prime numbers which are less than 100 making use of algorithm of the Sieve of Eratosthenes.

#include int main() { int num[100], i, j; #pragma omp parallel for for(i=0;i<=99;i++) /*Fill num with the first 100 numbers*/ { num[i]=i+1; printf("\nnum[%d]=%d,thread id=%d\n",i,num[i],omp_get_thread_num()); }

#pragma omp parallel for for(i=1;i<=99;i++) { if(num[i]!=0) { #pragma omp parallel for for(j=(i+1);j<=99;j++) { if(num[j]!=0) { if((num[j]%num[i])==0) num[j]=0; } } } }

for(i=0;i<=99;i++) { if(num[i]!=0) /*Print all non zero numbers (prime numbers)*/ printf(" %d",num[i]); } getch(); }