1 Introduction

This user reference manual gives an overview of the parallel version of SNEC, detailing the installation procedure, giving a brief tutorial and summarising the limitations of the package. The program interface (the commands used to drive the program) is identical to the sequential version, and thus the user is referred to that document for the program commands.

The parallel version of SuperNEC (SNEC) is an object-oriented program based on the FORTRAN program NEC-2.

The advantage of being able to operate in parallel is two fold. First, there is a reduction in the time taken to solve a problem. Secondly, a network of processors generally has more memory at its disposal than a single processor, thus much larger problems may be solved on a network of processors.

SNEC has been run on the distributed memory, parallel IBM-SP2 machine located at Southampton University. The principle problem analysed on this occasion was a DC-3 at 90 MHz. This structure was modelled using 5500 segments and generated an interaction matrix of 240 Mbytes. The simulation requested 3 radiation pattern cuts at 2 degree increments (181 points) and a single 3-D pattern in 4 degree theta and phi increments (4186 points). The run time on 12 processors was about 19 minutes.

The largest problem tackled with this software to date, was the simulation of the DC-3 at 160 MHz. Solving this problem on a sequential machine would require 2.3Gbytes of memory and take about 4 days of computer time to simulate. Distributing the problem onto 21 processors, reduces the memory per processor requirement to 110 Mbytes and a solution was found in 5.3 hours.

This manual is targeted at a network of Windows-NT machines. It is possible to connect UNIX workstations to the virtual machine as well, however, this requires that the SNEC executables be specially compiled for these platforms.

1.1 A Brief History of Parallel SuperNEC

In 1989 the FORTRAN program, NEC-2, was modified to operate in parallel on a network of transputers. During this project, all the numerically intensive routines were re-written to operate in parallel. The performance of the parallel algorithms was very satisfactory and it was clear that parallel processing was a simple means of reducing the time required to simulate a structure. As explained in the history of SNEC in the sequential manual, the original FORTRAN code was found to be difficult to modify. The code was therefore redesigned and re-implemented in C++. The result of this exercise was a computer code that was relatively easy to extend and maintain. One of the extensions made to the C++ code was parallel execution.

The original parallel code was implemented on a network of transputers. Transputer technology has, however, severe limitations. The most overbearing being that a transputer machine is not an all-purpose machine and hence is not widely available (when compared to workstations). Thus restricting SNEC to a network of transputers was not a good long term prospect for the program.

Currently there are a number of communications libraries that allow processors to communicate with one another over a local area network. One of these libraries is PVM (Parallel Virtual Machine). This library allows one to build up a parallel machine using a heterogeneous collection of existing processors. For example, one could connect a number of SUN 10’s and RISC 6000 machines to form a fairly powerful parallel machine.

SNEC was freed from the limitations of the transputer by using PVM to implement the required communications. Subsequently, SNEC has been adapted to incorporate a hybrid MOM-UTD method, MBPE, fast iterative solvers and many other features. These features are in prototype form and will be added to the core program in due course.

2 Installing Parallel SuperNEC

This manual describes the process required to get the parallel SuperNEC running on Windows-NT. It assumes that you have installed the software from the SuperNEC CD.

It is suggested that you follow the process described in this manual with the aim of getting a two-processor parallel machine up and running. Once you have achieved this, it is relatively easy to set up the other machines.

The set up procedure involves the following :

1. Setting your NT user rights

2. Installing the TCP Services

3. Running PVM

2.1 Setting your NT user rights

NOTE : This is NOT applicable to Windows 2000 users.

For set-up purposes, you are required to have administrator rights on the machine. If you do not, then you must log in as administrator.

Open the “User Manager”. This is done by going to Start | Programs | Administrative Tools | User Manager.

2.1.1 Ensure that “Power Users” are able to log on as a service

Click on the Policies | User rights menu item.

Tick the “Show Advanced User Rights” check box.

Select the right “Log on as a service” from the top “Right” pull down menu at the top of the window.

Ensure that Power Users exists in the list box labelled “Grant To”. If it does not then click on the “Add” button and add Power Users. Then close the “User Rights Policy” dialog box.

2.1.2 Make yourself a user

Ensure that you are a user on this machine and that you are a member of the “Power Users” group.

This process must be repeated on all machines that you wish to include in the network.

2.2 Installing the TCP Services

Open a DOS box and change directory to INSTALL_DIR\pvm\atrls (where the INSTALL_DIR is the directory in which you installed SuperNEC, e.g., c:\snec).

Type atrls install

Go to the Control Panel (Start | Settings | Control Panel) and open the “Ataman TCP R.L. Services”.

Click on the “Users” tab :

Click on the “Add User” button :

Fill in the following :

1. User Name : The user name that you use to log onto your computer.

2. NT User Name : Also set to the user name that you use to log onto your computer.

3. NT User Domain : Type in a dot (.)

4. Home directory : c:\temp

5. Interactive Cmd Processor : Leave blank

6. Batch Cmd Processor: Leave blank

7. Environment File: Leave blank

8. NT Passwork : Enter the same password that you use to log onto this machine.

9. Host Equivalence List : In this edit box you must type in the IP addresses (or host names) of all machines that will make up the parallel machine (separated by commas). If you are not concerned about which machines may access the computer, you may type *.*.*.* into the host equivalent box. The star acts as a wild string, and therefore entering the above string will allow all computers to link to your computer. If the machines you wish to add to the parallel machine have common numbers in their IP addresses, then you may hard code part of the IP address and use the wild string to designate the variable part. For example, assume the IP addresses of the machines you have are 146.141.16.89, 146.141.16.101 and 146.141.16.32, then you can enter 146.141.16.* as the host equivalent list. This will allow all three of the above machines to access your computer. In addition all other machines that have the same first 3 numbers in their IP address will also be able to access your computer (assuming you use the correct login name and password for these machines).

The dialog box should look something like this :

Push OK to accept the data.

Each user wanting to use the Parallel SuperNEC program must be added as users of the Ataman service.

Once you have finished entering all users, you must type “atrls start” at the command prompt (in the same directory as you typed “atrls install”).

This process must be repeated on all machines that you want to add to the network.

2.2.1 Checking the installation of the TCP Services

Once you have set yourself up as a user on all the machines you wish to include in the network and set up yourself as a TCP user, then perform the following test.

Let us assume that you have two computers set-up. I will assume that your user name is fred and that the (DNS) names of the two machines are Maxwell and Bohr. I will assume that you are logged in on the Maxwell machine.

1. Open a DOS box

2. Type “ping bohr” at the command line. Ensure that you get text that looks like the following:

Pinging bohr.ee.wits.ac.za [146.141.16.104] with 32 bytes of data:

Reply from 146.141.16.104: bytes=32 time<10ms TTL=128

If you get the text

Bad IP address bohr

then your domain name server is unable to locate your machine and you must ask your network supervisor to fix the problem.

If this command does not result in a reply from the bohr machine then you must request help from the network supervisor.

DO NOT GO PASSED THIS POINT UNTIL YOU GET THIS TO WORK

If the ping test is successful, then try the following :

Type “rsh Maxwell -l fred -n dir” at the command line (without the quotes).

You should get a directory listing of the c:\temp directory on your own machine (Maxwell).

If you do not then check that you have followed the above instructions properly. If this still fails then read the document called userman.doc in the INSTALL_DIR\pvm\atrls directory.

Try this same test on the bohr machine. i.e type “rsh bohr -l fred -n dir” at the command line of the machine named bohr.

DO NOT GO PASSED THIS POINT UNTIL YOU GET THIS TO WORK

Then try the following :

Type “rsh bohr –l fred –n dir” at the command line on the Maxwell machine (without the quotes).

You should get a directory listing of the c:\temp directory on the machine named bohr.

If you do not then check that you have followed the above instructions properly. If this still fails then read the document called userman.doc in the INSTALL_DIR\pvm\atrls directory.

DO NOT GO PASSED THIS POINT UNTIL YOU GET THIS TO WORK

Type the following at the command line

“rsh bohr –n dir” at the command line, without the quotes.

Again you should get a directory listing of the temp directory on the bohr machine.

This completes the installation of the TCP services.

DO NOT GO PASSED THIS POINT UNTIL YOU GET THIS TO WORK

2.3 Checking the Installation of PVM

2.3.1 Checking the registry settings

PVM requires three variables to be set in the registry. You can check that they are correct by doing the following (this is not really necessary as it should have been done when you installed SuperNEC).

1. Open a DOS box.

2. Type “regedit” at the command prompt, this opens a window that looks like :

3. Find the text “PVM” in the registry using the Edit | Find menu.

4. The variables should be set as follows :

PVM_ARCH=WIN32

PVM_ROOT=c:\snec\pvm (Assuming you installed SuperNEC in c:\snec)

PVM_TEMP=c:\temp

Check that they are correct.

2.3.2 Starting PVM

Start PVM by going to Start | Programs | SuperNEC | PVM Console.

Two DOS prompts should appear. The one displayed above will disappear after a few seconds.

The other DOS box will remain open and holds the PVM prompt “pvm>”

Typing help at the PVM command prompt gives a list of PVM commands that one can use.

Do the following :

1. Open another DOS box

2. Change to the directory INSTALL_DIR\bin

3. Run the hello program by typing “hello” at the command prompt.

The output of the program should be something like :

i'm t40003

from t40004: hello, world from Maxwell

To exit PVM you MUST TYPE halt at the PVM command prompt. If you kill the application in any other way, then pvmd.* files will be left in the PVM_TEMP directory. PVM will not start again while these files exist. You will therefore have to manually remove them before starting PVM again.

2.3.3 Adding another machine to the parallel machine

Type the following at the PVM command prompt :

add “bohr dx=c:\snec\pvm\lib\win32\pvmd3.exe” (including the quotes)

The dx variable must be equal to the location of the pvmd3.exe file on the bohr machine. If you installed SuperNEC in the c:\snec drectory, then the above is correct. If not, then you must modify the text that you type accordingly.

If the command is successful then you will get a screen that looks like the above.

If it is not successful then see section 2.3.4 for some suggestions as to how to proceed further.

DO NOT GO PASSED THIS POINT UNTIL YOU GET THIS TO WORK

Typing “conf” at the PVM command prompt gives you the configuration of the parallel machine. It should look something like :

Run the hello program from a DOS prompt a few times. The output should alternate between “hello, world from Maxwell” and “hello, world from Bohr”.

If you get to this stage and everything is working, then you are ready to run parallel simulations.

To exit PVM you MUST TYPE halt at the PVM command prompt. If you kill the application in any other way, then pvmd.* files will be left in the PVM_TEMP directory on all connected machines. PVM will not start again while these files exist. You will therefore have to manually remove them before starting PVM again.

Note : There is a timeout problem on WinNT systems that results in one having to wait for about 5 minutes after shutting down PVM before being able to run it again. It is therefore suggested that PVM is not shut down after performing a simulation if you intend running a number of simulations one after the other.

2.3.4 Suggestions for when you have problems adding another machine

1. PVM creates temporary files in the directory set by the PVM_TEMP variable in the registry. Check that this directory exists (normally c:\temp) on all machines.

2. Check whether there are any files called pvmd.* in the PVM_TEMP directory of the machine you are trying to add. When PVM exits it should clean up all pvmd.* files. If, for some reason, these files are not removed, then PVM will not be able to start on that machine. You must manually remove these files.

3. Check that you have write permissions in the c:\temp directory on all machines.

4. Ensure that the name of the machine and your login name are NOT the same.

5. If all else fails contact us at SuperNEC@poynting.co.za.

2.3.5 Setting up the host file for parallel SuperNEC simulations

The process described above is the manual method for running a parallel code. You will not have to go through the effort of having to add machines to the network manually when using SuperNEC.

SuperNEC uses a hostfile to define the machines that are to be networked. The hostfile exists in the INSTALL_DIR\bin directory. You must edit this file and add your own machines.

The hostfile would contain the following if I assume the machine names of Maxwell and Bohr and the INSTALL_DIR was c:\snec.

maxwell dx=c:\snec\pvm\lib\win32\pvmd3.exe

bohr dx=c:\snec\pvm\lib\win32\pvmd3.exe

Once you have written the hostfile you can run a parallel simulation with the following command :

snec –h hostfile input.nec output.out

Where input.nec is some SuperNEC input file that you have generated.

Note : You do not have to first start the PVM console. The SuperNEC program starts the PVM deamons on all machines.

Note : If you wish to terminate a simulation then the correct way to do this is to start the pvm console and type halt at the pvm command prompt. If you use CTRL-C to kill a parallel simulation, then the pvm processes will not be terminated. You will then have to manually kill these processes (on all machines) and remove all pvmd.* files from the TEMP directory on all machines.

3 Running Parallel SNEC from the command line.

The SNEC executable is run from the command line as follows :

snec -h hostfile input file [output file]

Notes :

· SNEC geometry and simulation controls should be specified in the “input file”.

· The simulation results will be written to the file specified by “output file”.

· If an output file is not specified, then the output is sent to the screen.

· If no parameters are specified, then a program usage statement is given

3.1 Tutorial - Simulating a monopole over ground

This tutorial is partially aimed at users who have not used NEC-2 or SNEC before. It illustrates the process of writing a SNEC input file manually and running the executable in parallel. If you wish, you can generate the structure using the GUI.

3.1.1 Writing the SNEC input file

Using a text editor write the following in a file named dipole.nec

CM Tutorial - Simulation of a monopole over a Sommerfeld ground
CE
GW         1              5              0.0           0.0           0.0           0.0           0.0           0.5           0.0001
GE
GN          2              0              1              0              4.0           0.01
FR           0              1              0              0              150.0
EX           0              1              1              0              1.0
RP           0              91            1              1500        -90.0        0.0           2.0
XQ
EN

1. The CM command takes a line of text as a parameter. This text is used as a comment in the SNEC output file.

2. The CE command may take a line of text as a parameter and indicates the end of the comment section of the input file.

3. The GW command specifies a wire of 5 segments (tag=1) starting at the origin and ending half a meter up the z-axis. The radius of the wire is 0.0001 meters.

4. The GE signifies the end of the geometry.
Note for NEC-2 users : A ground is specified for the simulation, however, one does not indicate this in the GE command.

5. The GN command specifies a Sommerfeld ground with relative dielectric constant 4.0 and conductivity 0.01 mhos/meter. The third integer field is set to 1 which implies that the current of all segments touching the ground are interpolated to the image below the ground.
Note for NEC-2 users : There is no need to generate a Sommerfeld ground file in SNEC.

6. The FR command specifies that the simulation is to be done at 150 MHz.

7. The EX command excites the antenna at the base with an applied field voltage source of 1Volt.

8. The RP command requests a radiation pattern of 91 points starting at Q = -90° and incrementing in 2° steps. The parameter 1500 indicates that vertical and horizontal gain is to be computed and that a table of total gains normalised to the maximum total gain should be printed at the end of the output file.

9. The XQ command indicates to the program that the simulation is to be run. The XQ command is redundant in this case, since the execution will have been performed after the RP command had been read.

NOTE : The parameters associated with each command are delineated by tabs or spaces. It is not necessary to line the parameters up in columns as is required in NEC-2. It is necessary, however, to put zeros for parameters that are not required if parameters are specified after those that are not required.

3.1.2 Running Parallel SNEC from the command line

Type the following command to run the simulation :

snec -h hostfile dipole.nec dipole.out

This will result in the simulation taking place on processors specified in the hostfile. You will find that the execution time of this simulation is longer than running the program sequentially. The reason for this is that the simulation is so small and hence the communication overhead is far greater than the computational effort.

3.2 Program Limitations

· There is no parallel out-of-core solution. Thus one must ensure that the processors making up the virtual machine have enough memory to store the interaction matrix.

· Each processor in the virtual machine is given the same amount of work to do (execution wise and memory wise). Thus if one processor is much slower than all the others, the run time will be governed by this processor. In a similar fashion, you are limited by the machine with the least memory. The total memory available for a parallel simulation is equal to the amount of memory in the machine with the least memory multiplied by the number of processors in the virtual machine.