Software Primitives for Emulation of

Multiprocessor Architectures

Anders Svensson

Department of Computer Engineering, Lund University

P.O. Box 118, S-221 00 Lund, Sweden

Abstract

In order to evaluate design principles for MIMD multiprocessors with distributed shared memory, an experimental emulation facility has been developed. The hardware of the 38 processing elements was designed to support efficient emulation of mechanisms which would be implemented in hardware in the real system. One processor in each processing element executes the application while another processor runs the emulation software.

This paper presents a software environment and primitives to simplify efficient implementation of emulators. A flexible system call mechanism is provided for exchanging messages between the execution and the emulation processors. A separate bus communication mechanism is provided for implementation of interprocessor protocols and 'Software probes' are used for monitoring purposes. Emulation principles are illustrated by implementation of a virtual memory system.

1 Emulation of multiprocessors

A system for experimental evaluation of design principles for MIMD multiprocessors with distributed, shared memory has been developed [15, 13]. The hardware of the system was designed to support efficient emulation of mechanisms which would otherwise be implemented in hardware in the real system.

Emulation is an alternative to the traditional approaches of either building a prototype of the real system or simulating it in software on a commercially available single or multiprocessor computer. Emulation is potentially much faster than simulation but still has much of the same flexibility as a simulator.

A commercial multiprocessor can be instrumented to collect information at a logical level by Software monitors [4], which interfere with the measured system since extra code must be inserted. Hardware monitors [4] are external to the measured system, and are therefore

more accurate, but have no knowledge of operating system activities. Hybrid monitors [9] try to combine the favorable properties of software and hardware tools.

One of the basic ideas behind the experimental system was that there should be a close correspondence between the various parts of the target system and the emulated multiprocessors. Even if some of the hardware in the target corresponds to software modules in the emulated version, the principles for interaction with other parts should be identical. This makes it possible to change one part without affecting any of the others and makes it easier to interpret the results of emulation in terms of the real system.

The goal has been to make it possible to execute parallel application programs on the emulated multiprocessor and study the dynamic effects of implementing various design principles throughout the system. In this way a wide range of interacting mechanisms, including effects of program behavior, can be studied with very limited effort.

There is a tradeoff between the range of architectures to be reasonably well supported by the emulation hardware and the performance of the system. If the variations are limited, the hardware support can be much more specific which will lead to an efficient emulation system performance-wise. In this case an initial decision was made to support architectures consisting of a number of interconnected processing elements (PEs), each one containing a processor and part of the memory. The communication scheme could be based on memory accesses and/or messages. This structure is closely reflected in the physical structure of the experimental system (see figure 1).

There are 38 identical PEs specially designed for this multiprocessor [13]. The number of PEs is restricted due to physical restrictions and each PE contains two microprocessors. One executes the application (the execution processor) and the other runs the emulation software (the emulation processor). A bus interface in each PE has been designed to support efficient em-