################################################ # # # ## ## ###### ####### ## ## ## ## ## # # ## ## ## ## ## ### ## ## ## ## # # ## ## ## ## #### ## ## ## ## # # ## ## ###### ###### ## ## ## ## ### # # ## ## ## ## ## #### ## ## ## # # ## ## ## ## ## ## ### ## ## ## # # ####### ###### ####### ## ## ## ## ## # # # ################################################ The following paper was originally published in the Proceedings of the USENIX Microkernels and Other Kernel Architectures Symposium San Diego, California, September 20-23, 1993 For more information about USENIX Association contact: 1. Phone: 510 528-8649 2. FAX: 510 548-5738 3. Email: office@usenix.org 4. WWW URL: https://www.usenix.org .ce 10 Cohabitation and Cooperation of Chorus and MacOS Christian Bac, Institut National des Tlcommunications, rue Charles Fourier, 91000 Evry, France. + 33 1 60 76 45 39, E-mail: chris@int-evry.fr Edmond Garnier, Alcatel Alsthom Recherche, Route de Nozay, 91460 Marcoussis, France. + 33 1 64 49 15 77, E-mail: garnier@aar.alcatel-alsthom.fr Abstract This paper describes experimental work on cohabitation and cooperation between a distributed operating system (Chorus1) and an event driven operating system (MacOS2). Our aims were to exploit the graphical and the musical capabilities of Macintosh hardware and software directly from Chorus applications, while minimizing our efforts in the field of device drivers and hardware interfaces. The work was carried out in four major stages. The first stage was to port the Chorus kernel on the Macintosh hardware. In the second stage we changed the way Chorus managed the hardware in order to keep the MacOS system alive. Conversely, we modified slightly the way Chorus was booted so as to present it as an application to MacOS. This led us to the third stage, which was to share system events (e.g. hardware interrupts) between the two systems. The Chorus system allows one to have multiple functions connected to an interrupt. This feature was used to connect both an internal Chorus driver and a low level function to an interrupt. The low level function leads to the MacOS interrupt driver. The fourth stage is currently being carried out. It consists in the design and implementation of an interface permitting user level events (as system calls) to cross the borders of the two systems. This paper describes each stage and draws lessons about system software cohabitation and reusability. 1. Introduction The technology of distributed systems [1] [2] has grown and matured since the early eighties [3]. Today it is used to support applications based on the UNIX1 operating system interface [4] [5] and also new features such as lightweight processes and Inter Process Communications with message passing. Over the same period personal computers became widely used. These computers offer graphical and musical interfaces, and have partially read only memory based operating systems. Our goal was to combine the two technologies and see whether they worked well together or not. It seemed particularly important to us to be able to use existing technology, including machines and operating systems, and to integrate them into our base of distributed computers. We decided to use a cooperative approach by allowing the two systems (Chorus and MacOS) to cohabit in memory space and to share the CPU. Our cooperative approach was slightly different from the Mach and Chorus operating systems. Mach, up to the 2.5 version, was embedded in the UNIX system. It has been rewritten in the 3.0 version to support system servers offering a UNIX interface at user level. An early version of Chorus [6] (Chorus V2), cooperated with the UNIX system, but it was on a multiprocessor machine. Chorus V2 ran on several processors and UNIX ran on one. A Chorus driver was integrated into the UNIX kernel and communicated with the Chorus system by using a common memory space. This approach was abandoned because it required very specific hardware architecture. A subsystem (Chorus/Mix) of servers emulating a UNIX system on top of Chorus was created instead. Both Chorus and Mach achieve binary compatibility at the application level but they use source code of the UNIX operating system in their servers. However we were not interested in another UNIX like operating system but wished to add more graphical capabilities, as in the V kernel [7], to the Chorus distributed system. Similar experiments have been done over Mach 3.0 [8] with the DOS or MacOS running as an application. 2. Chorus & Mac project The Chorus & Mac project began three years ago with the port of a Chorus simulator in the AU/X2 environment. The Chorus simulator is composed of a UNIX process and a library that can be used to program and test Chorus applications in the UNIX environment. This port was done to gain experience in the Chorus area. This was slightly more difficult than expected, mostly because the simulator used certain BSD features and made assumptions about memory allocations that were different in AU/X (which is System V based). During this port, we explored many parts of the source code, common to the simulator and the Chorus kernel, and debugged C++ code with simple debuggers like adb and sdb (which obviously were not suitable). The simulator was later used to develop graphical applications using the distributed IPC of the Chorus system in an attempt to anticipate future research. We then began the major work; porting a Chorus kernel on the Macintosh hardware and trying to exploit MacOS features directly from Chorus applications. This work was done in four major stages. The first stage was to port the Chorus kernel on the Macintosh hardware, the second was to have both systems cohabit but ignore each other and in the third stage we changed the interpretation of the interrupts slightly to accord the Chorus system to the MacOS use of interrupts. This permitted both systems to share system events. The last stage, that is currently being carried out, will allow user applications events to cross the border between the two systems. In the following sections, we will describe, how each stage has been successfully achieved. 2.1. Porting Chorus on Mac Hardware The target machine is a Macintosh II CX. It is based on a MC68030 and many added chips. We used the Chorus sources for a Tadpole TP33M that is also based on the MC68030 chip. 2.1.1. Chorus kernel design As shown in Figure 1 below, the Chorus kernel [9] is composed of four independent elements: P The supervisor dispatches interrupts, traps and exceptions delivered by the hardware. P The real-time executive controls the allocation of the processor. P The virtual memory manager is responsible for manipulating virtual memory hardware and local memory resources. P The inter-process communication manager provides message passing facilities. Figure 1 - Chorus Kernel Architecture Thus most of the system is portable. This part was not modified because we used the sources for the same kind of architecture (MC68030). 2.1.2. Macintosh hardware As shown in Figure 2 below, the Macintosh II CX hardware [10] that was used, is composed of: P a Motorola MC68030 microprocessor which integrates a Paged Memory Management Unit equivalent to the MC68851, P a Motorola MC68882 floating point unit, P eight megabytes of RAM, P 256 KB of ROM which contains a large part of the MacOS software, P six extension slots using the NuBus standard: one is used for the Ethernet board and one supports the video board, the NuBus is interfaced to the main processor bus by logic circuits referred as Bus Interface Units (BIUs), P regular chips controlling hardware: % a Zilog Z8350 serial communication controller (SCC) providing two ports for serial communications, % an NCR 5380 Small Computer System Interface (SCSI) controlling an internal 80 MB hard disk and the external SCSI connector, P and Apple custom chips used in others Macintosh: % the GLUE, which provides address decoding and controls signals, % the Apple Desktop Bus (ADB), which handles communications with the keyboard and the mouse, % two Versatile Interface Adapters (VIA), based on the Rockwell 6522 chip, which support the ADB, the Real Time Clock, timers and other I/O devices, % a custom digital sound synthesizer chip, called Apple Sound Chip (ASC), % and the IWM (Integrated Woz Machine), which controls the floppy disk. Figure 2 - Macintosh II hardware 2.1.3. Chorus hardware requirements Chorus hardware requirements are light. To run, the kernel needs a timer that performs an interrupt every 1/100th second, and a terminal interface that reads and writes characters. It also requires correct initialization of the memory management unit, we will discuss this point later on. Timing has been achieved by using timer one of the second VIA. This timer is used in MacOS to generate a signal known as the VBL (Vertical Blanking Interrupt). In MacOS, the handler associated with this signal performs periodic events such as blinking or reading mouse position, etc. We decided to connect the terminal to port A of the SCC. This port is usually used to pilot a modem in MacOS or a terminal in AU/X. The low level routines allowing Chorus to control the port usage were written in C. 2.1.4. The development environment We were working in a cross development environment. We used a SUN 3 to compile and build the Chorus archive. This file contains a boot program, the Chorus kernel, and a minimal set of actors. We transferred this archive to the Macintosh using AU/X and TCP/IP, then we rebooted and used the Standalone Shell (SASH). SASH is a MacOS application which gives access to the UNIX partition. SASH is capable of starting applications that have been developed in the AU/X environment and linked to the SASH library. These applications are capable of using many UNIX system calls that are translated by the SASH library. These applications also have access to the UNIX disk partition. We developed a preboot program that was launched by SASH. This preboot program loaded the Chorus archive in the memory and gave control to the boot program (inside the Chorus archive). The boot program tested the memory size and set up the PMMU translation tables. The boot program then started the Chorus kernel. 2.1.5. Translation tables In general, the management of the MMU must be written; in our case the greater part had already been done, due to the sources used. The initialization of the translation tables remained the main problem. The physical address space of the Mac is described in Figure 3 below. F100 0000 - FFFF FFFF Standard NuBus space F000 0000 - F0FF FFFF Reserved 6000 0000 - EFFF FFFF NuBus Slot expansion space 5000 0000 - 5FFF FFFF I/O space 4000 0000 - 4FFF FFFF ROM 0000 0000 - 3FFF FFFF RAM Figure 3 - Physical address space on the Mac II Chorus on MC68030 uses a virtual address space composed of 4 K Bytes memory pages and four levels of memory index as shown in Figure 4. Figure 4 - Translation tables for Chorus The translation tables are set up at boot time by the boot program. When Chorus is running, the first entry describes the actor running in the user space, and the second entry describes the Chorus kernel space. This space includes the actors running in the supervisor or system mode. A user actor is allowed to use the address range from 0x0000J0000 to 0x07FFJFFFF. The Chorus kernel uses the address range from 0x0800J0000 to 0x0FFFJFFFF for its private space. In the Mac version, the other level A entries are reserved to the kernel and map the physical address space allowing the kernel to manipulate circuits like the VIA or the SCC, in the address range 0x1000J0000 to 0xFFFF JFFFF. 2.1.6. Conclusions of stage one The use of the Macintosh hardware by Chorus was limited to SCC and VIA. The SCC was used to transmit and receive characters. The first timer of the second VIA was programmed to emit clock ticks. Although our development environment was complicated, we succeeded in completing each stage necessary to bring Chorus into the Macintosh memory. Despite the complexity of the Macintosh hardware, Chorus has proved to be easily portable on it. In fact, we were lucky to have sources for MC68030. Stage one was completed and we had learned a lot about Chorus memory management and Macintosh hardware. We then tested the kernel with the "Chorus kernel tests". They ran successfully and we let them run more than one week with no error. At this stage the Chorus system had only been modified to support the hardware and no attention had been paid to the MacOS system. This was to be our next stage. 2.2. Bringing MacOS back to life In the second stage we changed the way Chorus managed the hardware to keep the MacOS system alive. Conversely, we slightly modified the way Chorus was booted so as to present it as an application to MacOS. Figure 5 - Sharing memory space 2.2.1. Sharing memory space To favour cohabitation of Chorus and MacOS, the physical memory is separated in three areas (see Figure 5 above): P the memory reserved for the MacOS system (composed of two physical parts), P the memory reserved for the Chorus system (where the kernel and actors are placed), P and a third area that initially allowed a correct alignment of Chorus memory space and that was later used to go from one system to the other, and that we will call "No Man's Land". We decided to limit the physical memory reserved for Chorus to 6 MB. Chorus can behave as if memory does not begin at physical address 0. After tests on memory allocation in MacOS, we chose to start the memory space reserved for Chorus at the physical address 0x80000. To share memory space with MacOS, we asked for memory allocation to it during the preboot phase. This was done by rewriting the preboot program. 2.2.2. New Chorus preboot Thus a new Chorus preboot was written. It reserved the 6.5 MB memory block at MacOS level. It installed the Chorus archive in memory, and it allowed a direct boot from MacOS without using the SASH. As explained below, this program was further extended to act in MacOS space for Chorus. This brought the MacOS system partly back to life. We were thus able to stop Chorus and return back to MacOS as in a Mac application. The first stage in collaboration had been achieved successfully and the two systems could share memory. The next stage was the sharing of system events. 2.3. Sharing system events Now that we had both systems in memory and Chorus running all the time, we wanted to give more control to MacOS. This was achieved by giving control to it when system events appeared. To allow system events to be treated either by MacOS or by Chorus, we needed some low level functions that enabled the CPU to commute from one system space to the other. 2.3.1. Crossing the frontier When it is running, MacOS uses a 24 bit address space. This address space is mapped to the physical address space as shown in Figure 6 below. The lower part of the address space is used for physical memory. F0 0000 - FF FFFF 50F0 0000 - 50FF FFFF I/O space E0 0000 - EF FFFF FE00 0000 - FE0F FFFF Standard NuBus space D0 0000 - DF FFFF FD00 0000 - FD0F FFFF C0 0000 - CF FFFF FC00 0000 - FC0F FFFF B0 0000 - BF FFFF FB00 0000 - FB0F FFFF A0 0000 - AF FFFF FA00 0000 - FA0F FFFF 90 0000 - 9F FFFF F900 0000 - F90F FFFF 80 0000 - 8F FFFF 4080 0000 - 408F FFFF ROM 00 0000 - 7F FFFF 0000 0000 - 007F FFFF RAM Figure 6 - Virtual to Physical address space under MacOS The memory in address range from 0X50000 to 0X80000 is used to place assembly functions that are called in order to commute from one system context to the other. Most of the work done in the context switches consists in changing the PMMU setting and the value of the interrupt base register. These modifications allow the Macintosh to act in both memory modes, either 24 bits when in MacOS or 32 bits virtual in Chorus. The assembly functions are placed in memory by an enhanced preboot program. They use a part of this memory space to save and restore the context registers in memory locations. There are three functions: P a boot function that saves the MacOS system context at boot time, before starting the Chorus system, P a Chorus to MacOS function that saves the Chorus system context and restores the MacOS context, P and a MacOS to Chorus function that saves the MacOS context and restores the Chorus context. The contexts are constituted by all the MC68030 registers plus the PMMU registers that point to the translation tables. To save or restore the contexts, an intermediate memory mapping is set up, which allows access to the physical memory and system kernel space. 2.3.2. Interrupts Interrupts were a kind of nightmare; at first glance we could not split the circuit usage between the two systems. We had to consider a new distribution of events because any event could appear in both systems. We knew what to do when MacOS interrupts arrived in Chorus space and we were able to relay the interrupt to MacOS, but we had no control over what was happening in MacOS. We decided to let MacOS take care of interrupts and just used one of them to trigger a vital part of Chorus, namely, the scheduler. This was possible at the Chorus level because we could connect an interrupt to multiple tasks. This means that we could connect the timer interrupt to the Chorus function timein() and to the jump in MacOS if necessary that leads to the MacOS interrupt handler. The Chorus function timein() counts clock ticks and schedules user tasks. The jump in MacOS was achieved by connecting the interrupt level to a function in "No man's land" that simulated the interrupt on the MacOS stack and jump to the MacOS interrupt handler. The interrupt frame must be duplicated from the Chorus interrupt stack to the MacOS stack. As MacOS was not able to acknowledge the timer interrupt correctly, the interrupt generated by the timer was blocked in the VIA before the jump in MacOS. The interrupt was released on the return from MacOS. We checked that very few ticks were lost over a long period by using this method. 2.3.3. Events in MacOS The jump into MacOS allowed it to acknowledge the interrupts correctly. However we wanted MacOS to do more. As mentioned previously, MacOS is an event driven system, and in any MacOS application there is an event loop that handles the keyboard event, and the pull down event, etc. We decided to call a function in our preboot program (MacOS space) to perform certain events on the VBL interrupt. This routine was called from the function in "No man's land" that was associated with the VBL interrupt. This brought MacOS back to life in many operations: P mouse moving was detected and the mouse pointer was displayed on the screen, P accessories like Clock and Super Clock displayed the correct time and changed every second, P the mouse position could be read from the program, P and keyboard events were recognized. 2.3.4. What events should be allowed? We were able to use MacOS features by adding them to the boot program that acted in the Chorus name in MacOS space. For example, we added a function that was triggered by the click in the close box. This function initiated the Macintosh sound manager so that it played music for 30 seconds. Once initiated, the sound manager returned and Chorus continued. The buffer of the ASC is automatically loaded with the bytes representing the sounds to play when the VBL interrupt occurs. The music was correctly played by the MacOS sound manager although Chorus was running. One question still remained: What should we do with long events? In MacOS, some functions wait for the user to terminate an action before giving the control back. For example, the function allowing the dragging of a window with the mouse or any event involving a long click on the mouse, does not give the control back until the mouse button is released. We decided to disallow these functions because they delayed the Chorus part of the system too much. At this point, the two systems were present in memory and they could share system events like interrupts. The next stage was to allow user events to be shared by the two systems. 2.4. Sharing user events We are currently working on the sharing of user events by the two systems. Our aim is to allow user tasks in Chorus to call functions in MacOS. To do this, there are two major problems: the system call must be recognized by Chorus as a MacOS one, and the parameters and results of the system call must be exchanged between the user task in Chorus, and the MacOS system. 2.4.1. Heterogeneous system calls The user level events or system calls are achieved differently in the two systems. In Chorus, a system call is made via a trap and the interpretation of certain CPU register values gives the system call to the kernel. In MacOS, system calls are implemented as illegal instructions and the system call number is included in the instruction. This allows a great number of system calls. When an illegal instruction can be interpreted as a MacOS system call in the Chorus space, the Chorus kernel must call the function associated in MacOS. 2.4.2. Parameters and results A system call is generally associated with arguments. In a classic operating system (like UNIX), the arguments are copied from user space to system space before the kernel function is called. When the function has been completed, the results are passed from the system space to the user space. In the case of system calls from Chorus actors to MacOS, the arguments must be copied from an actor user space to MacOS, and back from MacOS to the actor memory space. This leads to general problems in passing arguments and getting results from functions executed in different memory spaces. This kind of problem is encountered in remote procedure calls [11] and the first solution is to copy the arguments and the results from one space to the other. It seems that it is not the best solution for systems acting in close memory space, and before doing anything, we preferred to explore two different fields in this area: % a taxonomy of user to system functions calls and usage of parameters and results in these functions, % measurements of the time spent in system commutation and the construction of arguments and results. 3. Experience and future work We were able to use the Chorus kernel features to make it cohabit with MacOS. During this experience we have learnt much about Chorus and MacOS. We have proved that both systems were well designed. In fact, they are so well designed that they can be used to do things for which they were not initially built. In our most recent version, we make both systems co-operate so that Chorus uses MacOS to read from the keyboard and write to the screen. To achieve this co-operation, we need an application in the MacOS system space acting on behalf of the Chorus system. This application is responsible for the correct reservation of resources and for calling the MacOS system. The exchange of data between Chorus and MacOS is done by using a Macintosh feature in memory space addressing. The Macintosh gives a cyclic vision of its memory that allows the same physical memory locations to be accessed with different addresses from system space in Chorus and from MacOS. For example the physical memory address 0x60000 (corresponding to a physical location in the "No Man's land") is used as a buffer to exchange the characters between Chorus and the preboot application. The buffer can be accessed in Chorus at the address 0x10060000. This virtual address is mapped to the same physical address by the MMU, but the cyclic vision of physical memory given by the Macintosh results in the physical memory at the address 0x60000 to be accessed instead. Thus the exchange of data was done with no modification in the setting of the PMMU. The Chorus kernel on Macintosh appears robust. We have tried our system with the new version of MacOS (system 7) and it still works. Maybe cohabitation can be improved since the system context switch is slow. The two systems co-operate but they are still not mixed enough. We think that some kind of sub-system, like Chorus MIX, acting as MacOS would be faster and better designed. The problem of access to resources usage at the user level, is still not completely solved and some further work is required to have an efficient use of cohabiting system resources. 4. Conclusion We have demonstrated that system cohabitation is possible and that it is a convenient way to extend the functions of a system. The cohabitation results in many advantages compared to the usage of the ROM routines from Chorus: P by using the MacOS system in RAM, the system benefits from routines that are not in ROM. New versions of some ROM routines and bugs corrections are also accessible. P the system also uses the interrupt handler from MacOS. The handlers acknowledge the hardware interrupt and post the MacOS events in the event queues in RAM. P further releases of the MacOS operating system could be used in cohabitation. This would allow an evolving operating system which take advantage of future developments in MacOS software. Although this work is very specific, the idea of mixing technologies from different areas in the same computer at the lower level is both interesting and advantageous. Cohabitation will allow Chorus real time processes to act in an event driven environment while using a well-known graphical interface with very few developments at the system level. The Chorus operating system can use the MacOS device drivers to manage hardware. The Chorus operating system thus acts as a MacOS application to access peripherals through MacOS. References [1] M. Rozier, V. Abrossimov, F. Armand, I. Boule, M. Gien, M. Guillemont, F. Herrmann, C. Kaiser, S. Langlois, P. Lonard and W. Neuhauser. Chorus distributed Operating systems. Computing Systems, 1(4), 1988. [2] Accetta, M.J., Baron, R.V., Bolosky, W., Golub, D.B., Rashid, R.F., Tevanian, A., and Young, M.W. Mach: A New Kernel Foundation for UNIX development. In Proc Usenix Summer, July, 1986. [3] H. Zimmermann, J.S. Banino, A. Caristan, M. Guillemont, and G. Morisset. Basic concepts for the Support of Distributed Systems: the Chorus approach. InProc. 2nd IEEE Int. Conf. on Distributed Computing Systems, Versailles (France), April 1981. [4] F. Hermann, F. Armand, M. Rozier, M. Gien, P. Lonard, S. Langlois and W. Neuhauser. Chorus, a new technologie for building UNIX systems. In Proc. EUUG Autumn '88 Conference, Cascais (Portugal) 1988. [5] Guide to OSF/1: A Technical Synopsis. O'Reilly & Associates, Inc. 1991. [6] F. Armand, M. Gien, M. Guillemont, and P. Lonard. Toward a Distributed UNIX system: The Chorus approach. In Proc. EUUG Autumn '86 Conference, Manchester (UK) 1986. [7] E.J. Berglund, An introduction to the V-System, IEEE Micro vol. 6, no 4, August 1986. [8] G. Malan, R. Rashid, D. Golub, and R. Baron, DOS as a Mach 3.0 Application. [9] Chorus Systmes, Overview of Chorus Distributed Operating Systems, from Chorus v.3 Programmer's reference Manual. [10] Apple MacIntosh Family Hardware Reference, Addison Wesley ref. 19255, 1988. [11] Sun Microsystems, Inc., "RPC: Remote Procedure Call Protocol specification: Version 2", RFC 1057, 1988. 1 Chorus is a registered trademark of Chorus Systmes. 2 MacOS is a registered trademark of Apple Computers Inc. 1 UNIX is a registered trademark of AT&T USL. 2 AU/X is the UNIX system on MacIntosh and a registered trademark of Apple Computers Inc..