A Grail Millennium Project

White Paper

History | Preface | TOC

This document is maintained by the author, Lewis A. Sellers, somewhere in the mountains of East Tennessee, the United States of America. It is an informal technical document for a works in progress project called Grail Millennium, or fully, The Minimal Operating System of Object Class Interfaces Holy Grail for the Millennium. In other words, for a new, easy to use, long-lived operating system.

Copyright Notice

This document and all material therein, unless otherwise stated, is Copyright © 1995,1996, Lewis A. Sellers. All Rights Reserved. Permission to distribute this document, in part or full, via electronic means (emailed, posted or archived) or printed copy are granted providing that no charges are involved, reasonable attempt is made to use the most current version, and all credits and copyright notices are retained.

Distribution Rights

All requests for other distribution rights, including incorporation in commercial products, such as books, magazine articles, CD-ROMs, and or computer programs should be made to the primary author Lewis Sellers.

Warranty and disclaimer

This document is provided as is without any express or implied warranties. This is a work-in-progress, and as such will contain outdated or as yet uncorrected or substanstiated assumptions. The author, maintainer and/or contributors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.

WWW Home Sites

You can currently find home sites to this project at If you can not reach them, or they seem to be down, do a key word search on AltaVista, Lycos, or the Web Crawler search engines.

Contact Email

The primary author of Grail Millennium should be reachable at lsellers@usit.net.

Preface | TOC

OCI/Spec | C O R E i386

DESIGNER

Lewis A. Sellers (aka Minimalist) lsellers@usit.net

CRITIQUED BY

REFERENCES

VALID PROCESSORS

DRAFTS

INSPIRATIONAL MUSIC for this Document:

TOC

(a0.90)

[DANGER WILL ROBINSON!

This document is currently (as you read it) being updated and revised. The author reserves the right to pretend he never was an English Major if you find a spelling or grammatical mistake.
This is by no means a finished document. It is being converted to HTML and brought up to current design specs as you read it. It may change daily infact. This is mostly stream-of-consciousness writing, and I've taken little effort to clean up explanations since the details are apt to change at random times, and thus those efforts wasted would be. Being asleep while I work on it doesn't help any either. Eventually of course this will be a much better read, but bear with the state of things for a while longer.]
--min

What is this document about?

This is part of the description of the Grail Millennium CORE, discussing Processor Specific elements for the x86 IBM PC clone platform. It is intended that a version of Grail also be ported eventually to the Macintosh system, hopefully by the time they get Copland straightened out.

At one time this document was called KERNEL.API and was 7-bit text only. The distributed version was then promoted to RTF, while it's native format remained Word 2 (.DOC). All OCI/Spec documents are exclusively maintained in HTML 3.2 format. (Or will be... as soon as they are all translated.(

Ok, but is there a version of this document for other processors?

Not yet. Currently there are plans to make versions of this document for the Motorola PowerPC 601, Motorola PowerPC 620, Motorola 68020/60 and Intel/HP P8/PA-RISC (not available till '98 or so).

The Grail Philosophy

"Know your enemy and thus yourself"-min 95

Grail is for Entertainment and Educational purposes. It is not meant to be a server platform, run business software or be a secure anything. I don't believe in a one-world one-OS philosophy anymore than I believe in communism.* (They made very good enemies though, and it is only against the worthy foe that your own greatness can be know.)

To that end, it is imperative that Grail be able to gracefully kill itself off and switch to another OS at will, and visa versa. We have reverence to all UNIX clones and respect for OS/2 and Mac System 7 and how it will be technically possible to form gateways back and forth to them.

Grail exalts power slightly above speed. Raw speed is not always the point. Grail tries to create a host of discrete functions that may be at some point supported by hardware, but if not, are supported in software. Where an operation or function is considered overhead (ie, something that probably won't be in some form of a loop) flexibility and power win out over speed. Where the function will probably be in tight loops, speed is everything and all our powers of coding are bent to optimization.

For flexibility, Grail is highly modular, and is object-oriented. There are no .EXEs, .BINs, .COMs, .VBX, or .DLLs. There is only the object and it's functions, and that object may or may not be a task. See the section on the Grail Script for example code.

[ed: revise this]

Table of Contents
Legal NoticesWho gets sued and why
HistoryChanges and who made them
PrefaceAbout the CORE i386 and the people that make it

OverviewAn Operating System Overview
CORE SYSTEM FunctionsWhat are they? What can they do?
Supervisor ObjectsDe/coupling Intimate Objects
Exceptions: A/Syncronous
Memory Management Unit
Virtual Memory Management Unit
Distributed Communcations
Symmetric Multithreading
User Authorization
AfterwordThe P8/PA-RISC and Beyond

TOC

An Operating System Overview

BY | Lewis Sellers lsellers@usit.net

"I must not fear.
Fear is the mind-killer.
Fear is the little-death that brings total obliteration.
I will face my fear.
I will permit it to pass over me and through me.
And when it has gone past I will turn the inner eye to see its path.
Where the fear has gone there will be nothing.
Only I will remain."
-- The Litany against Fear, DUNE

The Grail Millennium Operating System does not have a kernel per-say, nor is it any form of a monolithic structure. It utilizes a set of required supervisor objects known as the core-set or the core-objects. Above all other classes there is __SYSTEM. This class operates beyond the restrictions of most others as it is responsible for creating the conditions that allow them to begin functioning. It also is responsible for termination of the operating system and low-level communications between the symmetric system processors.

  • (MULTI-)PROCESSOR

    • __SYSTEM
    • __SUPERVISOR
    • __EXCEPTION
    • __MMU
    • __VMMU
    • __OBJECT
    • __TASK

  • FILE SYSTEM

    • __POLICY
    • __SESSION
    • __DEFRAGMENT
    • __ARCHIVE
    • __FOLDER
    • __FILE
    • __FS
    • __FSDEVICE

  • INSTANTIABLE FILE SYSTEM

    • __COMPRESS
    • __DECOMPRESS
    • __ENCRYPT
    • __DECRYPT

The Past : What was it like?

Once upon a time there were up to three separate environment models for Grail (Black, Gray, and White). After a lot of long hard thought upon this, the concept has been essentially dropped. The amount of complexity it would added to the core and all the support code was, in the final analysis, deemed to be more trouble than it was worth. What was wanted was to be able to disable paging (and therefore not suffer the potentially crippling wrath of sequential TLB misses) and secondly prevent your applications performance to degrade or be choppy because of multitasking.

Why not allow such? If we did go with the ability to do mode switches as such, more code would be required and thus the core would bloat and be less optimized. It was thought that, in the end, only democoders and a few hacker types would opt for these more primal modes.

--min '96

To start Grail all of these classes are loaded in a proper format and all the initial memory structures created. This is partially the responsibility of the core loader and partly that of the __SYSTEM class. Once started, the __SYSTEM scans the root area and loads the NULL user and thus asks for a login name and authorization before completely starting up.

As shown there are three major sets of core objects: Those that are processor specific, those relating to file systems, and those relating to file systems that are instantiable (expandable to many object instances). They are all of supervisor level security. Note that while most of the objects will have only one processor specific instance, it is entirely possible that other non-family processor sets of the objects may exist. That is to say, there can be two core-sets in memory for a multi-processor system for both an X86 processor and a PowerPC processor.

All objects can have threads of execution running within them. Object and thread alike are all the same type of task, though probably running at different priority levels. Tasks come in several different flavors, but all have 16 major levels of priority. On the X86, tasks are divided up by priority level into separate GDTs. That is, normally Grail has 16 GDTs each corresponding to a different task priority. If a priority level has more tasks than its GDT can hold (over 8000) then another GDT will be created and linked to, and so on as needed. There is thus no limit on the number of tasks Grail can have running at one time.

While objects and threads may run fully in the symmetric multi-processor shared-memory model that Grail was originally designed as, there are various layers of protection and security that may be invoked. To elaborate, let us say that Grail expects all the processors of a system to completely share the same linear memory. An object may however be declared as secure and it's private parts will be protected by paging them away from other processes, but the addressing will still remain valid. All threads of an object are assumed to run in the shared-memory of the object and handle their own communications. On the thread level there is no security.

On the object level however, there is CORE supported supervisor-level communications and security available. Objects communicate via defined Grail messaging mechanism. Thus, these objects may exist in the same intimate SMP system, across ethernets to other close-by distributed systems or even across global internet TC/IP connections. The same interface is used at this distributed object level.

To reiterate; Grail exists at both a symmetric multi-processor shared-memory level and a distributed systems level.

Grail does use the paging mechanism. Quite a bit in fact, though it tries to keep only one such structure in existance because of the overhead involved. If an object has a security level of Secure Private User however, then steps are made to deny the objects private parts from snooping through paging. A hermetic object also is similar in the efforts made to isolate it, but these are to protect the rest of the system from it instead of visa versa.

For various reasons, compatibility being among them, Grail operates at two levels -- Supervisor which is CPL0, and User which is CPL3. Only a small part of the objects run at CPL0, but that is ok since there is little restrictiveness on the User level objects. User access to the CORE objects that run at Supervisor level are made through CALLGATES. Naturally the core and all supervisor level data and code is protected by the U/S protection mechanism.

To clarify, Grail as said, uses an absolutely flat linear memory model. The same address may be used by various objects to the same refer data or code.

TOC

CORE __SYSTEM FUNCTIONS

DESIGNER | Lewis Sellers lsellers@usit.net

The core is not a single monolithic structure. It is a set of supervisor classes, __SYSTEM being above all others. It operates beyond their restrictions as it is responsible for creating the conditions that allows them to begin functioning and is also responsible for termination of the Grail operating system and low-level communications between the symmetric system processors.

SYSTEM BASE

At the base of all memory there is an area known as the system base area. Grail does support the possibility of multi-processor systems using processors from non-related families (ie, a Pentium and a PowerPC) as is evidenced here. The struct's first variable defines the pointer bitwidth for the entire system. For all current Intel X86's and their clones this is 5, ie 2^5 or 32-bits. The base pointer Backbone points to the backbone structure itself. Processors defines how many processing units there are in the system (usually 1). Processor_Families defines how many different familes of processors are in the current system, which is almost always without exception one.

DATA ALIGN STRUCT d.8 pointer_bitwidth ptr Backbone dptr processor_familes dptr processors STRUCT END DATA END

PROCESSOR FAMILIES

If you want to know the build, or other other such information of the core objects, simply access their public data structures for this information. For example, if you want to know what processor an object is for you can thus query it. The following are currently valid strings:

PROCESSORS

Every single processor in the system looks down at the base structure and uses it to coordinate with its brethern.

CLASS STRUCT ptr ID ptr GDT ptr LDT ptr IDT ptr Semaphore ptr Capability STRUCT END

__SYSTEM.Initiate

The function that will start it all. After the core loader has aquired all the core objects and placed them into memory, __SYSTEM included, this will finish the initializations and start Grail running. If the policy is set, a session resume will be attempted first.

__SYSTEM.Terminate

Closes everything down, then performs a reboot. If policy is so set, then a session save will first be performs.

__SYSTEM.Reboot

Performs some combination of bus or triple-fault failure forcing the CPU to reinitialize and a cold boot to occur.

__SYSTEM.Memory

This functions returns the amount of memory available to the symmetric system.

In EAX the total amount of memory on-board the system.
In EBX is returned the L1 cache.
In EDX is returned the L2 cache.
In ECX is returned the CMOS.

__SYSTEM.Processor processor

Returns information to you about the selected processor. The bitwidth of the FPU is returned in EDX. If 0, then FPU is not available. If 32 then it is a i386 or i486. If 64 it is a Pentium compatible. The total number of processors in the system is returned in ECX. Usually this is 1, except for the lucky few.

__SYSTEM.MultiProcessors

Reports back the number of processors locally in the current system. Usually this is one, though on some Intel Pentiums you will get back 2 or 4. Also returns back (in EBX) the number of disparate processor types.

__SYSTEM.Date

Some processors keep thier own internal clock date. Juilian calender. AD. Year, month, day of month. hour. minute. This function is not supported by the x86 line, so it returns NULL always.

EAX (hour/minute)

EBX (month/day)

ECX Year

EDX ?

__SYSTEM.Time

Returns the current time/date in the following format. This function is not supported by the x86 line, so it returns NULL always.

EAX Seconds

EBX BH=Hours BL=Minutes

ECX Year, 0 base, A.D. julian calender

EDX DH=Month DL=Day

__SYSTEM.Elapsed

Returns in EAX the seconds that have elapsed since the core initalization.

__SYSTEM.Build

Not a very useful function. It returns build number of Grail itself. This is not the same as the objects build.

TOC

Supervisor Objects: De/coupling Intimate Objects

DESIGNER | Lewis Sellers lsellers@usit.net

Supervisor objects are those at run at the highest privilege level (CPL0) and are protected from the rest of the system through various mechanism such as CALLGATEs and the U/S paging mechanism. These include the core objects, the file-system objects and all hardware debugger and monitor programs.

De/coupling is an act that automatically occurs with supervisor objects, especially the file-system objects. Within these are a few mandatory functions that are linked directly into the core itself when they are loaded. Because of this, and the fact that they run at Supervisor Privilege Level (CPL0) with the core, it is not entirely wrong to think of them as modular enhancements/upgrades/patches to the core.

Intimate (File-system Device/Supervisor) Object Illustration

______________________
|                     |
|       CORE          |
|_____________________|________________
| Supervisor Monitor Object            |
|______________________________________|
| Supervisor file-system Device Object |<==> HARDWARE
|______________________________________|
| Supervisor file-system Device Object |<==> HARDWARE
|______________________________________|

FILE-SYSTEM OBJECTS

A file-system device object is an intimate object class which is subsumed by the __FS object class proper. There can only be one instance of a File-system Device Object. Their code and internal data is protected to a degree from failure during development stages or errant program hacks. Otherwise, if the code or data of any of these drivers were compromised there could be severe repercussions upon system integrity.

Adding File-system Device Objects

If you have non-standard file-system devices that the core doesn't directly support then you have problems. Your copy of the Grail's operating system

MUST

reside on a standard supported mass-storage device (EIDE or SCSI). If that much is true, then any other strange, non-standard device you wish to have can be used by adding the names of the file-system objects to the Policy / Navigation and moving the actual file (the object) into root space (file-system device objects are ignored if they are not in root space).

The most common file-system device objects you'll be looking for are probably those for your CD-ROM. In any event, all the externals that are listed in the PNF are loaded into secure memory and links to their code added to Grails own. Then all data about devices that was created at core load is deleted and the device detection/mapping routines are run again.

Removing File-system Device Objects

When the object is discarded, its mandatory functions are automatically decoupled.

Mandatory Intimate Functions

This replaces what was known as a code-bridge in an earlier incarnation of Grail. The mandatory public functions in any file-system device object are:

DEVICE_Acquisition DEVICE_Information DEVICE_Status DEVICE_Setup_Environment DEVICE_Setup_Device DEVICE_Operation File-system device objects do not usually have any public functions except these. They have access to a bridge of functions which are analogous to the CALLGATEs that normal objects use. These functions are called directly by the __FS object, which sits over top of it. Each of the elements in the code-bridge is simply a pointer for a CALL [absolute] a function. The actual structure of each bridge is described in the section on Objects.

 File-system Device Object Bridge Illustration

Instance   BRIDGE NAME
_________________________________________________ 
|0 | NULL                                      |  \
|..|...........................................|   |
|1 | FDC (1.44mb floppy)                       |   |
|..|...........................................|   |
|2 | IDE                                       |   |
|..|...........................................|   |
|3 | EIDE (Western Digital)                    |   |
|..|...........................................|   |
|4 | ATAPI-2 (Seagate, Quantum)                |   |
|..|...........................................|   |
|5 | SCSI                                      |   | 
|__|___________________________________________|__/
|6 | Interdeck 320A 2xCDROM                    |     
|__|___________________________________________|All of these are
device
                                             objects that have
                                    /|\      automatically had a
                                     |       code bridge created
for
                                      \----- them at core-up.

DEVICE_Acquisition

Performs the hardware level detection and evaluation of equipment that the fs device object is designed for and places the relevant information back in it's internal structure. The device structure is different for file-system interface and communications protocol interface. As you're aware, Grail Design Philosophy dictates that we don't continually test for new equipment. We assume that the devices handled by our device objects exist and attempt to acquire them.

Immediately after the core start-up, all device object detection routines are run. This can either be an active or passive scan. Active means the routine twiddles ports, possibly interfering with the device in an attempt to determine if it exists and assess it's capabilities.

Each File-system Device Object has it's own policy settings group which control whether acquisition is active or passive. Passive acquisitions means that core assumes your device information is currently correct and hasn't changed. Active acquisitions means the device object probes suspect ports looking for devices.

DEVICE_Information

Returns information about the device, such as it's memory capacity, speed, etc.

Group 0: total memory Group 1: access time Group 2: I/O transfer rate

DEVICE_Status

Returns the current mode or operation status of the device.

DEVICE_Setup_Environment

Configures and maintains the data structures on the system for the device.

mode 0: device mode 1: exterior-file-system mode 2: file-system

DEVICE_Setup_Device

Talks to the device setting it up for various modes or operations.

stream block

DEVICE_Operation

Performs the low-level read and/or write operations.

mode 0: ? mode 1: read mode 2: write

SUPERVISOR OBJECTS

The supervisor object couples directly into the core and operates at CPL0. Though it is not required that __SUPERVISOR be present to run most Grail programs, many may take advantage of a few of the functions the object makes available. In short __SUPERVISOR provides an enhancement to the core that allows it's performance and operation to be watched and reported back to normal user programs. Software developers will make the most, almost exclusive, use of this object class, but functions gauging the amount of processor time used are also excepted to be popular.

__SUPERVISOR.Monitors

Returns 4 for Intel.

__SUPERVISOR.Monitor code, n

?

__SUPERVISOR.MonitorUsage

boolean

Would you like to know how much processor power a task is using? You can toggle this function on or off at will. Regardless, it allocates a small amount of user accessable memory at coupling to which it can echo it's internal calculations.

__SUPERVISOR.MonitorUsageAddress

This function returns the memory handle to the user echoed monitor and performance variables.

DATA ALIGN CLASS STRUCT d.16 processors d.8 bitwidth dptr[processors][tasklimit] task STRUCT END DATA END __SUPERVISOR.MonitorPaging boolean

off, on

Would you like to know which active object is causing all the page misses?

exceptions override...

__SYSTEM.ResourceFault_VMMU

__SYSTEM.CatastrophicError

This function is automatically executed when a totally uncontrollable error takes place. At least, the core tries to execute it. Generally we're taking about a triple fault on the CPU here. The response is to place up a register dump and CPU error message in text mode, and ask for permission to reboot.

This function is aliased as __SYSTEM.CrashAndBurn.

__SYSTEM.FatalError

This function is usually executed when some form of unrecoverable error takes place. It does a text register dump, and then issues:

FE_INFINITE_LOOP: CLI HLT MOV AL,FEh OUT 64h,AL JMP FE_INFINITE_LOOP Hopefully, the information that is so frozen on the user/developers screen will lead to quick insight into the problem. Generally a fatal error refers to one that affects the current application only.

This function is aliased as __SYSTEM.CrashCrashCrash.

TOC

EXCEPTIONS | A/SYNCRONOUS

DESIGNER | Lewis Sellers lsellers@usit.net

"Your enemies are one of your most valuable assets.
They will glady point out a products short-comings and weaknesses
where others might not."
 --min 2/15/96, very loose paraphrase of Sung Tzu

The __EXCEPTION object class handles both synchronous and asynchronous interruptions to the normal flow of program execution. On the IBM PC clones with Intel compatible processors, there are placeholders reserved for 64 exceptions, but only 48 are currently used. The first 32 are traps and exceptions reserved by the Intel CPU. The next 16 (20h to 2Fh) are IRQs mapped by PICs (programmable interrupt controllers) 1 and 2. The remaining 16 are reserved by the core and are not used. No other exceptions are defined or allowed for use.

INTERRUPT DESCRIPTOR TABLE (IDT)

Interrupt Gate and Trap Gate Descriptor Format Illustration

 63              48 47                32 31          16 15             0
------------------+--------------------+--------------+------------------
|                 | |   |S|      |     |              |                 |
| Offset          |P|DPL|=| Type |  0  | Selector     |  Offset         |
|       16-31     | |   |0|      |     |              |         0-15    |
------------------+--------------------+--------------+------------------
                    ^  ^  ^    ^
   P=1 -------------'  |  |    '----
                       |  |
   Always present      |  '---------
                       |
                       '------------

SYNCHRONOUS: CPU INTERRUPT HANDLERS

The 32 interrupts reserved by the CPU itself can either use the default handlers as provided by GRAIL or custom ones, the pointer to which the application supplies to the core.

The default handler functions that come assembled with the __EXCEPTION class object are simply called NULL, DUMP, and DEBUG. NULL just returns from the interrupt. DUMP displays all the registers to the text mode screen and waits for a keypress before attempting to reload the current Metaphor. DEBUG puts up a text mode register dump, waits for a keypress, and then keeps going. If you want better more extensive routines, you may of course write your own supervisor level functions.

Only INT 3 (BRK) is directly supported by Grail's own assembler, since calling any other interrupt directly is virtually useless. Besides the debugger breakpoint, all other INT are assumed to be external in nature. (Grail uses CALLGATEs instead of INTs).

80386/80486

00h 	Fault 	Divide by Zero
01h 	Fault or trap Debugger interrupt (Single Step)
02h 	Interrupt Nonmaskable interrupt
03h 	Trap 	Break point
04h 	Trap 	(INTO) Interrupt on overflow
05h 	Fault 	(BOUND) Bound range exceeded
06h 	Fault 	Invalid op-code
07h 	Fault 	Coprocessor not available Processor extension not available)
08h 	Abort 	Double fault (exception)
09h 	Abort 	Coprocessor segment overrun (reserved on 486DX)
0Ah 	Fault 	Invalid TSS (task-state segment)
0Bh 	Fault 	Segment not present
0Ch 	Fault 	Stack segment overrun (exception)
0Dh 	Fault 	General protection fault (violation)
0Eh 	Fault 	Page Fault
0Fh 	Reserved
10h 	Fault 	Coprocessor error
11h 	Fault 	Alignment check
->1Fh Reserved

ASYNCHRONOUS: PROGRAMMABLE INTERRUPT CONTROLLER (PIC 1 & 2)

The entries in the interrupt table from 20h to 2Fh are controlled by PICs 1 and 2. These are programmable interrupts controllers.

GRAIL has hardcoded all 16 of these interrupt handlers, but each of them actually in turn sequentially call up a list of pointers to object functions that you can manage. Just be aware that although you can have several functions called whenever the interrupt is generated, the Grail's routines will service any hardware accociated with it before your routines.

For instance, the keyboard interrupt gets the new scan code, places it in the keyboard routine buffer, acknowledges the interrupt, THEN runs though the list of routines you wish to run. This space for this list of routines is allocated dynamically by standard __MMU functions. Each entry merely consists of the two dwords Function and ID:

STRUCT ptr Function dptr ID STRUCT END Function is a pointer to the object function that is to be called. ID is the ID of the object that created the chain entry in the first place. (1 always indicates the CORE, and NULL (0) that this chain is invalid.)

When an external interrupt is generated, then every one of the subroutines listed in the interrupts chain is executed in sequence. That is, the interrupt vector for the PICs is hardcoded into the core, but the routine that is called, executes several of your object functions that you put in it's chained list. The actual code that is executed is something similiar to: 

; ack hardware
; enable interrupts
PUSHA
MOV ESI,[irq1_handle]
MOV EBX,[ESI]
WHILE EBX
 CALL [EBX]
 ADD ESI,8
 MOV EBX,[ESI]
ENDW
POPA
IRET

This list is separate for each of the interrupts, and created by the secure __MMU functions, therefore they can not be directly altered or corrupted by user programs. Initially enough space is allocated for 4 entries, but more will be allocated as needed.

HARDWARE INTERRUPT HANDLERS (Programmable Interrupt Controller PIC 1&2)

20h *Interrupt IRQ0 Metronome (system timer) 21h *Interrupt IRQ1 Keyboard interrupt 22h Interrupt IRQ2 Second 8259A (vertical retrace interrupt on some cards) 23h *Interrupt IRQ3 (serial 2) COM2 & COM4 24h *Interrupt IRQ4 (serial 1) COM1 & COM3 25h Interrupt IRQ5 Sound card (LPT2) 26h *Interrupt IRQ6 Floppy Controller 27h Interrupt IRQ7 LPT1: 28h Interrupt IRQ8 Real-Time Clock Alarm (RTC) 29h Interrupt IRQ9 Redirected IRQ2 (cascade) 2Ah Interrupt IRQ10 ??sound card 2Bh *Interrupt IRQ11 SCSI controller card ?? (9-12,14-15) 2Ch Interrupt IRQ12 ??mouse 2Dh Interrupt IRQ13 Math Coprocessor 2Eh *Interrupt IRQ14 IDE hard drive controller card 1 (1fx/3fx) 2Fh *Interrupt IRQ15 IDE hard drive controller card 2 (17x/37x) 30h-3Fh RESERVED FOR FUTURE UPGRADES BUT NOT USED The interrupts marked by * have device object prefix-code.

__EXCEPTION.Info

This returns the number of possible known synchronous exceptions in EAX and asynchronous in EDX. Current this always returns 32 in EAX. For asynchronous there is returned 16 in EAX on Intel machines, though it possibly could return 8 on some. Take this script fragment as an example:

(sync,async)=__EXCEPTION.Info Print "Sync#",sync,"\tAsync#",async,"\n"

__EXCEPTION.Synchronous_Function

no, ptr_function

This changes the handler for the specified CPU interrupts. By decree, all pointers in Grail that are less than 1024 are considered invalid. By default, all exceptions point to a function called __EXCEPTION.Dump. It switches to textmode, displays all registers and HLTs the system.

CPU INTERRUPT
Divide by Zero
Debugger interrupt
Nonmaskable interrupt
Break point
(INTO) Interrupt on overflow
(BOUND) Bound range exceeded
Invalid op-code
Coprocessor not available
Processor extension not available
Double fault (exception)
Coprocessor segment overrun
Invalid TSS (task-state segment)
Segment not present
Stack segment overrun
General protection fault
Page Fault
Reserved
Coprocessor error
Alignment check

__EXCEPTION.Synchronous_Get

no, ptr_function

Returns the pointer of the requested CPU interrupt function.

__EXCEPTION.Synchronous_Replace

no, ptr_vector

Returns the pointer of the requested CPU interrupt, and changes it to the new supplied pointer.

__EXCEPTION.Asynchronus_ClearChain

async

Removes all links in the specified PIC IRQs interrupt chain.

__EXCEPTION.Asynchronus_ChainCount

async

Returns number of links in the interrupt chain in EAX.

__EXCEPTION.Asynchronus_AddChain

async, ptr_handler

Adds handler routine to the chained list of the specified interrupt.

__EXCEPTION.Asynchronus_RemoveChain

async, ptr_handler

Removes one of the links in the interrupt chain.

TOC

Memory Management Unit

DESIGNER | Lewis Sellers lsellers@usit.net

[min: The subjects of the data Godet and Secure User multiple Page Table protection are reserved for later gamma releases of the Grail Operating System.]

Memory management handles on the Intel Architecture are all long words (32-bit dwords) which actually are the addresses to the start of the allocated block of memory in question. Each allocated block of memory has a structure associated with it which is in a contiguous list called the MMU list or the backbone. There is a pointer to this MMU backbone at the exact base of memory. This is discussed in the __SYSTEM class section.

The MMU backbone is a collection of N elements called either vertebra or simply elements similar to the following structure. The exact structure used is considered internal and may vary from build to build as Grail evolves.

STRUCT MMU_element ptr handle ;handle is the 32bit offset to memory dptr limit dptr objectID ;ptr to object handle that created it d.8 alignment d.8 area dbit.16 quality 2 security ;supervsior, private user, public user, hermetic 1 dma ;normal, DMA dbit end STRUCT END Each of the __MMU handles is dynamically allocated sequentially to this list in memory. Obviously, the MMU can't use it's own self to resize it's internal handle list. That aside, the internal handle list looks just like any other allocated block of memory. The first entry (1) in the MMU list is always the MMU list itself. The second (2) is the base SYSTEMS area. Always. Neither can be deallocated no matter what function you use. The zeroth entry is NULL. The core sets follow after at element three (3) starting with the __SYSTEM class.

Currently each handle uses 16 bytes per allocation for bookkeeping purposes. If that's too much for you, you can always write your own higher-level memory handling functions, such as MPOOL (See the MPOOL specs).

The MMU allocates space for everything in Grail, excepting some core structures that are handled by the __SYSTEM class. Ie, all data, all objects, file handles, et al. Everything that allocates memory must go through the MMU, and as it does, so the ID of the object which precipitated this action is stamped into those elements.

SECURITY/ACCESS

An allocated area of memory has may affect the accessibility of data through the Security settings. Normally most areas are technically accessible. You may read or write data at will throughout most of memory, though what good that does is an open subject of debate. Nevertheless, for various reasons you may wish stronger more active protection of certain areas. The two prime reasons for this are protection from erratic programs being developed that have not been completely debugged and, secondly, to keep suspect and possible harmless programs from intentionally corrupting your system.

SUPERVISOR

All such areas are forced to a physical-to-linear mapping, and have their paging tables flagged so as to prevent them from being able to be swapped out. The area is also shielded under the CPL0 Supervisor paging mechanism. Page alignment is enforced.

SECURE/PRIVATE USER

This may be exactly the same as PUBLIC USER if the policy setting is so set, except that page alignment is enforced. Otherwise, some scheme is attempted whereas to deny access to this area except by the object to which it belongs. Probably this will entail the use of separate page tables for each objects privates. This may vary across processors and implemenations however.

PUBLIC USER

This is the normal state of affairs for Grail.

HERMETIC

A hermetic user object is limited in many senses. Several of the core object functions will return a NULL and perform no action. The hermetic object is limited to accessing only the folder defined by the current User Resource File. If no such folder is defined, then a hermetic object is forbidden from using any local files. Depending upon your implementation of Grail, it is likely that the rest of the system memory will hidden from the object as well using a hermetic set of paging tables.

ELEMENT ATTRIBUTES

HANDLE

The handle that is created for the memory block. As stated, the handle is also the beginning of the block of memory.

LIMIT

The size of the block of memory to allocate in bytes.

OBJECTID

The object variable points back to the __MMU handle of the object that created it. Used when an object is discarded to clean up after it. An object can usually only access a handle if it's source matches it's own.

ALIGNMENT

This byte is a reference to the beginning and ending alignments. It can be computed by 2^(n-1), so an ALIGNMENT of 1 means 2^(1-1) or 1 byte. An ALIGNMENT of 4 means 2^(4-1) or 8 bytes alignment.

0 default (default to byte) 1 byte 1 2 word 2 3 dword 4 5 paragraph 16 13 page 4096

AREA

Specifies what the area is used for:

0 device (MMIO) 1 data 2 stack 3 code 4 object structure 5 thread structure 6 task semaphore 7 task stream 8 file system structure 9 Godet structure 10 Godet data Device is an area that is mapped by hardware devices and is beyond the __MMU's allocation schemes. Areas that are typically denied are the video and text memory spaces.

SECURITY

If Supervisor security is set, then only core routines have access to this area (read or write). Any area that is so allocated also has it's linear memory forcibly paged in and all the pages occupying it's space have bit 9 of their PTE set to 1, meaning that it can not be swapped out under any circumstance. This both defeats possible crashes, thrashing and security risks. Secure supervisor areas should be as small as possible.

DMA

The DMA flag indicates that the entire block of memory is located at a physical address below 1mb and therefore accessible by DMA controllers (drives and sound cards, etc).

THE PAGING MECHANISM

Paging is completely transparent to users and tasks. For non-core functions, the only way to ensure that a block of memory is mapped linear-to-physical is to use __MMU.AllocateLinear. For the most part this is only used by S/VGA graphics routines to block off 0A0000h to 0AFFFFh. etc. Supervisor memory blocks will have only linear to physical translations.

Paging requires 4kb base to hold the page directories, plus 4kb per 4mb of memory in the system. This applies to REAL memory, to virtual swap-memory, Godet-memory and SVGA linear-mapped memory.

Intel Page Directory Entry (PDE) Illustration

31 21 12 11 0 -------------------------------------------------------------------- | | | | | | | |P|P|U|R| | | Page Frame Address 31..12 of PTE Group |Avail|0 0|0|A|C|W|/|/|P| | | | | | | | |D|T|S|W| | -------------------------------------------------------------------- *486 only * * A Accessed PCD Page Cache Disable PWT Page write-through U/S User/Supervisor (always 1 in Grail) R/W Read/Write P Present

Intel Page Table Entry (PTE) Illustration

31 12 11 0 -------------------------------------------------------------------- | | | | | | | |P|P|U|R| | | Page frame address 31..12 of linear mem |Avail|0 0|D|A|C|W|/|/|P| | | | | | | | |D|T|S|W| | -------------------------------------------------------------------- *486 only * * D Dirty A Accessed PCD Page Cache Disable PWT Page write-through U/S User/Supervisor R/W Read/Write P Present If we held the entire page directory/table structure in memory it would take 4MBs. And if we had a separate address space for each task, that would be 4MB per task. Grail does neither though. Each 4MB is split into one entry in the Page Directory (a PDE). These 4MB blocks have 1024 page table entries (PTE) which control a 4096 byte page. Thus if you have a 8mb system, you must have two valid PDEs and their supporting 1024 PTEs each.

The Page Directory requires 4kb and is not optional. Each 1024 set of PTEs are required only if you plan on using that address space. Any PTE marked as U/S = 0 under Grail WILL NEVER be swapped out in a virtual-memory mechanism. The area under it is also protected from access by application programs.

SYMMETRIC MEMORY ACQUISITION

When you wish to allocate memory through the MMU it must look through the backbone for an appropriate section of free memory. In doing so, it uses a few simple structures to help decide which section of memory is best.

FRAGMENTATION

At one time Troy Selyem (tselyem@slonet.org) and I were mulling over different memory fragmentation solutions for EOS including how Macintoshes handle the problem. I eventually decided that for Grail it would produce the fastest code to not allow automatic relocation... though I still occasionally toy with the idea of such things.

It is yet to be seen whether objects will produce greater or lesser overall fragmentation.

MEGABYTE PHASE

The first phase contains sets of bytes, one for each megabyte of main memory. This structure holds eight bit-sized variables: DEVICE, DMA, PRIVATE, FULL, FREE, CLUTTER, PHYSICAL. FULL gives immediate response indicating whether the megabyte is completely full already or not. UNUSED states that the megabyte is completely unused is might make a good candidate for long contiguous streamed data.

 PHASE 1 Megabyte Byte Illustration

 ___________________________________________________________
| Device | DMA | PRIVATE | Full | Free | Clutter | Physical |
|________|_____|_________|______|______|_________|__________|
DEVICE

The memory area has devices mapped into it.

DMA

The area supports DMA transfers.

PRIVATE

The area has secure areas.

FULL

Area is full.

FREE

Area is completely free.

CLUTTER

There are many clusters of small data allocations here.

PHYSICAL

Entire area is in physical memory

The First Phase structure is small and used to rapidly scan the entire memory for a "general" area that may be suitable for the kind of area we wish to allocate.

PAGE PHASE

The second phase gets down to specifics and maps memory on a page per page basis. Each page has a boolean pair of flags. USED simply indicates whether the sector is used or not. DEVICE indicates the page is allocated to a device.

PHASE 2 Phase Pair Illustration

___________________ | USED | DEVICE | |_________|_________|

SCOPING MEMORY ALLOCATION (GARBAGE COLLECTION)

When an object or task removes itself from memory __OBJECT runs across the backbone freeing up any memory allocated by that object. Generally it makes two levels of passes. The first one looks for other objects or tasks called up by the original object and issues a PriorityOverride against them. The GC must then traverse the backbone looking for any data/objects, etc that belong to those objects. And so, down and down the tree until you reach an object that has not called up any other objects.

The second level of GC is simply deallocating everything else (data, not code).

The scoping mechanism of Grail is a mixed blessing. Even the unskilled or talentless programmer can get away with not deallocating any memory s/he requests since Grail will probably clean up after them. The downside of course is that removing an object from memory is probably not the fastest thing you've ever witnessed. However, as long as you don't try to write inner loops requesting and discarding objects the hit should be minimal.

USER FUNCTIONS

__MMU.Allocate

memory, alignment, quality, acquisition

Requests up to 2^32 bytes of memory using the memory acquisition algorithm. Memory allocated is normally byte aligned, but you can specify more performance friendly alignments.

Alignment
0default (non-aligned, default to dword on the Intel processors)
1byte 1
2word 2
3dword 4
5paragraph 16
13page 4096
Quality
0Normal (default)
1DMA

If the quality is DMA then the function allocates memory from a section that is hardware addressable by the DMA electronics. Normally DMA user space exists only up to the first physical megabyte of RAM so this is a precious resource. Returns NULL if insufficient DMA memory available. Notice that DMA memory is excluded from virtual-memory swapping.

The acquisition parameter sends a suggestion to the memory acquisition algorithm on how the memory will be used or change at future times. These are the currently recognized values:

0 Default

The acquisition algorithm can place the memory however it wants. 1 Static The size of the memory area will not change. We can allocate a very snug section of memory. 2 Shrinkage The memory area is expect to shrink. 3 Change There will be some changes going on in size. Find some space with room to grow. 4 Steady Growth Steady growth should be expected.

5 Streaming Growth

A great deal of change should be expected. This area is receiving a stream of data of unknown size. Find a large contiguous tract of memory. Memory is in EAX, alignment in EBX, quality in ECX and acquisition in EDX. The handle, if successful, is returned in ESI. Otherwise a NULL is returned.

__MMU.Resize

handle, size, lock

Allows you to specify a new size for a block of memory. This can be smaller or larger. If there is enough contiguous space available, then the handle will remain the same. But if you request more space and it is only available in some other part of memory, then the handle may change. Remember this, least problems occur.

If lock is 0 then if there is not enough contiguous memory to enlarge the current memory block, the function will look elsewhere and change the handle (address) if it can. When lock is 0 and there is not enough contiguous memory starting at the current address, the functions fails with a NULL.

Handle should be passed in ESI, size in EAX and lock in EBX. This function returns either the new handle (in ESI) or a NULL if it fails (in EAX and ESI).

__MMU.Free

handle

Removes a previously requested region of memory from the MMU backbone. The space is now free for other use. Only affects __MMU.Allocate. Does not affect any handles with linear or supervisor flags.

Handle is pased in ESI and result returned in EAX.

__MMU.Memory

In EAX the total amount of virtual memory, in EDX the maximum physical memory.

__MMU.MemoryUsed

Returns the total amount of memory used by everything (in EAX), and for DMA only (in EBX). ECX returns with the total amount of supervisor space that is occupied. EDX contains the total bytes currently swapped out to the swap-file.

__MMU.MemoryRemaining

Returns the total amount of free memory (everything, including DMA space and swap) in EAX. Also returns in EBX the total DMA viable space remaining in the system. ECX has the remaining physical memory, ie that discounting swapable memory. When ECX drops to zero system performance starts to degrade.

__MMU.Handles

Returns (in EAX) a count of all the handles currently being used system-wide and (in EDX) the number of those that are being used by the current object, and (in ECX) those being used by the current object and all it's children.

__MMU.Verify

handle

Verifies that handle is being used is the base of a memory block by scanning the MMU handle list. Returns negative truth in EAX.

__MMU.Info

handle

By supplying the memory block handle, information on the status of the area is returned. If this fails, a NULL is returned. Otherwise you receive a pointer-width value in byte components. Byte 0 is alignment, Byte 1 is area, Byte 2 is security, and Byte 3 is quality.

__MMU.InfoAddress

address

If you supply the pointer into the backbone for the description of an allocated memory area, this function will return back to you the pointer to the memory area itself (in EDI) and the limit of the area (in EBX). Returns flag structure in EAX.

__MMU.InfoHandle

handle

The inverse function of __MMU.InfoAddress. By giving the address an area of allocated memory, you will get a looked-up pointer (EDI) to the element in the backbone describing the area and the limit of the area (in EBX). Returns flag structure in EAX.

__MMU.Walk

handle, direction

You feed this function with a handle and it will return the next handle in the backbone list (in EAX). Note that if the current object does not have access to the next element, it will be skipped, and so on till you reach an element you can access. Direction of 0 is backwards, while 1 is forwards. A handle of 0 always points you towards the first element of the MMU backbone which is both the multi-processor backbone area and off-limits to user objects.

__MMU.AllocateLinear

memory, memory

Used to declare places in memory that are off-limits to allocation schemes. Typically this includes all areas that are mapped by hardware devices including the SVGA linear buffer and MMIO. This function does the same as __MMU.Allocate in actuality, except that it marks the area as belonging to a device and uses the start and stop addresses of the physical memory area denied.

__MMU.DeallocateLinear

handle

Removes __MMU.DeallocateLinear allocation of physical memory space. Generally, this function is hardly ever used.

__MMU.Map_PageGroupBitmap

destination_pointer, arg

This is an extremely useful and comprehensive function for monitor programs. It scans the backbone and generates a binary stream that contains status data of the entire of memory. This would be difficult otherwise because most objects would have parts of the backbone restricted from them. This function will scan the entire thing inclusive... and, since it does so directly, quite quickly too.

If the destination pointer is NULL then the structure is not created. It's size in bytes is computed and returned (in EAX) so that memory can be allocated for it. In all other cases, excepting the few map types that return single computed variables, all generated structures are created at this pointer area. Also return (in EBX) is the size of each element in bits.

Arg is EAX. Destination_Pointer is EDI.

Creates a byte-element map of all pages in the system.

STRUCT page_map dbit.8 byte 1 supervisor 1 hermetic 1 dma 1 device 1 virtual 1 used 1 object code 1 object data dbit end STRUCT END

__MMU.Map_Handle

destination_pointer, arg Scans the backbone and creates a map of n elements describing how memory is currently used. This function uses arg to determine how many divisions to make. For instance, on a 32MB system, using an arg of 32 would create a map of 32 elements, each 1MB in size. Such a rough map may or may not be useful.

STRUCT handle_map dbit.8 byte 1 supervisor 1 hermetic 1 dma 1 device 1 virtual 1 used 1 object code 1 object data dbit end STRUCT END Each element in the map determines the boolean truth by preponderance, ie if at least %50 of the memory has such a quality.

__MMU.Map_HandleAccessBitmap

destination_pointer, arg

Creates a bitmap showing whether the current object has access to each MMU element or not.

__MMU.Map_HandleOwnerBitmap

destination_pointer, arg

Creates a bitmap showing whether a MMU element was created or part of the current object.

__MMU.Map_HandleAlignmentExcess

destination_pointer, arg

This generates a list of pointer-width variables, one for each MMU entry, which has the number of bytes allocated but not used for the element.

__MMU.Map_HandleDistribution

destination_pointer, arg

You can use this function to create a list of all the MMU elements and how much of the total memory they are each using in comparison. The arg is used as a modulo. Most commonly arg is 100, thus giving you back a percentage use map for all elements. The size of each element in the map will be pointer-width, unless arg is 256 or less, wherein each element will be byte size. Just ask the function.

__MMU.Map_UsedLength

destination_pointer, average, quality

Creates a list of all MMU elements giving the length of each. If the average variable is true (1) then the list consists of only one element which is the average of all the elements that would normally be in the list. Returns the number of elements. If quality is NULL (0) then all memory areas are considered. If it is 1 then only DMA is looked at.

__MMU.Map_FreeLength

destination_pointer, average, quality

Creates a list of all MMU elements giving the length of each. If the average variable is true (1) then the list consists of only one element which is the average of all the elements that would normally be in the list. Returns the number of elements.

SUPERVISOR FUNCTIONS

The following functions are available only to other supervisor objects. They are not CALLGATEs. These functions will completely ignore you unless your object security level is Supervisor.

__MMU.SupervisorAllocate

memory

Attempts allocate secured memory for supervisor object functions. Only a __MMU.SupervisorFree function can release this memory. Depending upon core implementations, the core will probably align this data and/or allocate more space than you've requested automatically in an attempt to isolate it. For instance, the Intel IBM PC platform uses 4096 byte pages, and will page align the space, and only give up RAM in 4k blocks so as supervisor/user pages do not mix data.

__MMU.SupervisorResize

handle, memory

Resizes the secure memory area.

__MMU.SupervisorFree

handle

Releases memory allocated by __MMU.SupervisorAllocate.

TOC

Virtual Memory Management Unit

DESIGNER | Lewis Sellers lsellers@usit.net

The Virtual Memory Management Unit (VMMU) is used to provide both Virtual Memory for systems when they are low on RAM memory and Mass Storage Caching when there is a surplus of memory. Virtual Memory is switched on shortly after core start-up if there is a swap-file. When VMMU virtual memory managing is enabled then the size of the swap-file is counted as the size the system actually sees when looking at how much memory it has. The swap-file is mapped under real memory. When Grail is first started, the pages are organized in a linear to physical contiguous fashion to the end of real memory, and the swap buffer is wiped to NULL. (As per security specs, so is the unused real memory, FYI).

REAL <--> SWAPFILE VIRTUAL MEMORY RELATIONAL ILLUSTRATION

0 4 8mb |-----------------------------| | Real memory | |_____________________________| /\ || \/ |------------------------------------------------------| | VMMU Swapfile | |------------------------------------------------------| 0 4 8 12 16mb To clarify, say we have a 4MB system. Then we create a 12MB swap-file. As far as our programs can tell, we have a machine with 12MBs of RAM.

----------------  12mb-----
|       /\       |         |
|      /||\      |         |
|       ||       |         |
|       ||       |         |
|       ||       |         |
| Virtual Memory |         |
|       ||       |         |
|................|         |
|                |  4mb--  |
|                |      |  |
| Real Physical  |      |  |
|     Memory     |      |  |
|                |      |  |
|________________|  0mb----|

Everything will work just fine until the first 4mb of space are allocated for one thing or another. But when we start trying to address beyond that we will start grabbing real memory to stand on for our illusionary address space. That is, we will look for a page of real physical memory that is not used much, and use that for our page frame address, then mark the page in and read in the data from our swapfile to that location. Whatever page was using that memory before will first of course have it's contents swapped out to the swapfile at the logical place and then marked not present. 

Intel Page Table Entry (PTE) Illustration
31                                       12 11                     0

--------------------------------------------------------------------
|                                          | | | |   | | |P|P|U|R| |
| Page frame address 31..12 of linear mem  |Avail|0 0|D|A|C|W|/|/|P|
|                                          | | | |   | | |D|T|S|W| |
--------------------------------------------------------------------

I know that looks as little confusing, but VMMU operations is not as hard as they look. The swap-file is directly mapped from swapfile by using the page frame address as a lookup, and visa versa. The page tables naturally keep track of which 4096 byte pages are in memory and which are on disk. For each of the 4096 byte work pages, a flag is kept informing the swap routine whether it is dirty or not. That is to say, has it been written to? If so, then it must be written out to the swap file.

How does the core know which pages of real memory to grab when paging in from the swap file? Good question. If the core is smart enough to grab only linear areas that have low usage on them it can significantly reduce thrashing in some situations. Unfortunately we only have 3 bits in which to quantify usage rates. The first bit is used to determine whether we can swap the page or not. If the bit is 1, then it will never be swapped and it has a direct linear to physical addressability: Ie, it is memory that is used by the core and drivers for SECURE and core operations.

This leaves us with two bits for the paging scheme.

THRASHING BIT GROUP 11 10 9 ---------- | | | | | | | | ---------- |___| ^- SECURE Bit \ \-- THRASH LEVEL Bits 0=virgin/low 1=mid 2=high 3=thrash fault Operation isn't too complicated. All pages start out as virgin (ie, 0). A pointer, the thrash-point, is initialized to the beginning of the page tables. Each time there is a page fault, the PTE at the thrash-point, is checked to see if the page is secure. If it is, it is ignored, and the thrash-point moved to the next PTE, and it is checked, etc. If the PTE we're looking at is ok to use, we check the thrash-level.

If it is 0, we immediately use it (that physical page). If there are no 0 thrash-levels, then we look for 1s, if no 1s, we look for 2s, if no 2s... we issue a __SYSTEM.ResourceFault_VMMU.

Assuming we can ever find a non-secure PTE, with a thrash-level less than or equal to 2, we will use it, as we said. We will also increase it's thrash-level by 1, topping off at 3.

To counter the fact that eventually all the pages will reach a thrash-level of 3, and then issue CORE_ResourceFault VMMU, we use a floating adjustment variable (TFAV). It starts out at 0, but is increased by one everytime a thrash level is found to be saturated (ie, can't find any PTEs with thrash-levels of 0,etc).

Currently the size of the swap file is limited to 4GB, or 2 to the 32 power bytes solely because that is the upper limit of flat addressability. However, Grail is dying to evolve to 64-bit addressability to be in sync with it's default 64-bit file system. As soon as the CPUs move up, so will Grail. See __SYSTEM.AddressSize, etc.

To set or change the size of the common virtual memory swap file you must use __POLICY.VMMU_Size. Though it does accept any value, currently the size if rounded up to the nearest megabyte.

__VMMU.Create

Creates the VMMU swap-file in the primary root binding along side of the core, and NULLs the entire thing.

Uses the size as defined by __POLICY.VMMU_Size.

This function also turns virtual memory swapping on, if it wasn't already. If you need to change the swap-file space you must first delete it (__VMMU.Remove) and then recreate it in the new size.

__VMMU.Uncreate

Turns off VMMU, purges system, and deletes the swap-file (in that order).

__VMMU.Verify

Verifies that the swap file exists and is properly created. Boolean response in EAX.

__VMMU.Space

Returns current size of the swapped out data in EAX and the swapped in data in EDX. The total size of the swap-file is returned in EBX.

__VMMU.Swaps

Returns count of page swaps since last time VMMU was enabled (ECX), the count of writes to the swap file (EAX), the count of reads from the swap file (in EDX), and the size of each page (in EBX). You can of course compute data transferred by multiplying by the number in EBX (which should always be 4096, but to be on the safe side....)

TOC

Object-Orientation Background

DESIGNER | Lewis Sellers lsellers@usit.net

The fundamental design of Grail is decidedly object-oriented. All programs for Grail come in object modules, which are, when you get down to it, essentially class objects (a grouping of common code functions, data and system control information structures). You should reference the Grail Assembler specs for more information, but the basics follow.

Object modules will be one of four types: Supervisor, Device-Specific, Platform-specific or non-platform specific Script. Confused? Alright, here's the hierarchy again:

supervisor object device object (platform) object script (object)

The supervisor objects are either part of the core-set, file system device objects or supervisor add-ons for debugging. All supervisor essentially upgrade or enhance the core-set in one way or another. Device objects are those that talk directly to the hardware and support the required objects classes as defined by the Grail specs. They however are not critical to the operation of Grail. Most of the program you will run will be platform specific objects -- object compiled explicitly for your machine. And everything else is script -- code and data that is usable on any Grail platform.

Those object classes that are part of the core set must have a capitalized prefix starting with two underlines. Those that are considered essential to normal Grail operations must have a capitalized prefix (object name).

THE OBJECT TYPES

Supervisor Objects

Core Objects

The following are core-set object classes:

__NULL __SYSTEM __POLICY __EXCEPTION __MMU __VMMU __OBJECT __TASK

File-system Device-dependant Object

File System Interface __FLOPPY __IDE __EIDE __SCSI Virtual File System Interface __SERIAL ;serial null-cable __MODEM ;modem com-port __ETHERNET

Device-dependant Objects

A device object is a module designed to work with only one specific piece of hardware. When the module is loaded, the object is added to the list, and the code-bridge and ...

There are two types of device objects: Those relating to the file system and all others.

These device and platform object classes are mandatory:

MONITOR VIDEO GRAPHIC DIGITALAUDIO GMIDI FMAUDIO FS SCRIPT ASSEMBLER

Platform-dependant Objects

An object that does not reply on (or drive) a specific hardware device but who's code is binary compatible for only one platform is a platform object.

Cross-platform Objects (SCRIPT)

This not a module. This is a plain text file with the Grail script pseudo language which provides for a fully cross-platform program. When a script is loaded in it is quickly converted to a binary platform object, so it is slower when loaded, but it should work fine, without any change to the code on every platform Grail runs on.

PREDEFINED FUNCTION NAMES

?

~Creation

~Destruction

~Main

Objects with Main functions can be run as independent tasks (programs), but they don't have to be. You can load such an object into memory just use it's public functions without actually running the program.

~Polymorphism?

Does this object support polymorphic transformations? Aside from returning back a value indicating whether the object can or not, this also monitors the "powerdown" to a transitional state.

~Polymorph

~Polymorph2

~NULL

Normally ....

Mandatory Intimate Functions

This replaces what was known as a code-bridge in an earlier incarnation of Grail. The mandatory public functions in any file-system device object are:

~DEVICE_Acquisition

~DEVICE_Information

~DEVICE_Status

~DEVICE_Setup_Environment

~DEVICE_Setup_Device

~DEVICE_Operation

THE NATURE OF OBJECTS

Object Creation/Destruction

When any instance of an object is created it's creation function is automatically called. And naturally, when it is discarded it's destruction function is called. Typically, these functions, if they do anything, call into memory the objectsthat they themselves use, and then release them. It is curious to note that running a very high-level, abstract application can cause a curious chain-reaction of object-loading... one object is called, that calls up a dozen others that it uses, and those may call up a few others.

Object Function Returned Variables

Grail functions are designed to pass back multiple variables and structures of theoretically unlimited size. It's typically bad form to pass back more than a couple dozen bytes however. The data will either come back in CPU registers or streamed to a block of memory.

Unless otherwise stated, you can assume the following about the conditional return variable.

-2 Induced or artificial failure -1 Failure 0 Non-commital (neutral, no comment on whether success or failure.) 1 Success 2 Induced or artificial success type, number, variable_width, arg............

This defines what is to be returned when the object function ends. The type illustrates the form the data will take:

0 none 1 conditional 1 variable 2 fixed-variables 3 mixed-variables 4 structure A fixed-variable type indicates that the variables are all the same size. A mixed-variable type indicates that the list of variables have different sizes............

When variables are returned, the number parameters tells how many of them are to be.

THE OBJECT LOAD/UNLOAD PHASES

How are objects modules loaded?

When you request that module be loaded, the OS looks for it by first looking in the current tasks folder for the module file. If it can not be found there, then it looks in the policy / navigation file for a listed location. If it still can not be found then it returns an error, unless you've set the NULL_Object policy. In that case a NULL object is automatically created AND and error returned.

A module is a collection of code, data, lists of how the previous two link together and their names, as well as lists of debugging and info data. A module can be loaded in and then granted status as an independent running process (ie task). A module can also be loaded into memory to be used as a common set of functions by other objects or task objects.

To load an object into memory, an internal Load_Module function allocates space for a module then loads it into that area, doing relocation fix-ups as needed. If it is detected that the module is a device object then it is appropriately registered.

How are objects modules unloaded?

The Unload_Module function removes an object module from the task list, if it is found there. Then it navigates the __MMU memory block handle list (the backbone), freeing all blocks which have the same ID as the module being unloaded. It also removes all PIC chains created by the module by checking the chain IDs.

So what is loaded into memory?

Essentially everything, though the structures will change slightly. Some of the INFO, DEBUG and PARAMETER data may or may not be loaded depending on your current language and optimization policies.

SECURITY: DO OBJECT HAVE ANY?

Yes. They can make themselves somewhat secure against each other or hermetically contain suspect objects. There are in all four levels of security, though user object have only three available. The top-most level is supervisor security. This is reserved by the core and means that these objects run at CPL0 and hide behind supervisor page security.

In the middle there are two levels called secure user and public user security. Secure objects take more overhead than public, and thus, while more secure, are more bloated and slower. The most noticeable attribute of secure objects is that they make their private areas truly private through the paging mechanism.

Lastly there is hermetic user security. An object with hermetic security status is forcibly contained, limited in it's usage of filespace and system resources. Hermetic objects typically come from the 'net. They can not create or destroy folders or any of the like. They can open/read/write/close data files, but only those in it's own current folder and also the hermetic_folder folder if it exists. They can only change or create the associatives of a user profile, not delete them. They also can not use scripts, only precompiled objects. A hermetic object can be a script however.

Security Levels
Supervisor
Secure User
Public User
Hermetic User
An allocated area of memory has may affect the accessibility of data through the Security settings. Normally most areas are technically accessible. You may read or write data at will throughout most of memory, though what good that does is an open subject of debate. Nevertheless, for various reasons you may wish stronger more active protection of certain areas. The two prime reasons for this are protection from erratic programs being developed that have not been completely debugged and, secondly, to keep suspect and possible harmless programs from intentionally corrupting your system.

SUPERVISOR

All such areas are forced to a physical-to-linear mapping, and have their paging tables flagged so as to prevent them from being able to be swapped out. The area is also shielded under the CPL0 Supervisor paging mechanism. Page alignment is enforced.

SECURE/PRIVATE USER

This may be exactly the same as PUBLIC USER if the policy setting is so set, except that page alignment is enforced. Otherwise, some scheme is attempted whereas to deny access to this area except by the object to which it belongs. Probably this will entail the use of separate page tables for each objects privates. This may vary across processors and implemenations however.

PUBLIC USER

This is the normal state of affairs for Grail.

HERMETIC

A hermetic user object is limited in many senses. Several of the core object functions will return a NULL and perform no action. The hermetic object is limited to accessing only the folder defined by the current User Resource File. If no such folder is defined, then a hermetic object is forbidden from using any local files. Depending upon your implementation of Grail, it is likely that the rest of the system memory will hidden from the object as well using a hermetic set of paging tables.

THE NAMES OF THINGS

Title (Filename) / Instance Name

The filename is the typical name of the file as is stored in the associative branch. The instance name is usually the exact same thing as the filename. Instance name is part of the object structure in the data branch however.

Subtitle (Subject) / Object Class Name

Class names are things like __SYSTEM, __OBJECT, TIFF, PNG, etc. The name of a class of functions is in the data area and part of the object structure.

Function Name

Simply the name of a function of an object. Part of the object structure.

Object Pointer Sizes are dynamic (Infra-family compatibility)

All pointer types or pointer-related types that resides in an object modules file are kept track of in two pointer lists.

Listen up, because this is very important. Grail is designed so that it will ...

For example, say that you write several objects for the 32-bit PowerPC 601 running Grail. Then you upgrade to the 64-bit PowerPC 620. All you must do is get the new 64-bit core for the processor and the few supporting device objects. When you load your 32-bit objects or applications on the 64-bit machine, Grail will automatically promote all the data and code to 64-bits, as well as perform some data aligning.

The equivalent assembler instructions that are effected are:

PTR

DPTR

DB

DW

DL

JMP

CALL

Bcc

Thus, it is entirely hoped that you may write a 32-bit application today and still have it run on the 64 and 128-bit machines of the next decade. Software will have a greatly extended lifespan in comparison to other operating systems.

Simultaneous Multiple Architecture Provisions

Yes. Some thought has gone into providing not only intra-family compatibility, but allowing you to bind code into an object module for more than one processor family. It is thus possible to compile an object that will run on a x86 processor, an MC'60, a PPC601, and a PP620. Grail will query the object to see if it has code for the current processor, and if so, choose that.

Note that, since Grail does provide for the possibility of running in a system using processors from different families, different code may be loaded depending on which processor is selected.

The Stack

Each active instance (task object) has it's own stack space. All object functions active instances call use this stack space. The size of the stack space is defined in the object module format. This size...

OBJECT CACHING

Objects are cached. By that, we mean that if there is enough free non-virtualizing memory available on your system, then an object that is discard remains in memory, even though it is otherwise forgotten (except by the backbone). If the object is requested again.....

THREADS

?

HANDLING OBJECT/FUNCTION-LOAD FAILURE

Handling the situations that occur where a object function is not available or where whole objects are missing is generally up to the object that needs these services in the first place. This is quite different than from before the massive 0.6 revision of the core and the introduction of a fully object-oriented operating system.

It is generally suggested that you fully list the name of the functions that are missing and the objects associated with them such as:

POLYMORPHISM

Grail supports the ability of objects to change almost at will. Specifically this means that one object can be loaded into memory replacing one currently in memory. All pointers to functions are updated automatically, allowing other applications that may be using the object to continue on as if nothing happened. This is, for your information, how metaphors are changed while you're running Grail.

See the Function Call List for more information.

FUNCTION CALL LIST

Each object has a list of pointers to all its functions situated in the Object Module Header. The function pointers are accessable through the __OBJECT.Function CALLGATE.

data test: text "TEST.TestMyFunction", NULL data end code mov esi, test CORE __OBJECT.Function call [edi] code end After the CALLGATE function __OBJECT.Function we get a value in the register EDI. This is a pointer into a list of pointers to the function you want. To call the function you must do an indirection call such as

call [edi] The reason indirect calls are used is solely to allow polymorphism. If you switch an object for another, the pointer to the function will remain valid. It will almost assuredly point to someplace different in memory, but it is guaranteed to point to the new function or to the ~NULL function.

The Function Call List is dynamically generated. A static pointer list would be fine if your polymorphed objects all had the same functions in the same order, but the likelyhood of that happening would be low even if you tried to keep the functions even. Once a pointer for a specific function name is given, that pointer in the list always belongs to that function, even if the function disappears in the new object.

THE TASK (OBJECT/THREAD) LIST

Obviously Grail keeps a list in memory of all the objects being used, but where? Pretty simple. It keeps them listed in the MMU with everything else of course. Off hand, about the only list of consequence in Grail that is not kept track of by the MMU is the task list. That's separate on all platforms.

1. Thread

OMF Thread Header

2. Object

OMF Header

PROCESSOR SECTION

PROCESSOR SECTION

PROCESSOR SECTION

TOC

Object Format

DESIGNER | Lewis Sellers lsellers@usit.net

[min: Complicated isn't it? :) ]

For Grail there is but one, and only one format that executable code dwells in, and that is the Object Format. There are not .OBJ, .EXE, .DLL or .VBX files as most people understand them. 

 _________________
| OBJECT HEADER   |
|_________________|
 |  ______________________
 |_ | PROCESSOR SECTION    |
 | |______________________|
 |  ______________________
 |_ | PROCESSOR SECTION    |
 	|______________________|

PERSISTANCE

All objects possess the ability of persistance. They may retain data that is changed from session to session. This is in ADDITION to any of the transitory associative branch data. Object persistance directly changes structures within the object itself, if it all possible. Refer to the FLUSH commands of discussed in this document and the Grail Assembler docs.

THE OBJECT HEADER

All objects have a relatively simple header structure. It uses only three pointers to strings (which come directly after the header). 

DATA ALIGN
CLASS STRUCT object_header
d.8 pointer_bitwidth
d.8 task_type
STRUCT Internal_build
	d.32 Object
	d.32 Structure
STRUCT END
STRUCT Grail_Build
	d.32 Minimum
	d.32 Assembled
STRUCT END
ptr object ;title
ptr instance ;subtitle
d.64 Certificate_ID
ptr Certificate_Verfication
UTCAD_TIMECODE Conception
UTCAD_TIMECODE Assembly
d.8 objecttype
d.8 Instantiation
d.8 Security
d.8 Legality
d.8 Quality
d.8 Origin
d.8 Hierarchy
d.32 language
d.32 character_set
ptr ptr_language_name
ptr ptr_character_set_name
ptr NEXT
STRUCT END
DATA END

POINTER_BITWIDTH

The importance of this variable can not be stressed enough. It defines the bitwidth that the object is expecting for all it's pointers. The actual number is computed just as FILESPACE_WIDTH in the file-system document with 2^pointer_width giving the true bitwidth of the for a pointer. See the table below. The default for the Intel CPU is currently only 32-bits.

Width Actual Bitsize

0 1 1 2 2 4 3 8 4 16 5 32 <-- default 6 64 7 128 8 256 Bit 7 (ranging 0 to 7) is used to denote whether the pointer_width is locked. If bit 7 is TRUE (1) then Grail will return an error if you try to load an object with a different pointer_width than your current architecture (ie, trying to run 32-bit PPC601 objects on a 64-bit PPC620).

TASK TYPE

0=object

1=thread

MINIMUM GRAIL BUILD

The minimal Grail build as specified by the source code. It is assumed that if you try to run a program on a build lower than this that it won't work.

ASSEMBLED GRAIL BUILD

The Grail OCI set build number under which the object was assembled. Looked up by asking the core your compiling under what it's revision is. Supposedly this is the highest known build under which the (commercial) object should work, but not necessarily valid if assembly just occurred via a script on your end.

INTERNAL OBJECT BUILD

The revision number for the object itself. This number essentially only matters to the object and it's friends. Mainly for reference.

INTERNAL STRUCTURE BUILD

The structure build number lets other objects know if the public structures have changed, and thus how they should be accessed.

OBJECT NAME/TITLE

The name of the object class.

INSTANCE NAME/SUBTITLE

The name of this instance of the object. Usually just a number. Sometimes a specific name.

OBJECT TYPE

0 Supervisor Object 1 Device Object 2 object

INSTANTIATION

0 Unique 1 Multiple

SECURITY

0 Supervisor 1 Secure User 2 Public User 3 Hermetic User

HIERARCHY

Defines the objects place in the general hierarchy of things.

0 unknown 1 core 2 device object 3 support function 4 group function head 5 application head 6 metaphor application head test

LEGALITY

This defines the legal stance of object/application.

0 Unknown 1 Private 2 Public Domain 3 Freeware 4 Shareware 5 Commercial 6 Site Commercial 7 Restricted

QUALITY

Defines how finished of a product the object is:

0 Unknown 1 In-house 2 Alpha 3 Beta 4 Wide Beta 5 Gamma 6 Final Gamma

ORIGIN

What are the general origins of this object?

0 Unknown 1 OEM 2 Vendor 3 Grail Team 4 Grail Associates 5 Hobbyist (3rd party) 6 Professional (3rd party)

CERTIFICATE ID/VERIFICATION

Every version of the commercial assembler has a specific Certificate ID. This is stamped here. Only the commercial assembler by The Minimalist Group's Grail Team or by other authorized persons and/or companies should use this area. The freeware assembler always places a 0 in this area. All third-party assemblers should use 1. Ie:

0 Freeware Assembler 1 Third-party It is also optional to have an encrypted verification certificate stream follow. Typically this some form of public-key encryption data, but this is not specified. Third-party assemblers can use this area pointed to for whatever they want.

CONCEPTION TIMEDATE

Your assembly or higher level source will have a conception timecode (or should). This is compiled into the object module format. By conception, we mean the earliest date at which the object or it's predecessors were conceived.

ASSEMBLY TIMEDATE

The timedate that the object was assembled/compiled. Looked up by asking the core for the current timedate

LANGUAGE

This defines the language/character the object was designed under. That is, all the public function names, etc. can be used to flag someone that they can not read some aspects of the program. Bit width will either be 8 for ASCII or 16 for unicode. The type will be 0 for undefined, 1 for ascii or 2 for unicode.

PROCESSOR SECTIONS

Data is grouped separately from code, and may further be divided into Initialized and Uninitialized Data Blocks. Data in each structure, if it contains even just one initialized variable, is included in the initialized data block. Uninitialized data in a STRUCT is actually filled with zeros currently, but do NOT count on this always being true.

Uninitialized data that is not in a STRUCT is squeezed out of the rest of the data block and placed at the end of the data block in the uninitialized block. The size of this area is computed and saved in the header as the variable UNINITIALIZED_LENGTH. When the application is loaded this extra amount of space if allocated with the rest of the data block.

There is one very important thing to remember. VERY IMPORTANT. As anyone who has also read the CORE document knows, Grail will attempt to convert between bitwidths of objects in the same instruction architecture (ie, from a 32-bit PPC601 to a 64-bit PPC620). Unless you specifically declare an object module to be restricted to a specific bitwidth, all ptr, and pointer derived data statements as a well as JMP, CALL, etc code instructions make increase or decrease in size no matter what section subdirectives you use.

CODE
 _______________________________________
| CODE BLOCK                            |
| ..................................... |
| CODE FIXUP LIST                       |
| ..................................... |
| PRIVATE CODE BLOCK                    |
| ..................................... |
| PRIVATE CODE FIXUP LIST               |
| ..................................... |
| PUBLIC CODE FUNCTION LIST             |
| ..................................... |
| SUPERVISOR CODE FUNCTION LIST         |
| ..................................... |
| USER CALLGATE FUNCTION LIST           |
|_______________________________________|
DATA
_______________________________________
| READONLY INITIALIZED DATA BLOCK       |
| ......................................|
| FLUSH INITIALIZED DATA BLOCK          |
| ......................................|
| PRIVATE INITIALIZED DATA BLOCK        |
| ......................................|
| INITIALIZED DATA BLOCK                |
| ......................................|
| DATA FIXUP LIST                       |
| ......................................|
| ENDIAN FIXUP LIST                     |
| ......................................|
| PRIVATE DATA FIXUP LIST               |
| ......................................|
| PUBLIC DATA VARIABLES LIST            |
|_______________________________________|
OTHER
_______________________________________
| OBJECT data                           |
| ......................................|
| TASK data                             |
| ......................................|
| DEBUG data                            |
| ..................................... |
| FUNCTION PARAMETER LIST               |
| ..................................... |
| FUNCTION DESCRIPTION LIST             |
| ......................................|
| DECRYPTION data                       |
|_______________________________________|

STRUCT
d.32 processor_family
ptr architecture
STRUCT code
	STRUCT public
	 ptr code
	 ptr fixup
	STRUCT END
	STRUCT private
	 ptr code
	 ptr fixup
	STRUCT END
	STRUCT function_list
	 ptr public
	 ptr supervisor
	 ptr user
	STRUCT END
STRUCT END
STRUCT data
	STRUCT public
	 ptr data
	 ptr readonly
	 ptr flush
	 ptr endian
	STRUCT END
	STRUCT private
	 ptr data
	 ptr readonly
	 ptr flush
	 ptr endian
	STRUCT END
	dptr uninitialized_length
	STRUCT stack
	 ptr length
	 ptr handle
	STRUCT END
STRUCT END
STRUCT other
	ptr decryption
	ptr task
	ptr debug
	ptr parameter
	ptr description
STRUCT END
 	ptr offset_ptr
dptr offset_count
dptr stack_handle
dptr ?
STRUCT END

PROCESSOR FAMILY

ARCHITECTURE

This is text with the actual name of the processor family this code was designed for.

CODE BLOCK

Dwelling place of immutable code.

CODE FIXUP LIST

?

PUBLIC FUNCTIONS LIST

The public function list consists of a list of two components: the entry offset, and the ASCIIZ/UNICODE NULL-terminated name of the function.

SUPERVISOR PUBLIC FUNCTIONS LIST

?

USER CORE FUNCTIONS LIST

?

READONLY INITIALIZED DATA BLOCK

All of your initialized data that should not be changed resides here.

INITIALIZED DATA BLOCK

Here sleeps all initialized data that may change at some point in time (and, for speed reasons, some intermixed uninitialized data). If it bugs you that too much of your uninitialized data is being caught in the RIDB then remember you're free to have the Object compressed on-disk. Perhaps you should also try to arrange your data better.

DATA FIXUP LIST

Contains a specific number of pointers (32-bit dwords for intel) to the location in the file that needs to be fixed-up. Simply put, the fix involves adding the dword ptr to the beginning of the module to each fixup location. Very fast to implement.

DECRYPTION_PTR/_LENGTH

Objects that are copyrighted or part of another set can be secured against illegal or undesired copying by the appropriate code within their initialization function. However, that does not stop anyone from easily spoofing or reverse engineering the object. To that end each object can have its own separate decryption sequence.

One run-of-the-mill protection scheme is to do an __OBJECT.Verify for an object you know must be running (such as the main application object) and to refuse to run otherwise. This can be spoofed obviously, but it is a good cheap deterrent against the computer illiterate.

As you know you can externally compress and/or encrypt any file. However Objects also possess the ability to internally control an additional layer of encryption. The mechanism for encrypting an object is handled through the __OBJECT.Encrypt function which calls upon the standard encryption routines.

Decryption is also handled through standard encryption objects but here you have two avenues which to cause decryption. You can have a key-file which the object checks for and then uses to decrypt itself. Or you can pass the decryption string directly to an object. The advantages of having a key-file for decryption is as obvious to anyone with such experiences as is it's disadvantages. As for the passing the decryption string to the object... this technique is primarily for use in two situations: Where you are needing to send authorizations to remote objects (across the 'net, etc) or need the utmost in security and so use different encryption sequences for different objects in an application.

TASK_PTR/_LENGTH

Points to a data set used only by taskable objects. The structure may differ with different builds, but it will always start out with:

d.32 Scheduling dptr Out_Data_Buffer_Size dptr In_Data_Buffer_Size The rest of the structure is none of your business. :)

THREAD

DEBUG_PTR/_LENGTH

If NULL there is no debug information. Otherwise, there is. See DEBUGGING below. This information is stripped from an object as it is loaded if the system is currently set to memory conversation. This information is stripped from an object as it is loaded if the system is currently set to memory conversation.

FUNCTION DESCRIPTION

function description

PARAMETER LIST

This lists each function name in an object, and each parameter passable to it, and, for each of those, a list of possible values and their text names. This is somewhat of a luxury this is usually used only at compile time or with scripts. It allows you to get a text name equivalent to values for function parameters.

function parameter value, name value, name *function description...

This information is stripped from an object as it is loaded if the system is currently set to memory conversation

THE THREAD HEADER

All objects have a relatively simple header structure. It uses only three pointers to strings (which come directly after the header). DATA ALIGN CLASS STRUCT object_header d.8 pointer_bitwidth d.8 task_type STRUCT stack dptr length ptr handle STRUCT END ptr object ;title ptr instance ;subtitle STRUCT END DATA END

POINTER_BITWIDTH

The importance of this variable can not be stressed enough. It defines the bitwidth that the object is expecting for all it's pointers. The actual number is computed just as FILESPACE_WIDTH in the file-system document with 2^pointer_width giving the true bitwidth of the for a pointer. See the table below. The default for the Intel CPU is currently only 32-bits.

Width Actual Bitsize

0 1 1 2 2 4 3 8 4 16 5 32 <-- default 6 64 7 128 8 256 Bit 7 (ranging 0 to 7) is used to denote whether the pointer_width is locked. If bit 7 is TRUE (1) then Grail will return an error if you try to load an object with a different pointer_width than your current architecture (ie, trying to run 32-bit PPC601 objects on a 64-bit PPC620).

OBJECT NAME

The name of the object class.

INSTANCE NAME

The name of this instance of the object. Usually just a number. Sometimes a specific name.

Typically a combination of the manufacturers name and the product line -- "Generic Huffman", or "Stacker Technologies, Huffman-7" for instance.

TOC

Instantiation Background

DESIGNER | Lewis Sellers lsellers@usit.net

With Grail there may be multiple instances of the same object. As an illustration of this concept, assume you have designed a virtual arena or cyberspace meeting place of some fashion. You may have several Persona, Greeters or "NPC"s roaming the area. They could all be different objects (ie, different data and code) but they could all just as easily be difference instances of the same object. Since Grail prohibits self-modifying code (as a rule, not by force) the code of all the different Personas could simply be shared from the first Persona object called, saving memory. The non-shared readonly data (and also stack area) would be separate of course.

Instances come in two types: Passive and Active.

The passive instance is an object that is called into memory and is used by another object, or several other objects. Its functions are there to serve other objects. It does not have a main function and does not have a life of its own as it were, since it is not known by the task manager.

An active instance is an object that has it's name passed on to the multi-tasking unit. The object runs as a process. It has a main function. Whether the instance of an object is active or passive it has a few common attributes: object class name, instance name.

Object Class name

This is not the same as a file's name (ie, it's Title). It is the name describing it's class, or what it does, or what it controls. An object class with two underlines prepended to it signifies that the object belongs to the core object set such as the file-system device objects.

Instance name

This is not the same as a filename. Many times, this is a simply a text representation of a number. Sometimes it will be a distinct name such as "Vincent" or "Sissy", "Bird" or "Cat", etc. You could for example, have an object class named "Animals" with two instances called "Cat" and "Bird". Ie, "Animals:Cat" and "Animals:Bird".

Another more practical example might be that of the __DECOMPRESSION object class. Grail comes with the unique instance called "Standard Fixed Huffman", but it might not be too much to expect that Stac Electronics someday might work up a version as well. You might then have "__DECOMPRESSION:Standard Fixed Huffman" and a "__DECOMPRESSION:Stac Electronics series G", for example.

Typically, when talking about object/instance names you use a semi-colon ":" to separate the two.

Instances also use a binary numbering system as part of the refID system.

The Topology of Multiple Instances

Assuming that an object is declared as being able to have multiple instances, then you should be able to as many (passive) instances of an object as you have memory. When an object is loaded into memory more than one, then the second instance is loaded same as the first excepting that the code sections and readonly data sections are not loaded into the new instances space. The object reference header instead points to the first instance in regards to them. 

Multiple Instances Illustration
_________________________________________ | Instance 1 | Instance 2 | Instance 3 | |_____________|_____________|_____________| 
Multiple Instances Areas Illustration

 _____________________________________________________________________________
|     Instance 1          |       Instance 2        |      Instance 3         |
|_________________________|_________________________|_________________________|
| Code | Readonly | Other | (links to 1)    | Other |  (links to 1)   | Other |
|______|__________|_______|_________________|_______|_________________|_______|

Say for example, oh... you load in seven instances of Povray 3.0 for Grail. Its code and read-only data will only be loaded into memory once. If we just pick some number at random and say that the total size of the application is 1MB, then normally this might mean that you'd need 7MB to have them all running at once. If however, we assume that the code and all of Povray's readonly data (strings and data tables) takes up 800kb, then the total amount of space required to hold 7 instances of Povray 3.0 under Grail would be 2.2MBs. Ie, 1MB for the first instance and 200kb for each additional one. Roughly.

Grail Prohibits Self-modifying Code?

Yes. Though Grail does not attempt to enforce this policy, you should not try to modify code in memory. If you were to change the code of one instance, you'd be changing the code of ALL such instances. Under Intel-Grail you can modify most code however -- just don't do it.

Also, on the same subject, you should not modify data sections marked as READONLY for the same reason.

The Topology of Unique Instances

While most object allow multiple instances of themselves, some do not. These are almost without exception all device objects, ie objects that control specific pieces of hardware at the lowest level. They are called unique instances because, while some of them may belong to the same object class, their code and data must be different from all other instances.

If that is confusing, just take for example your file devices such as your old IDE Hard-drive, your new EIDE hard-drive, those two-SCSIs you also have, your CD-ROM and that aging floppy drive. All of these are of the __FSDEVICE class, but they are also separate instances with names like __IDE, __EIDE, __SCSI, and __FLOPPY. Unique instances.

The ability to have a class that can access several disparate pieces of hardware or code through the same interface is rather important to Grail, to say the least.

Creating an Object Task

This is achieved indirectly through the __OBJECT.Initiate functions which have subsumed the equivalent of a __TASK.Create function. This function takes an object in the list of currently loaded objects and makes it a separate running task. It returns NULL if there are already the maximum number of tasks present or there is insufficient memory to create the buffers. Otherwise you get back the selector for the new task TSS. This adds an entry (a TSS) to the GDT.

The size of the in buffer and out buffer, as well as its scheduling, is normally stored in the object's associative branch.

If at anytime the target app can not be found or run, and we are in a we are the only app in an LOCKED TASK MODE then the previous app is loaded. If it can not be found then the default metaphor is loaded. If that can not be achieved then the operating system issues a reboot.

An object must have a function called Main in order to use the function.

Uncreating an Object Task

This function is subsumed by __OBJECT.Terminate which removes an object from the list of currently running tasks (the GDT). then a priority message through the tasks semaphore, and waits a full second for a response. Hopefully by that time the application has cleaned up it's business, because if no response is forth coming in that time, it proceeds ahead anyway and does an Unload_Module.

TOC

Object-Oriented Instancing Functions

DESIGNER | Lewis Sellers lsellers@usit.net

CRITIQUING | John Fine john.fine@channel1.com (a0.60)

__OBJECT.VerifyObjectModuleFormat

object_name

This verifies that the file you have specified is actually a valid object module and functional. The codes returned are:

0 NULL 1 File/Object does not exist 2 File not a valid Grail object 3 Supervisor Object 4 Device Object 5 Platform Object 6 Script

The object name is pointed to by ESI.

__OBJECT.Verify

object_name

Verifies that the named object existences in memory. The name is passed in ESI, and the result returned in EAX.

__OBJECT.Request

object_name, security, priority, instance_name

First off, this performs an __OBJECT.Verify, and assuming all is ok, it does a module load of the object and adds it to the object list. If it is a Platform Object then nothing else happens. If the object requested is a Supervisor Object however, then the CALLGATES and code-gates are updated.

Each object has a request counter, which keeps the current balance of requests and discards. A request increases the counter by one. A discard decreases it. If the object is requested and does not exist it is loaded into memory. If the object is discarded and the counter is 0 the module is removed from memory.

A module can, though is not required, have a list of other modules that are loaded after it is.

If the object already exists and is noninstantiable then the function will fail. If the object class already exists but is instantiable then a new instance of it will be created.

If the object needs a key-file for decryption, the object will automatically use it under this function. If the object requires the decryption stream to be passed to it, you must use __OBJECT.RequestDecryption to access it.

As said before, all memory allocated by an object, including other objects, is tagged with it's unique ID. When it is discarded, everything in the MMU that has this tagged is also automatically freed after the objects destruction function has executed. If you are wanting an object you've called to stay in memory after the object that summons it has terminated you must abandon it.

You can define the security of the new object, if you wish. Or not. Use 0 for no change. You can only suggest lower or equal levels. If the level requested is higher than you are allowed, the object will load, but at your current level.

0 No change 1 Supervisor 2 Secure/Private User 3 Public User 4 Hermetic User This function returns the following values:

-2 Failure, instance already in existence -1 Failure, requires decryption 0 Status unknown 1 Success The instance will be ignored if it is NULL or if the object is non instantiable. Otherwise the instance name of the object as specified by its file will be superceeded by the name pointed to here.

Object name is a pointer using ESI, security is EAX, priority is EBX and instance name is EDI.

__OBJECT.RequestDecryption

object_name, type, decryption, security, priority

If the object you are loading requires a decryption stream to be passed to it for unlocking, you must use this version of __OBJECT.Request.

Object name is a pointer using ESI, type is EAX, decyption is EBX and instance name is EDI.

__OBJECT.Discard

object_name

This function looks for the object in the object list, and if it is there, it checks to see if it is in the task list. Objects running in the task list can not be discarded. Assuming we can discard, then we do by removing it from the object list and then performing a module unload.

The object name is ESI.

__OBJECT.Polymorph

object_name, object_name, security, priority, instance_name

This function will stop one object from operating, discard it, then request another one in its place. Note that it does verify the target file before starting. This is useful for changing characteristics of device objects on-the-fly, etc.

Object name is a pointer using ESI, security is EAX, priority is EBX, polymorph object name is ECX and instance name is EDI.

__OBJECT.Function

objectfunction_name

This retrieves the direct address pointer to a specific function within the specified instance of a class. For instance the psuedo-code

showit = __OBJECT.Function "TIFF.Convert_to_Raw" loads the showit variable with a pointer to the function Convert_to_Raw in the TIFF class. So, from now on when you wanted to convert a TIFF file to a raw you'd use the function showit. Got that?

Pointers to functions in a class are absolutely guaranteed not to change unless you discard the object the function belongs to. Naturally. For speed, this function is usually called in the automatic creation function of an object. That way all further uses of object functions within an object are direct calls to the function.

Objectfunction_name is passed in ESI. Status of the result is returned in EAX, and a pointer to the function returned in EDI. If function fails, EDI will be NULL and so will EAX.

__OBJECT.Initiate

object_name, instance_name

Similar to __OBJECT.Request, but after returning values from the Creation function, it initiates the Main function as an independent task. This object takes care of it's own termination. If the object requires a key-file for decryption, then this will be taken care of automatically.

By default, you should receive back the following status values:

1 Success 0 -1 Failure, General -2 Failure, decryption key-file missing -3 Failure, decryption key-file invalid -4 Failure, requires decryption stream -5 Failure, decryption stream invalid Object name is a pointer using ESI, and instance name is EDI.

__OBJECT.InitiateDecryption

object_name, type, decryption, instance_name

This version of __OBJECT.Initiate passes a stream of data to the specified object which it will use to initiate a stream decryption.

Object name is a pointer using ESI, type is EAX, decryption is EBX and instance name is EDI.

__OBJECT.Task

object_name

If your object has a Main function and has been started up as an independent task, then this will give you back it's task id so you can directly refer it it, or any other application currently running in the task list.

__OBJECT.Instance

object_name, instance_name

You want a specific instance of an object? This will find it for you, if it exists, by name.

The name name is ESI and the instance name is EDI.

__OBJECT.Encrypt

object_name, encryption_name

This uses the __ENCRYPT object class to encrypt an object.

__OBJECT.Null

object_name

This creates a special object. __OBJECT.Function will always report that the named function exists no matter what the name is. Any attempts to use these requested functions will always execute the same bit of code which simply returns the value 0 (NULL/neutral). The creation and destruction functions return NULL.

Object name is ESI.

__OBJECT.Section

object_name

This function looks through active memory for the specified object, and if found returns the following structure:

STRUCT STRUCT ptr ptr Language ptr Debug ptr Decryption ptr Parameter ptr Description STRUCT END STRUCT length dptr Language dptr Debug dptr Decryption dptr Parameter dptr Description STRUCT END STRUCT END Object name is in ESI.

__OBJECT.Instances

object_name

How many instances of an object are there currently active? Pass the object name in ESI and receive the answer in EAX.

__OBJECT.Count

Makes a count of all the objects currently available (in EAX) as well as a separate count of all that are encrypted (in EBX) and those that are Secure Private (ECX) and Hermetic (EDX).

__OBJECT.Flush

Forces data areas marked with FLUSH to be forcibly written out to the object file itself.

__OBJECT.Terminate

Removes the current object from the task list and then __OBJECT.Discards it.

__OBJECT.Detach

All objects and memory blocks are tagged with the ID of the object that created them. They terminate when their creator object terminates, unless you remove the tag. This function NULLs that tag and sets the object free. Formally called __OBJECT.unID.

TOC

Multi-tasking and Threading: An Overview

DESIGNER | Lewis Sellers lsellers@usit.net

CRITIQUING | John Fine john.fine@channel1.com (a0.60)

Grail supports pre-emptive multi-tasking and multi-threading. The multi-threading occurs at the most intimate symmetric level, while the multi-tasking occurs at the more secure distributed. Threads and tasks, no matter what the name, are all TSS (Task State Segment) entries in one or more GDTs (General Descriptor Tables). They are all the same on the hardware level. The only real difference is their scope and origin.

The theoretical absolute maximum number of tasks that current 80386/486/Pentium processors can handle without creative intervention is 8192. With Grail there are no such limitations. You may have as many tasks as you have memory and processor cycles. This comes about because Grail reserves at least one GDT table for each of it's 16 priority levels.

All task handles (or IDs) are native pointer-width. They point to a list of small data structures containing information on each task. The structure pointed to for threads is a simple affair such as:

STRUCT Thread_Struct dptr thread_code dptr threadID dptr status/processor dptr priority STRUCT END For objects the situation is much more complicated. An entire section dedicated to the Object Module Format is included later in this manual.

Selector Illustration

15                 4 3 2 1 0
-----------------------------
| INDEX             |T| RPL |
|                   |I|     |
-----------------------------
  INDEX (0-8191)
  TI  Table Index (0=GDT, 1=LDT)
  RPL Requested Privilege Level (0-3)

Multiprocessor Symmetry

Grail does support multiple processors, as long as the architecture is symmetric, ie, multiple processors sharing the same memory space. While currently the Grail design works only with multiple processor of the same sub-family (ie, all Pentiums, or all powerpc 601s) it is fully intended to tweak things so that any mix of processors may be used, so long as they are of the same pointer width. That is, PowerPC 601s, i486's, and 68ks could be used in the same system. You could not mix in a 64-bit PowerPC620 for the obvious reasons.

Before this may become a possibility several revisions to the object module format will have to be made. 

 _______________
|  PU1  |  PU2  |
|_______|_______|
 ______________________________________________________________
|  PU1  |  PU2  |  PU3  |  PU4 |  PU5  |  PU6  |  PU7  |  PU8  |
|_______|_______|_______|______|_______|_______|_______|_______|

MULTI-THREADING (SYMMETRIC THREADS)

it's core Grail assumes (or emulates) a shared-memory symmetric multi-processor environment. Whether a system has one processor or sixteen, they all operate within the same memory space, executing none, one or more tasks.

The thread is a low-level task, also called a symmetric thread (or symmetric task) in Grail. At the thread level there is no innate security, except through obscurity. Communications between threads is handled almost entirely through software. While it is not enforced, it is assumed that all threads within the same object will not communicate with threads from other objects. Object to object communications should be handled by the multi-tasking functions explained following. Threads should communicate only with threads within the same object.

System1 _______________ | PU1 | PU2 | |_______|_______| Communications between processors, at least on the thread level, occurs entirely by memory semaphores. Threads are scoped to the object they were created in. When the object goes out of scope, so do all it's threads.

The core handles only the creation, destruction and sleeping habits of threads. All other actions are handled by atomic software instructions. In Grail Script, for example, this would create a thread, do something useful, then remove the thread.

threadID = __TASK.CreateThread Renderline, 0, Modest, 0 ;do important things.... __TASK.DestroyThread threadID

MULTI-TASKING (DISTRIBUTED OBJECTS)

At a higher and more secure level than threads, are tasks, also known as distributed tasks or active objects. Unlike threads, all task operations are handled by secure core functions. A distributed task is always associated with an active object in Grail, so it is not unreasonable to think of them as essentially the same thing.

Object tasks may interact with other object tasks running on a processor in your single symmetric system, or with other objects on remote systems, whether they be across the room or on the other hemisphere of the planet (or other planets, as the case may be). The only real difference is the communications lag.

While threads have no security, object tasks using a messaging system which protects the data being transmitted by hiding it behind the supervisor page and restricting code access via CALLGATE mechanisms. Data being used to copied from one object to the other, forbidding direct changes to the original source.

      Finland          USA Room1        USA Room2
      System1          System2          System3
 _______________  _______________  _______
|  PU1  |  PU2  | |  PU1  |  PU2  | |  PU1  |
|_______|_______| |_______|_______| |_______|

Communications between objects on the same processor use core memory copies and semaphores. Those with processors in the same symmetric system may also make use of memory mapped I/O ports (MMIOs). Communication with objects on other systems will make use of a subset of the virtual filespace subsystem to transmit messages across modems, ethernet cards, etc.

QUEUING PRIORITIES

There are three to four queuing classes a task may use. A task may be invoked by an external event. It may be invoked directly by the request of software. Most often a task is invoked in a fairturn, priority-based manner. The fairturn priority may be the usual fixed type, or the occasionally useful autoadaptive.

When requesting a task priority you must use the a nibble/nibble pair of class and priority such as that which follows:

QUEUE BYTE ILLUSTRATION

7 4 3 0 |----------|----------| | Class | Priority | |__________|__________|
CLASSPRIORITY
04 Event00 SCHEDULER
03 Invocation01 CoreCritical
02 Timer02 Core
01 Adaptive03 FileSystemHigh
00 Fairturn04 FIleSystemLow
05 RealtimeDeviceHigh
06 RealtimeDeviceLow
07 ObjectHigh
08 ObjectLow
09 DeviceHigh
10 DeviceLow
11 ThreadHigh
12 ThreadModest
13 ThreadLow
14 Idle
15 Sleep

Scheduling has 256 different distinctions. There are 16 major named priority levels (high nybble) and 16 unnamed sublevels to each of those (low nybble).

QUEUING CLASSES

Fair-turn (fixed) Timeslicing

Normally a task you create has a fixed priority and there is nothing it can do to change this. This is called the fair-turn timeslicing scheduling method. You can alternatively use adaptive timeslicing as explained below.

Adaptive Timeslicing

Autoadaptive timeslicing uses the same priority levels as fixed timeslicing. The only difference is that this task is allow to change it's priority at will to anything up to it's dispatched maximum priority level. This is for tasks/threads whose processing requirements tend to change over time.

Event/Timer

The metronome (system timer) has been programmed to run at 60hz (60 times a second). Normally timer tasks get 10/60th of a second each sequentially.

Invocation

?

PRIORITY

SCHEDULER

Tells interrupts in multi-processor systems to bugger off. Already busy.

CoreCritical

The task occuring is extremely important an part of the supervisor level functions.

Core

A core object thread. The core comes before all other considerations.

FileSystemHigh

High priority filesystem streaming task.

FileSystemLow

Background filesystem tasks.

RealtimeDeviceHigh

A high priority real-time device.

RealtimeDeviceLow

A lower priority but real-time device object.

ObjectHigh

High priority object.

ObjectLow

Low Priority object.

DeviceHigh

High priority user-level device object.

DeviceLow

Low priority user-level device object.

ThreadHigh

High level thread.

ThreadModest

Modestly high priority thread.

ThreadLow

Relatively low priority thread.

Idle

Are there no tasks ready to run? I mean, besides those with ThreadNegligible priority? If not, then they are up for processing. Extremely low level thread. The lightest background task that will be encountered and still be competing against other active tasks.

Sleep (Suspend)

This task will never be executed. It is in sleep mode or sleep priority.

***

CR0--Control Register 0 Illustration

*This bit is only available on 80486 or better. 31 30 29 18 16 5 4 3 2 1 0 ----------------------------------------------------- |P|C|N |A|W| |N|E|T|E|M|P| CR0 |G|D|W |M|P| |W|T|S|M|P|E| ----------------------------------------------------- PG Paging *CD Cache disable *NW No write-through *AM Alignment mask *WP Write protect *NE Numerics exception *ET Extension type TS Task switched EM Emulate math coprocessor MP Monitor coprocessor PE Protect enable

Selector Illustration

15 4 3 2 1 0 ----------------------------- | INDEX |T| RPL | | |I| | ----------------------------- INDEX (0-8191) TI Table Index (0=GDT, 1=LDT) RPL Requested Privilege Level (0-3)

80386/80486 Descriptor Illustration

63 48 47 44 43 40 39 32 31 16 15 0 ------------------------------------------------------------------------- |Base | |B| |A|Limit | | |S| | | | | |address|G|/|0|V|16..19|P|DPL|=|Type|0| Base address | Limit | |24..31 | |D| |L| | | |1| | | 0..23 | 0..15 | ------------------------------------------------------------------------- G Granularity 0 byte 1 page (4096 byte) B/D 0 16 bit mode 1 Native (32-bit) mode DPL Descriptor privilege level (0-3) S Segment 0 Not valid (exception), object 1 Valid, memory TYPE if S==0 0 Unused (Invalid Descriptor) 9 TSS 11 Busy TSS 5 Task gate 12 Call gate 14 Interrupt gate 15 Trap gate if S==1 1 Read/write data segment 5 Execute/readable code segment

PRIVILEGE RINGS

In general, the GRAIL core and its functions run in ring 0. If present, supervisor debugging and monitoring code executes at CPL0 with the core. Everything else runs in ring 3 at IOPL 3, including all applications and their objects. Only a very few functions and data areas are secured. This is mainly to provide debugging crash protection, and should not degrade normal performance except during initialization procedures (or badly written, CALLGATE recursive code).

Privilege Ring List

Ring 0 Core/FS device objects/Supervisor Objects Ring 3 All else

SELECTORS

The GRAIL memory model is strictly flat. All memory is linearly addressable from 0 to 4 gigabytes. All the selectors (CS, DS, ES, SS, FS and GS) have exactly the same address area. This is the descriptor structure of all used data/code segments.

RING 0 (Core, fs device objects and supervisor debugger/monitors)

EXECUTABLE (CS) G=1, B=1, TYPE=5 execute/readble code segment DATA (DS,ES,FS,GS,SS) G=1, D=1, TYPE=1 read/write data segment

RING 3 (Object)

EXECUTABLE (CS) G=1, B=1, TYPE=5 execute/readble code segment DATA (DS,ES,FS,GS, SS) G=1, D=1, TYPE=1 read/write data segment

GENERAL DESCRIPTION TABLE (GDT)

User objects do not assign IDT, GDT or LDT values. They are created, modified and destroyed only by the core directly or through user CALLGATE functions. The GRAIL GDT currently starts out with 5 descriptors allocated. GDT(4) is an LDT entry to the list of all CALLGATEs that user objects can access to perform core functions.

GDT(0) NULL

Core: Ring 0

GDT(1) CS GDT(2) DS,ES, FS, GS, SS

Object: Ring 3

GDT(3) CS GDT(4) DS,ES, FS, GS, SS

CallGate Core Functions

GDT(5) LDT

Task/Object Descriptors

GDT(6) TSS 0-31 S=0, TYPE=9 <-- non-multi-tasking TR GDT(6) || \/ GDT(n)

LOCAL DESCRIPTOR TABLE (LDT) / CALLGATE FUNCTIONS

Grail uses only ONE local descriptor table. This holds all of the CALLGATEs that user objects may call from time to for services from the core. The LDT is at GDT(5).

80386/80486 CALLGATE Illustration

63 48 47 32 31 16 15 0 -------------------------------------------------------------------------- | Offset |P|DPL|S|Type| | Dword | CS | Offset | | 16 ... 31 |=| = |=| == |000| Count | Selector | 0..15 | | |1| 0 |0| 12 | | | | | -------------------------------------------------------------------------- DWORD COUNT The dword count of each call-gate specifes the size of the portion of stack that should be copied to the CPL0 ring stack in dword. This is usually 0 for most Grail functions. SELECTOR The selector for the SUPERVISOR ring 0 CODE (CS), which is GDT entry 1. OFFSET The offset point to the 32-bit address of the function in the core this is to be called.

TASK DESCRIPTORS

Grail supports an unlimited number of tasks through chaining of multiple GDTs in priority strands. A task is an object with a main function or one of it's threads. All tasks decriptors lie in the GDT from GDT 6 and up. The descriptor defines an area called the Task State Segment (TSS) which holds all the real information about the task, including the Grail semaphore data area.

Task Descriptor Illustration

63 48 47 44 43 40 39 32 31 16 15 0 ------------------------------------------------------------------------- |Base | | | |A|Limit | | |S|Type| | | | |address|G|-|0|V|16..19|P|DPL|=| = |0| Base address | Limit | |24..31 | | | |L| | | |0| 9 | | 0..23 | 0..15 | -------------------------------------------------------------------------

The size of the TSS is 104 plus the Grail Semaphore Structure and padding for future upgrades and alignment. The total is 160 bytes per. 

TASK STATE SEGMENT (TSS)

------------------------
|   0      | Back Link | 0
|----------------------|
|         ESP0         | 4
|----------------------|
|   0      |    SS0    | 8
|----------------------|
|         ESP1         | 12
|----------------------|
|   0      |    SS1    | 16
|----------------------|
|         ESP2         | 20
|----------------------|
|   0      |    SS2    | 24
|----------------------|
|         CR3          | 28
|----------------------|
|         EIP          | 32
|----------------------|
|        EFLAGS        | 36
|----------------------|
|         EAX          | 40
|----------------------|
|         ECX          | 44
|----------------------|
|         EDX          | 48
|----------------------|
|         EBX          | 52
|----------------------|
|         ESP          | 56
|----------------------|
|         EBP          | 60
|----------------------|
|         ESI          | 64
|----------------------|
|         EDI          | 68
|----------------------|
|   0      |     ES    | 72
|----------------------|
|   0      |     CS    | 76
|----------------------|
|   0      |     SS    | 80
|----------------------|
|   0      |     DS    | 84
|----------------------|
|   0      |     FS    | 88
|----------------------|
|   0      |     GS    | 92
|----------------------|
|   0      |    LDTR   | 96
|----------------------|
|I/O bitmap|       | T | 100
|----------------------|
|                      | 104
~                      ~
 GRAIL SEMAPHORE STRUCT
       (32 bytes)
~      +pad            ~
|______________________| 160
STRUCT dptr processor ;which processor is handling the task? (starting at 1) STRUCT area ptr Header ptr processor ptr Communications STRUCT END ;... STRUCT END Application programs can not directly invoke task switches. That is handled by CALLGATEs to the core which abstractly manage all such matters. This is a TASKGATE descriptor:

80386/80486 TASKGATE Illustration

63 48 47 32 31 16 15 0 -------------------------------------------------------------------------- | |P|DPL|S|Type| | Dword | TSS | | | U N U S E D |=| = |=| = |000| Count | Selector | U N U S E D | | |1| 0 |0| 5 | | | | | -------------------------------------------------------------------------- DWORD COUNT The dword count of each call-gate specifes the size of the portion of stack that should be copied to the CPL0 ring stack in dword. This is usually 0 for most grail functions.

TOC

Multitasking

DESIGNER | Lewis Sellers lsellers@usit.net

Communications between objects is via core callgates. Active objects are all tasks. Each task is handled by the Task Managing Unit (TMU) which uses both a semaphore and a variable-length data area for inter-task communications.

__TASK.Limit

Stops Grail from allowing new tasks to be created.

__TASK.Unlimit

Allows Grail to create new tasks as desired.

__TASK.Limited

number

Limits the number of tasks Grail will accept per priority level. The default number is 500 (mostly arbitrary).

__TASK.Enable

Changes TASK_BLOCK to 0, which means multi-tasking occurs as normal.

__TASK.Disable

Changes the TASK_BLOCK flag to 1. No more task switches will occur until you reenable it, and no more requests to create more tasks will be honored. It's not nice to use this unless you are the current metaphor, and thusly does a check to see if the current task is SECURE. If not, then the request is ignored.

__TASK.Count

Counts the number of TSSs in the GDT in EAX, and the maximum number of tasks available for usage under the current build version in EDX such as:

(tasks, max_tasks) = __TASK.Count()

__TASK.Verify

taskID

Scans the GDT looking for the task. If it doesn't find it a NULL is returned. Otherwise it returns it's task number (GDT). Cousin to __OBJECT.Task.

if (__TASK.Verify(task_1)) { task_1_name = __TASK.Object(task_1) __OBJECT.Discard(task_1_name) }

__TASK.Type

Is the task an active object or a thread?

__TASK.ObjectID

task

Returns the object name of a task if it exists. If the task is actually a thread, no name is returned.

__TASK.ThreadID

task

Returns the thread name of a task if it exists. If the task is actually a object, no name is returned. A thread ....

__TASK.Suspend

task

Changes the tasks status to suspended. Anytime this task comes up in fair-turn or through time-based scheduling it is skipped. The request is ignored unless the calling task is SECURE.

__TASK.Revive

task

Changes the tasks status to not_suspended.

__TASK.Scheduling

taskID, class, priority, type, schedule_time

Changes the scheduling of a task from it's default.

If scheduled then you must state the schedule time. This sets the size of the chunk of time that each task receives to process in. Because of task-switch overhead, the higher this is the better, as far as performance goes. Conversely of course, if it's set too high then things may get a bit choppy. This function should really be only used by the metaphor but anybody can use it. The input is a long word measured in sixty-thousands of a second. The default is 10000 (1/6th of a second).

__TASK.PowerDown

__TASK.PowerUp

TOC

Distributed Communcations

DESIGNER | Lewis Sellers lsellers@usit.net

This set of functions is one of those that you try to ignore at first, but eventually have to sit down one night and stare pitifully at the obtuse and bewildering reference documents. I'll try not to fuzz your brain up to the point you start wondering if maybe there's still one last cigarette in the house left over from before you gave them up years ago.

Anyway, to begin with: Tasks often will want to spy on one another or attempt to coordinate among themselves by sending inter-task communiques. The __TASK object class provides compact, straight-forward functions to help in this endeavor.

You can send and listen to both system signals and inter-task data streams. The system signals are well defined and used mainly for control and handshaking. Most of your communications will take place through definable data streams. Data streams, ie TASK communications buffers, are definable for outgoing information and for incoming. They can be defined as public and thus available to all tasks or as singularly private, checking the ID of any task trying to access them.

The streams can be (are) automatically defined by your task object when you compile it based on your code (at least in the Grail Assembler). You can of course explicitly create or modify these structures in your program.

The outgoing stream may have either one or no public areas. It may also have either no or many private areas.

All stream areas are individually defined as to their size, and thus each have assigned to it a starting pointer, a length, a pointer to the beginning of unused data, and a pointer to the last new byte of data.

Obviously to keep track of all these areas a dynamic list of structures must be kept for each task. This dynamic list of task stream areas is called the ... and a pointer to it is kept in the task object structure named ...

On a side note, if you noticed, most operating systems have the ability to send arguments or strings to a program when it starts up. There is no direct equivalent to this command-line legacy in Grail. You can however effect the same result by sending communiques to the task object after it has started up, thus shaping it's operating parameters.

There are two structures you should be aware of that are used by TASK functions: The Semaphore_Struct and the Stream_Struct. 

STRUCT semaphore_struct
 ptr Object_MMU
 ptr Stream_MMU
 dptr processor
 d.32 schedule
 d.8 suspended?
 d.8 priority
 d.8 finished
 d.8 busy?
 d.8 security
STRUCT END
STRUCT Stream_Struct
 ptr ID ;0=public, otherwise private
 ptr data
 dptr length
 dptr position
 dptr end
STRUCT END

Object_MMU

This is the MMU handle to the tasks' object structure (See __OBJECT), ie, a pointer into the backbone.

SCHEDULE

Schedule is in 1/60th of a second intervals wherein the task is awakened. If schedule is NULL then the task is not time-based, but uses the fair-turn scheduling system.

SUSPENDED?

Suspended tells other tasks if the task is even running. When this flag is other than 0, then task does not exist in the GDT and is completely inactive.

FINISHED?

The finished semaphore indicates that the program has ended. To determine if it ended well or because of some form of error, check the value:

0 not finished 1=no problem. termination was 'natural' 2=general corruption 3=general stack failure When finished is set, the suspend semaphore is also set and the task is suspended. Once the complete semaphore is set, the task will not accept further commands. If you wish to reuse it, use the __TASK.Reinitialize function.

BUSY?

The busy? flag is Non-0 when the task is busy processing and not accepting anymore data.

SECURITY

This is merely a reference to the security status of the task. It's one of the four: Supervisor, Secure, Public, Hermetic.

STREAM

Controls access to the stream. 0 means there is no stream. 1 is public. 2 is private.

**

This area is composed of a header, and two data buffer (ingoing and outgoing). The size of the data buffers are determined at the creation of a task and can NOT be changed thereafter.

If outgoing_position is not equal to outgoing_end then the task has information available. As to WHOM the data is meant, notice that depends on 'info'. If it is 0, the information is public. Otherwise it's private.

A public stream status means that outgoing stream data is not lost when another task reads it. It also means there is only one set of data that can be read at a time. The data can of course be a series of values with IDing substrings that are meant for specific groups of tasks of course (ie, the BLACK_TEAM picks out info for it, and the RED....)

private

OUTGOING

Holds an indicator for other tasks to read. Most usually it is a number 0 to 100 showing the percentage the application is complete with it's task. It is also known to hold a frame-count for sync in some programs, or just a general coded response. Useful for some tasks that are, for instance, decoding a JPG and want to inform the main task of their progress for their update bar.

INCOMING

You can tickle a task and causing it's incoming to be increased by 1.......

Some tasks use this for recording polls.

Ack is used acknowledge that the output has been seen. Use __TASK.Ack.

PRIORITY

The Priority_Message (ingoing) and Priority_Response (outgoing) are used mostly by the core to inform a task that it should disengage all it's critical hardware routines because there's a very good chance it is about to be terminated or at least suspended for a while. If this message was not sent, then a termination could leave running communications hardware or sound-cards, etc.

The priority flag is normally 0. The message is sent by changing the priority flag to -1. If this is received, then a task should change the priority to a 1, then shut-down all it's hardware communications, then change priority to a 2, then issue a __TASK.Wait.

After issuing the message, the core watches the priority flag for a change to 1. If it is then the task HAS seen the priority and is attempting to comply. Usually anything above 1 (say 2) indicates the task has closed down any hardware operations that it may be busy with (playing a MIDI song, for instance.)

Once the core detects that a priority message has been responded to, the task is usually either deleted, suspended or saved as a core image for some form of quick resume program. Usually core functions give all the tasks involved a second or so to respond, then do what they were going to do anyway.

OBJECT IDENTIFICATION

ObjectID is derived some several variables, including the objects priority, security, and symmetric memory address. If the object is non-local to the current system, it also....

__TASK.PriorityOverride

task

Informs a task that something horrible is about to happen and to get it's affairs in order as quickly as is possible. Only secure object can issue this. Others get a noncommittal return (0).

__TASK.PriorityResponse

0, 1 or 2, etc

__TASK.Wait

Waits til priority is NULL again.

__TASK.Outgoing

task

Increases the outgoing flag by 1.

__TASK.Incoming

task

Decreases the incoming by 1.

__TASK.Status

Allows a task to query Grail about it's own status. Returns in EAX a 0 (NULL) if there is no Priority or Ingoing message, etc. Otherwise bit 0 if busy, bit 1 priority, bit 1 suspended. Frames is returned in EDX. MMU is returned in EBX.

__TASK.Info task

__TASK.Get_Semaphore

ptr_semaphore, task

Fills the semaphore structure of the current task to the specified ptr address ptr_semaphor. You may then inspect the data at your leisure.

__TASK.Busy?

Change busy? to 1 if not accepting new data. Use frame to indicate completion status.

__TASK.Stream public, privates

__TASK.ID task, io, privno

This selects the object ID that a private message area in either incoming or outgoing should use.

__OBJECT.OtherUsage

ptr_handle

Used to ask another task about the usage of it's data. Returns outgoing in EAX and incoming in EDX.

Used to ask another task how big it's buffers are. Ingoing is returned in EAX and outgoing in EDX.

__OBJECT.OwnUsage

Returns outgoing in EAX and incoming in EDX. EBX is 0 if there is no outgoing data left, and Non-0 if there is. Used to ask another task how big it's buffers are. Ingoing is returned in EAX and outgoing in EDX.

SENDING MESSAGES

__OBJECT.SendMessage

ptr_data, data_length, objectID

Sends a variable length stream of data to the task. Most commonly these are either filenames or data structures of some kind. The data area is a circular-buffer. So, this first checks that your new data will not overwrite anything the task has not gotten to yet. If so, it merely returns a NULL.

If data is sent, the outgoing_end ptr is increased appropriately, and the function returns a number representing the amount of information (in bytes) that was already in the buffer before you sent data.

__OBJECT.RecieveMessage

ptr_data

Retrieves data sent to the current task and copies it to the area pointed to by 'data'. __MMU.Alloc at least the same as the outgoing data areas size.

If there is any data to receive, then afterwards the tasks ack counter is increased by 1 to indicate the action.

Once you get the data this does not necessarily mean it's erased. This depends on the status of 'info'. When it is 1, then there is public information available, which will be updated as soon as it can be, and you will need to reference 'frame' to see if there's newer data available then from the last time you checked. When 'info' is 0, then the info is generally private and after you get the data it is generally lost.

__OBJECT.SendPrivateMessage

ptr_data, data_length, task

__OBJECT.RecievePrivateMessage

ptr_data, from_task

POSTING MESSAGES

__OBJECT.Post

ptr_data, data_length

Places data in your (you the task) own outgoing buffer.

__OBJECT.ReadPost

__OBJECT.PrivatePost

ptr_data, length

Places data in your (you the task) own outgoing buffer.

__OBJECT.ReadPrivatePost

BROADCASTS

__OBJECT.Broadcast

__OBJECT.ReceiveBroadcast

TOC

Symmetric Multi-threading

DESIGNER | Lewis Sellers lsellers@usit.net

Grail does support threads. The threads may run on one single processor, or be spread about on multiple processors. Threads are scoped with the object that creates them. The functions that create, destroy, sleep and wake threads are handled by the core. All others are provided through software macros.

CORE

The core function that support threads are fairly straight forward. There are three creates, two destroyers and a couple controlling hardware enforced sleep.

__TASK.CreateThread

thread, processor, priority, unique

This function creates a single thread of code on a certain processor, at a specific priority, and possibly giving it unique information to begin the thread with. On the x86 this function passes all information through registers with this assignment plan: thread=ESI, processor=ECX, priority=EBX, unique=EAX. The function returns the threadID in EAX.

The thread parameter is a pointer to the beginning of the code the thread is to execute.

The processor parameter allows you to specify which processor you wish the thread to run on. The default of 0 leaves the choice up to the threading function itself, which generally tries to distribute the load evenly. On single processor systems 0 and 1 are effectively the same. If the number is out of range, it is modified by modulo.

With the

priority

parameter....

All threads may be passed by register a single variable that is pointer width. How it is interpreted is completely up to the thread. This

unique

parameter, if it is not ignored, is usually a simple variable or a pointer to a structure of some kind.

STRUCT dptr thread_code dptr threadID dptr processor dptr priority STRUCT END

__TASK.CreateParallelThreads

thread_struct, n

You can create any number of disparate threads of operation in parallel with this function. Information concerning the threads starting code, processor number, and priority are stored in a list of the following small structures:

STRUCT Thread_Struct dptr thread_code dptr threadID dptr processor ;and status dptr priority dptr unique ptr thread_header STRUCT END The arguments passed to this function by register are: thread_struct=ESI, sizeof_struct=ECX, n=EBX and unique=EAX.

__TASK.SpawnThread

processor, priority, unique

__TASK.DestroyThread

threadID

Terminates a thread by giving it a threadID in EAX.

__TASK.DestroyParallelThreads

thread_struct, sizeof_struct, n

Destroys several threads at once by accessing a structure containing a list of thread IDs.

__TASK.SuspendThread

threadID

The thread Priority is dropped to NEVER.

__TASK.ResumeThread

threadID

The thread priority is restored to normal.

__TASK.SleepmsThread

threadID, ms

Wait n number of milliseconds.

__TASK.WaitThread

threadID, UTCAD_TIMEDATE

Wait for specific timedate.

__TASK.Yield

The current task (object/thread) yields.

**

__TASK.ThreadStructures

Returns size of thread structure used by core. etc

MACROS

MUTUAL EXCLUSION LOCKS

Mutal exclusion locks are used to block access to code or data that must be atomic in nature, or in other words, must be allow to execute or change in a certain manner without outside intervention for a specific period. They operate as processor dependant MACRO constructs with Grail, and are software only for the x86 processors.

The locks are priority sensitive.

MUTEX_Clear m1

MUTEX_Lock m1

;fiddle with a set of graphic card ports

MUTEX_Unlock

MUTEX_Clear

mutex

Unlocks a mutex, absolutely

MUTEX_Lock

mutex

Tries to unlock the mutex (mutex++>0?). If mutex is locked, then the thread goes to sleep (via hardware), yielding to the multitasker.

MUTEX_SpinLock

mutex

Tries to unlock the mutex. If mutex is locked, then the thread spins busy until the mutex go unlock.

MUTEX_SkipLock

mutex, branch

Tries to unlock the mutex. If mutex is locked, then the code section is skipped. A short or long branch/jump is used to pass over the exclusive section.

MUTEX_Unlock

mutex

Unlocks the mutex, absolutely.

MESSAGING

MESSAGE_Send

MESSAGE_Wait

TOC

User Authorization

DESIGNER | Lewis Sellers lsellers@usit.net

__USER_AUTHORIZATION_Stream

USERstream, profilestream_length, stream, stream_length

This is the one and only function in this object class. It is used to change profiles (ie, your user identity). This is accomplished by sending both a stream of data identifying the profile you wish access to and it's authorization stream.

The identity stream is usually text entered from the keyboard (your name), and the authorization is usually more of the same (your password). There is absolutely no reason however that you can not set up your profile you require voice identification and/or authentification... or visual. The way this works you can have your profile specify an object that should process the stream inputs and return Truth on success.

This function returns NULL if authorization fails for any reason. If successful it returns a count of the number of times that access has been attempted on the profile.

If the return value would indicate failure, Grail pauses (typically) 5 seconds before returning to discourage brute-force hacking.

__USER_AUTHORIZATION is one of those rare objects (like __SCRIPT) that is automatically called by the core. It resides always in root space for security reasons. You can use the PNF to alias it to something else besides the default.

TOC

Afterword: The P8/PA-RISC and Beyond

BY | Lewis Sellers lsellers@usit.net

Grail at this moment is targeting the Intel x86, the Motorola 68k, the PowerPC and ARM x10 processor architectures. There is also the coming of the P8/PA-RISC to deal with. Machine languages change between platforms, and that's ok. The item that will affect Grail the most is the going from a 32-bit to a 64-bit architecture.

How will that be handled? As quitely as possible. Just as there are versions of the Grail core that come out for the 386, the 486, the Pentium and Pentium Pro's, so will there be those especially tweaked for the P8/PA-RISC chips. And as for Grail Script programs, once you get a new version of the core and it's accompanying assembler, they will all be automatically run at 64-bits.

But do you have to recompile your old Pentium Pro programs for the P8/PA-RISC? No. The core will probably have to change of course, and some new device objects, etc. But all your old programs should still work because Grail specific puts the weight of all the hardware interfacing into the device objects.

Beyond that, we'll just have to wait and see.

This document should be fully HTML 3.2 compliant
(excepting some Javascript timedate stamping).
If you encounter problem while browsing,
then email Lewis Sellers.
Copyright © 1995,1996, Lewis A. Sellers. All Rights Reserved.

TRADEMARKS

The Pentium and Pentium Pro are trademarks of the Intel Corporation.
TASM is a trademark of Borland International.

Grail Millennium Project