Virtual Reality known also as:Synthetic Environments, Cyberspace, Artificial Reality, Simulator Technology.. is a way for humans to visualize, manipulate and interact with computers and extremely complex data. The visualization part refers to the computer generating visual, auditory or other sensual outputs to the user of a world within the computer. This world may be a CAD model, a scientific simulation, or a view into a database. The user can interact with the world and directly manipulate objects within the world. The applications being developed for VR run a wide spectrum, from games to architectural and business planning. Many applications are worlds that are very similar to our own, like CAD or architectural modeling. Some applications provide ways of viewing from an advantageous perspective not possible with the real world, like scientific simulators and telepresense systems, air traffic control systems. Other applications are much different from anything we have ever directly experienced before. These latter applications may be the hardest, and most interesting systems. Visualizing the ebb and flow of the world's financial markets. Navigating a large corporate information base, etc.

Types of VR Systems A major distinction of VR systems is the mode with which they interface to the user. This section describes some of the common modes used in VR systems.

Window on World Systems (WoW) Some systems use a conventional computer monitor to display the visual world. This sometimes called Desktop VR or a Window on a World (WoW). This concept traces its lineage back through the entire history of computer graphics. In 1965, Ivan Sutherland laid out a research program for computer graphics in a paper called "The Ultimate Display" that has driven the field for the past nearly thirty years.

"One must look at a display screen," he said, "as a window through which one beholds a virtual world. The challenge to computer graphics is to make the picture in the window look real, sound real and the objects act real." [quoted from Computer Graphics V26#3]

Video Mapping A variation of the WoW approach merges a video input of the user's silhouette with a 2D computer graphic. The user watches a monitor that shows his body's interaction with the world. Myron Kruger has beenthe pioneer ( exhibited in Virtuality&Interactivity'98) a champion of this form of VR since the late 60's. He has published two books on the subject: "Artificial Reality" and "Artificial Reality II". At least one commercial system uses this approach, the Mandala system. This system is based on a Commodore Amiga with some added hardware and software. A version of the Mandala is used by the cable TV channel Nickelodeon for a game show (Nick Arcade) to put the contestants into what appears to be a large video game.

Immersive Systems The ultimate VR systems completely immerse the user's personal viewpoint inside the virtual world. These "immersive" VR systems are often equipped with a Head Mounted Display (HMD). This is a helmet or a face mask that holds the visual and auditory displays. The helmet may be free ranging, tethered, or it might be attached to some sort of a boom armature. A nice variation of the immersive systems use multiple large projection displays to create a 'Cave' or room in which the viewer(s) stand. An early implementation was called "The Closet Cathedral" for the ability to create the impression of an immense environment. within a small physical space. The Holodeck used in the television series "Star Trek: The Next Generation" is afar term extrapolation of this technology.

Mixed Reality Merging the Telepresence and Virtual Reality systems gives the Mixed Reality or Seamless Simulation systems. Here the computer generated inputs are merged with telepresence inputs and/or the users view of the real world. A surgeon's view of a brain surgery is overlaid with images from earlier CAT scans and real-time ultrasound. A fighter pilot sees computer generated maps and data displays inside his fancy helmet visor or on cockpit displays.

Fish Tank Virtual Reality The phrase "fish tank virtual reality" was used to describe a Canadian VR system reported in the 1993 InterCHI proceedings. It combines a stereoscopic monitor display using LCD Shutter glasses with a mechanical head tracker. The resulting system is superior to simple stereo-WoW systems due to the motion parallax effects introduced by the head tracker. (see INTERCHI '93 Conference Proceedings, ACM Press/Addison Wesley , ISBN 0-201-58884-6)

VR Hardware There are a number of specialized types of hardware devices that have been developed or used for Virtual Reality applications.

Image Generators One of the most time consuming tasks in a VR system is the generation of the images. Fast computer graphics opens a very large range of applications aside from VR, so there has been a market demand for hardware acceleration for a long while

Silicon Graphics Inc. has made a very profitable business of producing graphics workstations. SGI boxes are some of the most common processors found in VR laboratories and high end systems. The simulator market has produced several companies that build special purpose computers designed expressly for real time image generation. These computers often cost several hundreds of thousands of dollars.

Manipulation and Control Devices One key element for interaction with a virtual world, is a means of tracking the position of a real world object, such as a head or hand. There are numerous methods for position tracking and control. Ideally a technology should provide 3 measures for position(X, Y, Z) and 3 measures of orientation (roll, pitch, yaw). One of the biggest problem for position tracking is latency, or the time required to make the measurements and preprocess them before input to the simulation engine. The simplest control hardware is a conventional mouse, trackball or joystick. While these are two dimensional devices, creative programming can use them for 6D controls. There are a number of 3 and 6 dimensional mice/trackball/joystick devices being introduced to the market at this time. These add some extra buttons and wheels that are used to control not just the XY translation of a cursor, but its Z dimension and rotations in all three directions.

The Global Devices 6D Controller is one such 6D joystick It looks like a racket ball mounted on a short stick. You can pull and twist the ball in addition to the left/right & forward/back of a normal joystick. Other 3D and 6D mice, joystick and force balls are available from Logitech, Mouse System Corp. among others. One common VR device is the instrumented glove. The use of a glove to manipulate objects in a computer is covered by a basic patent in the USA. Such a glove is outfitted with sensors on the fingers as well as an overall position/orientation tracker. There are a number of different types of sensors that can be used. VPL (holders of the patent) made several DataGloves, mostly using fiber optic sensors for finger bends and magnetic trackers for overall position. Mattel manufactured the PowerGlove for use with the Nintendo game system, for a short time.

This device is easily adapted to interface to a personal computer. It provides some limited hand location and finger position data using strain gauges for finger bends and ultrasonic position sensors. The gloves are getting rare, but some can still be found at Toys R' Us and other discount stores. Anthony Clifton recently posted this suggestion for a" very good resource for PowerGloves etc.:

The concept of an instrumented glove has been extended to other body parts. Full body suits with position and bend sensors have been used for capturing motion for character animation, control of music synthesizers, etc. in addition to VR applications.

Position Tracking Mechanical armatures can be used to provide fast and very accurate tracking. Such armatures may look like a desk lamp (for basic position/orientation) or they may be highly complex exoskeletons (for more detailed positions). The drawbacks of mechanical sensors are the encumbrance of the device and its restrictions on motion. Exos Systems builds one such exoskeleton for hand control. It also provides force feedback. Shooting Star system makes a low cost armature system for head tracking. Fake Space Labs and LEEP Systems make much more expensive and elaborate armature systems for use with their display systems. Ultrasonic sensors can be used to track position and orientation. A set of emitters and receivers are used with a known relationship between the emitters and between the receivers. The emitters are pulsed in sequence and the time lag to each receiver is measured. Triangulation gives the position. Drawbacks to ultrasonics are low resolution, long lag times and interference from echoes and other noises in the environment.

Magnetic trackers use sets of coils that are pulsed to produce magnetic fields. The magnetic sensors determine the strength and angles of the fields. Limitations of these trackers are a high latency for the measurement and processing, range limitations, and interference from ferrous materials within the fields. However, magnetic trackers seem to be one of the preferred methods. The two primary companies selling magnetic trackers are Polhemus and Ascension. Optical position tracking systems have been developed. One method uses a ceiling grid LEDs and a head mounted camera. The LEDs are pulsed in sequence and the cameras image is processed to detect the flashes. Two problems with this method are limited space (grid size) and lack of full motion (rotations). Another optical method uses a number of video cameras to capture simultaneous images that are correlated by high speed computers to track objects. Processing time (and cost of fast computers) is a major limiting factor here. One company selling an optical tracker is Origin Instruments. Inertial trackers have been developed that are small and accurate enough for VR use. However, these devices generally only provide rotational measurements. They are also not accurate for slow position changes.

The following defines a number of levels of VR hardware systems. These are not hard levels, especially towards the more advanced systems. 1. Entry VR (EVR)

The 'Entry Level' VR system takes a stock personal computer or workstation and implements a WoW system. The system may be based on an IBM clone (MS-DOS/Windows) machine or an Apple Macintosh, or perhaps a Commodore Amiga. The DOS type machines (IBM PC clones) are the most prevalent. There are Mac based systems, but few very fast rendering ones. Whatever the base computer it includes a graphic display, a 2D input device like a mouse, trackball or joystick, the keyboard, hard disk & memory . 2. Basic VR (BVR) The next step up from an EVR system adds some basic interaction and display enhancements. Such enhancements would include a stereographic viewer (LCD Shutter glasses) and a input/control device such as the Mattel PowerGlove and/or a multidimensional (3D or 6D) mouse or joystick.

Advanced VR (AVR) The next step up the VR technology ladder is to add a rendering accelerator and/or frame buffer and possibly other parallel processors for input handling, etc. The simplest enhancement in this area is a faster display card. For the PC class machines, there are a number of new fast VGA and SVGA accelerator cards. These can make a dramatic improvement in the rendering performance of a desktop VR system. Other more sophisticated image processors based on the Texas Instruments TI34020 or Intel i860 processor can make even more dramatic improvements in rendering capabilities. The i860 in particular is in many of the high end professional systems. The Silicon Graphics Reality Engine uses a number of i860 processors in addition to the usual SGI workstation hardware to achieve stunning levels of realism in real time animation. An AVR system might also add a sound card to provide mono, stereo or true 3D audio output. Some sound cards also provide voice recognition. This would be an excellent additional input device for VR applications.

Immersion VR (IVR) An Immersion VR system adds some type of immersive display system: a HMD, a Boom, or multiple large projection type displays (Cave). An IVR system might also add some form of tactile, haptic and touch feedback interaction mechanisms. The area of Touch or Force Feedback (known collectively as Haptics) is a very new research arena.

Cockpit Simulators A common variation on VR is to use a Cockpit or Cab compartment to enclose the user. The virtual world is viewed through some sort of view screen and is usually either projected imagery or a conventional monitor. The cockpit simulation is very well known in aircraft simulators, with a history dating back to the early Link Flight Trainers (1929?). The cockpit is often mounted on a motion platform that can give the illusion of a much larger range of motion. Cabs are also used in driving simulators for ships, trucks, tanks and 'battle mechs'. The latter are fictional walking robotic devices (i.e. the Star Wars films). The BattleTech location based entertainment (LBE) centers use this type of system.

Available VR Software Systems There are currently quite a number of different efforts to develop VR technology. Each of these projects have different goals and approaches to the overall VR technology. Large and small University labs have projects underway (UNC, Cornell, U.Rochester, etc.). ARPA , NIST, National Science Foundation and other branches of the US Government are investing heavily in VR and other simulation technologies. There are industry supported laboratories too, like the Human Interface Technologies Laboratory (HITL) in Seattle and the Japanese NTT project. Many existing and startup companies are also building and selling world building tools (Autodesk, IBM', Sense8, VREAM). There are two major categories for the available VR software: toolkits and authoring systems. Toolkits are programming libraries, generally for C or C++ that provide a set of functions with which a skilled programmer can create VR applications. Authoring systems are complete programs with graphical interfaces for creating worlds without resorting to detailed programming. These usually include some sort of scripting language in which to describe complex actions, so they are not really non-programming, just much simpler programming. The programming libraries are generally more flexible and have faster renders than the authoring systems, but you must be a very skilled programmer to use them. (Note to developers: if i fail to mention your system below, please let me know and I will try to remember to include it when, and if, i update this document again) Here in Virtuality&Ineractivity in Section III you will have the opportinity to experimen VR interactive accomplish throurg different softwares.

Freeware VR Programs At the low end of the VR spectrum are the freeware products and garage or home-brew VR hackers . There are currently a few fast rendering programs that have been released with source code and no charge. These programs are generally copyrighted freeware, which means that the original creators retain the copyright and commercial use is restricted. They are not polished commercial programs, and are often written by students. However, these programs exist to give people a very low cost entry into the VR world.

Aspects of A VR Program Just what is required of a VR program? The basic parts of the system can be broken down into an Input Processor, a Simulation Processor, a Rendering Process, and a World Database. All these parts must consider the time required for processing. Every delay in response time degrades the feeling of 'presence' and reality of the simulation. I.. Input Processes The Input Processes of a VR program control the devices used to input information to the computer. There are a wide variety of possible input devices: keyboard, mouse, trackball, joystick, 3D & 6D position trackers (glove, wand, head tracker, body suit, etc.).

A networked VR system would add inputs received from net. A voice recognition system is also a good augmentation for VR, especially if the user's hands are being used for other tasks. Generally, the input processing of a VR system is kept simple. The object is to get the coordinate data to the rest of the system with minimal lag time. Some position sensor systems add some filtering and data smoothing processing. Some glove systems add gesture recognition. This processing step examines the glove inputs and determines when a specific gesture has been made. Thus it can provide a higher level of input to the simulation. .

Simulation Process The core of a VR program is the simulation system. This is the process that knows about the objects and the various inputs. It handles the interactions, the scripted object actions, simulations of physical laws (real or imaginary) and determines the world status. This simulation is basically a discrete process that is iterated once for each time step or frame. A networked VR application may have multiple simulations running on different machines, each with a different time step. Coordination of these can be a complex task. It is the simulation engine that takes the user inputs along with any tasks programmed into the world such as collision detection, scripts, etc. and determines the actions that will take place in the virtual world.

Rendering Processes The Rendering Processes of a VR program are those that create the sensations that are output to the user. A network VR program would also output data to other network processes. There would be separate rendering processes for visual, auditory, haptic (touch/force), and other sensory systems. Each renderer would take a description of the world state from the simulation process or derive it directly from the World Database for each time step.

Visual Renderer The visual renderer is the most common process and it has a long history from the world of computer graphics and animation. The reader is encouraged to become familiar with various aspects of this technology. The major consideration of a graphic renderer for VR applications is the frame generation rate. It is necessary to create a new frame every 1/20 of a second or faster. 20 frames per second (fps) is roughly the minimum rate at which the human brain will merge a stream of still images and perceive a smooth animation. 24fps is the standard rate for film, 25fps is PAL TV, 30fps is NTSC TV. 60fps is Showscan film rate. This requirement eliminates a number of rendering techniques such as raytracing and radiosity. These techniques can generate very realistic images but often take hours to generate single frames. Visual renderers for VR use other methods such as a 'painter's algorithm', a Z-Buffer, or other Scanline oriented algorithm.

There are many areas of visual rendering that have been augmented with specialized hardware. The Painter's algorithm is favored by many low end VR systems since it is relatively fast, easy to implement and light on memory resources. However, it has many visibility problems. For a discussion of this and other rendering algorithms, see one of the computer graphics reference books listed in a later section. The visual rendering process is often referred to as a rendering pipeline. This refers to the series of sub-processes that are invoked to create each frame.

A sample rendering pipeline starts with a description of the world, the objects, lighting and camera (eye) location in world space. A first step would be eliminate all objects that are not visible by the camera. This can be quickly done by clipping the object bounding box or sphere against the viewing pyramid of the camera. Then the remaining objects have their geometry's transformed into the eye coordinate system (eye point at origin). Then the hidden surface algorithm and actual pixel rendering is done.

The pixel rendering is also known as the 'lighting' or 'shading' algorithm. There are a number of different methods that are possible depending on the realism and calculation speed available. The simplest method is called flat shading and simply fills the entire area with the same color. The next step up provides some variation in color across a single surface. Beyond that is the possibility of smooth shading across surface boundaries, adding highlights, reflections, etc.

An effective short cut for visual rendering is the use of "texture" or "image" maps. These are pictures that are mapped onto objects in the virtual world. Instead of calculating lighting and shading for the object, the renderer determines which part of the texture map is visible at each visible point of the object. The resulting image appears to have significantly more detail than is otherwise possible. Some VR systems have special 'billboard' objects that always face towards the user. By mapping a series of different images onto the billboard, the user can get the appearance of moving around the object.

I need to correct my earlier statement that radiosity cannot be used for VR systems due to the time requirements. There have recently been at least two radiosity renderers announced for walkthrough type systems - Lightscape from Lightscape Graphics Software of Canada and Real Light from Atma Systems of Italy. These packages compute the radiosity lighting in a long time consuming process before hand. The user can interactively control the camera view but cannot interact with the world.

Auditory Rendering A VR system is greatly enhanced by the inclusion of an audio component. This may produce mono, stereo or 3D audio. The latter is a fairly difficult proposition. It is not enough to do stereo-pan effects as the mind tends to locate these sounds inside the head. Research into 3D audio has shown that there are many aspects of our head and ear shape that effect the recognition of 3D sounds. It is possible to apply a rather complex mathematical function (called a Head Related Transfer Function or HRTF) to a sound to produce this effect. The HRTF is a very personal function that depends on the individual's ear shape, etc. However, there has been significant success in creating generalized HRTFs that work for most people and most audio placement. There remains a number of problems, such as the 'cone of confusion' wherein sounds behind the head are perceived to be in front of the head. Sound has also been suggested as a means to convey other information, such as surface roughness. Dragging your virtual hand over sand would sound different than dragging it through gravel.

Haptic Rendering Haptics is the generation of touch and force feedback information. This area is a very new science and there is much to be learned. There have been very few studies done on the rendering of true touch sense (such as liquid, fur, etc.). Almost all systems to date have focused on force feedback and kinesthetic senses. These systems can provide good clues to the body regarding the touch sense, but are considered distinct from it. Many of the haptic systems thus far have been exo-skeletons that can be used for position sensing as well as providing resistance to movement or active force application.

Other Senses The sense of balance and motion can be served to a fair degree in a VR system by a motion platform. These are used in flight simulators and some theaters to provide some motion cues that the mind integrates with other cues to perceive motion. It is not necessary to recreate the entire motion perfectly to fool the mind into a willing suspension of disbelief. The sense of temperature has seen some technology developments. There exist very small electrical heat pumps that can produce the sensation of heat and cold in a localized area. These system are fairly expensive. Other senses such as taste, smell, pheromone, etc. are beyond our ability to render rapidly and effectively. Sometimes, we just don't know enough about the functioning of these other senses.

World Space The virtual world itself needs to be defined in a 'world space'. By its nature as a computer simulation, this world is necessarily limited. The computer must put a numeric value on the locations of each point of each object within the world. Usually these 'coordinates' are expressed in Cartesian dimensions of X, Y, and Z (length, height, depth). It is possible to use alternative coordinate systems such as spherical but Cartesian coordinates are the norm for almost all applications. Conversions between coordinate systems are fairly simple (if time consuming).

.. World Coordinates A major limitation on the world space is the type of numbers used for the coordinates. Some worlds use floating point coordinates. This allows a very large range of numbers to be specified, with some precision lost on large numbers. Other systems used fixed point coordinates, which provides uniform precision on a more limited range of values. The choice of fixed versus floating point is often based on speed as well as the desire for a uniform coordinate field.

A World Divided: Separation of Environments One method of dealing with the limitations on the world coordinate space is to divide a virtual world up into multiple worlds and provide a means of transiting between the worlds. This allows fewer objects to be computed both for scripts and for rendering. There should be multiple stages (aka rooms, areas, zones, worlds, multiverses, etc.) and a way to move between them (Portals).

World Database The storage of information on objects and the world is a major part of the design of a VR system. The primary things that are stored in the World Database (or World Description Files) are the objects that inhabit the world, scripts that describe actions of those objects or the user (things that happen to the user), lighting, program controls, and hardware device support.

Storage Methods There are a number of different ways the world information may be stored: a single file, a collection of files, or a database. The multiple file method is one of the more common approaches for VR development packages. Each object has one or more files (geometry, scripts, etc.) and there is some overall 'world' file that causes the other files to be loaded. Some systems also include a configuration file that defines the hardware interface connections. Sometimes the entire database is loaded during program startup, other systems only read the currently needed files. A real database system helps tremendously with the latter approach. An Object Oriented Database would be a great fit for a VR system, but I am not aware of any projects currently using one. The data files are most often stored as ASCII (human readable) text files. However, in many systems these are replaced by binary computer files. Some systems have all the world information compiled directly into the application.

Objects Objects in the virtual world can have geometry, hierarchy, scripts, and other attributes. The capabilities of objects has a tremendous impact on the structure and design of the system. In order to retain flexibility, a list of named attribute/values pairs is often used. Thus attributes can be added to the system without requiring changes to the object data structures. These attribute lists would be addressable by name (i.e. cube.mass => mass of the cube object). They may be a scalar, vector, or expression value. They may be addressable from within the scripts of their object. They might be accessible from scripts in other objects.

Position/Orientable An object is positionable and orientable. That is, it has a location and orientation in space. Most objects can have these attributes modified by applying translation and rotation operations. These operations are often implemented using methods from vector and matrix algebra.

Hierarchy An object may be part of an object part HIERARCHY with a parent, sibling, and child objects. Such an object would inherit the transformations applied to it's parent object and pass these on to it's siblings and children. Hierarchies are used to create jointed figures such as robots and animals. They can also be used to model other things like the sun, planets and moons in a solar system.

Bounding Volume Additionally, an object should include a BOUNDING VOLUME. The simplest bounding volume is the Bounding Sphere, specified by a center and radius. Another simple alternative is the Bounding Cube. This data can be used for rapid object culling during rendering and trigger analysis. Objects whose bounding volume is completely outside the viewing area need not be transformed or considered further during rendering. Collision detection with bounding spheres is very rapid. It could be used alone, or as a method for culling objects before more rigorous collision detection algorithms are applied.

Object Geometry The modeling of object shape and geometry is a large and diverse field. Some approaches seek to very carefully model the exact geometry of real world objects. Other methods seek to create simplified representations. Most VR systems sacrifice detail and exactness for simplicity for the sake of rendering speed. The simplest objects are single dimensional points. Next come the two dimensional vectors. Many CAD systems create and exchange data as 2D views. This information is not very useful for VR systems, except for display on a 2D surface within the virtual world. There are some programs that can reconstruct a 3D model of an object, given a number of 2D views. The sections below discuss a number of common geometric modeling methods. The choice of method used is closely tied to the rendering process used. Some renderers can handle multiple types of models, but most use only one, especially for VR use. The modeling complexity is generally inversely proportional to the rendering speed. As the model gets more complex and detailed, the frame rate drops.

3D PolyLines & PolyPoints The simplest 3D objects are known as PolyPoints and PolyLines. A PolyPoint is simply a collection of points in space. A Polyline is a set of vectors that form a continuous line.

Polygons the most common form of objects used in VR systems are based on flat polygons. A polygon is a planar, closed multi-sided figure. They maybe convex or concave, but some systems require convex polygons. The use of polygons often gives objects a faceted look. This can be offset by more advanced rendering techniques such as the use of smooth shading and texture mapping. Some systems use simple triangles or quadrilaterals instead of more general polygons. This can simplify the rendering process, as all surfaces have a known shape. However, it can also increase the number of surfaces that need to be rendered. Polygon Mesh Format (aka Vertex Join Set) is a useful form of polygonal object. For each object in a Mesh, there is a common pool of Points that are referenced by the polygons for that object. Transforming these shared points reduces the calculations needed to render the object. A point at the edge of a cube is only processed once, rather once for each of the three edge/polygons that reference it. The PLG format used by REND386 is an example of a Polygonal Mesh, as is the BYU format used by the 'ancient' MOVIE.BYU program.) . The geometry format can support precomputed polygon and vertex normals. Both Polygons and vertices should be allowed a color attribute. Different renderers may use or ignore these and possibly more advanced surface characteristics. Precomputed polygon normals are very helpful for backface polygon removal. Vertices may also have texture coordinates assigned to support texture or other image mapping techniques.

Primitives Some systems provide only Primitive Objects, such as cubes, cones, and spheres. Sometimes, these objects can be slightly deformed by the modeling package to provide more interesting objects.

Solid Modeling & Boolean Operations Solid Modeling (aka Computer Solid Geometry, CSG) is one form of geometric modeling that uses primitive objects. It extends the concept by allowing various addition, subtraction, Boolean and other operations between these primitives. This can be very useful in modeling objects when you are concerned with doing physical calculations, such as center of mass, etc. However, this method does incur some significant calculations and is not very useful for VR applications. It is possible to convert a CSG model into polygons. Various complexity polygonal models (# polygons) could be made from a single high resolution ''metaobject" of a CSG type.

Curves & Patches Another advanced form of geometric modeling is the use of curves and curved surfaces (aka patches). These can be very effective in representing complex shapes, like the curved surface of an automobile, ship or beer bottle. However, there is significant calculation involved in determining the surface location at each pixel, thus curve based modeling is not used directly in VR systems. It is possible, however, to design an object using curves and then compute a polygonal representation of those curved patches. Various complexity polygonal models could be made from a single high resolution 'metaobject'.

Dynamic Geometry (aka morphing) It is sometimes desirable to have an object that can change shape. The shape might simply be deformed, such a bouncing ball or the squash/stretch used in classical animation ('toons'), or it might actually undergo metamorphosis into a completely different geometry. The latter effect is commonly known as 'morphing' and has been extensively used in films, commercials

Beginnings Virtual reality may have popped into the headlines only in the past few years, but its roots reach back four decades. It was in the late 1950s, jthat an idea arose that would change the way people interacted with computers and make possible VR. At the time, computers were hulking Goliaths locked in air-conditioned rooms and used only by those conversant in esoteric programming languages. Few people considered them more than glorified adding machines. But a young electrical engineer and former naval radar technician named Douglas Engelbart viewed them differently. Rather than limit computers to number crunching, Engelbart envisioned them as tools for digital display. He knew from his days with radar that any digital information could be viewed on a screen. Why not, he then reasoned, connect the computer to a screen and use both to solve problems?

Opportunity and timing At first, Engelbart's ideas were dismissed, but by the early 1960s other people were thinking the same way. Moreover, the time was right for his vision of computing. Communications technology was intersecting with computing and graphics technology. The fi rst computers based on transistors rather than vacuum tubes became available. This synergy yielded more user-friendly computers, which laid the groundwork for personal computers, computer graphics, and later on, the emergence of virtual reality. Several pivotal events marked the decade:

Fear of nuclear attack prompted the U.S. military to commission a new radar system that would process large amounts of information and immediately display it in a form that humans could readily understand. The resulting radar defense system was the first "real time," or instantaneous, simulation of data.

Advanced Research Projects Agency Aircraft designers began experimenting with ways for computers to graphically display, or model, air flow data. Computer experts began restructuring computers so they would display these models as well as compute them. The designers' work paved the way fo r scientific visualization, an advanced form of computer modeling that expresses multiple sets of data as images and simulations.

Massachusetts Institute of Technology An infusion of self-styled computer wizards strove to reduce the barriers to human interactions with the computer by replacing keyboards with interactive devices that relied on images and hand gestures to manipulate data. In 1962 Ivan Sutherland developed a light pen with which images could be sketched on a computer. Sutherland's first computer-aided design program, called Sketchpad, opened the way for designers to use computers to create blueprints of automobiles, cities, and industrial products. By the end of the decade, the designs were operating in real time. By 1970, Sutherland also produced a primitive head-mounted display and Engelbart unveiled his crude pointing device for moving text around on a computer screen -- the first "mouse." One of the most influential antecedents of virtual reality was the flight simulator. Following World War II and through the 1990s, the military and industrial complex pumped millions of dollars into technology to simulate flying airplanes (and later driving tanks and steering ships).

Evans & Sutherland Then as now, it was cheaper, and safer, to train pilots on the ground before subjecting them to the hazards of flight. The early flight simulators consisted of mock cockpits built on motion platforms that pitched and rolled. A limitation, however, was they lacked visual feeback. This changed when video displays were coupled with model cockpits.

Evans & Sutherland By the 1970s, computer-generated graphics had replaced videos and models. These flight simulations were operating in real time, though the graphics were primitive. In 1979, the military experimented with head-mounted displays. These innovations were driven by the greater dangers associated with training on and flying the jet flighters that were being built in the 1970s. By the early 1980s, better software, hardware, and motion-control platforms enabled pilots to navig ate through highly detailed virtual worlds. Of course, the "military-industrial complex" was not the only entity interested in computer graphics.

A natural consumer of computer graphics was the entertainment industry, which, like the military and industry, was the source of many valuable spin-offs in virtual reality.

By the 1970s, some of Hollywood's most dazzling special effects were computer-generated, such as the battle scenes in the big-budget, blockbuster science fiction movie Star Wars, which was released in 1976. Later came such movies as Terminator and Jurassic Park. In the early 1980s, the video game business boomed.

National Aeronautics and Space Administration One direct spin-off of entertainment's venture into computer graphics was the dataglove, a computer interface device that detects hand movements. It was invented to produce music by linking hand gestures to a music synthesizer. NASA Ames was one of the first customers for this new computer input device for its experiments with virtual environments. But the biggest consumer of the "dataglove" was the Mattel company, which adapted it into the PowerGlove, the pervasive mitt with which children conquered adversaries in the popular Nintendo game.

So you want to explore digital, 3-D worlds in real time? With appropriate hardware and software you can do just that. Choose between various head- or boom-mounted displays. Or, surround yourself in digital imagery by stepping into a virtual reality room, the CAVE. Described in the following documents are the most common virtual reality systems and their associated interface devices that, like 3D mouses, allow you to control and manipulate the virtual world. Also presented is a display of VR architecture. This example of a typical VR system will give you an idea of how computing hardware and software work together to generate a virtual world. In fact, a visit to a VR system would be a good place to start.

Interactivity would have remained wishful thinking if not for the development of high-performance computers in the mid-1980s. These machines provided the speed and memory for programmers and scientists to begin developing advanced visuali zation software programs. By the end of the 1980s, low-cost, high-resolution graphic workstations were linked to high-speed computers, which made visualization technology more accessible. All the basic elements of VR had existed since 1980, but it took high-performance computers, with their powerful image rendering capabilities, to make it work. Demand was rising for visualization environments to help scientists comprehend the vast amounts of data pouring out of their computers daily. As drivers for both computation and VR, high-performance computers no longer served as mere number crunchers, but became exciting vehicles for exploration and discovery.

"In the totally immersive virtual environment before you, you have the ability to walk around in any of the ancient structures at this site, pick up the artifacts, watch virtual inhabitants, study their behavior, and even rebuild parts of the ancient site based on the evidence you will find in the virtual scene. You may even redo all or part of your reconstruction over and over, testing alternative views, until the outcome matches the actual archaeological record as you have learned about it from your remote instructor or from speaking to the virtual inhabitants." Both situations just a bit too fantastic? Fine for the special effects wizards of science fiction, but not for serious public education? We will see how close reality is to imagination when we reach the end of this paper. " Donald Sanders -archeologist-learning site

"Since when Virtual Reality has been around??' . Since the 1920s when Link Corporation manufactured training devices that simulated fighter plane cockpits

The status of VR pioneer is often given to Ivan Sutherland, who first proposed the use of stereographic head-mounted displays in the early 1960s so that users could look around a computer-generated room simply by turning their heads.

In the early 1970s, Myron Krueger coined the term "artificial reality" and began developing computer-controlled responsive environments (Krueger 1993).

About the same time, the MIT Media Lab produced a simulated tour through Aspen Colorado, in which participants could drive down a virtual street and enter and explore virtual buildings. The 1980s brought rapid changes to VR technology.

Jaron Lanier, a founder of VPL Research, Inc., is credited with coining the expression "virtual reality" to distinguish between the immersive environments he was creating and traditional computer simulations.

Thomas Zimmerman, co-founder of VPL, worked with Lanier to develop a glove for grasping computer-generated objects in virtual worlds.

NASA developed goggles that allowed the wearer to look around a graphic landscape portrayed on a computer screen while hearing synthesized speech and 3D (binaural) sounds, and grabbing objects with their hands.

Communication and feedback with a computer-simulated environment was direct; no contact with the computer was needed.

In 1992, the movie Lawnmower Man introduced the concept of virtual reality to the public. By the mid-1990s it was possible to reach out and touch virtual objects, to feel different textures and sensations; and perfume companies experimented with virtual smells to send odors electronically from lab to lab.

There is no generally agreed-upon definition of virtual reality, and several distinct types of virtual reality have now emerged--artificial reality, augmented reality, immersive reality, telepresense, and CAVEs. All provide different degrees of immersion, interactivity, and unencumbered navigation

"virtual reality" means an interactive, self-directed, multisensory, computer-generated experience providing the illusion of participating in a synthetic 3-dimensional (3D) environment.

Virtual reality, Java, and the Internet make all that possible right now. We can re-create ancient built environments, and within them we can re-place the artifacts as they were found and link to those objects texts, photographs, and narrations to provide the virtual visitor with information tailored to specific grade levels or curriculum guidelines . Picture the new learning tool that results--the virtual world itself becomes the visual index to an entire dataset. Should an instructor or student desire information about any subject, object, or space, just walk up to that item and click on it to retrieve links to the photographic archive or written descriptions about that item and associated items at the site.

View maps, plans, early travelers' drawings, previous excavators' notebooks, or museum records, all with a click of the mouse. Interactive inquiry need not proceed in the mostly linear fashion of standard books, educational materials need not be limited by the cost of printing color plates, searches need not be restricted to words alone, since links can be made to and from 3D objects or pictures or virtual locations. Add to that the capability of listening as an expert explains about the building's spatial organization, function, or date. This we can do today.

Three-dimensional, interactive, archaeological digital databases, if accessed using virtual reality, thus become dynamic media re-creating an ancient world in its original complexity creating a near firsthand experience. Data about a site that has become globally distributed can be brought back into a single virtual environment allowing students and teachers to view the objects in their original architectural context for the first time since antiquity. This is an unprecedented educational opportunity.

VRML makes possible the inclusion of 3D models into multimedia environments on systems that are within the financial range of pubic institutions and may allow for fulfilling the promise of distance education. Right now, thanks to VRML, the full power of 3D environments can be combined with the full power of multimedia to create an unrivaled learning experience, promoting awareness of past civilizations, understanding of different cultures, and appreciation of different places, peoples, and their cultural heritage (Sanders & Gay 1996a). Previous virtual reality environments have emphasized real-time response and the immersive qualities of the experience. However, the wider application of those worlds for education or research is limited because: (1) expensive hardware and software are required; (2) text display is rudimentary; and (3) users are unable to browse related text or pictures.

VRML has changed that, making possible the integration of 3D data, standard HTML 2D text, pictures, and video into a World Wide Web page, permitting simultaneous viewing of 3D and 2D information. Three-dimensional environments can now be used for what they do best: allow users to gain a full understanding of a spatial structure through self-directed exploration while retaining all the power and detail provided by the 2D text and graphics of standard Web pages.

Links between 2D and 3D data make it possible to click on a specific detail in the 3D environment and either switch to a different 3D model or bring up supporting text or pictures in a separate scrollable frame on the same window or in a different window

Hot spots in the text can affect the 3D portion of the screen and hot objects in the 3D environment can access and change the text and graphics on the Web page. Objects can move, change and react to the user. Sound can provide ambient context and additional information through narration. HTML-coded pages can provide a curriculum framework. Anchored viewpoints and labeled locations allow the creation of self-guided or teacher-driven explorations of archaeological sites, each emphasizing different aspects of the data, from daily life of the ancient inhabitants to archaeological methodologies.

artificial reality	Simulated spaces created from a combination of computer and video systems
augmented reality	The use of transparent glasses on which a computer displays data so that the viewer can simultaneously view computer generated and real world scenes. See also avatar: A user's graphical persona inside a virtual world.
avatar	The term avatar is derived from Sanskrit and relates to a "mental traveller" in Indian fairy tales. An avatar - my virtual representative - communicates with others in a V environment.
binocular omni	Orientation Monitor (BOOM): A 3-D display device suspended from a weighted boom that can swivel freely so the viewer can use the device by bringing the device up to the eyes and viewing the 3-D environment while holding it. The boom's position and orientation communicates the user's point of view to the computer.

artificial reality: Simulated spaces created from a combination of computer and video systems.

augmented reality: The use of transparent glasses on which a computer displays data so that the viewer can simultaneously view computer generated and real world scenes. See also avatar: A user's graphical persona inside a virtual world.

avatar: The term avatar is derived from Sanskrit and relates to a "mental traveller" in Indian fairy tales. An avatar - my virtual representative - communicates with others in a V environment.

binocular omni-Orientation Monitor (BOOM): A 3-D display device suspended from a weighted boom that can swivel freely so the viewer can use the device by bringing the device up to the eyes and viewing the 3-D environment while holding it. The boom's position and orientation communicates the user's point of view to the computer.

browser: Overviews, such as indexes, lists or animated maps, to provide a means of navigating through the physical, temporal, and conceptual elements of a virtual world.

cave: VR world projected on the walls and ceiling of a room to give the illusion of immersion. cue conflict: A theory to explain the kind of motion sickness caused when the body tries to interpret conflicting clues being received by the senses. Frequent causes are faulty calibration of eye devices or delay between the sensory inputs and output display. computer graphics: (1) The branch of computer science concerned with methods of creating, modifying, or analyzing pictorial data. (2) The use of a computer in any discipline to create, modify, or analyze images. coordinates: A set of data values that determine the location of a point in a space. The number of coordinates corresponds to the dimensionality of the space.

cyberspace: A computer synthesized reality. Often a computer synthesized 3-D space. See also: virtual reality.

cyborg: A robotic humanoid modeled directly from digital readings of a real human and transformed into a photo realistic, animated character produced via illusionary metamorphosis. data sonification: Assignment of sounds to digitized data which may involve filtering to give illusion of localized sound. data spacialization: Assignment of orientation (yaw, pitch) and position coordinates (x,y,z) to digital sounds assigned to data.

data glove: A glove wired with sensors and connected to a computer system for gesture recognition and navigation through a virtual environment. Known generically as a "wired glove." deformable object technology

(DOT): Virtual objects which bend and deform appropriately when touched. depth cueing: Use of shading, texture, color, interposition, or other visual characteristics to provide a cue for the distance of an object from the observer.

dolly shot: Display of a scene while moving forward or backward. See also pan shot and track shot.

doppler effect (VR): An apparent increase in the frequency of sound or light as its source approaches an observer or a decrease if it moves away.

dynamic lighting: Changes in lighting effects as objects or the observer move. dynamics: The rules that govern all actions and behaviors within the environment.

effectors: Interfacing devices used in virtual environments for input/output, tactic sensation and tracking. Examples are gloves, head mounted displays, headphones, and trackers. egocenter: The sense of one's own location in a virtual environment.

environment: In VR terms, this is a computer-generated model that can be experienced by an observer as if it were a place. exoskeleton: mechanically linked structure for control of an feedback from an application.

eye clearance: The most accurate figure of merit used to describe the HMD positioning relative to the eye.

eye tracking: Measurement of the direction of gaze. eyeball in the hand: A metaphor for visualized tracking where the tracker is held in the hand and is connected to motion of the projection point of the display.

field of view (FOV): The angle in degrees of the visual field. Since a human's two eyes have overlapping 140 degree FOV, binocular or total FOV is roughly 180 degrees in most people. A feeling of immersion arises when the FOV is greater than roughly 60 to 90 degrees.

fish tank VR: The ego center of an observer looking "through" a computer monitor to a virtual outside world using a stereoscopic display system. That is, to a person looking through a stereo "window" to a virtual "outside", the person imagines him/herself to be in a fish tank. force feedback: An output device that transmits pressure, force or vibrations to provide the VR participant with the sense of resisting force, typically to weight or inertia. This is in contrast to tactile feedback, which simulates sensation applied to the skin. fractal: A self-similar graphical pattern generated by using the same rules at various levels of detail. That is, a graphical pattern that repeats itself on a smaller and smaller scale. frustum of vision: Three-dimensional field of view in which all modeled objects are visible.

gouraud shading: The shading of polygons smoothly with bilinear interpolation.

haptic interfaces: Use of physical sensors to provide users with a sense of touch at the skin level, and force feedback information from muscles and joints.

head tracking: Monitoring the position and orientation of the head through various tracking device.

head-coupled: Displays or robotic actions that are activated by head motion through a head tracking device.

head-related transfer function: A mathematical transformation of sound spectrum that modifies the amplitude and phase of acoustic signals to take into account the shape effects of the listener's head.

heads-up display: A display device that allows users to see graphics superimposed on their view of the real world.

hidden surface: A surface of a graphics object that is occluded from view by intervening objects.

head mounted display (HMD): A set of goggles or a helmet with tiny monitors in front of each eye to generate images seen by the wearer as three-dimensional. Often the HMD is combined with a head tracker so that the images displayed in the HMD change as the head moves.

image distance: Perceived distance to the object. (In contrast to the real object distance, if there exists a real object.)

immersion: The observer's emotional reaction to the virtual world as being part of it.

interactivity: The prefix "inter" means 'between' a or 'in the middle of a'. Activity signifies active behaviour and was borrowed from the Latin word 'agere' in the 17th century. In the context of digital media, the concept of interactivity signifies mutual communication between sender and recipient - the "author" can become the "user" and the "user" can become the "author".

interaural amplitude: Differences between a person's two ears in the intensity of a sound, typically due to the location of the sound.

interaural time: Differences between a person's two ears in the phase of a sound, typically due to the location of the sound.

interface (VR): A set of devices, software, and techniques that connect computers with people to perform tasks.

interface: A boundary across which two systems communicate. An interface might be a hardware connector used to link to other devices, or it might be a convention used to allow communication between two software systems. Often there is some intermediate component between the two systems which connects their interfaces together.

Internet: A world wide digital network capable of supporting shared virtual worlds.

inverse kinematics: A specification of the motion of dynamic systems from properties of their joints and extensions.

joystick: An input device that consists of a short lever gripped with one hand to be moved from side to side or towards and away from the person. Frequently it is used to navigate in a virtual space.

kinesthetic dissonance: Mismatch between feedback or its absence from touch or motion during VR experiences.

kinesthesis: Sensations derived from muscles, tendons and joints and stimulated by movement and tension.

latency (VR): Lag between user motion and tracker system response, sometimes measured in from as. Delay between actual change in position and reflection by the program. Delayed response time.

liquid crystal display (LCD): Display devices that use bipolar films sandwiched between this panes of glass. They are lightweight and transmissive or reflective, and are often used in HMDs.

level of detail (LOD) (VR): A model of a particular resolution among a series of models of the same object. Greater graphic performance can be obtained by using a lower LOD when the object occupies fewer pixels on the screen or is not in a region of significant interest.

magic wand: A three-dimensional input device used for pointing and interaction. A kind of three-dimensional mouse.

metaball: A surface defined about a point specified by a location, a radius, and an "intensity." When two metaballs come in contact, their shapes blend together.

metallic distortion: Noise interference or degraded performance in electromagnetic trackers when used near large metallic objects.

model (VR): A computer-generated simulation of something real. motion parallax: A means whereby the eyes can judge distance by noticing how closer objects appear to move more than distant ones when the observer moves. See also {{binocular parallax}} and parallax.

motion platform: A controlled physical system that provides real motion to simulate the displayed motion in a VR world.

navigation: The term "navigation" signifies the definition of and adherence to a course and is derived from the Latin "navigare" which can be translated as steering, sailing or travelling. The same symbols are used on the Internet as in real space - though virtual navigation involves the "re-configuring" - i.e. production - of a time process.

objects (VR): Discrete 3-D shapes within the virtual world that a user can interact with.

occipital cortex: The back of the brain receiving retinotopic projections of visual displays.

occlusion: Hiding an object or a portion of an object from sight by interposition of other objects.

pan: The angular displacement of a view along any axis of direction in a three-dimensional world.

pan shot: Display of a scene while moving about any axis. See also dolly shot and track shot.

parallax: The difference in viewing angle created by having two eyes looking at the same scene from slightly different positions, thereby creating a sense of depth. (Also referred to as binocular parallax.) See also motion parallax parietal cortex: An area of the brain adjacent and above the occipital cortex, though to process spatial location and direction information.

perspective: The rules that determine the relative size of objects on a flat viewing surface to give the perception of depth.

phong shading: A method for calculating the brightness of a surface pixel by linearly interpolating points on a polygon and using the cosine of the viewing angle. Produces realistic shading.

photo realism: An attempt to create realistic appearing images with great detail and texture.

pitch (VR): The angular displacement of the lateral axis about a horizontal axis perpendicular to the lateral axis.

pixel (VR): The smallest element of a display that can be adjusted in intensity (see IEEE dictionary).

platform (VR): Individual or shared VR displays built into physical mockups of vehicles and other physical settings.

polygon: A display element that consists of an area enclosed by a set of by a set of broken straight lines.

portal: Polygons or icons that a user can pass through in a virtual space to automatically load a new world or execute a user-defined function. A three-dimensional version of an interactive icon in multimedia.

position sensor: A tracking device that provides information about its location and/or orientation.

position trigger: A hotspot, sensitive spot, or button that causes a change in the computer program when touched in some way.

presence: A feeling of being immersed in an environment, able to interact with object there. A defining characteristic of a VR system.

radiosity: A diffuse illumination calculation system for graphics based on energy balancing that takes into account multiple reflections off many walls.

ray tracing: A technique for displaying a three-dimensional object with shading and shadows by tracing light rays backward from the viewing position to the light source.

real time (VR): Action taking place with no perceptible or significant delay after the input that initiates the action.

real-time imaging: Graphics or images synchronized with real-world time and events. reality engine. A computer system for generating virtual objects and environments in response to user input, usually in real time.

refresh rate: The frequency with which an image is regenerated on a display surface (see IEEE dictionary).

retinal binocular disparity (RBD): Ratio of the convergence angle of the image to the convergence angle of the object.

roll (VR): Angular displacement about the lateral axis. (See IEEE Dictionary.)

scenes view: Virtual display viewed on a large screen or through a terminal window rather than with immersive devices. semiocclusion: Occlusion to one eye only.

shared worlds: Virtual environments that are shared by multiple participants. shutter glasses: Glasses that alternately block out the left and right eye views in synchrony with the computer display of left and right eye images to provide stereoscopic images on the computer screen. {{photo}}

simulator sickness: Various disturbances, ranging in degree form a feeling of unpleasantness, disorientation, and headaches to extreme nausea, caused by various aspects of a simulator. Possible factors include sensory distortions such as abnormal movement of arms and heads because of the weight of equipment, long delays or lags in feedback, and missing visual cues from convergence and accommodation.

six degrees of freedom (6DOF): Ability to move in three spatial directions and orient about three axes passing through the center of the body. Thus the location and orientation are specified by six coordinates.

spatial navigation: Self-orientation and locomotion in virtual worlds. stereopsis: Binocular vision of images with different views by the two eyes to distinguish depth. tactile displays: Devices that provide tactile and kinesthetic sensations.

technsplanation: Use of VR technology and other communication technology to explain or teach.

teleos(TM): A tool to create Silicon Graphics computer-based real-time interactive environments with "life-like" Deformable objects.

teleoperator: Person doing telemanipulation. telepresence: Remote control with adequate sensory data to give the illusion of being at that remote location.

temporal lobe: An area of the brain in front of the occipital cortex and the parietal cortex which is the receiving site for hearing. terrain: Geographical information and models that can be either randomly generated or based on actual data.

three-dimensional graphics: The presentation of data on a two-dimensional display surface so that it appears to represent a three dimensional model.

track shot: Rotating display of the same scene. See also dolly shot and pan shot.

tracker (VR): A device that provides numeric coordinates to identify the current position and/ or orientation of an object or user in real space.

universe: The collection of all entities and the space they are embedded in for a VR world.

viewpoints: Points from which raytracing and geometry creation occurs. The geometric eye point of the simulation.

virtual MIS: Use of computer models and specialized interaction devices that mimic surgical tools to allow medical personnel to practice minimally invasive surgery (MIS) procedures.

virtual prototype: Simulation of an intended design or product to illustrate the characteristics before actual construction. Usually used as an exploratory tool for developers or as a communications prop for persons reviewing proposed designs.

virtual reality: A computer system used to create an artificial world in which the user has the impression of being in that world with the ability to navigate through the world and manipulate objects in the world.

virtual world: Whole virtual environment or universe within a given simulation. visualization: The ability to graphically represent abstract data that would normally appear as text and numbers on a computer.

world in the hand: A metaphor for visualized tracking where a tracker is held in the hand and is connected to the motion of an object in a display.