The origin of the computer
Digital Media, animation,simulation

DEVELOPMENT OF COMPUTER AND COMMUNICATIONS TECHNOLOGY
Historical development of computer interactivity historical context for the development and use of real time computing, timesharing, networking, software, and graphical user interfaces.

Computing Before Computers

While calculating technologies (numbering systems, counting stones, counting sticks, etc.) have existed since antiquity, machines for calculation date only from the seventeenth century. Until the late nineteenth century, these machines were curiosities found in gentlemen' s scientific cabinets.

In the late nineteenth century, however, big businesses emerged, management and ownership were separated, and tools were developed as control items for the professional managerial class. Mechanical desk calculators were one of these new office machines, together with the typewriter, cash register, duplicating equipment, and filing systems. In the half century from 1890 to 1940, the desk calculator was improved, with such new features as printing mechanisms and electromechanical power, and the technology was spread across a wide range of large and small businesses. In this same half century, three other calculating technologies emerged. A punched card tabulating system was developed for assembling and counting large amounts of data for the 1890 U.S. population census. This technology continued to be improved in this period and leased to government agencies and large companies, such as the railroads and insurance companies, to process their paperwork and keep track of inventories.

This was the origin of IBM. A completely independent tradition of analog computing developed in the engineering community, especially in the electric power industry, which became established in this same era.
Analog computing devices were useful for many physical problems involving continuous instead of discrete variable, but they tended to operate like laboratory equipment and be applicable to only one specific purpose. In the 1930s there were a few attempts to build automatic calculators that could conduct large numbers of calculations without human intervention, so as to meet the needs of scientists.

THE INVENTION OF THE COMPUTER

The Invention of the Computer All three of these calculating traditions influenced the electronic, stored program computer or "computer" for short. The computer emerged from the Allied effort in World War II to calculate ballistic tables for the many new guns and shells that were being introduced.

The story of the ENIAC calculator, developed at the University of Pennsylvania, and the follow-on stored-program computer EDVAC is well known and need not be repeated here. It is useful, however, to emphasize three features about the EDVAC: its electronic switching, which made it fast enough to carry out a wide variety of applications; its digital representation of numbers and method of calculation, which gave it enough precision and tractability to make it amenable to general- rather than special-purpose use; and its stored program capability, which allowed it to be programmed for different tasks and to carry them out automatically, without human intervention, once the instructions and data had been entered.

Its wide applicability is one main reason why the computer has been such an important and pervasive technology in the late twentieth century.

Many people use computers today to draw, or compose letters, or send electronic mail- none of which were anticipated applications in the early postwar years. Given the scientists, engineers, and military personnel who were involved in the design of the first computers, it is not surprising that there was only one kind of application in mind- a super calculator that could do the large numbers of calculations required for certain kinds of large military or scientific applications.

It is because of this conception that many people predicted that the United States would need no more than perhaps ten such machines to meet national demand. This is part of the same tradition as those people building automatic calculators in the 1930s. Although the computer eventually made obsolete all the older kinds of calculating technologies, except for inexpensive calculators and a few special-purpose calculating devices, these earlier types had a strong shaping force on the design, use, and understanding of computers.

Engineers from the burgeoning postwar aerospace industry built hybrid analog-digital computers to design aircraft wings. Computers were too expensive to be used in place of desktop calculators by most companies in the early postwar years, but this was to change in the 1970s when minicomputers and timesharing services became widely available.

Large businesses and government agencies that had been using punched card tabulating systems soon changed over to computers supplied by the computer manufacturing industry that grew up within a decade of the invention of the computer- an industry in which the most successful companies were the business machine manufacturers such as IBM, Burroughs, and NCR.

The design requirements for the data processing machine were different from those of the scientific calculator; the scientific calculator did many precise calculations with relatively little input and output of data, whereas the data processing machines did few, relatively imprecise calculations with large amounts of data input and output.

It was the market for data processing computers that allowed the computer industry to grow in the late 1950s and 1960s. There was one similarity between the scientific calculators and the data processing machines. Both operated in batch processing mode.

Large numbers of small problems were collected (placed together in a batch), generally on punched cards or punched paper tape, and run through the machine at one time.

There were two main reasons for this: it enabled the capital-intensive computers to be used efficiently, and it was somewhat easier to build machines that ran in this way. The efficiency of the operation of the machine came, however, at the expense of the efficiency of the user.

The user would submit his or her cards to the machine' s operator, and they would be stored until the machine was ready for the batch in which the program was included.

Usually a day went by before the user would get the results, and as often as not the computer would not have done any meaningful calculation because of the existence of a missing parenthesis in the instructions, or some other minor syntactical error in the machine- language that was the only language the machine would understand.

The batch mode was thus highly non-interactive: the time between human input and machine response was too great, the language of communication was too machine-like, and there existed an intermediary (the computer operator) between machine and user. Yet batch processing was entrenched as the mode of computing supplied by the computer industry.

REAL TIME!

Real-time Computing Perhaps the most important, but by no means the only, place in which the batch mode of operation was undermined and replaced by new modes of operating computers was MIT. MIT was at the cutting edge in computing. MIT had built its reputation as the leading engineering school in America on teaching and research that placed mathematical sciences at the core of engineering. Engineering problems became problems that were solved through calculation.

The electrical engineering department in the period between the two world wars was famous for its research on electrical power systems, its close connections with General Electric, and the multitude of analog and other computing devices it built and used. MIT resumed its work in computing after the war and continued to be one of the leading research and educational institutions in computing.

The first project at MIT that challenged the batch processing mode of operation had its origins in the war.
The MIT Servomechanisms Laboratory, which had been founded in 1941 to build computing and control devices used for such purposes as aiming guns and stabilizing aircraft, was asked by the U.S. Bureau of Aeronautics in 1943 to build a general purpose aircraft simulator, which could train pilots to fly any of the planes in the military' s arsenal.

A flight trainer was a mockup of a cockpit with instruments and controls attached to a control system. When a pilot "flew" the simulator, the control system sent appropriate data to the aircraft's instruments and mechanical arms would move the cockpit to simulate the pitch and yaw that would occur.Jay Forrester, the assistant director of the laboratory, began by designing a control system that was essentially an analog computer that could be programmed to simulate the particular plane that was being "flown". Forrester learned about digital computing at the end of the war and changed the control from an analog device to a digital computer.

Eventually the trainer became secondary and the goal was to build a digital computer that would serve this purpose. A batch processing computer was of no value for this application.

In order to provide a realistic simulation, as the pilot moved the throttle or the yoke stick the computer must be able immediately to process and communicate adjustments to the instrument panel and the arms that moved the cockpit.

Real time computing

This computation had to be done in so-called real time. Real time computing was much more difficult and expensive to implement. It took four times as long and forty times as much money as originally budgeted to complete this computer, known as Whirlwind. In the course of doing so, many innovations were introduced, such as magnetic core memory, light pens for entering data, and numerous new electronic circuits. Whirlwind served as the prototype for the control center computers in the SAGE air defense system.

The Russians had exploded their first nuclear bomb in 1949, and the Americans were worried that Russians could carry nuclear warheads in long-range bombers over the North Pole and into the United States.

It was decided therefore to establish a line of radar field stations that would communicate their raw data back to these control centers, where the computers would process the data from many field stations and display air traffic on an electronic screen.

This would be monitored to gain an early warning of an attack. The SAGE system was expensive, costing $8 billion by the time it was implemented- and immediately made obsolete by the introduction of Inter-Continental Ballistic Missiles. But the important point here is, it was critical that these computers operate in real time.

There was no value in using a batch processing computer to learn that a bomber had entered your air space the day before! IBM, which had built the SAGE computers, used the expertise to build an important civilian real-time system, the SABRE airline reservation system, which was fully operational in 1964, to track reservations as they were made by customers to airline representatives around the country and later the world. SABRE was interactive in some senses of the word.

The agent was able to communicate back and forth with the centralized reservation computer while the customer was there in person or on the telephone, and usually a booking could be made while the customer waited. From the company' s perspective, its booking department could interact with all of its field agents at one time, getting up to the minute information that would help it to maximize passenger loads and profits and limit irritations to customers by overbooking.

It was not interactive in the sense of allowing two human to interact. Thus, while communication was a critical part of the system, this was not a communication system for humans. .

TIMESHARING

Timesharing Another project at MIT at the same time as SAGE and SABRE were being developed, known as Project MAC, opened up another interactive alternative to batch processing. MIT was actively involved in using computers in graduate education and research in the 1950s.

Mixing charity with business purposes, IBM chairman Thomas Watson, Sr. had established a program to provide steep discounts- and occasionally free use- of IBM computers to universities. IBM had a long-term relationship with MIT and had supplied a computer to them that was used by both MIT and a New England regional consortium of colleges.

The MIT faculty had come to the conclusion that batch processing computers, as these IBM computers were, was not good for educational purposes. Programming lessons or research projects were most effective if students had many opportunities to run their programs, with short turnaround times.

Several members of the faculty began to develop plans for a computer that would serve this educational function well, with support from the National Science Foundation, which had begun to take over the role of support for university computing from the private sector as part of the overall national response to Sputnik.

The solution that MIT arrived at was timesharing. A number of different users, scattered around campus, could each sit at a terminal and type in his or her program.
The computer would move from one terminal to the next, devoting short bursts of attention to each terminal in term. If the number of users did not overload the system, the response time to each users was short- only a few seconds typically- so each user could feel as though they had the machine' s complete attention and that the programming and processing could go on in real time.

Timesharing was a very satisfactory technical solution to the educational application, but timesharing systems- especially ones with ample capacity for an educational institution- were very expensive.
The cost of developing a single timesharing system for MIT would have consumed most of the money NSF had allocated for providing computer facilities for the entire U.S. college and university system; so they reluctantly had to decline to fund the implementation of the research program they had initially supported.

At just this time in 1962, the Department of Defense Advanced Research Projects Agency (ARPA) had decided to open a computing office.

ARPA had been started by President Eisenhower in the late 1950s, partially in response to Sputnik, as a way to consolidate advanced technological research for the armed services in one place, so as to cut costs and avoid some of the intense interservice rivalries that had long existed. Within a year after starting the computing office, ARPA was spending more on computing research than the sum of all other government agencies combined.

MIT was one of the first recipients of ARPA computing support, receiving several million dollars for Project MAC to build the timesharing computer. The computer was a success, and it led the computer manufacturers- even a reluctant IBM- to begin building timesharing computers

COMPUTER UTILITY

The Computer Utility Some of the people associated with Project MAC began to speak of a computer utility. The choice of words was intentional, to associate with the water or electric utilities that people had in their homes. Computers were high capital items. Universities were unable to afford modest computers without government help.
Small businesses and individual homes could not possibly afford to buy their own computers. But terminals and dedicated telephone lines hooked up to a timesharing computer probably was something that colleges and even small businesses could afford, and some day the price might come down in price to a point where they could be installed in private homes.

A new computer service industry grew up in the late 1960s to provide timesharing services. IBM and General Electric, plus new companies such as Tymshare and University Computing Company, entered this business.
These companies were the darlings of the stock market for a few years, but in the early 1970s they experienced a quick and decisive market decline because they were all having trouble writing the software needed to operate their systems effectively.

The larger the number of users hooked to the system, the more difficult the software. Soon timesharing systems were being used only in special niche markets in science, engineering, and business, where there were no more than fifty users. Thus this mode of interactive computing was available then to a small community of users, but not to the wider community that the advocates of the computer utility had hoped for.

The prospect of having a computer utility in every home just like your electricity, natural gas, and water was not to happen at that time or through the technology of timesharing services. The next logical step after timesharing was networking.

Although the users of a particular timesharing system could be located anywhere (connected to the computer by a telephone line), in practice most of the users of a particular system were clustered in one small geographical region, such as on a university campus or on one campus of a industrial research laboratory. It was a centralized organizational structure, and the users could only interact with the one centralized computer.

However, if computers themselves could be networked together in some way, the users, from their terminals, could communicate through their computer to gain access to other computers at some distance- computers that had different data, different programs, or additional computing power or features. This was the basic idea of networking when it was first developed. Several groups independently came up with the idea of networking, but the organization that made it happen was ARPA.

ARPA was interested in enabling the different groups of researchers that it was supporting to make use of the software and hardware paid for by ARPA at other sites.

They also wanted to take advantage of the time zone differences across the country to get more use out of the facilities. East Coast researchers could make use of west coast computer facilities for several hours before west coast researchers would be getting up, and the reverse would be true at night.

The technology was developed with ARPA' s funding and strong encouragement, and the first system, hooking four computing centers, became fully operational in 1970. ARPANET proved to be extremely successful, and the number of nodes increased steadily throughout the 1970s. During the 1970s, ARPANET was restricted to use by certain military organizations and researchers at a handful of top research universities that had ARPA contracts. Other computer science researchers were clamoring to take advantage of the access to the net, as much for the email contact with the other senior members of the research community as for access to the facilities at other computer sites. Indeed, email had been a throw-in to the original design, almost an afterthought, but it proved to be perhaps the most popular aspect of networked computing in the 1970s and 1980s.

The National Science Foundation tried several times in the 1970s to build a network for the entire research community but were prevented from doing so for a long time by the Office of Management and Budget, which did not want NSF in the position of operating a service business, especially if it would be in competition with the private sector. Commercial network services first appeared in 1975, when Telcomp timesharing service was recast as Telnet networking service, under the chairmanship of Larry Roberts, one of the former program officers at ARPA.

By 1978 Telnet had nodes in 176 U.S. cities and 14 countries, as well as several competitors. NSF was able in 1978 to establish Theorynet to connect the researchers in theoretical computer science, and in the 1980s to build NSFNET, to connect all kinds of scientific researchers. Another important network was USENET, which was formed in 1978 for colleges that had been excluded from connection to ARPANET.

One important feature of USENET was its news system, which involved a kind of electronic bulletin board that enabled users to subscribe to news groups where like-minded people could exchange ideas.

By 1991 there were 35,000 nodes on the USENET and more than a million subscribers. Internet, Gopher, and World Wide Web One of the obstacles for the growth of networking was the military, which was concerned about the security of the information it sent over the ARPANET and hence was very cautious in accepting new users. This problem was resolved in 1982, when a special military network, MILNET, was established for secure military communications.

Another obstacle was that some organizations, notably including IBM and Digital Equipment Corporation, built proprietary networks built on technologies other than that used in ARPANET. People on DECNET, for example, were not able to communicate directly with those on ARPANET until internetworking was developed.

ARPA established protocols (i.e., ways of communicating) that enabled one network to talk with another, which resulted in the Internet. Although these protocols were developed in the early 1970s, they were not heavily used until the 1980s. In 1984 there were still only about a thousand host computers on the Internet (mainly for scientific and engineering researchers at universities), but by 1988 there were 50,000 hosts and by the following year there were 150,000.

The networked world had moved out of the military and the university and into the public domain. Today, users are familiar with using the Internet not only for email and news groups, but also for the ability to read and write electronic documents with multimedia enhancements. These are a recent development.

The World Wide Web, with its capability to incorporate multimedia into documents, was developed at the CERN High-Energy Physics Laboratory in Switzerland in 1989. And the first major search and retrieval tool was Gopher, developed at the University of Minnesota in 1991. Much more powerful search tools developed in the last few years have made Gopher obsolete.

Computer Access: Technological and Economic Issues One final issue that must be considered in any discussion of the history of computer interactivity is access. This has two dimensions, technological and economic. Unless computers were relatively easy to use, they would not be used by large numbers of people. Unless they were affordable, they also would not receive mass use.

There are many technological achievements that have gone into making computers more powerful, more easy to use, and more amenable to interactive use. These include the change from batch processing operating mode to real-time, networked mode. We will consider only two other innovations here. One is the development of software. One useful way of reading the history of software is as the story of the automation of computer programming and use.

The first computers were supplied by manufacturers with almost no software. This was in part because computers were regarded as fast calculators to be used by scientists, who were believed to have the capability of doing the programming themselves.

Even internal housekeeping chores, such as where inside the computer's memory to store a particular piece of data, had to be specified by the user. Software for these purposes and for debugging of programs began to appear in the early 1950s. In the late 1950s, the first programming languages, FORTRAN and COBOL, were developed, so that users could interact with the computer in languages that resembled scientific or business rather than machine language. In the 1960s an independent software industry grew up, which wrote programs for all sorts of applications. In the 1980s, with the advent of the personal computer, all kinds of software, such as word processing, became available off the shelf at reasonable prices.

Another advance was the graphical user interface that has been closely identified with the Apple Macintosh products- windows, icons, mouse, and pulldown menus- and is now available on all personal computers, which novice users find more intuitive and easier to use without training.
The original research in this area was conducted in the 1960s in two laboratories supported by ARPA. At the Human Factors Research Center at the Stanford Research Institute, Douglas Englbart and others built an Electronic Office, which would integrate text and pictures in an electronic format that was unprecedented then but commonplace today. One feature of the Electronic Office was the invention of the mouse.

The other group was located at the University of Utah, where ARPA supported a major group that was at the cutting-edge in research on computer graphics. In a doctoral dissertation completed in 1969, Alan Kay developed the idea of the Dynabook, which was a computer-driven, notebook sized device that could store vast amounts of data and incorporated sophisticated information-finding tools.

The problem was that the technology was not yet compact or inexpensive enough to devote such a machine to a single user, as Englbart and Kay had envisioned. With the invention of the microprocessor at Intel and the continuing drop in the price of semiconductor technology, it was finally possible to begin thinking in the early 1970s of building a small, personal computer.

The first group to do so was Xerox, in its research center in Palo Alto, California, where Robert Taylor (formerly of ARPA) led a group of computer scientists that included Alan Kay. They completed the Alto workstation in 1975, sold commercially in 1981 as a high-end personal computer known as the Xerox Star. It was a technical triumph, having all of the features we expect today, but it cost too much to succeed in the marketplace.

However, Steve Jobs of Apple toured the Xerox facilities and came away with the plans for the successful line of computers that Apple is now known for: the Macintosh computer, which was introduced in 1984 after making the same marketing mistake as Xerox with its too-expensive Lisa computer the previous year.

The other consideration about access is economic. Sitting at a terminal in a timesharing mode was better for interaction than working in the normal batch processing environment, but the best experience was to have your own computer.

In the 1950s, computers were affordable only to large organizations, and there the typical user did not have hands-on access to the computer, which was the only way to have an interactive experience in those days.

The only people who were able to gain hands-on experience were hackers at places such as MIT, where they could sign up for middle-of-the-night shifts during which they would have complete control of the machine.

In the late 1960s the first minicomputers were manufactured by companies such as Digital Equipment Corporation or Data General. These machines were affordable to research scientists for their individual laboratories or by small businesses.

A number of people became hooked on having their computer through their experience with minicomputers, and these people, such as Stephen Gray, who was the editor of Electronics magazine and founder of the Amateur Computer Society, were the ones who promoted the microcomputer industry.

what are computer graphics ?

The term 'computer graphics' as used in this context refers to a set of computer applications which can be used to produce images and animations which would have been impossible with the technology available only a few years ago.

what are computer graphics In the popular press, computer graphics sound wonderful. They mirror real life, if anything they are better. Computer users no longer have to struggle with arcane and cumbersome user interfaces.
Everything will be intuitive. Unfortunately it's not that simple. A recurring theme within discussion of this technology is that while there are some aspects of Computer Graphics and Virtual Reality which are easy there are some which present a host of new design problems (Wexelblat, 1993).

When we look at a TV screen or movie, it is much the same as looking through a window - except that the scenario and unfolding events are typically distant in place and time. When we look at a computer screen it is much the same, except that the scenario and events are now not 'real' but computer generated: the environment we are looking at is 'virtual', it is a representation of the real world (Slater and Wilbur, 1995).

To visualise is to bring something as a picture before the mind. This is exactly what the visualisation software and hardware systems are trying to achieve. For years, various visualisation tools have been developed to help scientists to have a better understanding of problems of their concern. These tools can be used to create something as simple as 2-D images such as graphs, or it can be used to generate complicated 3-D images (Jean et al, 1991). Advances in computer hardware technology have led to an increase in the power available to computer users. Major software houses have kept pace with these advances developing software which utilises the technology available. Many engineers are now familiar with three-dimensional CAD systems and use them routinely in their work place. Many specialised software/CAD packages have become available dealing with particular aspects of the minerals field, such as geology, mine design or land reclamation.

These packages also offer a range of features such as advanced graphical user interfaces, full three dimensional modelling and solid modelling options. These systems will continue to change as photo-realistic rendering and Virtual Reality options become more widespread. The term 'computer graphics' as used above refers to a set of computer applications which can be used to produce images and animations which would have been impossible with the technology available only a few years ago. To produce high resolution computer graphics a three dimensional geometry is defined using conventional CAD software.

A range of texture maps are then applied to create solid three dimensional objects (texture maps are the computer graphics equivalent of applying patterned wallpaper over an object). Lighting conditions are then defined and objects are viewed from a range of different camera positions. If enough time and effort is put into creating these worlds the images produced can be difficult to distinguish from photographic images. A more advanced feature available at the higher end of the graphics market is the ability to create sequences of rendered frames and thus display these as an animation, or film.

VR and Interactivity

I. What is Virtual Reality

Virtual Reality known also as:Synthetic Environments, Cyberspace, Artificial Reality, Simulator Technology.. is a way for humans to visualize, manipulate and interact with computers and extremely complex data. The visualization part refers to the computer generating visual, auditory or other sensual outputs to the user of a world within the computer. This world may be a CAD model, a scientific simulation, or a view into a database. The user can interact with the world and directly manipulate objects within the world. The applications being developed for VR run a wide spectrum, from games to architectural and business planning. Many applications are worlds that are very similar to our own, like CAD or architectural modeling. Some applications provide ways of viewing from an advantageous perspective not possible with the real world, like scientific simulators and telepresense systems, air traffic control systems. Other applications are much different from anything we have ever directly experienced before. These latter applications may be the hardest, and most interesting systems. Visualizing the ebb and flow of the world's financial markets. Navigating a large corporate information base, etc.

Types of VR Systems A major distinction of VR systems is the mode with which they interface to the user. This section describes some of the common modes used in VR systems.

Window on World Systems (WoW) Some systems use a conventional computer monitor to display the visual world. This sometimes called Desktop VR or a Window on a World (WoW). This concept traces its lineage back through the entire history of computer graphics. In 1965, Ivan Sutherland laid out a research program for computer graphics in a paper called "The Ultimate Display" that has driven the field for the past nearly thirty years.

"One must look at a display screen," he said, "as a window through which one beholds a virtual world. The challenge to computer graphics is to make the picture in the window look real, sound real and the objects act real." [quoted from Computer Graphics V26#3]

Video Mapping A variation of the WoW approach merges a video input of the user's silhouette with a 2D computer graphic. The user watches a monitor that shows his body's interaction with the world. Myron Kruger has beenthe pioneer ( exhibited in Virtuality&Interactivity'98) a champion of this form of VR since the late 60's. He has published two books on the subject: "Artificial Reality" and "Artificial Reality II". At least one commercial system uses this approach, the Mandala system. This system is based on a Commodore Amiga with some added hardware and software. A version of the Mandala is used by the cable TV channel Nickelodeon for a game show (Nick Arcade) to put the contestants into what appears to be a large video game.

Immersive Systems The ultimate VR systems completely immerse the user's personal viewpoint inside the virtual world. These "immersive" VR systems are often equipped with a Head Mounted Display (HMD). This is a helmet or a face mask that holds the visual and auditory displays. The helmet may be free ranging, tethered, or it might be attached to some sort of a boom armature. A nice variation of the immersive systems use multiple large projection displays to create a 'Cave' or room in which the viewer(s) stand. An early implementation was called "The Closet Cathedral" for the ability to create the impression of an immense environment. within a small physical space. The Holodeck used in the television series "Star Trek: The Next Generation" is afar term extrapolation of this technology.

Mixed Reality Merging the Telepresence and Virtual Reality systems gives the Mixed Reality or Seamless Simulation systems. Here the computer generated inputs are merged with telepresence inputs and/or the users view of the real world. A surgeon's view of a brain surgery is overlaid with images from earlier CAT scans and real-time ultrasound. A fighter pilot sees computer generated maps and data displays inside his fancy helmet visor or on cockpit displays.

Fish Tank Virtual Reality The phrase "fish tank virtual reality" was used to describe a Canadian VR system reported in the 1993 InterCHI proceedings. It combines a stereoscopic monitor display using LCD Shutter glasses with a mechanical head tracker. The resulting system is superior to simple stereo-WoW systems due to the motion parallax effects introduced by the head tracker. (see INTERCHI '93 Conference Proceedings, ACM Press/Addison Wesley , ISBN 0-201-58884-6)

VR Hardware There are a number of specialized types of hardware devices that have been developed or used for Virtual Reality applications.

Image Generators One of the most time consuming tasks in a VR system is the generation of the images. Fast computer graphics opens a very large range of applications aside from VR, so there has been a market demand for hardware acceleration for a long while

Silicon Graphics Inc. has made a very profitable business of producing graphics workstations. SGI boxes are some of the most common processors found in VR laboratories and high end systems. The simulator market has produced several companies that build special purpose computers designed expressly for real time image generation. These computers often cost several hundreds of thousands of dollars.

Manipulation and Control Devices One key element for interaction with a virtual world, is a means of tracking the position of a real world object, such as a head or hand. There are numerous methods for position tracking and control. Ideally a technology should provide 3 measures for position(X, Y, Z) and 3 measures of orientation (roll, pitch, yaw). One of the biggest problem for position tracking is latency, or the time required to make the measurements and preprocess them before input to the simulation engine. The simplest control hardware is a conventional mouse, trackball or joystick. While these are two dimensional devices, creative programming can use them for 6D controls. There are a number of 3 and 6 dimensional mice/trackball/joystick devices being introduced to the market at this time. These add some extra buttons and wheels that are used to control not just the XY translation of a cursor, but its Z dimension and rotations in all three directions.

The Global Devices 6D Controller is one such 6D joystick It looks like a racket ball mounted on a short stick. You can pull and twist the ball in addition to the left/right & forward/back of a normal joystick. Other 3D and 6D mice, joystick and force balls are available from Logitech, Mouse System Corp. among others. One common VR device is the instrumented glove. The use of a glove to manipulate objects in a computer is covered by a basic patent in the USA. Such a glove is outfitted with sensors on the fingers as well as an overall position/orientation tracker. There are a number of different types of sensors that can be used. VPL (holders of the patent) made several DataGloves, mostly using fiber optic sensors for finger bends and magnetic trackers for overall position. Mattel manufactured the PowerGlove for use with the Nintendo game system, for a short time.

This device is easily adapted to interface to a personal computer. It provides some limited hand location and finger position data using strain gauges for finger bends and ultrasonic position sensors. The gloves are getting rare, but some can still be found at Toys R' Us and other discount stores. Anthony Clifton recently posted this suggestion for a" very good resource for PowerGloves etc.:

The concept of an instrumented glove has been extended to other body parts. Full body suits with position and bend sensors have been used for capturing motion for character animation, control of music synthesizers, etc. in addition to VR applications.

Position Tracking Mechanical armatures can be used to provide fast and very accurate tracking. Such armatures may look like a desk lamp (for basic position/orientation) or they may be highly complex exoskeletons (for more detailed positions). The drawbacks of mechanical sensors are the encumbrance of the device and its restrictions on motion. Exos Systems builds one such exoskeleton for hand control. It also provides force feedback. Shooting Star system makes a low cost armature system for head tracking. Fake Space Labs and LEEP Systems make much more expensive and elaborate armature systems for use with their display systems. Ultrasonic sensors can be used to track position and orientation. A set of emitters and receivers are used with a known relationship between the emitters and between the receivers. The emitters are pulsed in sequence and the time lag to each receiver is measured. Triangulation gives the position. Drawbacks to ultrasonics are low resolution, long lag times and interference from echoes and other noises in the environment.

Magnetic trackers use sets of coils that are pulsed to produce magnetic fields. The magnetic sensors determine the strength and angles of the fields. Limitations of these trackers are a high latency for the measurement and processing, range limitations, and interference from ferrous materials within the fields. However, magnetic trackers seem to be one of the preferred methods. The two primary companies selling magnetic trackers are Polhemus and Ascension. Optical position tracking systems have been developed. One method uses a ceiling grid LEDs and a head mounted camera. The LEDs are pulsed in sequence and the cameras image is processed to detect the flashes. Two problems with this method are limited space (grid size) and lack of full motion (rotations). Another optical method uses a number of video cameras to capture simultaneous images that are correlated by high speed computers to track objects. Processing time (and cost of fast computers) is a major limiting factor here. One company selling an optical tracker is Origin Instruments. Inertial trackers have been developed that are small and accurate enough for VR use. However, these devices generally only provide rotational measurements. They are also not accurate for slow position changes.

Levels of VR Hardware Systems

The following defines a number of levels of VR hardware systems. These are not hard levels, especially towards the more advanced systems. 1. Entry VR (EVR)

The 'Entry Level' VR system takes a stock personal computer or workstation and implements a WoW system. The system may be based on an IBM clone (MS-DOS/Windows) machine or an Apple Macintosh, or perhaps a Commodore Amiga. The DOS type machines (IBM PC clones) are the most prevalent. There are Mac based systems, but few very fast rendering ones. Whatever the base computer it includes a graphic display, a 2D input device like a mouse, trackball or joystick, the keyboard, hard disk & memory . 2. Basic VR (BVR) The next step up from an EVR system adds some basic interaction and display enhancements. Such enhancements would include a stereographic viewer (LCD Shutter glasses) and a input/control device such as the Mattel PowerGlove and/or a multidimensional (3D or 6D) mouse or joystick.

Advanced VR (AVR) The next step up the VR technology ladder is to add a rendering accelerator and/or frame buffer and possibly other parallel processors for input handling, etc. The simplest enhancement in this area is a faster display card. For the PC class machines, there are a number of new fast VGA and SVGA accelerator cards. These can make a dramatic improvement in the rendering performance of a desktop VR system. Other more sophisticated image processors based on the Texas Instruments TI34020 or Intel i860 processor can make even more dramatic improvements in rendering capabilities. The i860 in particular is in many of the high end professional systems. The Silicon Graphics Reality Engine uses a number of i860 processors in addition to the usual SGI workstation hardware to achieve stunning levels of realism in real time animation. An AVR system might also add a sound card to provide mono, stereo or true 3D audio output. Some sound cards also provide voice recognition. This would be an excellent additional input device for VR applications.

Immersion VR (IVR) An Immersion VR system adds some type of immersive display system: a HMD, a Boom, or multiple large projection type displays (Cave). An IVR system might also add some form of tactile, haptic and touch feedback interaction mechanisms. The area of Touch or Force Feedback (known collectively as Haptics) is a very new research arena.

Cockpit Simulators A common variation on VR is to use a Cockpit or Cab compartment to enclose the user. The virtual world is viewed through some sort of view screen and is usually either projected imagery or a conventional monitor. The cockpit simulation is very well known in aircraft simulators, with a history dating back to the early Link Flight Trainers (1929?). The cockpit is often mounted on a motion platform that can give the illusion of a much larger range of motion. Cabs are also used in driving simulators for ships, trucks, tanks and 'battle mechs'. The latter are fictional walking robotic devices (i.e. the Star Wars films). The BattleTech location based entertainment (LBE) centers use this type of system.

Available VR Software Systems There are currently quite a number of different efforts to develop VR technology. Each of these projects have different goals and approaches to the overall VR technology. Large and small University labs have projects underway (UNC, Cornell, U.Rochester, etc.). ARPA , NIST, National Science Foundation and other branches of the US Government are investing heavily in VR and other simulation technologies. There are industry supported laboratories too, like the Human Interface Technologies Laboratory (HITL) in Seattle and the Japanese NTT project. Many existing and startup companies are also building and selling world building tools (Autodesk, IBM', Sense8, VREAM). There are two major categories for the available VR software: toolkits and authoring systems. Toolkits are programming libraries, generally for C or C++ that provide a set of functions with which a skilled programmer can create VR applications. Authoring systems are complete programs with graphical interfaces for creating worlds without resorting to detailed programming. These usually include some sort of scripting language in which to describe complex actions, so they are not really non-programming, just much simpler programming. The programming libraries are generally more flexible and have faster renders than the authoring systems, but you must be a very skilled programmer to use them. (Note to developers: if i fail to mention your system below, please let me know and I will try to remember to include it when, and if, i update this document again) Here in Virtuality&Ineractivity in Section III you will have the opportinity to experimen VR interactive accomplish throurg different softwares.

Freeware VR Programs At the low end of the VR spectrum are the freeware products and garage or home-brew VR hackers . There are currently a few fast rendering programs that have been released with source code and no charge. These programs are generally copyrighted freeware, which means that the original creators retain the copyright and commercial use is restricted. They are not polished commercial programs, and are often written by students. However, these programs exist to give people a very low cost entry into the VR world.

Aspects of A VR Program Just what is required of a VR program? The basic parts of the system can be broken down into an Input Processor, a Simulation Processor, a Rendering Process, and a World Database. All these parts must consider the time required for processing. Every delay in response time degrades the feeling of 'presence' and reality of the simulation. I.. Input Processes The Input Processes of a VR program control the devices used to input information to the computer. There are a wide variety of possible input devices: keyboard, mouse, trackball, joystick, 3D & 6D position trackers (glove, wand, head tracker, body suit, etc.).

A networked VR system would add inputs received from net. A voice recognition system is also a good augmentation for VR, especially if the user's hands are being used for other tasks. Generally, the input processing of a VR system is kept simple. The object is to get the coordinate data to the rest of the system with minimal lag time. Some position sensor systems add some filtering and data smoothing processing. Some glove systems add gesture recognition. This processing step examines the glove inputs and determines when a specific gesture has been made. Thus it can provide a higher level of input to the simulation. .

Simulation Process The core of a VR program is the simulation system. This is the process that knows about the objects and the various inputs. It handles the interactions, the scripted object actions, simulations of physical laws (real or imaginary) and determines the world status. This simulation is basically a discrete process that is iterated once for each time step or frame. A networked VR application may have multiple simulations running on different machines, each with a different time step. Coordination of these can be a complex task. It is the simulation engine that takes the user inputs along with any tasks programmed into the world such as collision detection, scripts, etc. and determines the actions that will take place in the virtual world.

Rendering Processes The Rendering Processes of a VR program are those that create the sensations that are output to the user. A network VR program would also output data to other network processes. There would be separate rendering processes for visual, auditory, haptic (touch/force), and other sensory systems. Each renderer would take a description of the world state from the simulation process or derive it directly from the World Database for each time step.

Visual Renderer The visual renderer is the most common process and it has a long history from the world of computer graphics and animation. The reader is encouraged to become familiar with various aspects of this technology. The major consideration of a graphic renderer for VR applications is the frame generation rate. It is necessary to create a new frame every 1/20 of a second or faster. 20 frames per second (fps) is roughly the minimum rate at which the human brain will merge a stream of still images and perceive a smooth animation. 24fps is the standard rate for film, 25fps is PAL TV, 30fps is NTSC TV. 60fps is Showscan film rate. This requirement eliminates a number of rendering techniques such as raytracing and radiosity. These techniques can generate very realistic images but often take hours to generate single frames. Visual renderers for VR use other methods such as a 'painter's algorithm', a Z-Buffer, or other Scanline oriented algorithm.

There are many areas of visual rendering that have been augmented with specialized hardware. The Painter's algorithm is favored by many low end VR systems since it is relatively fast, easy to implement and light on memory resources. However, it has many visibility problems. For a discussion of this and other rendering algorithms, see one of the computer graphics reference books listed in a later section. The visual rendering process is often referred to as a rendering pipeline. This refers to the series of sub-processes that are invoked to create each frame.

A sample rendering pipeline starts with a description of the world, the objects, lighting and camera (eye) location in world space. A first step would be eliminate all objects that are not visible by the camera. This can be quickly done by clipping the object bounding box or sphere against the viewing pyramid of the camera. Then the remaining objects have their geometry's transformed into the eye coordinate system (eye point at origin). Then the hidden surface algorithm and actual pixel rendering is done.

The pixel rendering is also known as the 'lighting' or 'shading' algorithm. There are a number of different methods that are possible depending on the realism and calculation speed available. The simplest method is called flat shading and simply fills the entire area with the same color. The next step up provides some variation in color across a single surface. Beyond that is the possibility of smooth shading across surface boundaries, adding highlights, reflections, etc.

An effective short cut for visual rendering is the use of "texture" or "image" maps. These are pictures that are mapped onto objects in the virtual world. Instead of calculating lighting and shading for the object, the renderer determines which part of the texture map is visible at each visible point of the object. The resulting image appears to have significantly more detail than is otherwise possible. Some VR systems have special 'billboard' objects that always face towards the user. By mapping a series of different images onto the billboard, the user can get the appearance of moving around the object.

I need to correct my earlier statement that radiosity cannot be used for VR systems due to the time requirements. There have recently been at least two radiosity renderers announced for walkthrough type systems - Lightscape from Lightscape Graphics Software of Canada and Real Light from Atma Systems of Italy. These packages compute the radiosity lighting in a long time consuming process before hand. The user can interactively control the camera view but cannot interact with the world.

Auditory Rendering A VR system is greatly enhanced by the inclusion of an audio component. This may produce mono, stereo or 3D audio. The latter is a fairly difficult proposition. It is not enough to do stereo-pan effects as the mind tends to locate these sounds inside the head. Research into 3D audio has shown that there are many aspects of our head and ear shape that effect the recognition of 3D sounds. It is possible to apply a rather complex mathematical function (called a Head Related Transfer Function or HRTF) to a sound to produce this effect. The HRTF is a very personal function that depends on the individual's ear shape, etc. However, there has been significant success in creating generalized HRTFs that work for most people and most audio placement. There remains a number of problems, such as the 'cone of confusion' wherein sounds behind the head are perceived to be in front of the head. Sound has also been suggested as a means to convey other information, such as surface roughness. Dragging your virtual hand over sand would sound different than dragging it through gravel.

Haptic Rendering Haptics is the generation of touch and force feedback information. This area is a very new science and there is much to be learned. There have been very few studies done on the rendering of true touch sense (such as liquid, fur, etc.). Almost all systems to date have focused on force feedback and kinesthetic senses. These systems can provide good clues to the body regarding the touch sense, but are considered distinct from it. Many of the haptic systems thus far have been exo-skeletons that can be used for position sensing as well as providing resistance to movement or active force application.

Other Senses The sense of balance and motion can be served to a fair degree in a VR system by a motion platform. These are used in flight simulators and some theaters to provide some motion cues that the mind integrates with other cues to perceive motion. It is not necessary to recreate the entire motion perfectly to fool the mind into a willing suspension of disbelief. The sense of temperature has seen some technology developments. There exist very small electrical heat pumps that can produce the sensation of heat and cold in a localized area. These system are fairly expensive. Other senses such as taste, smell, pheromone, etc. are beyond our ability to render rapidly and effectively. Sometimes, we just don't know enough about the functioning of these other senses.

World Space The virtual world itself needs to be defined in a 'world space'. By its nature as a computer simulation, this world is necessarily limited. The computer must put a numeric value on the locations of each point of each object within the world. Usually these 'coordinates' are expressed in Cartesian dimensions of X, Y, and Z (length, height, depth). It is possible to use alternative coordinate systems such as spherical but Cartesian coordinates are the norm for almost all applications. Conversions between coordinate systems are fairly simple (if time consuming).

.. World Coordinates A major limitation on the world space is the type of numbers used for the coordinates. Some worlds use floating point coordinates. This allows a very large range of numbers to be specified, with some precision lost on large numbers. Other systems used fixed point coordinates, which provides uniform precision on a more limited range of values. The choice of fixed versus floating point is often based on speed as well as the desire for a uniform coordinate field.

A World Divided: Separation of Environments One method of dealing with the limitations on the world coordinate space is to divide a virtual world up into multiple worlds and provide a means of transiting between the worlds. This allows fewer objects to be computed both for scripts and for rendering. There should be multiple stages (aka rooms, areas, zones, worlds, multiverses, etc.) and a way to move between them (Portals).

World Database The storage of information on objects and the world is a major part of the design of a VR system. The primary things that are stored in the World Database (or World Description Files) are the objects that inhabit the world, scripts that describe actions of those objects or the user (things that happen to the user), lighting, program controls, and hardware device support.

Storage Methods There are a number of different ways the world information may be stored: a single file, a collection of files, or a database. The multiple file method is one of the more common approaches for VR development packages. Each object has one or more files (geometry, scripts, etc.) and there is some overall 'world' file that causes the other files to be loaded. Some systems also include a configuration file that defines the hardware interface connections. Sometimes the entire database is loaded during program startup, other systems only read the currently needed files. A real database system helps tremendously with the latter approach. An Object Oriented Database would be a great fit for a VR system, but I am not aware of any projects currently using one. The data files are most often stored as ASCII (human readable) text files. However, in many systems these are replaced by binary computer files. Some systems have all the world information compiled directly into the application.

Objects Objects in the virtual world can have geometry, hierarchy, scripts, and other attributes. The capabilities of objects has a tremendous impact on the structure and design of the system. In order to retain flexibility, a list of named attribute/values pairs is often used. Thus attributes can be added to the system without requiring changes to the object data structures. These attribute lists would be addressable by name (i.e. cube.mass => mass of the cube object). They may be a scalar, vector, or expression value. They may be addressable from within the scripts of their object. They might be accessible from scripts in other objects.

Position/Orientable An object is positionable and orientable. That is, it has a location and orientation in space. Most objects can have these attributes modified by applying translation and rotation operations. These operations are often implemented using methods from vector and matrix algebra.

Hierarchy An object may be part of an object part HIERARCHY with a parent, sibling, and child objects. Such an object would inherit the transformations applied to it's parent object and pass these on to it's siblings and children. Hierarchies are used to create jointed figures such as robots and animals. They can also be used to model other things like the sun, planets and moons in a solar system.

Bounding Volume Additionally, an object should include a BOUNDING VOLUME. The simplest bounding volume is the Bounding Sphere, specified by a center and radius. Another simple alternative is the Bounding Cube. This data can be used for rapid object culling during rendering and trigger analysis. Objects whose bounding volume is completely outside the viewing area need not be transformed or considered further during rendering. Collision detection with bounding spheres is very rapid. It could be used alone, or as a method for culling objects before more rigorous collision detection algorithms are applied.

Object Geometry The modeling of object shape and geometry is a large and diverse field. Some approaches seek to very carefully model the exact geometry of real world objects. Other methods seek to create simplified representations. Most VR systems sacrifice detail and exactness for simplicity for the sake of rendering speed. The simplest objects are single dimensional points. Next come the two dimensional vectors. Many CAD systems create and exchange data as 2D views. This information is not very useful for VR systems, except for display on a 2D surface within the virtual world. There are some programs that can reconstruct a 3D model of an object, given a number of 2D views. The sections below discuss a number of common geometric modeling methods. The choice of method used is closely tied to the rendering process used. Some renderers can handle multiple types of models, but most use only one, especially for VR use. The modeling complexity is generally inversely proportional to the rendering speed. As the model gets more complex and detailed, the frame rate drops.

3D PolyLines & PolyPoints The simplest 3D objects are known as PolyPoints and PolyLines. A PolyPoint is simply a collection of points in space. A Polyline is a set of vectors that form a continuous line.

Polygons the most common form of objects used in VR systems are based on flat polygons. A polygon is a planar, closed multi-sided figure. They maybe convex or concave, but some systems require convex polygons. The use of polygons often gives objects a faceted look. This can be offset by more advanced rendering techniques such as the use of smooth shading and texture mapping. Some systems use simple triangles or quadrilaterals instead of more general polygons. This can simplify the rendering process, as all surfaces have a known shape. However, it can also increase the number of surfaces that need to be rendered. Polygon Mesh Format (aka Vertex Join Set) is a useful form of polygonal object. For each object in a Mesh, there is a common pool of Points that are referenced by the polygons for that object. Transforming these shared points reduces the calculations needed to render the object. A point at the edge of a cube is only processed once, rather once for each of the three edge/polygons that reference it. The PLG format used by REND386 is an example of a Polygonal Mesh, as is the BYU format used by the 'ancient' MOVIE.BYU program.) . The geometry format can support precomputed polygon and vertex normals. Both Polygons and vertices should be allowed a color attribute. Different renderers may use or ignore these and possibly more advanced surface characteristics. Precomputed polygon normals are very helpful for backface polygon removal. Vertices may also have texture coordinates assigned to support texture or other image mapping techniques.

Primitives Some systems provide only Primitive Objects, such as cubes, cones, and spheres. Sometimes, these objects can be slightly deformed by the modeling package to provide more interesting objects.

Solid Modeling & Boolean Operations Solid Modeling (aka Computer Solid Geometry, CSG) is one form of geometric modeling that uses primitive objects. It extends the concept by allowing various addition, subtraction, Boolean and other operations between these primitives. This can be very useful in modeling objects when you are concerned with doing physical calculations, such as center of mass, etc. However, this method does incur some significant calculations and is not very useful for VR applications. It is possible to convert a CSG model into polygons. Various complexity polygonal models (# polygons) could be made from a single high resolution ''metaobject" of a CSG type.

Curves & Patches Another advanced form of geometric modeling is the use of curves and curved surfaces (aka patches). These can be very effective in representing complex shapes, like the curved surface of an automobile, ship or beer bottle. However, there is significant calculation involved in determining the surface location at each pixel, thus curve based modeling is not used directly in VR systems. It is possible, however, to design an object using curves and then compute a polygonal representation of those curved patches. Various complexity polygonal models could be made from a single high resolution 'metaobject'.

Dynamic Geometry (aka morphing) It is sometimes desirable to have an object that can change shape. The shape might simply be deformed, such a bouncing ball or the squash/stretch used in classical animation ('toons'), or it might actually undergo metamorphosis into a completely different geometry. The latter effect is commonly known as 'morphing' and has been extensively used in films, commercials

Virtual Reality: History

Beginnings Virtual reality may have popped into the headlines only in the past few years, but its roots reach back four decades. It was in the late 1950s, jthat an idea arose that would change the way people interacted with computers and make possible VR. At the time, computers were hulking Goliaths locked in air-conditioned rooms and used only by those conversant in esoteric programming languages. Few people considered them more than glorified adding machines. But a young electrical engineer and former naval radar technician named Douglas Engelbart viewed them differently. Rather than limit computers to number crunching, Engelbart envisioned them as tools for digital display. He knew from his days with radar that any digital information could be viewed on a screen. Why not, he then reasoned, connect the computer to a screen and use both to solve problems?

Opportunity and timing At first, Engelbart's ideas were dismissed, but by the early 1960s other people were thinking the same way. Moreover, the time was right for his vision of computing. Communications technology was intersecting with computing and graphics technology. The fi rst computers based on transistors rather than vacuum tubes became available. This synergy yielded more user-friendly computers, which laid the groundwork for personal computers, computer graphics, and later on, the emergence of virtual reality. Several pivotal events marked the decade:

Fear of nuclear attack prompted the U.S. military to commission a new radar system that would process large amounts of information and immediately display it in a form that humans could readily understand. The resulting radar defense system was the first "real time," or instantaneous, simulation of data.

Advanced Research Projects Agency Aircraft designers began experimenting with ways for computers to graphically display, or model, air flow data. Computer experts began restructuring computers so they would display these models as well as compute them. The designers' work paved the way fo r scientific visualization, an advanced form of computer modeling that expresses multiple sets of data as images and simulations.

Massachusetts Institute of Technology An infusion of self-styled computer wizards strove to reduce the barriers to human interactions with the computer by replacing keyboards with interactive devices that relied on images and hand gestures to manipulate data. In 1962 Ivan Sutherland developed a light pen with which images could be sketched on a computer. Sutherland's first computer-aided design program, called Sketchpad, opened the way for designers to use computers to create blueprints of automobiles, cities, and industrial products. By the end of the decade, the designs were operating in real time. By 1970, Sutherland also produced a primitive head-mounted display and Engelbart unveiled his crude pointing device for moving text around on a computer screen -- the first "mouse." One of the most influential antecedents of virtual reality was the flight simulator. Following World War II and through the 1990s, the military and industrial complex pumped millions of dollars into technology to simulate flying airplanes (and later driving tanks and steering ships).

Evans & Sutherland Then as now, it was cheaper, and safer, to train pilots on the ground before subjecting them to the hazards of flight. The early flight simulators consisted of mock cockpits built on motion platforms that pitched and rolled. A limitation, however, was they lacked visual feeback. This changed when video displays were coupled with model cockpits.

Evans & Sutherland By the 1970s, computer-generated graphics had replaced videos and models. These flight simulations were operating in real time, though the graphics were primitive. In 1979, the military experimented with head-mounted displays. These innovations were driven by the greater dangers associated with training on and flying the jet flighters that were being built in the 1970s. By the early 1980s, better software, hardware, and motion-control platforms enabled pilots to navig ate through highly detailed virtual worlds. Of course, the "military-industrial complex" was not the only entity interested in computer graphics.

A natural consumer of computer graphics was the entertainment industry, which, like the military and industry, was the source of many valuable spin-offs in virtual reality.

By the 1970s, some of Hollywood's most dazzling special effects were computer-generated, such as the battle scenes in the big-budget, blockbuster science fiction movie Star Wars, which was released in 1976. Later came such movies as Terminator and Jurassic Park. In the early 1980s, the video game business boomed.

National Aeronautics and Space Administration One direct spin-off of entertainment's venture into computer graphics was the dataglove, a computer interface device that detects hand movements. It was invented to produce music by linking hand gestures to a music synthesizer. NASA Ames was one of the first customers for this new computer input device for its experiments with virtual environments. But the biggest consumer of the "dataglove" was the Mattel company, which adapted it into the PowerGlove, the pervasive mitt with which children conquered adversaries in the popular Nintendo game.

So you want to explore digital, 3-D worlds in real time? With appropriate hardware and software you can do just that. Choose between various head- or boom-mounted displays. Or, surround yourself in digital imagery by stepping into a virtual reality room, the CAVE. Described in the following documents are the most common virtual reality systems and their associated interface devices that, like 3D mouses, allow you to control and manipulate the virtual world. Also presented is a display of VR architecture. This example of a typical VR system will give you an idea of how computing hardware and software work together to generate a virtual world. In fact, a visit to a VR system would be a good place to start.

Interactivity would have remained wishful thinking if not for the development of high-performance computers in the mid-1980s. These machines provided the speed and memory for programmers and scientists to begin developing advanced visuali zation software programs. By the end of the 1980s, low-cost, high-resolution graphic workstations were linked to high-speed computers, which made visualization technology more accessible. All the basic elements of VR had existed since 1980, but it took high-performance computers, with their powerful image rendering capabilities, to make it work. Demand was rising for visualization environments to help scientists comprehend the vast amounts of data pouring out of their computers daily. As drivers for both computation and VR, high-performance computers no longer served as mere number crunchers, but became exciting vehicles for exploration and discovery.

"In the totally immersive virtual environment before you, you have the ability to walk around in any of the ancient structures at this site, pick up the artifacts, watch virtual inhabitants, study their behavior, and even rebuild parts of the ancient site based on the evidence you will find in the virtual scene. You may even redo all or part of your reconstruction over and over, testing alternative views, until the outcome matches the actual archaeological record as you have learned about it from your remote instructor or from speaking to the virtual inhabitants." Both situations just a bit too fantastic? Fine for the special effects wizards of science fiction, but not for serious public education? We will see how close reality is to imagination when we reach the end of this paper. " Donald Sanders -archeologist-learning site

"Since when Virtual Reality has been around??' . Since the 1920s when Link Corporation manufactured training devices that simulated fighter plane cockpits

The status of VR pioneer is often given to Ivan Sutherland, who first proposed the use of stereographic head-mounted displays in the early 1960s so that users could look around a computer-generated room simply by turning their heads.

In the early 1970s, Myron Krueger coined the term "artificial reality" and began developing computer-controlled responsive environments (Krueger 1993).

About the same time, the MIT Media Lab produced a simulated tour through Aspen Colorado, in which participants could drive down a virtual street and enter and explore virtual buildings. The 1980s brought rapid changes to VR technology.

Jaron Lanier, a founder of VPL Research, Inc., is credited with coining the expression "virtual reality" to distinguish between the immersive environments he was creating and traditional computer simulations.

Thomas Zimmerman, co-founder of VPL, worked with Lanier to develop a glove for grasping computer-generated objects in virtual worlds.

NASA developed goggles that allowed the wearer to look around a graphic landscape portrayed on a computer screen while hearing synthesized speech and 3D (binaural) sounds, and grabbing objects with their hands.

Communication and feedback with a computer-simulated environment was direct; no contact with the computer was needed.

In 1992, the movie Lawnmower Man introduced the concept of virtual reality to the public. By the mid-1990s it was possible to reach out and touch virtual objects, to feel different textures and sensations; and perfume companies experimented with virtual smells to send odors electronically from lab to lab.

There is no generally agreed-upon definition of virtual reality, and several distinct types of virtual reality have now emerged--artificial reality, augmented reality, immersive reality, telepresense, and CAVEs. All provide different degrees of immersion, interactivity, and unencumbered navigation

"virtual reality" means an interactive, self-directed, multisensory, computer-generated experience providing the illusion of participating in a synthetic 3-dimensional (3D) environment.

Virtual reality, Java, and the Internet make all that possible right now. We can re-create ancient built environments, and within them we can re-place the artifacts as they were found and link to those objects texts, photographs, and narrations to provide the virtual visitor with information tailored to specific grade levels or curriculum guidelines . Picture the new learning tool that results--the virtual world itself becomes the visual index to an entire dataset. Should an instructor or student desire information about any subject, object, or space, just walk up to that item and click on it to retrieve links to the photographic archive or written descriptions about that item and associated items at the site.

View maps, plans, early travelers' drawings, previous excavators' notebooks, or museum records, all with a click of the mouse. Interactive inquiry need not proceed in the mostly linear fashion of standard books, educational materials need not be limited by the cost of printing color plates, searches need not be restricted to words alone, since links can be made to and from 3D objects or pictures or virtual locations. Add to that the capability of listening as an expert explains about the building's spatial organization, function, or date. This we can do today.

Three-dimensional, interactive, archaeological digital databases, if accessed using virtual reality, thus become dynamic media re-creating an ancient world in its original complexity creating a near firsthand experience. Data about a site that has become globally distributed can be brought back into a single virtual environment allowing students and teachers to view the objects in their original architectural context for the first time since antiquity. This is an unprecedented educational opportunity.

VRML makes possible the inclusion of 3D models into multimedia environments on systems that are within the financial range of pubic institutions and may allow for fulfilling the promise of distance education. Right now, thanks to VRML, the full power of 3D environments can be combined with the full power of multimedia to create an unrivaled learning experience, promoting awareness of past civilizations, understanding of different cultures, and appreciation of different places, peoples, and their cultural heritage (Sanders & Gay 1996a). Previous virtual reality environments have emphasized real-time response and the immersive qualities of the experience. However, the wider application of those worlds for education or research is limited because: (1) expensive hardware and software are required; (2) text display is rudimentary; and (3) users are unable to browse related text or pictures.

VRML has changed that, making possible the integration of 3D data, standard HTML 2D text, pictures, and video into a World Wide Web page, permitting simultaneous viewing of 3D and 2D information. Three-dimensional environments can now be used for what they do best: allow users to gain a full understanding of a spatial structure through self-directed exploration while retaining all the power and detail provided by the 2D text and graphics of standard Web pages.

Links between 2D and 3D data make it possible to click on a specific detail in the 3D environment and either switch to a different 3D model or bring up supporting text or pictures in a separate scrollable frame on the same window or in a different window

Hot spots in the text can affect the 3D portion of the screen and hot objects in the 3D environment can access and change the text and graphics on the Web page. Objects can move, change and react to the user. Sound can provide ambient context and additional information through narration. HTML-coded pages can provide a curriculum framework. Anchored viewpoints and labeled locations allow the creation of self-guided or teacher-driven explorations of archaeological sites, each emphasizing different aspects of the data, from daily life of the ancient inhabitants to archaeological methodologies.

GLOSSARY OF VIRTUAL REALITY TERMINOLOGY .

.Robotic+VR+Interactivity

What is the definition of a 'robot'? "A reprogrammable, multifunctional manipulator designed to move material, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks" Robot Institute of America, 1979 ...
Webster says: An automatic device that performs functions normally ascribed to humans or a machine in the form of a human.

Where did the word 'robot' come from? The word 'robot' was coined by the Czech playwright Karel Capek (pronounced "chop'ek") from the Czech word for forced labor or serf. Capek was reportedly several times a candidate for the Nobel prize for his works and very influential and prolific as a writer and playwright.
Mercifully, he died before the Gestapo got to him for his anti-Nazi sympathies in 1938. The use of the word Robot was introduced into his play R.U.R. (Rossum's Universal Robots) which opened in Prague in January 1921.

The play was an enormous success and productions soon opened throughout Europe and the US. R.U.R's theme, in part, was the dehumanization of man in a technological civilization. You may find it surprising that the robots were not mechanical in nature but were created through chemical means. In fact, in an essay written in 1935, Capek strongly fought that this idea was at all possible and, writing in the third person, said: "It is with horror, frankly, that he rejects all responsibility for the idea that metal contraptions could ever replace human beings, and that by means of wires they could awaken something like life, love, or rebellion. He would deem this dark prospect to be either an overestimation of machines, or a grave offence against life."

[The Author of Robots Defends Himself - Karl Capek, Lidove noviny, June 9, 1935, translation: Bean Comrada] There is some evidence that the word robot was actually coined by Karl's brother Josef, a writer in his own right. In a short letter, Capek writes that he asked Josef what he should call the artifical workers in his new play. Karel suggests Labori, which he thinks too 'bookish' and his brother mutters "then call them Robots" and turns back to his work, and so from a curt response we have the word robot. R.U.R is found in most libraries. The most common English translation is that of P. Selver from the 1920's which is not completely faithful to the original.

The term 'robotics' refers to the study and use of robots. The term was coined and first used by the Russian-born American scientist and writer Isaac Asimov (born Jan. 2, 1920, died Apr. 6, 1992). Asimov wrote prodigiously on a wide variety of subjects. He was best known for his many works of science fiction. The most famous include I Robot (1950),
The Foundation Trilogy (1951-52), Foundation's Edge (1982), and The Gods Themselves (1972), which won both the Hugo and Nebula awards. The word 'robotics' was first used in Runaround, a short story published in 1942. I, Robot, a collection of several of these stories, was published in 1950. Asimov also proposed his three "Laws of Robotics", and he later added a 'zeroth law'. ?

Law Zero: A robot may not injure humanity, or, through inaction, allow humanity to come to harm. ?
Law One: A robot may not injure a human being, or, through inaction, allow a human being to come to harm, unless this would violate a higher order law. ?
Law Two: A robot must obey orders given it by human beings, except where such orders would conflict with a higher order law. ?
Law Three: A robot must protect its own existence as long as such protection does not conflict with a higher order law. An interesting article on this subject: Clarke, Roger,

When did robots, as we know them today
come into existence?

The first industrial modern robots were the Unimates developed by George Devol and Joe Engelberger in the late 50's and early 60's. The first patents were by Devol for parts transfer machines. Engelberger formed Unimation and was the first to market robots.

As a result, Engelberger has been called the 'father of robotics.' Modern industrial arms have increased in capability and performance through controller and language development, improved mechanisms, sensing, and drive systems. In the early to mid 80's the robot industry grew very fast primarily due to large investments by the automotive industry. The quick leap into the factory of the future turned into a plunge when the integration and economic viability of these efforts proved disastrous.

The robot industry has only recently recovered to mid-80's revenue levels. In the meantime there has been an enormous shakeout in the robot industry. In the US, for example, only one US company, Adept, remains in the production industrial robot arm business. Most of the rest went under, consolidated, or were sold to European and Japanese companies. In the research community the first automata were probably Grey Walter's machina (1940's) and the John's Hopkins beast.

Teleoperated or remote controlled devices had been built even earlier with at least the first radio controlled vehicles built by Nikola Tesla in the 1890's. Tesla is better known as the inventor of the induction motor, AC power transmission, and numerous other electrical devices. Tesla had also envisioned smart mechanisms that were as capable as humans. SRI's Shakey navigated highly structured indoor environments in the late 60's and Moravec's Stanford Cart was the first to attempt natural outdoor scenes in the late 70's. From that time there has been a proliferation of work in autonomous driving machines that cruise at highway speeds and navigate outdoor terrains in commercial applications.

Holograms

What is a Hologram? Holograms are amazing three-dimensional images created with the use of lasers. Holograms can be created from actual objects, or from film footage, providing that it meets certain specificiations. With the film-to-hologram technique, many different perspectives are combined in one holographic image. When you view the hologram, you see all the perspectives at one time and your brain reconstructs the scene as if you were actually there. If enough perspectives are included in the hologram you can actually look around objects and see what is behind them! You will be able to see the entire process of creating these holograms right here at this site. This site will be updated on a regular basis. The project will begin in October of 1998. Space III Holograms

In this section we will see the hologram, we will understand how its production is carried out, why we see it tridemensional...what is a laser...what technology generates this reality on an optical level.

At this point we present to you a sculpture of classical style in a very suggestive mode and which becomes animated as the public traverse the body the face is transformed into an unexpected muffin, the eyes roam the space as an enigmatic look as that of the mona lisa...the personages stuck in its rigid material murmer in low voices, ther converse among themselves, they project realities and the public..marveled understand in reality what is the virtual since the sculpture piece,, its temperament and its form which shows in a holofonic form and the imagination that makes emerge sentiments which confuses and enters in a trance in the digital and virtual.

What is a Hologram? Holograms are amazing three-dimensional images created with the use of lasers. Holograms can be created from actual objects, or from film footage, providing that it meets certain specificiations. With the film-to-hologram technique, many different perspectives are combined in one holographic image. When you view the hologram, you see all the perspectives at one time and your brain reconstructs the scene as if you were actually there. If enough perspectives are included in the hologram you can actually look around objects and see what is behind them!

Is the right word Hologram or Holograph? The preferred word is Hologram. The dictionary defines a Holograph as a hand written document, as in a holographic will or deed. A Holographer is someone who makes holograms. Holography is the word for the technology and artform. According to Isaac Asimov, a Holographist is a person who collects or studies holography but does not make holograms. Things pertaining to holography are said to be Holographic. ·

Are holograms projections? · No, holograms are not projected. Light fills up a hologram like plaster would fill up a cast. Technically, they are reconstructions of the light that reflected off the object. · If a hologram breaks is the whole image visible in each piece? · No, each broken piece would let you see the image from its own unique perspective.

Think of a hologram as a window. Anywhere you look through a window you see what's on the other side. If you were to paint the window black and scratch a hole in the paint on the left side of that window just big enough to look through, you would see everything on the other side of the window. Like looking through a peephole. If you then scratch another viewing peephole somewhere on the right side of the window, you still can see through, but from a different perspective.

This is the same effect that each broken piece of a hologram would display. Just remember that if you have two broken pieces taken from opposite sides of the hologram, and you are looking at an object that looks differently from each side, one piece may let you see just one of those sides while the other piece will let you view the other side. So, you might say that each piece of a hologram stores information about the whole image, but from its own viewing angle. No two pieces will give you a view that is exactly the same. ·

Can you suggest any good books on holography? · My favorite is "Practical Holography", by Graham Saxby, published by Prentice Hall, London. The "Holography Handbook"by Fred Unterseher, published by Ross Books, Berkeley, CA, is a good beginner's guide. These and other books on the subject can be found at Barnes & Noble. ·

How many lasers do you need to make a hologram? · One. However, you can shoot several different holograms on the same piece of film. Each holographic exposure can be shot with a different color laser if, for example, you are making a multi color image of red, green, and blue. A color hologram can also be made with a single laser using tricks of the trade like emulsion swelling or multiple reference angles. ·

What does the word LASER mean? · It is an acronym or abbreviation of the first letters of Light Amplification through Stimulated Emmission of Radiation. · What is color? · Light is a wave. We see differnt sizes of light waves as different colors. Its something like the sizes of the strings of a harp making different musical notes. The largest strings of a harp make the lowest pitch notes and the shortest strings make the highest pitched musical notes. A rainbow is like a harp with strings of light.

The largest visible light waves are called red. Those a little smaller are called orange. A bit smaller and we get yellow. Smaller still is green. Smaller once again and we have blue. And the tinyest visible light waves are violet. Light waves smaller than violet are invisible and called ultraviolet. Light waves larger than red are also invisible and called infrared. The visible spectrum is from 400 nm (nanometers - one billionths of a meter) for violet to 700 nm for red. ·

Is there a word to describe where an image appears in a hologram? · Yes, there are a few common ones that are quite helpful. If an image appears to be on the other side of the hologram, like looking through a window, it is called virtual. If an image jumps right out of the hologram and appears in front of the film, it is called real, since it has left the "virtual" world inside the film and entered the "real" world. When you flip a hologram over, the image is inside out and called pseudoscopic . Flip it back over and view it normally, right side out, and it is called orthoscopic.

An image can be orthoscopic and real or orthscopic and virtual. Or an image can be pseudoscopic and real or pseudoscopic and virtual. An image can be both real and virtual, as in the case of an image that starts behind the film and then protrudes right out of it. Holograms can be made (especially by artists) that have both orthoscopic and pseudoscopic images in them. Any combination of these terms is possible. So, to quickly rehash, Real = in front; Virtual = behind; Orthoscopic = right side out; Pseudoscopic = inside out. ·

How are images made to jump out in front of the holographic film? · As just explained in the previous response, images that protrude out in front of a piece of holographic film are called real images. Virtual image holograms are used as the masters for real image holograms. Most real image holograms are holograms of holograms. The basic concept is like the idea that a negative of a negative is a positive.

In effect, when you typically make a hologram it is orthoscopic (right side out) and virtual (the image appears behind the film). If you turn this orthoscopic and virtual image hologram over the image you see is both pseudoscopic (inside out) and real (in front) since the spatial relationship of where the image is seen has flipped. If you use this image to record a second hologram, that image will be pseudoscopic (inside out) because you are recoding the pseudoscopic image of the first hologram and virtual. If you then turn it over it is orthoscopic (right side out) because an inside out image of an inside out image is right side out and real because each time you flip a hologram over you reverse from virtual image to real image. Voila! .

What about Stereoscopy ??

What is a 3D Stereoscopic Image?
When humans perceive the environment around them, each eye sees a different perspective image. These two images give rise to the perception of true depth. Most printed images which look 3D are classified as monoscopic images. Monoscopic images try to simulate depth by using shadows and rendering techniques.

A stereoscopic image, on the other hand, appears to have real depth where objects can even seem to leave the surface of the display device and hover in the middle of the room. A stereoscopic image is composed of a right perspective frame and a left perspective frame - one for each eye. When the right frame is viewed by your right eye and the left frame is viewed by your left eye, your brain will perceive a true 3D view.

What is Stereoscopy?????????
The difference between simulated 3D and stereoscopic 3D is that in stereoscopic 3D a distinctly different image is being shown to each eye simultaneously. The images represent two different renderings of an idenical environment htat have been captured from positions that correspond to the location of the left and right eye.

How Stereo Vision is accomplished...??
This is accomplished by creating two different images of the world, one for each eye. The images are computed with the viewpoints offset by the equivalent distance between the eyes.
There are a large number of technologies for presenting these two images. The images can be placed side-by-side and the viewer asked (or assisted) to cross their eyes. The images can be projected through differently polarized filters, with corresponding filters placed in front of the eyes. Anaglyph images use red/blue glasses to provide a crude (no color) stereovision.

In the section VI of Virtuality&Interactivity II, the visitor will be able to enjoy a real and stereoscopic vision of the city of New York, Guatemala!! Wearing the the stereoscopic lenses, the visitors can have a real-stereo experience as as if you were actually there, close, within the Big Apple, the terrific experience of the wonders of the Grand Canyon not withstandidng you are comfortably sitting in the exhibit space of Virtuality &I Interactivity exhibit.
Moreover you will feel the actual emotions of travelling on a ballon, the impressive beauty of the nature of the grand canyon as if you were actually there, travelling on an icarius with stereoscopic lenses.

Ray Hanissan, the author of these stereoscopic videos will come in person to stage his own works and dialogue with the public about virtuality and interactivity and share his experience in the area of production.

Telepresence - Networking -Cyberspace

• Telepresence
  Telepresence is a variation on visualizing complete computer generated worlds. This a technology links remote sensors in the real world with the senses of a human operator. The remote sensors might be located on a robot, or they might be on the ends of WALDO like tools.
  
  Fire fighters use remotely operated vehicles to handle some dangerous conditions. Surgeons are using very small instruments on cables to do surgery without cutting a major hole in their patients. The instruments have a small video camera at the business end. Robots equipped with telepresence systems have already changed the way deep sea and volcanic exploration is done. NASA plans to use telerobotics for space exploration. There is currently a joint US/Russian project researching telepresence for space rover explorat
  
• "Cyberspace
  " is the `place` where a telephone conversation appears to occur. Not inside your actual phone, the plastic device on your desk. Not inside the other person's phone, in some other city. _The_place_between_ the phones The indefinate place _out_there_, where the two of you, human beings, actually meet and communicate." Bruce Sterling.
  The word "cyberspace" was coined by the science fiction author William Gibson, when he sought a name to describe his vision of a global computer network, linking all people, machines and sources of information in the world, and through which one could move or "navigate" as through a virtual space.
• The word "cyber", apparently referring to the science of cybernetics, was well-chosen for this purpose, as it derives from the Greek verb "Kubernao", which means "to steer" and which is the root of our present word "to govern".
  
  It connotes both the idea of navigation through a space of electronic data, and of control which is achieved by manipulating those data. For example, in one of his novels Gibson describes how someone, by entering cyberspace, could steer computer-controlled helicopters to a different target. Gibson's cyberspace is thus not a space of passive data, such as a library: its communication channels connect to the real world, and allow cyberspace navigators to interact with that world.
  
  The reference to cybernetics is important in a third respect: cybernetics defines itself as a science of information and communication, and cyberspace's substrate is precisely the joint network of all existing communication channels and information stores connecting people and machines.
  
  The word "space", on the other hand, connotes several aspects.
• First, a space has a virtually infinite extension, including so many things that they can never be grasped all at once. This is a good description of the already existing collections of electronic data, on e.g. the Internet.
• Second, space connotes the idea of free movement, of being able to visit a variety of states or places.
• Third, a space has some kind of a geometry, implying concepts such as distance, direction and dimension.
  
  The most direct implementation of the latter idea is the technology of virtual reality, where a continuous three-dimensional space is generated by computer, which reacts to the user's movements and manipulations like a real physical space would. In a more metaphorical way, the geometry (or at least topology) of space can be found in the network of links and references characterizing a hypertext (which can be seen as the most general form for a collection of interlinked data).
  
  Nodes in a hypertext can be close or distant, depending on the number of links one must traverse in order to get from the one to the other. Moreover, the set of links in a given node define a number of directions in which one can move. However, a hypertext does not seem to have any determined number of dimensions (except perhaps infinity), it is not continuous but "chunky", and the distance between two points is in general different depending on the point from which one starts to move.
  
  One of the challenges for the researchers who are trying to make present computer networks look more like a Gibsonian cyberspace is to integrate the intuitive geometry of 3-D virtual reality, with the more general, but cognitively confusing, infinite dimensionality of hypertext nets (see e.g. NCSA's Virtual Reality interface to the World-Wide Web are being discussed.
• As a description for what presently exists, the word "cyberspace" is used in a variety of significations, which each emphasize one or more of the meanings sketched above.
  
  Some use it as a synonym for virtual reality, others as a synonym for the World-Wide Web hypermedia network, or for the Internet as a whole (sometimes including the telephone, TV, and other communication networks). None of the uses already seems to incorporate the most intrinsically cybernetic aspect of the concept: that of a shared medium through which one can exert control over one's environment. Control can apply as well to objects in cyberspace (e.g. when you alter the information in database through a Web form interface), as to objects in the real world (telepresence or teleoperation).