a)
"Robots+Avatar dreaming with Virtual Illusions"

go to WEB PAGE

Author: Franz Fischnaller (F.A.B.R.I.CATORS/Milano)
Producer: F.A.B.R.I.CATORS, Milan/Italy
Collaboration with:
K-TEAM SA, SWITZERLAND
Copyrigtht: Franz Fischnaller 1998

This interactive installation integrates virtual reality, robotics and telepresence with a high aesthetical content as well as an suggestive and enigmatic interaction. The Real ambience represented by a physical structure under the form of an Arena. The Virtual Ambiences is articulated by 10 different worlds or cities contained in the "Mother City" Ying&Yang. The personages in this installation exist both in real space (Koala=robots) and in virtual space Ying and Yang: avatars. Each can control the other and yet both are controlled by the user in a local space and also through the net (in a non local space).

Components and personages of the installation:

A real ambience represented by an physical structure under the form of an Arena. This is the physic habitat of Koala. Virtual worlds, each of it with a different "reality".

The personages:
1)Koala (Robot) which lives in his real ambience and it is strongly related with the Virtual world through Ying. 2)Ying (avatar) and alter ego of Koala. Ying represents the feminine principle. She is strongly related to the physical world through Koala.
3)Yang (avatar) lives alone by itself in the limbo of the cyber space and represent the masculine

a) The installation consist of a real and highly aesthetical and astonishing scenography under the form of an Arena with labyrinthic pathways, made of mirrors, metal, wood, in a shiny colorful sand and stone-landscape. This is the habitat in which Koala lives. Real ambience represented by a physical structure under the form of an Arena. This is the "physical habitat of Koala ".

b) Virtual Ambiences is articulated by the mother city Ying Yang and 10 different worlds or cities contained in the mother city.

The worlds or cities and 10 in number and are known as: Avalon, Amazonia, Root of Knowledge, Black Hole, Lautriv's City, Bosh Hell, Ynte Colony, skin city,Boundry City, Amazonas City.

c) Koala (real personage) is a robot in the form of a cyber turtle. Its body is covered by a smart skin which is at the same time a catalyzer and mirror of his experience and emotions which he lives by means of the interactivity, since this skin is a porous and expressive entity... Koala lives in his real ambience which is strongly related to the virtual world through Ying (avatar).

d)Ying is the alter ego of Koala in the Virtual World. Its an antropomorphic human like creature Ying the avatar represents the feminine principle. She is closely related to the physical world through Koala. Each of them quite individual and unique yet integral in this robotic universe.

e)Yang (avatar) which also exist in the Virtual World navigate alone, with no alter ego in the physical world. However , Yang becomes connected to the real world by means of the manipulation of a joystick provided for the purpose.Yang, a virtual creature has no alter ego with the real world. He exists by himself in the limbo of the cyberspace and represents the masculine principle.

Description of the experience: The public can interact with each single robot in it's environment, by moving a joystick. This interaction generates a real time reaction in the behaviour of the robot as well in the avatars in the Vr-world, visible on the proyection screen at the top of the scenography of the arena " the real world".

There are two joysticks: one is to interact with Koala. It is possible to navigate and conduct Koala in the virtual world. And the other joystick with Ying and yang (avatars).

The public can command the actions of koala and in consequence the actions of Ying in the Virtual Worlds. ying the avatar will operate in the virtual worlds and koala will follow-up step by step what his alter ego does..and generate the same actions. Koala is cloned with it's Avatar Ying. Koala is present in the virtual world through his avatar ... Ying.

On some occasions it could be observed that Koala moves in a very particular way, in his habitat as if trying to avoid some obstacles, as if he is entering into a cave, trying to escape an object of obstruction.

The curious thing is that in the real world there is no apparent obstacles ... but in-fact there are obstacles. It is only that you do not see them but koala has the capacity to see them...since he has the virtual vision in his perception and he avoids the obstacles his avatar finds in the Virtual World. Koala the robot is seeing the obstacles since they become visible in his imagination these objects which exist in the virtual world.

Koala sees that which his avatar sees in the virtual world, it means that he sees through his avatar. Only that while this occurs, the public sees in the real world smooth sands, without obstacles, while in fact koala is navigating and acting as if it were in the virtual, it is only that his physical body is placed on the sands but his vision is focused in the virtual world.

Koala will replicate, in real space, the travels of Ying, drawing out the path traversed on the environment of sand. It will seem that he is a sort of somnambulist ... going up and down in a senseless manner. He however, knows where he goes it is only that the human eyes are not able to percieve it until Koala relives the same with traces in the sand.

This mean that the physical world is an illusion and that what we see in virtual and dreams are projections without consistency. That the avatar, the alter ego sees objects in the virtual world and reproduces it in the real world and the objects you do not see even are also there because Koala sees it through his avatar Ying.

Robots ... Avatars and the Net

In the web page that accompanies the installation ... Robot avatar dealing with virtual illusions there is a menu with diverse possibilities of action, view point, interaction...and of navigation. Koala can be commanded from different points of view and inputs. His actions exercise immediate responses in real time in his avatars and viceversa.

1. You could become the view point of Koala. This way you assume, by means of your interaction the position of the other to see the worlds around you.
   
2. You could also see the movements of Koala, Ying or Yang as if they were placed in a virtual chamber in frot of your eyes, and as a consequence you assume the view point of koala or that of Ying or Yang ... this is to say see the worlds, the cities from their view point... nevertheless, you are physically behind the joystick, this implies that a sort of telepresence is experienced, your body is there behind the joystick but your eyes see what could be seen by one of the three personages and from their own point of view.
   
3. You could also have an "external view point", that is to say see from the outside, the inside scenes...change radically from the perspective of that of the personages and assume the position of the view of a bird, for example: fly over the landscape, the visions, the lanscape of the virtual world.
   
4. By means of the web pages, from robot avatar dreaming with virtual illusion, you could enter and leave different cities, and enter again into the mother city which inturn contains the rest of the cities. This process of entering in contact and navegating and interacting with the city is being done through the net.
   
5. Let us speak about another avatar....of Yang. Yang could be activated and commanded by means of a second joystick. In reality, Yang represents in ludic terms the "view point" a "moving view point"...

There exist the possibility of attracting Koala cliking on the button of the joystick and causing Koala to move towards Yang, trying to find him either in the physical or virtual world.

In synthesis either Koala or the avatars Ying and Yang can interact with the visitors by means of the network or telepresence. Network which becomes a sort of prolongation.

More about the net from a non local space

The Avatar Yang reacts and acts in the virtual world with movements of coordinates X and Z. Between the coordinates X and Z of the virtual world there are invisible barriers, invisible zones that define a net of interactive knots.

Each knot is a point of interaction which in turn is connected to sites, webs, htpp.... This represents the world of the net, of telepresence. In the real human world these tangible realities are invisible, this is because the real world is not represented physically, in a concrete way since in our world they are, in effect, invisible realities, intangible nevertheless they exist. And they are of vital importance for humanity and man in the aspect of comunication.

On the other hand these interactive knots in the virtual world are represented by objects, cubes, geometric forms, symbols.

The interesting thing is that koala, through his alter ego, recognizes where these interactive knots are located and which contain a latent cyber reality which is activated only when touched or someone passes on top or very close to them. The presence of koala and his avatar creates a stimulus for their connection.

In technical terms when the robot or his avatar passes close to these knots whether in the physical world (invisible knots) or the virtual world (visible knots) a collision is brought about and the net of the world becomes connected, activating the knot and connecting at some place in the net.

If the connection obtains a positive result there emerges a tree in the virtual world and this signifies and/or symbolizes that the connection was activated. If the tree does not grow it means that the connection was not made.

Technical specification

The installation is carried out in the C language with Microsoft Visual C++ - Windows NT 4.0. The part dealing with the graphic has been developed with the VC Division library. For the execution of the application it is necessary that the DVS Division runtime component is installed with the required characteristics. Other components necessary for the adequate functioning of the application are: (two) analogous joysticks and the corresponding drives. A joystick permits the entry into interactive contact with Koala and its avatar Ying...whether in the real world or in the virtual world. The second joystick permits the entering in contact with the two worlds: the real world of Koala and the virtual world of Ying and Yang.

Interacting with these two joysticks and these worlds... interactions, collisions and encounters are generated which permit the entry, exit the discovery of new worlds all in funtion with the laws that govern actions and reactions of the three personages of this installation. The visitor has the capacity to intervene, interact, penetrate within new worlds, change the actions and movements of Koala and the avatars... command koala to move towards its avatar or Yang, enter into one of the cities, wander around the "Mother City".

During this walk, there is set in the form of an invisible grid a sort of "world-map" of sites, which will be called up with a ping to the corresponding positioned website existing in the database. The database of the weblinks: about (suggested) 100 http addresses will be defind and stored in a database and than localized in the invisible grid of the VR-world. For the very moment the avatar crosses over this link, he will create a connection to the site and if the connection happened to be sucessful a flag or something else will be visualized on the screen. The environment is a landscape wich is generated as a 3D surface mapped with a corresponding texture. Depending upon the different areas/zones the outlook changes. WE WOULD LIKE to offer the opportunity to the a vast public via Internet to participate and send inputs to the simulation. Virtual Ambiences is articulated by a "Mother City" Ying&Yang and 10 different worlds or cities contained in the Mother City.

Author: Franz Fischnaller
Producer: FABRICATORS ARCHITECTS OF CULTURE - FABRICATORS OF IDEAS
Via F.lli Bronzetti 6, 20129 Milano/Italy - Phone: 39-2-70128233 - Fax: 76110498
cell: 39-348-2310451 - fabricat@galactica.it - http://www.fabricat.com

Collaboration with:
K-TEAM SA, SWITZERLAND
Tel: 41-21-8025472 Fax: 8025471- info@k-team.com

b) Pinocchio Interactive

WEB PAGE

Production by
F.A.B.R.I.CATORS

Development of the experimental prototype in collaboration with
K-TEAM SA, Switzerland

Pinocchio Interactive's experimental prototype in its first phase of development will be presented for the first time in "Virtuality & Interactivity II",
. Pinocchio Interactive is a dynamic installation which integrates: robotic, animation, digital fantasy and interactivity. It is a ludicrous experience which is filled with surprises, tricks and wizards of the life of Pinocchio in the version 2000.More than an interactive piece it is an interactive cinematographic work of great humor and perplexity. The surprise is the outstanding strength of "Pinocchio Interactive".

The installation is comprised of a real stylized puppet,"Pinocchio Puppet"of 1.8 meters in height, made out of wood and metal. In the scenary where "Virtual Pinocchio" acts is integrated a screen measuring 3X3 mts and on which the new life of "Virtual Pinocchio" evolves.The personages are "Pinocchio Puppet","Virtual Pinocchio" and the Interactive Visitors.There is a dynamic relationship between the three personages and their realities.

1) Robotic physical World "Pinocchio Puppet" who acts in the middle of the scenery, between the digital world and the path of the interactive visitor.2) Digital World "Virtual Pinocchio"and its story, his friends and enemies which are projected on the screen integrated in the scenery.3) Public.. Interactive visitors

An interactive script has been written ad hoc, in which a very bold and free interpretation of the original story of Pinocchio is given. Nevertheless it is kept within the original essence of the story. "Virtual Pinocchio " and its story consists of eight chapters. The visitor slips in each of the digital chapters through "Pinocchio Puppet" as well as he interacts in real time with the digital world contained in the chapters.

A series of innovations have been added to the script. Great fantasy has been taken in each of the movements of the "Pinocchio Puppet", the interaction among the worlds, the sounds, the voice, the interaction between the digital and robotic worlds and the public.Within the Digital World "Virtual Pinocchio" appears to be a superimposed avatar of brilliant green but without a clear physiognomy. The outline of digital Pinocchio is a brilliant outline filled somehow by green intense shadow. It is "Pinocchio Puppet" which is of a decisive clear color and volume and exercises all the functions that the "Virtual Pinocchio" carries out, if it runs he runs, if it turns he turns. "Pinocchio Puppet" nose becomes lengthened when he lies. He is the interactive (animator) between the real world (the visitor) and the digital world ("Virtual Pinocchio's" story, friends and enemies)

"Pinocchio Puppet" has the basic characteristics of the classical puppets but with the difference that this Pinocchio is "robotized". The juggler's natural threads have been connected to the computer and the presence of the authorship has been translated to an electronic brain becoming a virtual extention of the author. The Juggler is the executive robotized system of the author. In synthesis his new source of life is the computer.The juggler has been susbtitute by a computer .The technical aspect of the movements of Pinocchio puppet maintains its original concept of being a puppet, but commanded through a robotized system. There are eight interactive points which are connected by means of threads to the computer and obey the impulse of the public...

"Pinocchio Puppet" is powerfull, sometimes, smart, sometimes sad, sometimes frustrated. It is a critical personage with his own character and identity, is a rebellious but enslaved being, is pampered by "Virtual Pinocchio" and besides that he acts in function of the interactivity that the public exercises on him.

He knows that he exists, but that he do not feel always confortable of his "being" robotized and depend of exterior presences. The juggler is an electronic brain instead of a human hand.. While the juggler's hand is connected by threads the electronic juggler is connected to a computer.

Bold, tired of portraying the part of the old jester, "Pinocchio Puppet" is a little jealous of the "Virtual Pinocchio" for whom he works and sometimes refuses to obbey. Often Pinocchio Puppet act like a "rebeld " clone which enters in conflict with his alter ego: "Virtual Pinocchio". The dialogue becomes intense, he speaks, acts, reacts, moves until he reaches a point that he reacts and enters in a phase of total contradiction in relation to his virtual counterpart thereby creating a story within the story.

It also occurs that at a certain point "Pinocchio Puppet" becomes conscious of his own being and he starts to act by himself as the whole entity and tells the story of his own life through the projection he himself orders to be projected.

Nevertheless his temperamental character he shows great courage for hard work in the interpretation. It is he who works raises and lowers the curtains of the theater.

The visitors will spend fascinating moments with this interactive digital theatrical interactive spectacle in which there is a real illusion to distinguish between the real and the virtual. Inasmuch as it enters in a vortex of unexpected sensations and reactions highly stimulating... That for a moment there arises an intense overlaping between the real and literary worlds.

Description of the Interactive Experience.

Upon entering, the visitor will find a space of darkness in which is seen an illuminated "Pinocchio Puppet", long and slim of two meters in hight in the posture of "The Thinker" (sculpture by the french artist Rodin) . On the screen there appear, in slow motion, several little Pinocchios coming out from a piece of wood, like the effect of the popcorn. "Pinocchio Puppet" says "ummm ..what a situation no one has come to interrelate with me... come foward my friends". Stumping his right foot, impatient in his posture of "The Thinker".

"Pinocchio Puppet" is one of the main interface of the installation. The visitor interacts with both worlds the robotic and the digital through him. The visitor can activate "Pinocchio Puppet" and command his actions through the joystick in front of the scenery, he reacts touching the "menu pannel" he has in the scenary, activating the Digital World and bringing up "Virtual Pinocchio" and its story to life.

The visitor has the freedom to explore and participate in an active mode with the installation through "Pinocchio Puppet", which speaks, moves, comments, obeys, becomes irritated as well as acts with great charm with the public and at the same time interacts with "Virtual Pinocchio" ans its story, its counterpart of the virtual world which is projected on the screen.

When a visitor gets close to the joystick, he/she moves and turns "Pinocchio Puppet". It is the visitor who decides to which chapter of the history he wants to enter. But it its "Pinocchio Puppet" to let him in. The technical aspect of the movements of Pinochhio puppet maintains its original concept of being a puppet, but commanded through a robotized system. There are eight interactive points which are connected by means of threads to the computer and obey the impulse of the public...

Production:
F.A.B.R.I.CATORS
Concept: Franz Fischnaller
Script: Yesi Maharaj
Design of the Interactive Scenography: Franz Fischnaller
Texture and Graphic : Franz Fischnaller
Pinocchio Puppet: Franz Fischnaller
Voices:
Pinocchio: Roberto Cajiafa (Italy)
Fata: Liaty Pisani (Italy)
Project Coordination:

Robot System: K-TEAM SA, Switzerland

Special thanks to Giuliano Bianchi who was one of the first key sources of inspiration in the development of this project.

FABRICATORS
ARCHITECTS OF CULTURE - FABRICATORS OF IDEAS
Via F.lli Bronzetti 6, 20129 Milano/Italy
Phone: 39-2-70128233 - Fax: 76110498
cell: 39-348-2310451
e-mail: fabricat@galactica.it
http://www.fabricat.com

c) Tracking the Net

Interactive Installation
Produced by: F.A.B.R.I.CATORS e Elekton s.n.c

Synthesis

Tracking the Net will be present in first vision in Virtuality&Interactivity 26-29 May 1999

In SIGGRAPH 99 Art Gallery:technOasis. 8-13 Agoust 1999, Los Angeles, USA.

Tracking the Net" is an installation under the form of an interactive netted cube of 3X3mts with rear-projection onto one wall, with high resolution image. It can host from 1 to 10 interactive visitors which can navigate and interact in real time.
The cube has its own electronic sensors and active notes which reveal the presence of the visitor. Movements of the visitor can be detected and measured over a wide space in order to control a real time animation.
The visitor can interact with the virtual environment by touching, pushing , manipulating the net. Just by interacting with the net the visitor can navigate and interact with the virtual objects, sounds, music.
Movement of the nets are detected by infrared cameras ( Qualisys) and identified by real time tracker.

Interacting with the net the digital environment morphs into a high speed vortex.The space morphs into a different environment. A highway of information, streams, discks, whir, layers of network cyber landscape appear, arriving, moving, crossing from all directions. Streaming towards different information nodes. From an electronic stream all expands into an intangible cyberspace, there are no boundaries...We are in the habitat of " Cybor Net".

The next step of Tracking the Net is an installation of 4X4 mts with rear-projection onto 4 walls, with high resolution images ( stereo and non), able to take form 1 to 20 interacitve visitor which can navigate and interact simultaneosly in real time.

The Netter cube is compose in a larga scale with the innovative cable: Live Wire!
"Live Wire is ELAM's innovative, cable like, electroluminescent (EL) lamp. It is a thin fibre, 0.7 mm (0.275")lighting diameter, emitting light when an alternating current is applied to the electrodes the fibre ends.Live Wire construction is a multi-layer coaxial cable with a dialectric layer made of electroluminescent phosphor particles

Tracking the net" is an installation under the form of an interactive Netted cube of 3X3 mts. This netted cube has its own electronic sensors and active notes which reveal the presence of the visitor and its sensitive to the human presence its action and interactions. Touching, pushing, pulling this net generates a different reaction. This reaction can be felt and seen through the sceen positioned in front of the net. The complete physical installation occupies an aprox space of 4X8 mts and it has the capacity to receive from 10 to 20 interactive visitors simultaneosly. Movements of the visitor can be detected and measured over a wide space in order to control a real time animation. This can be used to create visual effects in shows. The netted cube is composed in a large scale with the innovative cable: Live Wire.

"Live Wire is ELAM's innovative, cable like, electroluminescent (EL) lamp. It is a thin fibre, 0.7 mm (0.275") lighting diameter, emitting light when an alternating current is applied to the electrodes on one of the fibre ends. Live Wire construction is a multi-layer coaxial cable with a dialectric layer made of electroluminescent phosphor particles covered by a transparent electrode.

How does Live Wire Work?
Electrically, Live Wire is an EL light emitting capacitor operated by alternating current. When applied, the current operates a potential between the core of the fibre and the upper transparent layer and excites the in between electroluminescent phosphor layer.

Technical description
Special infrared cameras detect position of small IR light generators placed over human body segments. Computer elaborates three dimensional coordinates and generates a stream of real time data that can drive computer generated images.

Interactivity :
A machine detects and reacts to human movement. A system of a a very precise way to measure position (thus kinematic data) of human limbs. It can be used virtually in any application in which kinematic interaction is required.

Visitor experience :
"Tracking the Net" is immersed in a play of shadows. Breaking the darkness a net of intense colors, The net strings break up the shadows like virtual strings hanging in the emptiness. Beyond one of the faces of the net there is a screen of 3X2.25 mts. The visitor steps in and starts the interaction touching, tiering, moving, pushing the net. While this actions occurs the action will come up in real time, with sounds, vr projection on the screens. The visitor will get back in real time the feedback of the "Net sensibility". The visitor will be able to generate their sound, music, images.

"Cybor-Net"

The visitor becomes immersed in the relative worlds of the net, the cyber world the Cyberspace: A computer synthetized 3-D reality. "Cybor -Net" lives beyond this 3 D space, trapped in the net which will struggle to escape out of it with the help of the interactive visitors of tracking the net. "Cybor-Net" is a robotic humanoid modeled directly from digital readings of a real human and transformed into a photo-realistic animated character produced via an illusory metamorphosis.

He invites the visitors to rescue him...keeps saying when he gets the right inputs by the interaction of the visitor:. "welcome to my world... welcome to cyberworld". Cybor Net" is an androide personage. Perhaps it might remind you of the myth of the antique navigators transported to the cyber cosmos, floating in the intangible atmosphere of the net.

While the interacitve visitor interacts with the net, which is an elastic, cubicular and elastic mass, through which the visitor can see through to images projected on the screen the projection.

The virtual world ( sound -images) and surely "cybor net" will establish a dynamic interaction with the visitors in real time. The interaction will generate quite interesting effects. While the "physical net" is being manipulated, formed and deformed by the visitors the virtual world will react in consequence.

Tracking the Net at the present is a VR installation of 3X3 mts with rear-projection onto one wall,with high resolution images. It can host from to 1 to 10 interactive visitor which can navigate and interact in real time simultaneosly.

The next step of Tracking the Net is an installation 4X4 mts with rear-projection onto 4 walls, with high resolution images ( stereo and non) able to take from 1 to 20 interactive visitors which can navigate and interact in real time simultaneosly.

The user interact with the environment using their own body, their hands, just by interacting with the "net" and can navigate, manipulate and Traduzione.

Elekton

Commercial activity started in 1992 and has been developing in time with designing, software engineering, manufacturing and distribution of medical devices, motion capture and analisys systems and robotic devices for TV / cinema appliances.

Marco Monzani:
Born in Milan, Italy on August 17, 1967.
Electronic Engineer, master degree in Biomedical Appliances.
In 1992 starts his activities founding Elekton, a company involved in medical devices design and software. Since 1993 cooperates with various Italian scientists and issues scientific papers regarding human movement analysis and ergometrics. In 1994 starts Orbita project, with the aim to design and implement robotic devices for cinema and television shootings. Orbita project still develops in cooperation with a leading company of this market sector, now making these devices available on international basis.

Produced by:
F.A.B.R.I.CATORS e Elekton
Original Idea: Franz Fischnaller
Interactive content, texture, Graphic design and Virtual worlds: Franz Fischnaller F.A.B.R.I.CATORS
Programming and plug in development:Marco Monzani
Software assistance for the content development: Lorraine Paul F.A.B.R.I.CATORS

Tecnologia:
MOTION CAPTURE
Software: Division,softimage,etc
Elekton Real time tracker
Elekton-Hardware: Qualisis -
Elekton Sensori di posizione a infrarossi

Live Wire: ELAM's innovative, cable like, electroluminescent
Original idea of Ing. Bruno Latini.
Developed and Produced by Panerai Sistemi, SpA

Elekton
Via Favaron 66, 20054 Nova Milanese (MI/Italia)
elekton@cube.it
Tel.: 39 0362 387550 Fax: 39 0382 .
0348-8706831

Panerai Sistemi, SpA
Via Ponte a Iozzi 8/3
Cascine del Riccio. Impruneta (FI)

FABRICATORS
ARCHITECTS OF CULTURE - FABRICATORS OF IDEAS
Via F.lli Bronzetti 6, 20129 Milano/Italy
Phone: 39-2-70128233 - Fax: 76110498
cell: 39-348-2310451
e-mail: fabricat@galactica.it
http://www.fabricat.com

BACK TO INDEX