12 Steps toward Immersion
You feel like you’re there. That’s the power of immersion.
Immersive technologies aim to hijack your senses with signals which persuade you that you are embedded in a virtual environment. When it succeeds, you are promoted from mere observer to full inhabitant of the new world.
Immersion promises us new experience: We can explore and we can learn. We can visit worlds that are beyond our reach: remote locations, ancient times, worlds of fiction and every sort of fantasy. We visit them as lucid dreamers: Thanks to immersive technology, our “willing suspension of disbelief” requires very little will power.
This goal isn’t new. The Renaissance painters achieved it by inventing perspective. Early movies -grainy, jumpy and silent – were stunningly immersive to their new audiences. Today we seek immersion in the new medium of Virtual Reality.
Many varieties of experience are called Virtual Reality – ranging from simple viewers to large multiplayer spaces. At each level, immersion is deeper because more senses are engaged and the viewer becomes more of a participant. Meanwhile each level demands more skillful development. Let’s look at twelve features that promote immersion.
This 360 photograph shows the GamesThatWork production stage, with the author wearing a VR headset. (Hint: Try dragging it!)
When each eye sees a slightly different view of the world, depth perception becomes possible. This phenomenon of binocular vision has been exploited since the Victorian era. It sucked up pennies in 1900’s peepshow arcades and it will sell 3D movie tickets tonight. Virtual Reality headsets nearly always exploit stereo vision.
When the image contains everything that can be seen from a single point, it is called 360 (pano, photosphere, surround photo). You are the center of the scene. As in real life, you see only one sector of the scene at any time, but you can turn and tilt the view to see whatever part you like. Everyone who has used Google Street View is familiar with 360 vision.
360 photos work on various platforms. On a flat screen, you change the angle of view with your fingers. Virtual Reality systems, however, introduce the new feature of head tracking. To select the angle of view, just turn to face what you want to see. This direct, natural control of the field of view enhances immersion. It is associated with headsets, but it can be achieved with a bare cellphone. As you turn your head, you keep the cellphone screen in front of your face – offering a moving window into the virtual world.
The 360 image can be a video. In this case, the world contains moving images and the location of the point of view (the camera location) may also be moving. While viewing, you cannot control the location of this camera, but you can control its angle and tilt.
The sense of immersion is significantly enhanced by the addition of audio, particularly when the audio sources are located in space and change as the field of view turns. Audio production is far less expensive than producing visuals, yet it can be even more compellingly immersive.
The virtual world becomes far more fluid when it is live, rather than pre-recorded. You view fresh computer generated content, rather than static photography. The view is rendered in realtime, as in a videogame. Now, you can not only change the angle of view but also shift the location of your head to see around objects in the foreground. This effect, parallax, is a strong contributor to visual immersion.
Not only can you change your point of view, but the live computer generated world can respond to your actions.
This class includes the hybrid technology of photogrammetry, in which the synthetic world is not an artist’s model, but the result of scanning a real scene and its contents.
This Sketchfab model shows Claire Blare, GamesThatWork’s teen pregnancy prevention spokes-rapper. Cartoon worlds, ironically, are generally more immersive than realistic worlds with similar budgets. GamesThatWork’s teen pregnancy prevention spokes-rapper. Cartoon worlds, ironically, are generally more immersive than realistic worlds with similar budgets. Visitors in cartoon VR see a self-consistent world. By contrast, in phot0-realistic VR, the eye seeks out all the defects.
When your live virtual world includes a live view of your real world, you are in Mixed Reality. This category includes Augmented Reality, where the system embeds virtual images into the world around you, either by intercepting the video feed of a camera or by reflecting into the view you see through your glasses.
It also includes Augmented Virtuality, in which bits of your real realtime world can be seen embedded in the computer generated scene,
In Augmented Reality, you control the not only the angle of the camera, but the also its location. A geolocated application, using GPS for example, is responsive to location all over the world. Think Pokémon GO.
In ‘Crike’, Air Force trainees could see virtual anatomy superimposed on live standard patients. This GamesThatWork concept art shows the real scene and the augmented view with the player performing cricothyrotomy.
High precision location becomes possible when GPS satellites are replaced by local electronic beacons, like the Vive‘s tiny lighthouses. Alternatively, some sophisticated headset sensors (HoloLens, Tango) use the room’s natural landmarks. These technologies allow you walk around, stand up or crouch to see new things. This freedom of movement adds extraordinary verisimilitude to the virtual scene. Exploring on foot, rather than sitting, means you are far more present in the virtual space.
Different systems offer different volumes: The mass-market Vive operates at room scale. Special purpose systems can offer warehouse-scale play with multiple players.
Walking in a tightly registered virtual space makes it feel rock solid. But systems without this positioning technology often attempt virtual locomotion. You move in the virtual world, but your real body goes nowhere. The dissonance between visual cues of movement and the body’s sense of stasis frequently results in nausea.
A system that can precisely track your head can (with more hardware) also track other things. In particular, it can track your hands. The virtual world becomes more immersive as you can reach out to touch it. Handheld controllers also add haptic experience, immersing more sense in the virtual world.
When you touch the world, you change the world. The virtual world can respond to your direct actions – throwing an object, wielding a weapon, opening a door. You are not just present, you are participating, As you move, you enjoy the consequences of your action. A world that responds to you offers profound immersion.
This 360 photo shows an early phase in GamesThatWork’s Kitchen Designer, in which the player interactively equips her virtual kitchen.
It is not enough to be present. It is not enough to be interactive. A deeper immersion comes when you are motivated to achieve a goal in the virtual world. You will be fully immersed when you are in a world that presents challenges and you confront these to achieve an end.
This Korean fan site shows the award-winning Brush Up VR in action. It is GamesThatWork’s vigorous, humorous action game in room scale VR. Kids learn about conscientious dental hygiene – and everyone has fun.
Your goal in the virtual world may be entirely fictitious and self-contained (eg: you save the castle from attackers) or it can relate to back to challenges you face in real life.
In our studio, we prefer to close the loop. We want players in the world of virtual reality to return with new skills and insights that improve their experience of regular reality. These skills might include hygiene, first responder skills, social skills and others.
Before we start counting sophisticated computational techniques, let’s acknowledge that the most effective technology is a cheap bit of plastic (or cardboard). With hooded eyes, and covered ears, we achieve a lot of immersion just by masking out the competing signals of real reality.
The success of a new medium depends on everything from technological readiness to market conditions. It depends on creative zeal. But most of all, it depends all on its ability to meet human needs.
To this, we dedicate our efforts.