Predicting future technological trends can be a self-built trap. Some developments can turn out to be fads or dead ends (3DTV anyone?) or viable technologies can take a long time to become reality. That’s certainly true in the case of Thomas A Furness III, the “grandfather of virtual reality”, who told Howard Rheingold for the 1991 book Virtual Reality: “Don’t stay fixated on the idea of screens. Screens might become obsolete sooner than you think.”
The screen – or to use a more flexible term, the display – is still very much with us 30 years on. Not only are they there in our homes (TVs, computers) but also almost everywhere outside (digital signage and video displays in shops, railway stations, airports, museums, hospitals and bars). And, of course, just about everyone carries a screen with them: their smartphones.
Despite our reliance on a recognisable display and its current ubiquity, it is more than likely that Furness’ prediction will come to pass to some extent. Virtual technologies ç or, to be precise, 3D display technologies – are becoming more mainstream, finding their way into areas outside gaming, experiential events and experimental cinema. Education, museums, corporate businesses and public spaces such as shopping centres and urban spaces now feature some form of virtual display or projection.
The big question is whether the screen/display as we know it will continue to be the best way of presenting images. The means of watching virtual or 3D material is dictated, in part, by whichever technology is being used. Virtual reality (VR) is synonymous with wearing headsets and although development continues to make these smaller and less cumbersome, there is always a barrier between the wearer and other (real) people.
“VR is fantastic for a solo experience,” comments Andy Hook, technical solutions director of entertainment lighting and AV services company White Light. “It’s not collaborative and probably 99 percent of what we do as a business is about shared experiences. Whether that’s theatre, live events, performance, education or corporate installations, everything [we do] is about our shared collaborative experience. We’ve seen massive increases in VR quality, both from a rendering point of view and the weight and comfort of the headsets, but I think it has a limited lifetime, at least for mass consumer adoption.”
The virtual/3D field that Hook does see as “the next technical revolution” is augmented reality (AR). “It will impact almost every market we deal with and probably every person, in the same way that an iPhone or smartphone has changed the world,” he says. Instead of a wholly computer-generated world, as with VR, AR combines reality with 3D imagery, with information or images ‘overlayed’ on to what the viewer is seeing around them (buildings, streets and so on).
AR and the technology Hook regards as the ultimate goal XR (extended reality) can both be placed
under the general head of ‘mixed reality’. This term is used to describe White Light’s main product in the virtual/3D-imaging field, SmartStage. Essentially a way to perform green screen effects and presentations without a green screen. SmartStage was designed to allow anyone to produce sophisticated looks and effects without the need for full-time technical supervision. “We previewed our XR tech at IBC [International Broadcasting Convention] 2018 and initially thought it would be used purely for broadcasting,” Hook says. “Within months after that we were installing it in universities and corporate installations. They wanted people who weren’t broadcast professionals, like students and marketing executives, to create amazing looking content with minimal effort.”
SmartStage comprises a media server for each studio camera, the outputs of which feed into a vision mixer, which in turn feeds a series of LED video walls. This means the presenter and other people in the stage environment are able to see everything going on around them (rather than just a blank green screen and relying on small monitors) and interact with it more directly.
For AR, headsets in the most general sense can still be involved but these are more likely to be like regular glasses, albeit with ‘smart’ technology. Even these could become redundant, replaced by science fiction-style tech such as Sony’s compact retinal near-eye display and the Mojo AR contact lens. More recognisable displays are also involved, with head-worn mini-screens, optical projection systems, monitors and handheld devices.
Hilary McVicker, vice president of sales and marketing at virtual technologies design and engineering company The Elumenati, observes that wearable devices – not just glasses but gadgets such as fitness trackers – will continue evolving. “The agility and eventual affordability of these products will make them the right choice for many use cases,” she says. “But the social, collaborative aspects of large-scale displays and installations – both projected and LED – cannot be replaced.”
This, McVicker explains, is the driver behind The Elumenati’s immersive environments, which she describes as “VR beyond the headsets”. The company combines immersive projection systems and screens with interactive software to produce different display environments, including domes, 360-degree cycloramas and globes. New to the portfolio is compound-curved LEDs for large-scale installations in spaces with high ambient light levels. A recent project is the Morehead Planetarium and Science Centre GeoDome at the University of North Carolina. Through touch-free interactivity, visitors can access the star map created by the full-sky, giga-pixel Evryscope telescope. The exhibit is based on a 3-meter diameter OpenDome Portal illuminated by an OmniFocus 300 series 4K projector.
McVicker comments that communal experiences such as this are a major advantage of large-scale immersive displays. “It’s impossible to overestimate human connection,” she says. “In entertainment the ability to be with others for play and fun is key to bringing audiences back after the pandemic. Headsets have always presented an operational challenge in maintaining hygiene and this will be even more so post-Covid. Large-scale installations can solve this and offer an alternative way to provide contact-free immersion. Headsets will also be a tough-sell for experiential marketing, where hybrid events with virtual components accessible remotely, complementing in-person gatherings, might be the future.”
This mixed and extended form of virtual presentation is one that Pippa Bostock, business director of the Centre for Creative and Immersive Extended Reality (CCIXR), part of the Faculty of Creative and Cultural Industries at the University of Portsmouth, feels is the way ahead, with display technology changing to suit the purpose. “I think it [displays] will evolve, in the same way that none of us would have even conceived of the term XR five years ago,” she says. “To me, the boundaries between those three [VR, AR and XR] are going to shift and we’re not going to keep those defined terms necessarily. We will merge much more towards XR as encompassing all of these technologies.”
Among other facilities CCIXR features 12 XR labs and a White Light SmartStage. Bostock explains that the centre was created to “enable businesses to come and engage with cutting edge immersive technologies to understand what those can mean for them and the benefits they bring”. It will also be a R&D base, as well as nurturing a new generation of developers and researchers. “The graduates emerging into this field have been called unicorns, because they’re so rare and mythical,” Bostock says. “With the CCIXR team we are creating a unicorn farm because we’re filling that skills gap in the industry, with graduates that will have real-time engine skills and an understanding of the different sectors in which they will be used.”
XR is attracting not only a new breed of developer but also new entrepreneurs who are establishing companies with the express aim of producing groundbreaking technologies that will change how images are created and viewed. Among these is Voxon Photonics, based in Adelaide, South Australia. For over ten years this start-up has been working on volumetric 3D display technology, with the aim of creating something people can move around and look at from any direction, with each having a different view of it.
The company’s first commercial product – and its current flagship – is the VX1 3D volumetric display. This is intended for use in education, data visualisations, experiential marketing and medical imaging to provide a collaborative experience with the real-world experience of people having a unique perspective of the image because of their position. “If you and I look at another person, there’s no situation where two people have the same view of that person,” co-founder, CEO and CTO Gavin Smith explains. “We use a volumetric display to achieve that and in our case it’s a swept surface volumetric display. This means a screen is moving somewhere in the display. It’s like when you draw circles in the air with a burning stick round a campfire, you create a circle using persistence of vision. This is because of the humanised inability to see high speed images, which blurs these together and you see a circle. In our case we are projecting over four thousand images per second on to a moving surface.”
Smith believes volumetric technology will continue to grow, with the real challenge being scaling it. “Ultimately what we’re doing is creating an extension,” he says. “VR has taken people down the rabbit hole [into] kind of an insular, lonely experience. Maybe you’re sharing things with your friends in the online world but this Matrix-like future where everyone’s plugged into a pod is not necessarily somewhere everyone wants the world to go. I certainly don’t want that for my children, so we’re focusing on displaying 3D data in the real world with social values, face-to-face contacts and conversation, trying to make the kind of interaction of a board game like Monopoly but making it a digital experience with the interactivity of a video game.”
Prox & Reverie is another start-up that was founded to exploit the potential of XR and immersive technologies, with a focus on entertainment. Co-founder and director Martin Taylor observes that virtual technology has moved beyond the time when, around 2015, it was being dismissed by critics as just another fad like 3DTV. “These technologies are now crucially embedded in so many industries, providing content and platforms for consumers and businesses alike,” he says. “It’s a tool [and] in a wide array of industries such as MedTech and even space training, VR and mixed reality have become an incredibly valuable tool in research and application.”
The general enthusiasm for mixed reality experiences, Taylor points out, is reflected in how the terms VR, AR and real-time – which at one time would have been highly specialised or nerdy – are now used widely among the wider population. “Each technology has taken a targeted path over the past several years to solve specific challenges, with both independent developers and large technology companies succeeding in their own way,” he comments. “What we’re seeing now is a convergence of these efforts to create truly immersive and seamless virtual experiences.”
On the big question of what all this means for the humble display, Taylor leans towards the personal, wearable solution for augmenting viewing: “VR specifically is overcoming its stigma of being a lonely, individual experience. As a viewing technology it has had to wait a little for new social platforms, cloud services and connection speed to catch up. Even in the last 12 months that has improved dramatically. Whereas the kind of imagined mixed reality science fiction experience of Minority Report needs much more finesse to truly anchor synthetic experiences on to high expectations of physical reality. Again, it’s moving much faster than originally planned, with everyday wearable glasses coming in the next 18 to 24 months, which will signal the true beginning of the spatial computing era.”
Despite the promise of new wearables, Pippa Bostock of CCIXR does not see the faithful screen disappearing. “I think it will always have a place,” she says. “That’s not to say that place won’t change and evolve but look at the popularity of the cinema screen. After all these years, we still go to the cinema because it’s a shared experience. We saw the power of that with The Dream virtual production we worked on for the Royal Shakespeare Company in March. People could have watched individually on their phones but they still put it up on the TV screen and shared it with the family. Screens are such a user friendly and established part of our lives. It’s quite relaxing to sit and passively watch something on a screen or even interact by screens. It’s an important way of communicating information.”
As much as the plugged-in, always connected, enhanced human world of William Gibson’s cyberpunk vision now seems that much closer than it did in 1980s (when they appeared wildly futuristic), it is still not quite with us yet. “We are probably quite a long way from having an AI device projecting on to our retinas that will allow us to draw all of these displays that we need around us,” comments Andy Hook at White Light. “We will probably have for some period of time a physical display device, whether that’s an LCD panel or a projection or whatever, to create the main view of the content we’re looking at. But there will be a way of augmenting that content when we need to weave our mobile device with AR glasses or something that allows us to bring the content out.”
Right now, video panels continue to play a part in White Light’s move into more science realms, such as the current ability to ‘teleport’ the image of a person into a physical setting some distance away. “When we started the XR journey, in tandem with the explosion in virtual production, we did a lot of R&D and investment in LED video display technology,” Hook explains. “We were incredibly picky over viewing angle, colour and reproduction latency and worked with brands such as ROE Visual to push the quality of panels over what we’d needed before. But then there were thousands of manufacturers across China making cheaper but more efficient LEDs. Now we can use those cheaper LEDs as a building material to cover everything in video and the premium products for virtual production and broadcast and, to a degree, XR.”
The market is broadening at an increasingly fast rate, with the means of viewing 3D imagery ranging from glasses and headsets to screens such as the Looking Glass light-field display to the Japanese Aerial Burton system, which uses laser plasma technology to create images in the air. Gavin Smith at Voxon foresees a combination of presentation technologies, with headsets/glasses still in play along with partly translucent OLED matrices and, ultimately, volumetrics.
“The future of 3D imagery is never going to be owned by one particular technology,” he concludes. “Very specifically the future is going to be a combination, with the ubiquitous cell phone producing ground beams bouncing off something to produce images in the air. VR will continue because it’s a 100 percent immersive experience and then there’s what we do. We want a way of displaying that in a display that embodies creative sharing of information. This could be in a school, university, video arcade or even watching football in a pub on a giant volumetric display. That’s what we want to see and I firmly believe that’s going to happen.”
With innovation accelerating, it could be that this, and other predictions, happen a lot sooner than that of Grandfather Thomas Furness.