In December, a shuttle resupply mission successfully reached the International Space Station. Among the cargo were two Microsoft HoloLens devices for use as a part of NASA’s Sidekick project. The goal of Sidekick is to enable station crews with assistance when and where they need it. According to NASA, this new capability could reduce crew training requirements and increase the efficiency at which astronauts can work in space.
We were thrilled to see some early pictures today of astronaut Scott Kelly with HoloLens at the International Space Station!
To provide a little background on the project, Sidekick has two modes of operation. The first is “Remote Expert Mode,” which uses Skype, to allow a ground operator to see what a crew member sees, provide real-time guidance, and draw annotations into the crew member’s environment to coach him or her through a task. Until now, crew members have relied on written and voice instructions when performing complex repair tasks or experiments.
The second mode is “Procedure Mode,” which augments standalone procedures with animated holographic illustrations displayed on top of the objects with which the crew is interacting. This capability could lessen the amount of training that future crews will require and could be an invaluable resource for missions deep into our solar system, where communication delays complicate difficult operations.
In order to prepare for the mission, and what it would be like to use HoloLens at the International Space Station, NASA had a chance to experiment with it quite a bit at the Aquarius underwater research station as a part of NASA Extreme Environment Mission Operations NEEMO. Below are a few pictures of Astronaut Luca Parmitano using HoloLens at the underwater facility.
And we’re happy to report that HoloLens is mission operational at NASA’s Jet Propulsion Laboratory – exploring Mars using holograms of Mars Rover images.
We couldn’t be more thrilled about the work we are doing with NASA – I can’t wait to see more from the crew at the International Space Station!
How The Growth Of Mixed Reality Will Change Communication, Collaboration And The Future Of The Workplace
A recent report from investment bank Goldman Sachs predicted that within 10 years, virtual reality hardware will be an $80 billion industry. This “base case” forecast assumed that adoption will be slow, as compared to that of smartphones and tablets, but, the report noted, “as the technology advances, price points decline, and an entire new marketplace of applications (both business and consumer) hits the market, we believe VR/AR has the potential to spawn a multi-billion dollar industry, and possibly be as game changing as the advent of the PC.”
While the conversation around VR (virtual reality) and AR (augmented reality) often focuses on gaming and video entertainment, the Goldman report theorizes that these use cases will account for less than half of the software market.
As a sometimes-gamer, it’s fun to think about strapping on a headset and diving headfirst into my favorite virtual worlds. But to limit our imagination to these applications is ignoring the unlimited potential of a hybrid reality created by augmented and virtual technology to affect every business and industry.
By combining analog, two-dimensional ways of working with new mixed–reality experiences, we can transform our ability to communicate, collaborate and create. The challenge for businesses will not be to provide a more immersive experience, but a more valuable experience.
The continued disruption of communication modalities
Message carriers were put out of work by the telegraph, the telephone was disrupted by the Internet and the good old-fashioned conference call was replaced by VoIP video conferences and screen-share-enabled unified communications systems.
Before the Internet, the historical evolution of long-distance communication technology was always toward replicating human connection in its clearest form: a face-to-face conversation. The telegraph may have missed the human voice, but its relative speed was a step toward an immediate verbal response.
Ironically enough, the first words spoken across a telephone line in 1876 by Alexander Graham Bell to his assistant Thomas A. Watson were, “Mr. Watson, come here — I want to see you.”
Most digital communication across the Internet lacks the verbal, facial and body language cues of a face-to-face conversation, but the reach of our messages and the media at our disposal (photos, videos, memes, gifs, articles, etc.) has made it a medium of undeniable allure and value.
Why would I call a friend on the phone and tell them about a great concert when I can post a status and let all my friends know at once, all while showing them a video of me belting out my favorite song with the performer?
That being said, to say there is sometimes communication breakdown across the Internet is an understatement that requires no further explanation for anyone that has ever read a Comments section.
Don’t get me wrong, a connected world is undoubtedly a better world. I defer to the mission statement of the Mark Zuckerberg-led Internet.org for a perfect summation:
“The internet is essential to growing the knowledge we have and sharing it with each other. And for many of us, it’s a huge part of our everyday lives. But most of the world does not have access to the internet. Internet.org is a Facebook-led initiative with the goal of bringing internet access and the benefits of connectivity to the two-thirds of the world that doesn‘t have them. Imagine the difference an accurate weather report could make for a farmer planting crops, or the power of an encyclopedia for a child without textbooks. Now, imagine what they could contribute when the world can hear their voices. The more we connect, the better it gets.”
But the more we connect, the more important it is that we connect better.
Virtual, augmented and mixed experiences that exist at the intersection of our physical and digital worlds will bring the humanity of the face-to-face conversation back into the evolution of our communication.
Don’t make the mistake of equating these virtual experiences solely with sci-fi and gaming applications in which you have a surrogate and exist in a different, alternative reality system.
Mixed reality, or hybrid reality, merges real and virtual worlds to produce new environments where physical and digital objects co-exist and interact in real time.
I’m not talking about plugging into the Matrix as a means for improved communication. I’m talking about the ability for two people across the world to put on a headset and share any experience they choose — whether it’s to sit next to each other and physically flip through a photo album or to visit their dream destination.
Five or 10 years ago, we used text to communicate. Today, we communicate and share with photos and videos. Tomorrow, with VR, we’ll be able to communicate with experience.
What does this mean for the future of the workplace?
For one, it means improved collaboration. Mixed reality has the potential to allow a global workforce of remote teams to work together and tackle an organization’s business challenges. No matter where they are physically located, an employee can strap on their headset and noise-canceling headphones and enter a collaborative, immersive virtual environment.
Language barriers will become irrelevant as AR applications are able to accurately translate in real time. Imagine Google Translate acting in real time between two or more people.
It also means a more flexible workforce. While many employers still use inflexible models of fixed working time and location, there is evidence that employees are more productive if they have greater autonomy over where, when and how they work. Some employees prefer loud workspaces, others need silence. Some work best in the morning, others at night.
Employees also benefit from autonomy in how they work because everyone processes information differently. The classic VAK model for learning styles differentiates Visual, Auditory and Kinesthetic learners.
Visual learners will appreciate the immersion and optic stimuli of mixed reality. If nothing else, auditory learners will benefit from the reduction in auditory distractions that plague the modern open office space. Kinesthetic learners that learn best by moving, touching and doing will benefit from being able to explore and collaborate in mixed reality. Conference calls that cause kinesthetics to tune out can be replaced by interactive, tactile modes of work-like whiteboarding sessions.
This greater autonomy in where, when and how employees work will serve to maximize productivity by empowering them to complete tasks in the manner that is best for them. It will allow employees to enter and work in “flow” states of complete absorption.
Named by renowned psychologist Mihaly Csikszentmihalyi, flow refers to “the mental state of operation in which a person performing an activity is fully immersed in a feeling of energized focus, full involvement, and enjoyment in the process of the activity.”
Video gamers should immediately recognize this mental state, as game design is particularly adept at inducing flow states where hours and hours fly by and the player is completely enveloped in the game.
Csikszentmihalyi theorizes that in order to retain flow and “stay in the zone,” the activity must reach a balance between the activity’s challenges and the participant’s abilities. If the challenge is too great, it promotes anxiety — too easy, and it promotes boredom.
The seesaw between anxiety and boredom is far too familiar to the modern workforce. Without fail, we try to get heads down on a project, and the emails, slack messages and “do you have a minute?” desk drive-bys keep us from ever being able to focus. Anxiety rears it ugly head.
We finally get the project done and while we are waiting for feedback from the client or organizational leadership, the communication channels miraculously quiet down. This is where boredom comes in.
Mixed reality is conducive to inducing flow states because of its ability to immerse employees in designed experiences that match their learning styles, preferences for stimuli and ability. But perhaps more importantly, it can serve to limit the distractions that cause anxiety and the latency that leads to boredom.
Distractions are eliminated by the worlds we are able to design that only push the messages imperative to the work we are doing.
Latency, or the time between an action and its response, is eliminated when our work is memorialized digitally as we complete it. A client or supervisor is able to join our work process digitally at any time to track and review progress.
Last, but certainly not least, mixed reality creates solutions for the universal problem of finite resources.
Aside from eliminating the monetary travel cost and the opportunity cost of time spent on red-eye flights and in jet-lagged meetings that plague global business, mixed reality reduces an even more sparse resource — real estate.
On a macro level, population is increasing and space is not. Reducing the need for large offices by creating virtual workspaces will make the office park a relic.
On a micro level, just think about your own office. There are never enough conference rooms, and never enough workspaces. That awesome whiteboard you just covered with great ideas? Your colleague is coming in 30 seconds after you finish for a client call and needs it erased.
Mixed reality workspaces that memorialize our work while we complete it will not require furious note taking and cell phone picture snapping in those 30 seconds.
In fact, those 30 seconds will not exist, because whether we are sitting at our desk, in our home or in Starbucks, accessing a perfectly designed virtual workspace is as simple as putting on your headset.
The future of communication and collaboration at work will be defined by virtual, augmented and mixed reality experiences that provide economic value. To equate this collision of our physical and digital worlds solely with play and entertainment is to miss one of the great upcoming technological evolutions of our workforce.
Mobile Virtual Reality Is The Ghost Of Christmas Past
The worlds of gaming, sports, live music, commerce and adult entertainment are all primed for a revolutionary consumption shift with the advent of virtual reality (VR). Fantasy shall become reality; a new age of immersive entertainment lies just around the corner.
And just as the world progressed from radio to television, so too shall it do so with VR.
Lo and behold, we have the Samsung Gear VR as this Christmas’ must-have VR toy. Virtual reality, nestled under your tree, for the low, low price of only $99 (plus the $500 Samsung smartphone you need to actually power the Gear VR headset).
Whoa, whoa, hold up just a minute. We are a long ways off from the aforementioned VR Shangri-La with Samsung’s current VR headset. This “VR,” dubbed “mobile VR,” relative to true VR, is like thinking of your car phone (brick phone) relative to today’s powerful smartphone.
Gear VR allows us to sit front row at an NBA Warriors game or go backstage at a Nicki Minaj concert (thanks to content from NextVR and Jaunt) — but, it is a passive experience. Being able to move your head around does not make for much interaction, and that’s all you can really do with the Gear VR.
More importantly though, I could reach my hand out to high-five Stephen Curry, but Curry would just run right past me. I’m invisible…the people in this virtual world can neither see nor hear me. It’s like I’m Ebenezer Scrooge, floating alongside the ghosts of Christmas past, present and future, through Charles Dickens’ ethereal journey in A Christmas Carol.
While I am excited to show my family many Gear VR experiences this Christmas, Christmas 2016 is the real holiday to watch. The big push in consumer marketing from Oculus, Sony and HTC will be coming then. We’ll have an install base of about 25-35 million Sony PlayStation 4s, and Sony’s VR headset, PlayStation VR, will have an avid and waiting fan base of gamers to buy it. Go to Best Buy, pick up a discounted PlayStation 4 and VR headset, plug it in and you’re good to go. Easy.
Marketing the headset as a peripheral plug ’n play device is easier than the uphill battle Oculus and HTC/Valve may have ahead. That said, Oculus, HTC and Sony have developed “true VR” — i.e., games and content that allow the user to feel truly immersed, so far as they can interact with the world and other VR users around them — a stark difference to the largely passive experience offered by mobile VR to date.
While the VR experience from Oculus, Sony and HTC/Valve VR headsets will be leagues ahead of that provided by the Gear VR, such headsets warrant another Scrooge analogy at this point. It is unlikely that the miserly Ebenezer would ever consider paying the non-trivial $1,000 (or more) to buy an appropriate PC to run these advanced VR headsets.
The high price point (and more technical user requirements for a PC-based headset) presents an impediment to building and scaling an audience quickly. This means that any kind of ad-supported, free-to-play business models are a ways off; you need lots of eyeballs and players before you can make meaningful revenue as a content creator.
A rich, easy-to-monetize content ecosystem, like that provided by Google and Apple on mobile, is necessary for the flywheel of content creation and consumption to spin faster and faster. In the near term, we will only see VR content funding from the headset manufacturers themselves, and from brands who see VR as a way to look cool and edgy.
But for now, we have 360-degree videos. Yes, they provide a more immersive viewing experience than watching your living room TV. But you are stuck viewing the action from a fixed position. You can spin around on your office chair or stand up and turn around to get 360-degree views from where you’re seated, but try rolling forward. Things do not look quite like they should in real life, which is enough to shatter the illusion and immersion.
John Carmack, chief technical officer at Oculus (a partner of Samsung’s on the creation of the Gear VR device and software ecosystem), stated that “positional tracking” will be possible on future versions of the Gear VR. A chance to move around freely within replayed memories would be pretty cool.
For example, imagine having free range of motion inside a family video replay, walking around your child’s birthday party video and seeing the looks on everyone’s faces, enjoying these memories in a whole new way.
True social and environmental interaction in VR will come from Oculus, HTC/Valve and Sony. Oculus’ in-house content team has tight links to the world’s largest social network through its parent company, Facebook. Valve has deep gaming chops with hardcore PC gamers. And Sony has a long history of success in developing hit triple A games for its PlayStation console.
All of them stack up better than Samsung when it comes to making gamers happy. And gamers are usually some of the earliest adopters of new technology platforms, so it is important to win them over at VR’s launch. The Xbox did well because of Halo, and gamers await VR’s Halo equivalent with bated breath. Frankly, I don’t think it’s going to come from a Samsung platform.
Expanding VR to the masses will take experiential demos at Best Buy and Target. It will require ladies of The View, hosts of Sports Center and stars on YouTube and Snapchat to evangelize the hardware and platform.
More importantly, if Google’s failed augmented reality headset, Glass, serves as any kind of example, you can partner with fashionistas such as Diane von Furstenburg, have beautiful ad campaigns with beautiful models, et cetera, et cetera, but asking people to put something “techy” on their face can be very weird.
So, getting the hardware right is important. That means a design sense like that which Apple’s Jony Ive brings to technology products, making a device that is ergonomically and aesthetically beautiful. All the Madison Avenue ad dollars in the world cannot make VR cool if it follows Google Glass’ path.
For example, photos of old tech guys in the shower, wearing Glass, did not exactly help market the device beyond its nerdy, Mountain View roots. And we really need folks to think this is for everyone — not just geeks!
The good news is that mass-pleasing sports, music and games are all high on the roster for major VR headset manufacturers at launch. Oculus, HTC and Sony are funding top-tier content creators to stock digital store shelves with fun experiences, bundled and ready-to-go on 2016’s ship date.
Mobile VR is a great stepping stone to true VR, which, in itself, is a stepping stone to even more mass-pleasing platforms such as augmented reality (a discussion for another day).
In the meantime, enjoy your Gear VR this Christmas. Float through Victorian England with Ebenezer Scrooge and the ghosts of Christmas past, present and future. Just know that you, too, are a ghost in there, paralyzed from the neck down, and mute.
“These are but shadows of the things that have been,” said the Ghost. “They have no consciousness of us.” Wait until next year for that to change.
