It’s just about impossible to get your hands on Microsoft’s impressive mixed-reality HoloLens platform these days — unless you’re a computer science student at the University of Washington. Then you get to play with them whenever you want.
At least that’s the case for the students in CSE 481V, in which, according to the course description, you will “learn a ton about Virtual and Augmented Reality, get familiar with the latest technology and software, and build an app in 10 weeks.”
This is the first time the course has been offered in this fashion, with generous underwriting by local VR/AR players Microsoft, Oculus and Valve/HTC. The 36 students in the course had access to the HoloLens dev team and all the major headsets — there were 25 HoloLenses involved, which is probably more than have ever been in one place. Students also got to hear from guest speakers like Oculus Chief Scientist Michael Abrash and author Neal Stephenson — whose “Snow Crash” was required reading for the course.
All in all, it’s enough to make a guy want to matriculate.
“We pitched the idea of a VR/AR class last year to HoloLens leadership and they immediately got excited and were eager to make it happen,” wrote Steve Seitz, one of the class’s instructors. “I was initially quite worried about the idea of relying on a brand new device and development platform for a 36 person class. But I’m extremely impressed with the development environment… it was good enough that students with no prior experience could get up and running quickly and make some really compelling applications in just a few weeks.”
You can see what those applications were at the course webpage, complete with weekly blog posts showing progress from concept to execution. There’s augmented reality cooking, a painting app and the clever idea of gamifying the process of scanning a room so it can be used in other apps.
The class culminated in a sort of open demo day at the UW campus, where students could show off their work to the general public and serious players like Microsoft Research’s CVP Peter Lee.
It’s a great opportunity for students, no doubt, but also a fertile testing ground for the companies in the space. How did these fresh young minds interact with the technologies? What did they run up against? What tools did they wish they had? This kind of extensive focus testing is always valuable, not to say this was an ulterior motive, just that it was no doubt a fruitful collaboration.
“For the HoloLens team, this was an opportunity to evaluate the platform in a focused educational settings, and get early feedback,” wrote Seitz. The team also provided technical support and training.
Seitz and the class’s other instructor, Ira Kemelmacher-Shlizerman, aim to offer the class again next year. UW is, of course, a convenient location for Microsoft to work with, but the institution is also a hub for research in this area, having pioneered many VR and AR ideas early on in its famous HITLab.
[vc_row css=".vc_custom_1470325448228{background-color: #1e73be !important;}"][vc_column][vc_column_text]
[/vc_column_text][/vc_column][/vc_row]
[vc_row][vc_column][vc_column_text]
“Experiencing mixed-reality is truly mind-blowing and I can’t wait to see the looks on the faces of these area execs as they complete their holographic mission.” -Jackie Wiener, VP Marketing & Customer Engagement, Managed Solution
[/vc_column_text][/vc_column][/vc_row]
[vc_row][vc_column][vc_column_text]
[/vc_column_text][/vc_column][/vc_row]
By Brian Barrett as written on www.wired.com
Help me, HoloLens. You’re my only hope.
OK, so maybe it’s not quite time to write R2-D2 out of Star Wars quite yet. But Microsoft researchers have created something that brings one of the droid’s best tricks to our present-day lives. It’s called holoportation, and it could change how we communicate over long distances forever. Also, it makes for one hell of a demo.
It started, though, as a response to homesickness, says project lead Shahram Izadi. His Cambridge (UK)-based team, which focuses on 3D-sensor technologies and machine learning among other next-generation computing concerns, had spent two and a half years embedded with the HoloLens team in Redmond, Washington. Izadi is a father—that’s his daughter in the demo video—and when the time came to dream up the next challenge, they turned to the one that most affected their personal and professional lives during that stretch: communication.
“We have two young children, and there was really this sense of not really being able to communicate as effectively as we would have liked,” Izadi says. “Tools such as video conferencing, phone calls, are just not engaging enough for young children. It’s just not the same as physically being there.”
So they created something that, in several key ways, is. Holoportation, as the name implies, projects a live hologram of a person into another room, where they can interact with whomever’s present in real time as though they were actually there. In this way, it actually one-ups the classic Star Wars version, in which a recorded message appears in hologram form. Holoportation can do that, too, but the real magic is in what basically amounts to a holographic livestream.
As you might imagine, that takes a lot of horsepower.
I’ll Take You There
The Holoportation system starts with high-quality 3D-capture cameras placed strategically around a given space. Think of each of them as a Kinect camera with a serious power-up. “Kinect is designed to track the human skeleton,” says Izadi. “We’re really about capturing high quality detail of the human body, to reconstruct every feature. That has required a rethinking of the 3D sensor from the ground up.”
Once those cameras have captured every possible viewpoint, custom software stitches them together into one fully formed 3D model. This process is ongoing, Izadi says, as more frames of data make for a higher-quality model. The accumulated data results in an incredibly lifelike hologram that can be transported anywhere in the world that has a receiving system, like, say, a HoloLens. And it can do it fast.
“We want to do all of this processing in a tiny window, around 33 milliseconds to process all the data coming from all of the cameras at once, basically, and also create a temporal model, and then stream the data,” says Izadi, whose team leans on a small army of off-the-shelf (but high-end) Nvidia GPUs to crunch the relevant numbers.
But wait, you’re saying, that must be an insane amount of data to transmit. You’re right! Not only does the holoportation process generate mountains of data, Izadi points out that most streaming video codecs aren’t particularly 3D-friendly. That makes compression, which in this case transforms gigabytes into megabytes, a huge part of making holoportation work.
Aligning Worlds
To be clear, what you’re seeing in this video is real. It actually does work. There are still some hurdles to overcome before holoportation becomes a part of our everyday lives, though. You’ll notice, for instance, that the furniture in the two rooms that Microsoft uses is identical, making interaction much more seamless than it would be with the furniture from two rooms overlapping, or people walking through desks, and so on. Fortunately, there’s a straightforward solution: Train the cameras to only focus on the items you want to holoport, rather than an entire room.
“The user could potentially decide that they don’t want to replicate any furniture,” says Izadi. “We have this notion of background segmentation, where you capture the room just with the furniture in it, and that means that only the foreground object that goes into the room after will be of interest for the stream.” You could also strategically incorporate certain pieces of furniture by deciding how the two rooms align. Take, as an example, two grandparents holoporting in from their couch so that they can experience Christmas morning. Rather than let them float in mid-air, one could decide to orient their seated holograms onto the couch in their own living room.
This gets to be fairly heady stuff. For now, it’s probably enough to know that Izadi’s team is aware of potential spatial problems, and sees them instead as opportunities. After all, Izadi ultimately sees the project as a consumer device. We already have dedicated home theaters; why not dedicated home holoportation rooms, as well?
“The end goal and vision for the project is really to boil this down to something that’s as simple as a home cinema system” says Izadi. “You walk in to a number of these almost speaker-like units, in a way that you would set up surround sound, but this is giving you surround-vision.”
Long before then, maybe even within a couple of years, Izadi expects that you might find holoportation rigs in meeting rooms. And while it would be expensive—that’s a lot of cameras, and a lot of GPUs—he points out that global business travel costs about a trillion dollars a year. Holoportation starts to look a lot less expensive next to a round trip ticket to Shanghai.
Also intriguing? While Izadi’s team has worked closely with HoloLens, their system doesn’t play hardware favorites. All you’d really need to enjoy, at the receiving site, is an virtual reality or augmented reality headset.
“We’re very much agnostic to what we call the ‘viewer’ technology,” says Izadi. “Obviously we feel like there are some unique scenarios with HoloLens, but we would like to leverage as many display technologies as possible.”
Including, one suspects, plucky little droids.
About Managed Solution
We're technology enthusiasts with a people-first approach. For over two decades, we've witnessed the profound impact that the right technology and support can have on businesses and individuals. Success, to us, is seeing our clients, partners, and team conquer challenges to achieve their greatest goals and build lasting connections. This relentless pursuit of inspiration drives us forward, pushing us to deliver innovative solutions that empower growth and lasting success.
By Alex Kipman as written on blogs.windows.com
In December, a shuttle resupply mission successfully reached the International Space Station. Among the cargo were two Microsoft HoloLens devices for use as a part of NASA’s Sidekick project. The goal of Sidekick is to enable station crews with assistance when and where they need it. According to NASA, this new capability could reduce crew training requirements and increase the efficiency at which astronauts can work in space.
We were thrilled to see some early pictures today of astronaut Scott Kelly with HoloLens at the International Space Station!
To provide a little background on the project, Sidekick has two modes of operation. The first is “Remote Expert Mode,” which uses Skype, to allow a ground operator to see what a crew member sees, provide real-time guidance, and draw annotations into the crew member’s environment to coach him or her through a task. Until now, crew members have relied on written and voice instructions when performing complex repair tasks or experiments.
The second mode is “Procedure Mode,” which augments standalone procedures with animated holographic illustrations displayed on top of the objects with which the crew is interacting. This capability could lessen the amount of training that future crews will require and could be an invaluable resource for missions deep into our solar system, where communication delays complicate difficult operations.
In order to prepare for the mission, and what it would be like to use HoloLens at the International Space Station, NASA had a chance to experiment with it quite a bit at the Aquarius underwater research station as a part of NASA Extreme Environment Mission Operations NEEMO. Below are a few pictures of Astronaut Luca Parmitano using HoloLens at the underwater facility.
And we’re happy to report that HoloLens is mission operational at NASA’s Jet Propulsion Laboratory – exploring Mars using holograms of Mars Rover images.
We couldn’t be more thrilled about the work we are doing with NASA – I can’t wait to see more from the crew at the International Space Station!
Chat with an expert about your business’s technology needs.
