At Workstreampeople I have been researching the capabilities of Microsoft’s augmented reality device, the HoloLens. Since Workstreampeople is in the Dialogue Management business – e.g. in remote support scenario’s – I am primarily looking for innovation opportunities around Skype and custom apps on HoloLens. Working with this amazing device certainly has inspired me; here are some of the things I learned:
What Skype on HoloLens is all about… for real
The core scenario targeted by the Skype client on HoloLens starts whenever you say something like: “show me how…” or “can you look at this…”.
Here is a recent real-world example, where an elevator service technician from thyssenkrupp asks a remote expert to do a safety double-check:
The work thyssenkrupp is doing with HoloLens centers on the use of Skype. With Skype on HoloLens, technicians can be hands free while on the job, even when making remote calls to subject-matter experts and sharing holographic instructions between users. This enables more flexibility while also complying with safety regulations. In initial trials, use of HoloLens has reduced the average length of thyssenkrupp’s service calls by 4X. Those are impressive results indeed; this case is an indication for the potential value and market size for this type of AR collaboration.
HoloLens plugin for the Windows Skype client
If you use the Skype client for Windows when you connect to a HoloLens skype client, you are asked if you want to download the HoloLens plugin for Skype. Once the plugin is installed, you get an Edit Scene button overlaid on the video (which shows what the HoloLens user is seeing):
Clicking this button will freeze the video and give you a dial of controls, which allow you to place arrows, draw freehand and place pictures; all as holograms. When you are done, you click Stop Editing and the video unfreezes. The HoloLens user can also add holograms and you both see all holograms, but you can only change your own holograms:
As always in the HoloLens, all holograms stay rock-solid rooted to their exact location in the real world; if you like, you can fill up your entire environment. Here a I walked up close to some holograms I added from the Windows client:
You can also share your desktop or a window from the Windows Skype client, and it appears in the HoloLens:
The remote desktop view is fairly live but of limited dimensions and resolution (for the keen observer, there is currently a bug that flips the remote screen horizontally).
Using Mobile Skype clients
While the Windows Skype client offers the most functionality, the HoloLens Skype client works fine for video calls with the mobile Skype clients. Here I am calling myself on the Android Skype client from Skype on HoloLens:
The mobile client shows all that you as HoloLens user see including holograms that you add, and you see what the remote mobile device camera sees (the remote expert can select front or back facing camera as needed).
So even when the remote expert that you as a HoloLens user need help from is on the move, he or she can still assist you – although without creating holograms in your world.
Potential
Technically, the HoloLens is a first device on the Windows Holographic platform, which Microsoft recently opened up to partners to allow AR devices of multiple brands to collaborate in the same augmented (mixed) realities. Windows Holographic which will also be available on all mainstream Windows 10 PC’s next year.
The HoloLens today can run normal Universal Windows Platform apps in 2D windows. I could run a Xamarin Forms sample app that targets Android, iOS and UWP mobile devices just fine on the HoloLens:
So developing for the HoloLens can start really simple, with existing, proven (cross-platform) technologies which require no new skills.
In addition to this, VR is about to go mainstream in the mobile world: on October 4 Google announced the Pixel phones and the $79 Daydream View headset that support the standard Android VR platform, Daydream. Once VR goes mainstream in the consumer space, it is a small step to AR in the business space. A first step could be adding support for mixing the real world into the virtual world via the phone camera (as is currently done by the HTC Vive).
Functionally, the current HoloLens device and Skype app already enable valuable scenario’s, as the thyssenkrupp case demonstrates. But however great the communication tools are, if you can’t get the right person to assist you, no value will be realized. This is where a good dialogue management solution can help; to get you the right expert in your moment of need. More on that some other time…
All in all, AR is all set up to grow tremendously really soon. These are interesting times.
(this post was also published at Anywhere365.IO)
