Here’s a video that chriszaharia uploaded showing how he integrated Sony’s HMZ-T1 head display, Kinect, speech recognition software and more to play Skyrim in a virtual reality environment. Video and details after the break!
Via video’s description:
Skyrim played through a simulated Virtual Reality environment using a head mounted display, head tracking, motion sensing and speech recognition.
I’ve also written up a walkthrough on how you can do it yourself too:
http://zookal.com/blog/2012/03/how-to-play-skyrim-in-virtual-reality/This is what was used for the demo:
– Skyrim – A first person action role-playing game for the PC.
– Sony HMZ-T1 – High definition head mounted display with Stereoscopic 3D.
– Microsoft Kinect – Track movement and gestures.
– TrackIR 5 – Head tracking.
– TrackClip Pro – Used with the TrackIR 5 for more accurate head tracking.
– Shoot – Speech recognition software.
– FAAST 0.9 – Software used to map Kinect gestures to the keyboardI’ve setup the Sony HMZ-T1 head mounted display to use Stereoscopic 3D as well as attached the TrackClip Pro on it for head tracking. I had to place the TrackIR 5 on a wire hanging from the ceiling as it needed to be around head level to track my head movements properly.
The Kinect was setup on the PC using PrimeSense’s OpenNI drivers. I used FAAST 0.9 with a custom script to map certain gestures with the keyboard, such as walking on the spot to move in the game, leaning left, leaning right, jumping and moving my right arm forward to use the sword.
Shoot was used to map several speech commands with actions in the game as well, such as saying ‘Open Menu’ and ‘Close Menu’ to trigger the in-game menu, as well as using commands to navigate through the menu.
All the devices to do this cost less than $1,500, although it is still not smooth and responsive enough yet for a decent gaming session. Hopefully in the near future this concept of gaming will be more refined.
Amazing stuff!