//****************************************************************************//
//******************* Virtual Reality - April 17th, 2019 ********************//
//**************************************************************************//
- Don't be happy it's today because it's not tomorrow; be happy that it's today. And when tomorrow comes, even if it feels rough, be happy for that, too; it'll pass!
- Also, fill out the CIOS! Greg Turk's done an awesome job teaching throughout this whole semester, and he deserves some credit for that!
--------------------------------------------------------------------------
- Alright, we're gonna talk about VIRTUAL REALITY today, and there're 3 BIG pieces any VR system needs:
- We need to track the user's head motion
- We need to create images of the virtual world in real time
- We need to display those images convincingly to the user
- The history of VR is actually a rather long one, with 3 sort of distinct "waves" in history
- In the 1960's, Ivan Sutherland first started looking into the fundamentals of VR
- Ivan Sutherland (who's still alive as of 2019!) is considered the "father" of computer graphics as an academic field, and he famously created the world's first GUI-based program (a constraint-based design package called "Sketchpad," which directly inspired CAD) and arguably the first object-oriented programming language
- He also famously described computer graphics as "a looking glass into a mathematical wonderland"
- After winning a Turing Award for his research, he worked at the University of Utah for several years trying to create an "Ultimate Display" that would be as close to real life as possible
- As a prototype, they built a device with all 3 of the key VR components (a tracking device, a head-mounted display, and real-time rendering), but it was actually an AUGMENTED REALITY device, projecting virtual stuff onto the real world
- However, the device was bulky and INCREDIBLY unwieldy, and required a ceiling-mounted pipe to track movements, earning it the nickname "The Sword of Damocles"
- Several research labs continued pursuing this kind "virtual environment" work (such as Henry Fuchs at UNC (Professor Turk's Phd. advisor)), but it certainly wasn't mainstream
- In the late 1980's, Jaron Lanier founded a company called VPL - "Virtual Programming Language" - and coined the term "virtual reality"
- VPL sold complete virtual reality systems that had all the key components, a magnetic motion tracking system, and a "data glove" that let people use their hands to interact with the virtual world
- ...it wasn't very accurate at ALL, but it did work!
- The work Lanier did here drew a LOT of attention from the tech field, so what happened?
- Well, the technology at the time was still a little crude, and it didn't live up to the hype: the images weren't realistic, the tracking wasn't good enough, it was heavy, and motion sickness was a constant issue
- MOTION SICKNESS is when there's a disconnect between how fast your eyes see you moving and the acceleration your body feels
- Some people have thought this is because the body thinks you've been poisoned when you get dizzy, leading to good ol' vomit-on-the-wall syndrome
- There was also no obvious market for the tech - everyone thought it was cool, but no one wanted to actually buy it!
- So, once again, VR work died off, and was confined to a handful of research labs
- Then, in 2010, Palmer Luckey made a custom head-mounted display prototype that had surprisingly good motion tracking - enough to significantly decrease the motion sickness problem
- After a successful Kickstarter campaign, he founded Oculus VR to expand on his display, and in 2014 Facebook bought Oculus for ~3 BILLION dollars, basically confirming that this was a field they wanted to take very seriously
- That's some of the history, but what about the tech that goes into VR systems today?
- Nowadays, we have GPUs EVERYWHERE that can render millions of polygons at well over 60 Hz, which is a MASSIVE advantage we have over the early 1990s VR attempts
- Because of this, image synthesis itself is mostly a solved problem, perfect photorealism notwithstanding
- More tricky is the Head-Mounted Display, which'll be how we display these images to the user
- We need the HMD to deliver separate images to each eye so that we can get a 3D parallax effect
- Oftentimes this is done using 2 small LCD or OLED displays, with lenses to make the image appear at a comfortable distance (instead of being mashed directly into your eye)
- Ideally, this HMD will also have a wide field-of-view
- Having it be as light as possible would also be nice, both for user comfort and because heavy headsets can contribute to motion sickness
- Wireless headsets would also be a plus, so that users can't trip on wires as they walk (that's an actual concern!)
- Let's take a closer look at the Oculus Rift's headset, specifically
- The headset is able to obtain a pretty wide field-of-view using FRESNEL LENSES to focus the light
- This means that, instead of a traditional thicker lens, we can get a similar focusing effect by having "slices" in the lens; in exchange for a little bit of image quality, we get a much smaller, lighter lens!
- The last piece of the puzzle is tracking systems - and there's still a LOT of variability in how these work!
- Our main concern here is to figure out the x,y,z position of the user's head AND their head's orientation, all at 60 Hz or faster
- We then need to pass this information to the rendering system, render from the user's position, and put that image on the screen
- As mentioned, there's several techniques we could use:
- Magnetic tracking systems have fallen out of favor, since metal inside the room can mess up their calibration
- Nowadays, OPTICAL systems are pretty popular
- "Outside-in" systems uses an external camera to track "beacons" placed strategically on the HMD
- This is the system Oculus uses, where they have invisible Infrared LEDs scattered around the HMD, which the camera observes
- The camera then uses traditional computer vision techniques (triangulation, etc.) to figure out where the HMD is; the more beacons it observes, the higher its accuracy
- However, the accuracy here decreases as we get farther away from the camera, making this technique best for seated positions where the user isn't moving around much
- "Inside-out" systems put the camera on the HMD itself, and then try to observe reference points placed throughout the room to figure out where it is
- An example of something that does this is the HTC Vive, which has a larger tracking area than the Rift
- Here, there are two "lighthouses" that sweep the room with a sheet of laser light: one horizontal sheet, one vertical
- The headset then has a bunch of light sensors on it (not true cameras per se) that track this laser light to figure out its position
- So, what're the challenges for VR today?
- Well, motion sickness still remains a problem; it's getting better, but it's not perfect by any means
- Some people have almost no motion sickness issues, while for other people it's debilitating
- There's also the lack of a "killer app" that'd give a reason for everyone to buy this
- Video conferencing, video games, data visualization, medical operations - there're some use cases, but there hasn't been anything so compelling yet to justify everyone buying a $300 headset
- Augmented reality might also be a thing to watch out for - the two big systems around today (Magic Leap and Hololens) are both pretty similar and aren't perfect, but it's still early days
- Is this 3rd round of VR going to be "the one?" I don't know, and the market certainly doesn't know, but there are some very large companies pushing to make this a reality
- Alright, two more lectures to go! Come Friday to hear the next one!