Interview of Shanee Nishry, Games Developer & Advocate at Google

At the Devoxx Anvers conference, I met Shanee Nishry who talked about Sense override (Cardboard).

Ludwine : Can you introduce yourself in a few words?

Shanee : I am a game developer, technology enthusiastic and futurist. My current focus is on games, graphics and virtual reality, but artificial intelligence and bio­technology also holds a special spot in my heart.

Ludwine : You talked about Virtual Reality and presented “Senses Override (Cardboard)” at the Devoxx conference, Anvers. Could you tell us more about it?

Shanee : Definitely! I talked about the current developments in virtual reality (VR), where it is now, what’s out there, and how people can start working on it. I am very excited about the growth in adoption of VR, as I feel it can completely transform media consumption and entertainment and so much more. In one specific future use case, imagine surgeons trained via extremely accurate computer simulation or even operating by controlling a robot with camera, which moves its tools according to motion controlled tool the doctor is holding. This could be the case when a patient cannot be transported near a qualified doctor.

Simpler implementations would include games and complete 3D movies where you are transported into the scene and follow the characters, completely immersed by the action and atmosphere. There are applications for education, where you can teach a class about space, physics and chemistry through the safety of their seat, taking them to places which would be too dangerous or impossible. Speaking of impossible, it can also help people with disabilities travel, even if they normally can’t.

This technology is evolving so fast now, but not many people know of it. It still has quite a way to go, but people can already create their own virtual reality applications and help us evolve this field.

 

 

Ludwine : This year at the Google’s booth, the cardboard was the star. I tested it and it liked to contrast between this cardboard and the VR you can see inside. It’s very intriguing. How does it work?

Shanee : It’s quite simple actually! There are three main components: the cardboard headset, lenses and your phone.

The Cardboard is just a cover to fit the screen, block environmental light and host the lenses. The lenses then focus your large field of view into the screen which sits inside and generates pictures for your eyes to see. The last bit is the software itself which anyone can make. The idea is to create a split view rendering, drawing what the left eye would see on half the screen and the right eye’s image on the other half. These images are generated by placing a virtual camera in a scene, for example a 3D game. The camera is offset in position and orientation for each eye, creating an accurate representation for what it would see and resulting in the stereoscopic image.

There is a tiny bit more improvement due to the distortion of the lens and each of those images can benefit by applying Barrel Distortion to correct for it but that’s the main story. The last interesting bit is the magnet on the side of the Cardboard device which is used for input, which I thought is really smart. As the screen can’t be accessed when it is in the headset, a different solution is required. Some people might pair their phone with a bluetooth controller but not everyone has one. On the other hand, most phones have a magnetometer used for compass so by moving a magnet near the sensor you can create a noise in the magnetic field, this noise is then converted into user input, acting as a trigger.

In addition to this, Cardboard also contains an NFC sticker to detect when the phone is inside and a rubber band to keep the phone in place.

 

Ludwine : You love working on games, graphics and virtual reality. What is the last surprising things you’ve seen/discovered about it?

Shanee : Unfortunately, there aren’t many virtual reality games available yet, as both hardware and software still needs to catch up. It is very exciting to see all kind of peripherals coming along to enhance virtual reality, such as three dimensional treadmills from Virtuix, head mounted motion cameras like the LeapMotion and different type of controllers like gloves and wearable sensors. There are also very interesting games. I’m currently playing Dragon Age Inquisition, which isn’t a virtual reality game, but I really love the level of detail and graphics there. It is amazing how much graphics progressed in just about a decade and how much it is still improving. This also makes me excited about mobile. It’s amazing that the latest smartphone has about the same computing power as the Xbox 360 and Playstation 3. I can’t wait to see where we will be in just a year or two.

 

Ludwine : One last message?

Shanee : Keep a look for what’s out there! Things aren’t always obvious. I work with game developers and a lot of people think virtual reality is only for games, but it can be for all kind of things from architecture design and social medium to even robots, like in Avatar. Make exciting things and help shape our future. Someone’s got to do it and waiting for it to happen is boring.

 

Thank you Shanee !

Her talk about Senses Override (Cardboard) on Parley’s.