VECTOR SPACE BIOSCIENCES: THE PROTEIN NETWORK

With the intricate nature and breadth of DNA data, space bioscientists expressed the concern of needing to interact with information spatially to find discoveries of different aspects of DNA that can be activated when radiation tears down proteins in the body. I led the XR product strategy and development of an AI-driven protein-discovery platform that turns a correlation matrix dataset into visualizing a language model spatially in XR or traditional screen - which allows scientists at NASA GeneLabs, ICL London, AstraZeneca, etc. to dive in and interpret known, and most importantly, hidden relationships between proteins, pathways, drug compounds, and molecular sequences which enabled novel discoveries.

Built foundationally for AR with webXR, AR Foundation and three.js — dyanmic frontends to work with both XR and traditional screens.

Collaborated on with fellow key stakeholders of the Vector Space Biosciences product and engineering teams, primarily Kasian Franks, Mark Lam, and Arina Maltseva.

 THE PROCESS

The vision disability I focused on for this scope is retinitis pigmentosa. People with this condition slowly lose their vision over time from outward in. There is a pit of clarity in the center, which shrinks to increased blurriness, to eventually loss of sight.

I decided to work towards creating the vision effects to be applied directly to the headset - or the camera in Unity - and to keep them as a completely separate layer from the environment. I felt this to be a scalable idea for the breadth of this project, so essentially the environments can be changed and swapped out separately.

I spent a lot of time thinking through how to achieve the pivotal blurry bulk of the vision giving way to the clear center. And I worked on scripting a shader to create this, which essentially adds different pixels together for a coordinate, and then consecutively does that throughout a region to create a wider blur effect:

 
 
 
 
 
 

As a comparison, the upper image below is what the scene would look like standard in the headset with no effects…and the image below that is what the scene looks like with the retinitis pigmnetosa vision effect I created:

 
 

From there I set out to accomplish the next layer of the project: creating as photorealistic environment visuals as I could – in this case being on a subway train, as the crux of the narrative is around managing and adapting for wayfinding public spaces.

I first built the structures of the subway train, and subsequently worked on materials, textures, and lighting to provide as realistic an impact as possible. Finally, I conceptualized a live-action shot, 360-degree image of a subway tunnel nearing a station as the skybox of the Unity scene to set the environment seen through the windows of the train to be etched in reality. Pictured below is the evolution of that work:

 
 

And as a comparison, the upper image below is what the built environment scene would look like standard in the headset with no vision disability effects…and the image below that is what the scene looks like with the retinitis pigmnetosa vision effect from before atop the created environment:

 
 

And I am really excited to present for the first time what it is truly like to see with retinitis pigmentosa.

 
MySight R.gif