The Pitch of Red was a VR experience meant simulate the synesthesia, a neurological condition that couples two senses at once. This experience was conceived and designed in a team of five, at Simon Fraser University which consisted of myself, Leo Danenkov, Vlad Ryzhov, Amber Shao, Sheri Wong. Designed for use on the Oculus Go, The Pitch of Red was made in Unity during an intensive 1-month development cycle from at Simon Fraser University. I contributed towards designing the opening tutorial level and programming the gaze-based user interaction.
Very early in the development, I knew that a tutorial stage would be necessary for the overall user experience because awkwardly having to explain the controls to the users AFTER they have worn head mounted display would negatively affect the user’s presence in the virtual environment. Thus, the user had to learn two tasks in the tutorial, how to move through the environment and interact with an object by looking at it.
To keep the user’s attention focused, I wanted to keep the visuals of the tutorial level to be as minimal as possible. The color blue was used sparingly and only used when either guiding the user to where they had to move to or help them identify what they had to look at in the environment. This use of visual contrast was both unobtrusive to the interaction and explicit enough for the user to direct their attention.
The type of synesthesia my team and I wanted to evoke was the association of specific sounds when looking at specific colors. Since gazing at colors served as the primary method of interaction between the user and the virtual environment, I created a prototype where a sound would play when the user looks at an object in the environment. I then iterated on this prototype to play different sounds depending what color the user was looking at, incorporating various instruments you would hear during the .
Overall we thought that the project was successful in communicating synesthesia given the positive response from the users on the day of the showcase. Pitch of Red was even covered by my faculty's official Facebook page.