Abstract

The use of virtual and augmented reality (XR) technology in both gaming and professional applications like sports, training, and design presents significant challenges at the intersection of engineering and creative endeavors. Both human and hardware limitations must be overcome through design and iteration in order to effectively develop XR applications. We present a series of learnings gained through observation, iteration, and empirical evidence, and develop a set of rules for creating deeply interactive applications in these mediums. These components of a larger developing grammar are borrowed from the respective grammars of previous media, such as games and film, but are evolved to leverage the strengths unique to XR. Through using case studies from consumer products like Robo Recall and Bullet Train, as well as examples from other media, we will show how we start with a theory of interaction, and refine it through a combination of observation and iteration towards something that is accessible and enjoyable by users. By adapting to the players’ experiences, we can also improve their performance at the task, which suggests that these same techniques can be leveraged for training in sports or other skill-based endeavors. Finally, we will highlight how technological tools can help democratize this iterative process, to allow cross-discipline team members to collaborate more quickly and effectively to achieve high-quality results.