Imagine a world where the boundaries between the physical and digital realities blur, where you can see objects and information floating in front of you, and where you can enter whatever digital environment you want to. This might sound like science fiction, but this is already happening thanks to technologies such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). These technologies are quickly evolving, changing the way we work, learn, play, and interact with the world around us. From the use of VR headsets that allow us to explore new worlds and enter new realities, to AR apps that show information onto our physical surroundings, to MR that blends the real world and virtual world, we are now entering an era populated by human-computer interaction. As these technologies evolve, they aren’t just changing what we see, but they’re also changing how we experience and interact with the world around us.

Defining the Reality Spectrum

Before diving into their potential, we must first understand what VR, AR, and MR are. Virtual Reality (VR) is the most popular and most known out of the three, being used in gaming, education, and industry. It immerses the user in a digital environment, fully disconnecting them from the physical world around them. Augmented Reality (AR) overlays digital information and/or objects onto the real world, with the means of enhancing the way we interact with our physical environment. Mixed Reality (MR) blends real world and digital elements, allowing the user to not only see the digital information and content like AR does but also allows the user to interact with digital objects in a physical space like you can in VR.

Current State and Uses of Technologies

Today’s current state of VR/AR and MR offer many different ways that the digital world can blend with our physical world, with each technology finding a specialized use across industries.

VR, used with devices like the Meta Quest 2, allows the user to experience a fully immersive virtual world. VR excels in gaming and entertainment currently, but it is also finding its way into education, healthcare, and training for jobs. Surgeons could practice procedures, people can go to therapy and work on their phobias in a controlled environment, and workers can go through safety training risk free in virtual environments. The user experience in VR is unique, because it completely replaces the user’s visual and auditory senses. The immersion that VR can bring can lead to a strong sense of presence, like you are really there, but it can also cause motion sickness which is a big problem UX designers need to look at when it comes to VR.

AR has become widely used in smartphone apps, allowing us to enhance our viewing of the real world with digital information or overlays. Although, there are popular games like Pokémon Go that are examples of the use of AR, AR is also being used in retail with virtual try-on experiences, aiding with directions in real time, helping surgeons by assisting them during complex surgeries, and making immersive learning experiences for students. AR’s user experience is all about blending digital information into the real world. UX designers focus on creating interfaces that don’t overwhelm the user’s natural interaction with the environment.

MR, used with Microsoft’s HoloLens 2, can blend virtual objects with your physical space. It is playing a critical role in industrial and architectural design, allowing them to interact with 3d models in real time which not only helps in general but can help with the creative process as well. It is also being used in maintenance and repair, where they can work on machinery with digital instructions. MR is currently more limited than AR and VR as it isn’t as accessible because of hardware and cost. MR presents the most difficult UX challenge of blending digital and physical interactions. UX designers have to create interfaces that feel natural and good in both areas.

As these technologies evolve, user interfaces are becoming more natural to people by adding hand tracking, eye tracking, and voice commands. UX design will play a crucial role in making these technologies more accessible. AI will also play a crucial role in the development of these technologies. AI can enable these technologies to understand the real world more, help recognize objects, and interact with the users which will overall create a more realistic and engaging experience for its users. As ergonomics and software improve, these technologies will integrate even more into our daily lives, potentially changing how we go about our days as a whole. In the future, these technologies can change how we work, learn and interact with the world around us.