Meta AR – Interaction Design

The most fundamental interaction that signified the usage of Augmented Reality at Meta, was known as Hand Grab, the motion of the user closing their hand around an object visible through the Meta headset. In the Meta Workspace, the visual design of how this UX interaction worked was critical to ensuring proper visual feedback and affordances necessary to familiarizing the user with the interaction.

In the Workspace, the launcher for applications or 3D models was a Shelf within the AR space, containing visual 3D representations of each content. The user would need to reach out towards one with an open hand, and then close their hand around the desired object in order to "grab" it. The sensors and cameras on the headset would track the user's motion and the system would interpret those motions to set the values which trigger that reaction to connect the object to the user's hand. The nature of this system made it so that it was necessary to incorporate many visual responses to the reaction to alert the user that hand grab was achieved. These included visual changes on both the Shelf and Object, such as animated scale enlargement, lighting glows and visual effects.

Video showcasing the Hand Grab Interaction:

Task

Design the UX and Visual interactions for Hand Grab in the Meta Augmented Reality software