Designing the spatial interaction language for ARious, an iOS AR learning app for young learners, by combining principles from architecture, environmental psychology, and HCI to help users transition from 2D screens into 3D interactive content.
ARious had a clear content advantage in the AR learning category. The lessons themselves were beautifully crafted, the 3D models were accurate, and the educational design was strong. What it had was a usability gap that no amount of content polish was solving.
First-time users would scan a target, see the 3D content appear in front of them, and then freeze. They did not know whether to move their device, walk around the content, tap it, or wait for something to happen. The transition from the familiar grammar of 2D apps (taps, swipes, scrolls) to the unfamiliar grammar of AR (physical movement, spatial relationships, gestural interaction with 3D content) was producing a learning curve that the product could not afford.
The broader product world in 2020 was at an inflection point with AR. Apple had released ARKit, the underlying tech was rapidly improving, and several AR learning apps were trying to crack consumer adoption. The category had a chicken-and-egg problem: users had no mental model for AR because few products existed to teach them, and products struggled to retain users because users had no mental model. ARious needed to solve the mental-model problem inside its own product without waiting for the broader category to mature. The brief I took on as a UX apprentice was to design a spatial interaction language that users could pick up in seconds rather than minutes, and that would work for young learners specifically rather than for adopters in general.
I researched two adjacent disciplines that have decades of thinking on spatial human behaviour but that the AR product community in 2020 was rediscovering from scratch. Architecture and HCI each had frameworks that mapped almost directly onto problems ARious was trying to solve, and a lot of the research phase was identifying the right concepts in those disciplines and translating them into AR design vocabulary.
Architecture taught us how physical environments suggest paths, focal points, and zones without explicit instruction. A well-designed building does not need signage to guide first-time visitors through it, because the architecture itself communicates where to go. The same principle applies to AR content: the placement, scale, and visual treatment of 3D objects in space can suggest interactions without requiring explicit tutorials.
HCI gave us frameworks for affordances and mental models in spatial computing. The literature on direct manipulation, on natural mapping, and on perceived versus actual affordances had been developed for 2D interfaces but transferred remarkably well to 3D ones. The translation work was identifying which existing AR conventions matched HCI principles and which were violating them.
The research output was a spatial vocabulary specification: a small set of interaction patterns, with explicit rules for when each pattern applied, what it should look and feel like, and how it should teach itself to first-time users. This specification became the central document the rest of the team used for any AR design decision.
The interaction language for ARious combined principles from both disciplines into a small vocabulary that was deliberately small enough to learn in one session.
Subtle visual anchors tied the content to a physical plane so users felt invited to walk around it. The anchors used soft shadows, ground contact lighting, and gentle motion cues to create a sense of physical presence. Users moved their device naturally to look around the content because the content felt like it was actually there, rather than feeling like a screen they needed to navigate via gestures.
Gestural affordances were borrowed from familiar AR conventions where possible (pinch to scale, two-finger rotate, tap to select) and explicitly taught only where the convention did not yet exist. The principle was that users should encounter the minimum number of unfamiliar gestures, and any unfamiliar gesture should be introduced in a context that made its function obvious.
Focal-point design drew on architectural principles of visual hierarchy. Each AR scene had a clear primary focal point that drew attention naturally, then secondary elements arranged in a way that suggested an order of exploration. Users moved through the scene in a predictable progression because the scene itself directed their attention, not because the app instructed them to do so.
The result was a small vocabulary of spatial gestures and affordances that users could learn in one session, that scaled gracefully across the variety of lessons ARious offered, and that did not require explicit tutorialisation to introduce.
Reject heavy tutorialisation. The default approach in AR onboarding circa 2020 was heavy tutorialisation: long overlay sequences, animated guides, explicit instructions for every gesture. We went the opposite way. Every tutorial element added is one a user has to read instead of doing, and tutorials in AR contexts have particularly poor retention because the user is already cognitively loaded by the novelty of the medium itself.
Design the content to suggest interactions. I argued for designing the AR content itself to suggest interactions through its placement, lighting, and motion, and using overlays only as a last resort for genuinely novel interactions. This put more pressure on the spatial design but produced a much faster path to confidence for first-time users.
A small vocabulary, deliberately constrained. The temptation in defining an interaction language is to include every possible interaction. We deliberately constrained the vocabulary to under ten patterns total, on the principle that a small learnable vocabulary serves users better than a comprehensive one. Lessons that needed interactions outside the vocabulary were redesigned to fit the vocabulary, rather than the vocabulary being extended to fit them.
Borrow from disciplines that have already solved this. Architecture had spent centuries thinking about how built environments guide human movement and attention. HCI had spent decades thinking about affordances and mental models. AR product design did not need to reinvent these wheels; it needed to translate them. Most of the design output came from translation rather than invention.
Anchors, paths, focal points
The spatial design vocabulary became reusable across new AR lessons the team built after the original work, which was the most valuable outcome. Future lessons did not have to reinvent the interaction model, which compounded the original design effort across all subsequent content. The team could focus on educational content rather than rebuilding interaction patterns lesson by lesson.
Beyond ARious itself, the work shaped how I think about spatial computing in general, which became more relevant as the broader product world started taking AR more seriously in the years after the project. The principles of borrowing from architecture and HCI, of designing to teach rather than tutorialising explicitly, and of constraining vocabularies to learnable sets have all stayed with me across other projects in different domains.
The specification document itself outlived the apprenticeship and continued to be referenced by the team after I had moved on, which is the truest test of whether a design system survives or quietly dies with its original author.
Working in AR forced me to look outside design's own canon for answers. Architecture, environmental psychology, and HCI all had decades of thinking that the AR product community was rediscovering from scratch in 2020. The mistake I would have made if I had not done this research is to invent design solutions that were already invented elsewhere, just dressed in product-design vocabulary.
I have stayed curious about adjacent disciplines ever since this project, because design's hardest problems often have been quietly solved next door by people we do not usually read. Architects, environmental psychologists, theatre directors, and game designers all have things to teach product designers, and the willingness to look outside the canon has produced some of my best work since.