Unity, C#
Project Overview
Elemental Worlds, an interactive exhibit at Science Museum Oklahoma, is a two-part hardware and software system co-authored with Pete Froslie. Designed as both a playful environment and a research platform, the installation immerses visitors in a dynamic, ultra-wide panoramic projection where custom-designed digital creatures inhabit a living, responsive virtual landscape. Visitors move fluidly between designing their own creatures at touchscreen stations and seeing those creations come alive within a richly simulated ecosystem projected across a 180-degree panoramic screen.
Interaction Design
The experience begins with drawing: visitors use a touchscreen interface to design digital creatures through intuitive tools for shape, pattern, and color. Touch gestures are employed to sketch forms, apply colors, and select from a library of drawing tools. Once complete, the 2D drawings are dynamically mapped as surface textures onto procedurally generated 3D bodies, instantly transporting the visitor’s creature into the surrounding environment. This act of creation bridges personal authorship with collective play, as multiple creatures from different visitors populate the same shared world.
Virtual Environment and AI Ecology
The immersive projected environment is a dynamic simulation that evolves across time and condition. Day and night cycles, changing weather, and atmospheric effects modulate light, shadow, and color, producing a sense of environmental continuity. AI-driven wildlife behaviors animate the creatures, allowing them to graze, wander, or interact with one another in ways that feel spontaneous and lifelike. The system blends authored rules with emergent unpredictability, offering visitors the experience of watching their creations adapt within a collective ecosystem.
Technical Development
A key research contribution of Elemental Worlds was the development of custom processes for real-time texture mapping — consuming two-dimensional visual input and wrapping it dynamically onto three-dimensional geometry. This workflow required bridging drawing systems with generative modeling, creating a seamless pipeline from human gesture to real-time simulation. Additional technical layers included AI behavioral modeling, environmental simulation, and the integration of multi-projector edge-blended panoramic visuals, producing a cohesive immersive experience.
Spatial and Experiential Design
The environment itself was conceived as both spectacle and stage. Visitors stand within a circular platform enclosed by railings, surrounded on all sides by projected landscapes. Blacklight-reactive murals and sculptural interventions frame the entry sequence, extending the sense of immersion into the physical space. Once inside, visitors are enveloped in the projected world, their silhouettes often overlapping with the virtual terrain — blurring distinctions between physical presence and digital environment.
Impact and Engagement
Though playful and lighthearted in tone, Elemental Worlds functions as a deeper exploration of human–computer interaction, collective authorship, and ecological simulation. For children, it is a place of delight and discovery. For researchers, it demonstrates how digital systems can capture and replay public creativity, transforming ephemeral gestures into persistent, evolving artifacts within a shared world. The installation reframes the museum as a laboratory for experimentation, where visitors are both audience and co-creators.
Conclusion
Elemental Worlds demonstrates how immersive media, responsive ecosystems, and participatory design can converge into an architectural-scale experience. By turning drawing into digital life, and spectatorship into ecological participation, the project challenges the boundary between creator and environment, offering instead a vision of museums as places where play and research fuse into a single, shared experience of wonder.
